Object Storage v1 to Object Storage v2

Ask the Community
, ,
1

Hello everyone,

My team manages Ontology for several clients, and we are currently planning a migration from Object Storage v1 to Object Storage v2.

I wanted to reach out to the community to see if anyone has already gone through this migration and would be willing to share their experience. Specifically, I’m interested in:

  • How long the migration took in your environment
  • Any unexpected issues or challenges you encountered beyond the requirements and limitations documented by Palantir
  • Any lessons learned or best practices you would recommend

We have been asked to provide an estimated downtime window to our clients. Since this is our first Object Storage migration, we currently have a preliminary estimate of 3 hours, but we’d love to hear from others who have gone through the process and can provide real-world insights.

Thanks in advance for any advice or experiences you can share!

2

Hi @matchatea,

The migration time will depend solely on the resources available on your enrollment, as well as the size and shape of the data you are migrating.

If knowing the migration time is essential, you could try to set up a new Object Type, backed by an existing dataset, convert that to OSv1 and (as this defaults to OSv2) then convert that back to OSv2 and track that migration as a measure. Keep in mind my previous point regarding the size and shape of the data.

Note that the migration will still allow users to access data, but not edit it.

For one migration of a production Object Type, I chose to stand up a new OSv2 object type and swap that one in, as this was receiving a lot of continuous edits.

My approach was as follows:

  • Map out how the current v1 Object is being used, capturing all applications, use cases, etc. that consume it and all actions/functions tied to these.
  • Stand up the new v2 Object, using the same data source as the v1 Object.
  • Create actions that mirrored the ones on the v1-backed Object.
  • Tag and release new functions where code updates were required.
  • Set up tests for all functions and actions (can be done locally) – a good tip here is to go through the edit history, to properly map out the types of edits the Object is receiving in prod.
  • Rename the v1 Object and Actions in the Ontology (don’t save yet!).
  • Rename the v2 Object and Actions in the Ontology to the API names used by v1.
  • Arrange for a short downtime window.
  • Commit the changes to the Ontology.

The bad thing about the above method is that it can obviously break things if you are not careful. The good thing about the method is that you have minimal downtime.

Alternatively, you can also trial all of this out in a different namespace and then either use the “Migrate Resource” functionality of the Ontology Manager or Marketplace.

Make sure you read the docs and are aware of the changes between OSv1 and OSv2, especially if your applications are currently using the API to edit the Object Types.

My general piece of advice is to sandbox as much as you need to feel comfortable with this, before moving forward. Start small, build experience, catch edge cases and have a plan for reverting back if you run into trouble. Fear is the mind killer and doesn’t help when debugging.

3

Thank you, this is extremely helpful for consideration as we are starting the process for multiple clients soon.

Due to the specific data, we work with, we avoid a lot of the migration limitation.

Still the biggest part is announcing downtime and estimating how long it’s going to be as different client has different amount of data.

Hopefully I can get a little more insights as we work through this next month and share it here for other community members.