I have a Marketplace product containing edits applied through user actions and would like to lift-and-shift the entire product, including said edits, via Marketplace. Is this supported?
I’m not sure if you’re trying to just move over ontology objects but something that we were considering doing when doing something similar was to just export the ontologies as CSVs and write adhoc code to load that into the ontologies in the new Marketplace installations.
(disclaimer, I’m not associated with Palantir, just offering an alternate solution if you haven’t thought of it yet)
Hi @psarnicki
The suggestion with a CSV is a good one. Other ways that can be adopted are:
- You materialize the edits into a dataset and link the object in your target project to the materialization (note you will lose the edits history though).
- You use a function that, whenever an edit is made to the object, automatically applies the same change to the corresponding object in the other installation. However, this approach is controversial if the object is backed by Marking, as it could potentially breach the security perimeter.
1 Like
Thank you for your comments, CodeRepoEvangelist and HugoRodrigues! After confirming with the Marketplace team that there is no built-in feature to support this, we ended up taking a very similar route. We created a “difference dataset” for each materialization dataset, containing only the rows that either differed from or did not appear in the backing dataset at migration time T0. We then merged those “difference datasets” into pipelines just ahead of the datasets that back object types in our new installation. This merge was done so that rows from the “difference dataset” always win.
Besides the limitations described by CodeRepoEvangelist and HugoRodrigues, this blocks upstream changes on any object that appears in the “difference dataset”, preventing pipelines from updating rows that users had previously edited (more details below, if useful). That trade-off was acceptable for us because our pipelines do not update historical rows and our system does not have object types that depend on materializations of other object types. A more complex transform could reproduce two conflict-resolution modes, but this was not needed in our case.
Details on possible updates to new installation’s object_type_a:
-
Case 1: pipeline_1 updates a row present in frozen_materialization_a: the frozen row wins and the update is discarded. This will disable the Apply Most Recent conflict-resolution mode and is the accepted limitation of our approach.
- Related sub-case: at T0 a row exists only in frozen_materialization_a and later re‑appears from the pipeline: this will not happen in our application but we added logic to our merging pipeline to monitor for this.
-
Case 2: pipeline_1 updates a row not present in frozen_materialization_a: the update propagates normally and whichever conflict‑resolution mode is configured on object_type_a decides the winner. This is the desired behaviour for us.
-
Case 3: User edits an object instance that maps to a row in frozen_materialization_a: user edit wins regardless of conflict-resolution mode because it happened after any row in frozen_materialization_a. This is the desired behaviour for us.
-
Case 4: User edits an object instance that does not map to a row in frozen_materialization_a: whichever conflict‑resolution mode is configured on object_type_a decides the winner. This is the desired behaviour for us.
Our system does not use object types that rely on materializations of other object types (object_type_y on the attached diagram), but similar analysis should be performed for such cases. Cases 1 and 2 are especially important as updates can originate from pipelines (dataset_x → dataset_y) as well as from Ontology Actions on object_type_a (live_materialization_a → dataset_y).
1 Like