I have an ontology object called reconciliations. The ontology object has a backing dataset. In workshop I use an object table and create new/edit actions to create new reconciliation entries and update existing ones.
I am expecting the backing dataset to also reflect these new entries and updates but I do not see any new data in the backing dataset. I read the docs on materializations but I’m still having a bit of trouble understanding how to implement this solution where I am able to retain edits and new entries in the original dataset.
Are there any examples or best practices to follow for this?
Why do you want to retain edits in the backing dataset?
That dataset, will be overriden the next time your pipeline runs - thus losing the edits again or potentially having conflicts between user edits and updated data in the backing data sources.
In the Ontology Architecture, Edits are handled by the underlying Funnel service and you can’t easily access them.
However, you can create a materialized dataset, which shows the current state of the objects, including user edits.
You can find the documentation here.
This materialization should contain the exact information you are looking for.
I’m not sure the purpose of a backing dataset then unless it’s to display initial information.
And with the materialization it’s just a current view right? So if I need to maintain a history of edits somewhere accessible by a sql editor or even write to another database, what is the option?
The purpose of the backing dataset is to represent the data from your source system(s).
Unfortunately, there isn’t a native to get all edits on an object.
One possible way is to use Action Logs, for all actions that are linked to an object.
1 Like
got it, i’ll look into action logs. Mostly I just need to retain a history of the changes since my users want an audit trail.