I have recently changed the semantic version of my incremental dataset, which has caused the outputs to run as a snapshot.
My transform is configured to accumulate files in the output dataset, by reading in the previous output at runtime.
Is there a way that I can safely store a previous transaction in another dataset for reference in the future? I am worried that the previous transaction in my dataset is going to get cleaned up and deleted since it is no longer part of the latest view.
You should be able to use https://www.palantir.com/docs/foundry/api/datasets-resources/branches/create-branch/ to create a new branch pointing at the transaction in-question. As long as that branch exists, Retention should keep that transaction (and all parent transactions in its view).
There may be certain combinations of custom Retention policies that would result in the transaction being deleted at some point even with the abovementioned branch-creation, so out of an abundance of caution, I suggest that you double-check your Retention configuration.