When you replay a streaming pipeline from an amount of time ago, do you lose the already-processed data from before that time window? For example if I have a pipeline with an output object type. The pipeline has been running for a week, so I have object A in the ontology from 5 days ago. If I replay the pipeline from 2 days ago, does object A disappear from object explorer? Or does object A just not get reprocessed with whatever pipeline change may or may not have affected it?
Hey @groth! From the docs : “When you replay a pipeline, all data in the output stream is lost. If you wish to retain the data from the previous stream, you can direct the output to a new destination.”
https://www.palantir.com/docs/foundry/pipeline-builder/outputs-deliver-pipeline/#redeploy-vs-replay
This won’t delete your object though
So the object would still persist?
What would directing the output to a new destination look like? Swapping the pipeline output for a new stream so that you still have the old output?
Yes the object won’t be deleted.
You could create a new streaming output with the replay if you didn’t want to delete the data from the original dataset
1 Like