I have a streaming pipeline that applies a filter on a stream and produces a time series sync. I want to remove the filter, and redeploy the pipeline, however, I would like to avoid replaying the whole stream, as it would take a while and I am happy with the filter not being applied for future data only. Is this the expected behaviour if I don’t select “replay on deploy”, or will not replaying the stream cause the existing output data to be lost?
Hi! If you do not press replay then the output dataset is not reset. Therefore, the behavior you mention will persist, i.e. the filter will not be applied to future data only.
1 Like