Sometimes I rewrite code transforms for a system that has operated for a long time. I can’t easily run the code incrementally because there is no new data to process, and it’s not easy to turn incremental off because the backing data set is now too large to fit in memory.
It would be nice if there were a setting where I could test incremental code transforms by rerunning the last batch of data that came in incrementally. You could even have a nice comparison of the output data with the two methods