I have a pyspark transform in a code repo and I want to confirm if a particular build of this transform ran incrementally. Based on the run time, i believe that it is running incrementally, but I wanted to be sure.
Here is what I see in the logs when I search messages for “incremental”
The jobspec contains this:
But I don’t seen anything in the snapshot/incremental section of the build report:
Thanks,
Andrew
Hello @agirvin
It will say ‘Update’ in the build history tab of the dataset, or if you look at ‘Current transaction view’ in the dataset’s details, as below, you’ll see “Update”, with “Snapshot” being the opposite, e.g. running a complete build, and in that case you would only see the one box in the transaction view.
@agirvin Cool, if that solved your question, mind clicking that this was the solution please? Then other people can see in the summary it’s solved, and quickly see the solution. Thanks.
