I have datasets A → B that both incrementally get built. The last transaction of A was UPDATE, despite that, B now ran as a snapshot, and I’m not sure I understand why. I see the message below, but this doesn’t tell me much:
It just says that it’s caused by dataset A, but I don’t see any more details why. I do see that the input is “Range of previous view”, however, my dataset B already built after the second to last transaction of A, so technically it should only take this latest UPDATE transaction in consideration, if I’m not mistaken.
Any guidance on how I can debug and understand this would be appreciated.
If it’s of any importance, dataset A is built within Code Repositories and dataset B in Pipeline Builder.
Build logs for dataset A. Search for “snapshot” / “incremental” in the message column and there should be a row stating whether the build ran in incremental or snapshot fashion and the reason for it. This might contain more info.
In the history page of dataset A, select the transaction that triggered the snapshot of dataset B. On the right, check the Transaction details, Job spec and Files section. This could give more insight on whether the build ran in snapshot or not.
I suspect the code for dataset A might not handle a case where ctx.is_incremental is false and that could have caused this. Worth checking that.
