Allow Upstream Retention in Pipeline Builder

Ask the Community
1

I am looking to allow for my incremental pipeline builder to continue working incrementally even when my upstream dataset has data retentioned out. In python this can be done with allow_retention=True. Is there an equivalent of this for Pipeline Builder?

2

This is the default incremental option for Pipeline Builder, which you can think of as the equivalent to the IGNORE_INCREMENTAL_DELETES option in Java Transforms.