I’m trying to write an Incremental Java Transform, and want to understand what the best practices are for doing so.
In the documentation here, it seems like we have to write methods to handle and decide ReadRange and WriteModes based on the input. This seems to reproduce Python’s @incremental annotation logic.
Is the best practice then to re-do this logic via these methods or is there something smarter to do here? It feels like the alternative is hardcoding WriteMode and ReadRange which doesn’t feel as flexibile as the @incremental python annotation
Fwiw, AIP assist does the switch/case methods from the docs (as expected)
The flexibility and verbosity of Java transforms do require you to implement your own incremental logic. You can make it slightly less verbose by using fall throughs for conditions you want to be handled in the same way.
The upside is that you can implement more opinionated logic for how to handle incrementality than would be available to you in Python Transforms using the @incremental decorator (e.g. if you want to snapshot if more than X% of your inputs change or on the third Tuesday of every month).