I have an incremental pipeline where my source system sends a full snapshot every sync. My transforms apply CDC (Change Data Capture) to detect inserts, updates, and deletes, then write only the changes as APPEND transactions to the output dataset. Each row has a last_upd_tmstmp (timestamp) and is_deleted (boolean) column. Over time, when a row is updated, the new version is appended but the old version still exists in a previous transaction — so the same primary key appears across multiple transactions. This works fine for the Ontology (it deduplicates by PK, latest transaction wins), but I now want to use this dataset as input for an ML model. For the first run, I need to read the full dataset and deduplicate (keeping the row with the highest last_upd_tmstmp per PK, filtering out is_deleted=true), then for subsequent weekly runs I only need the latest APPEND (which is already unique within the transaction). My question is: should I deduplicate in my ML transform code using a Window function, or should I create a View with primary key deduplication on top of the APPEND dataset? And if I use a View, the documentation says deduplication is only applied within the incrementally read transaction range — does that mean I can only use the View as a snapshot input for downstream transforms?