I need a way to periodically do a snapshot load of a raw dataset that gets loaded incrementally via a Data Connection Sync. I have not yet found a way to do this other than manually going into each Sync, switching to snapshot mode, commented out the where clause for the time filter and disabling incremental.
Does anyone have a possible solution there.
Reasons are plenty, but here’s a couple:
- Schema changes. (or include a column we previously ignored) We often get new fields from the source and there is now way that I know of to back populate existing rows with these new fields.
- We have multiple environments. We sometimes need to refresh Test from Prod, etc. IN this case I need a complete reload of source data.
The only solution I’ve been able to find is to literally update EVERY row in every table with a meaningless change to uptick the timestamp field. Very long running process and seems like overkill.