When using Data connection to automatically extract data periodically, is it necessary to consider transaction isolation levels such as READ COMMITTED or REPEATABLE READ?
I assume you’re referring to periodic incremental data ingests, as transaction isolation levels won’t matter if you’re snapshotting periodically.
For incremental data ingests, Foundry relies on the value of column used for incremental in the last ingested row. The next run always queries for rows with value higher than the last ingested value in Foundry. We also only run on SELECT query in the transaction. So in general, transaction isolation levels shouldn’t matter - we should not run into situations where we ingest duplicate rows or miss rows.
1 Like
Thank you for your thoughtful response.