Hi All,
In the Data Connection - Export, we have the option of exporting dataset to a snowflake or any other connection’s table. However, how can we implement pre SQL and post SQL in the Data Connection Table Exports.
Thanks
This is not supported in Table Exports - it is supported in legacy export tasks though.
We have the same feature request open.
ok, Thanks!
Is it possible to programmatically check the status of Table export task, like its history/run status.
The API document has no url for Table Exports.
https://www.palantir.com/docs/foundry/api/general/overview/getting-started/
Can you provide more context on the need to run pre-sql and post-sql queries for exports? We want Exports to be a point-and-click workflow, and prefer users to use external transforms for any usage that requires running custom queries on the source. Allowing arbitrary pre and post sql queries can loosen the transactionality guarantees we can provide, e.g. the tables affected by pre and post queries may not support transactions, and a failed job can result in a bad state in the DB.
To my knowledge external transforms do not support agent-bound JDBC Sources. Correct me if I am wrong here.
In our case we are exporting data to operational databases into staging tables. The database team provides a procedure which will move the data into the final, production table.
We use export tasks to call this procedure after the export. So far we can’t move to Exports due to this missing feature.
Yes, we are having a similar issue.
We use pre and post sql to add audit logging for each export which is not possible with the new table export.
We though about using the Table export API to check the status of export task and trigger code repository for processing the post sql however the APIs are not available and is a new feature request as per the Palantir team.
We need pre sql and post sql to audit the export completed successfully.
It is not resulting in any bad state of DB.