I encountered an error stating ‘Failed to set checkpoint directory’ when trying to export a file to S3. I’ve checked my AWS permissions and export configuration options, but the issue persists. How can I resolve this?
Hey @raybellwaves , could you describe your setup in more detail? Do you have a batch or streaming pipeline? Also I’m assuming you have an output dataset in Foundry that then is being exported to S3 through a Data Connection file export? Is this error being thrown as a part of building the Foundry output from the pipeline, or during the downstream export task?
Batch pipeline. It’s a one off export of a Foundary dataset. It’s the downstream export task as i’m trying it in the data connection app. I’m able to read from the s3 bucket ok but can’t write back to the same bucket.
Gotcha, I believe data-connection would be the correct tag for this question
Hey @raybellwaves,
Could you share the error stack trace if you’re seeing one with the error message?
Further, are you using export-tasks or file exports in this pipeline? If you’re using the former, it is recommended to switch to file exports, as they’re much easier to set up and more stable.