Use API to copy foundry dataset to another location

Do you want to copy datasets or files in datasets?

Do you really want to physically create a copy or move a dataset?

If you want to copy I am afraid you need to create a new dataset as target and than iterate all files to copy them over using either transforms or a script that runs outside of foundry and leverages any of the available APIs. I think easiest would be to setup aws cli and use the aws s3 sync command which will nicely handle the listing and parallelisation for you.

The simplest would likely be to run an identity transforms from your source datasets to your targets dataset. In other words an “empty transform” that simply do copy paste without any transformation.
You can do that in pipeline builder or in code repository.

You can as well leverage “Views” which is a way to union multiple dataset, without physical copy. Which means that in your case, it can make your source dataset available to another location.
https://www.palantir.com/docs/foundry/data-integration/views

Alternatively, you can move the datasets, as suggested, but I would understand that in your use-case you actually want a copy.

I don’t said you need to save it to S3 you can actually sync from one dataset to the other:

aws s3 sync s3://<foundry-dataset-rid1> s3://<foundry-dataset-rid2> --profile foundry