# \#transforms-python

**URL:** https://community.palantir.com/tag/transforms-python/84.md

[Latest](https://community.palantir.com/latest.md) · [Categories](https://community.palantir.com/categories.md) · [Tags](https://community.palantir.com/tags.md)

---

## [Developer Tier: TLS timeout connecting to OpenSky authentication API](https://community.palantir.com/t/developer-tier-tls-timeout-connecting-to-opensky-authentication-api/7528)

<div class="topic-metadata">

**Author:** [@DhanushBB](https://community.palantir.com/u/DhanushBB)\
**Replies:** 0\
**Last updated:** [November 16, 2026, 1:56am UTC](https://community.palantir.com/t/developer-tier-tls-timeout-connecting-to-opensky-authentication-api/7528 "2026-11-16T01:56:15Z")

</div>

I’m building a Python external transform in Foundry Developer Tier to retrieve aircraft observations from OpenSky Network. Configuration: Two single-domain REST API sources: authentication and aircraft data. Approved…

---

## [Lessons from building with the AI FDE: what it nails, and what to tell it up front](https://community.palantir.com/t/lessons-from-building-with-the-ai-fde-what-it-nails-and-what-to-tell-it-up-front/7377)

<div class="topic-metadata">

**Author:** [@2ffa006a15b15457d492](https://community.palantir.com/u/2ffa006a15b15457d492)\
**Replies:** 0\
**Last updated:** [October 14, 2026, 3:14pm UTC](https://community.palantir.com/t/lessons-from-building-with-the-ai-fde-what-it-nails-and-what-to-tell-it-up-front/7377 "2026-10-14T15:14:43Z")

</div>

I wrote up some lessons from building with the AI FDE on Foundry: Blog Link Sharing the key points here too, in case they save someone else some time. On documented platform paths it’s genuinely fast. It mapped twenty-…

---

## [\[BUG\] Media Reference returned in wrong format](https://community.palantir.com/t/bug-media-reference-returned-in-wrong-format/4817)

<div class="topic-metadata">

**Author:** [@project\_dreams](https://community.palantir.com/u/project_dreams)\
**Replies:** 1\
**Last updated:** [August 19, 2025, 5:44pm UTC](https://community.palantir.com/t/bug-media-reference-returned-in-wrong-format/4817 "2025-08-19T17:44:47Z")

</div>

I want to generating PNG images in a Python transform and save them as a Media item. The inputs to the transform are datasets, and there are then media outputs and datasets outputs. No media input. It’s not clear to me…

---

## [Restrict Build Permissions for a Transform](https://community.palantir.com/t/restrict-build-permissions-for-a-transform/7257)

<div class="topic-metadata">

**Author:** [@ishavasil](https://community.palantir.com/u/ishavasil)\
**Replies:** 1\
**Last updated:** [September 23, 2026, 12:58pm UTC](https://community.palantir.com/t/restrict-build-permissions-for-a-transform/7257 "2026-09-23T12:58:27Z")

</div>

hi deltas! was wondering if anyone knows how to restrict build permissions for a specific transform? we have a transform that writes back to one shared drive across dev stage and prod spaces and want to restrict permissi…

---

## [Applying a schema to an already structured dataset of Parquet files?](https://community.palantir.com/t/applying-a-schema-to-an-already-structured-dataset-of-parquet-files/915)

<div class="topic-metadata">

**Author:** [@vidak](https://community.palantir.com/u/vidak)\
**Replies:** 3\
**Last updated:** [August 2, 2024, 7:05pm UTC](https://community.palantir.com/t/applying-a-schema-to-an-already-structured-dataset-of-parquet-files/915 "2024-08-02T19:05:00Z")

</div>

I have a Python External Transform that downloads Parquet files from an external bucket which are already structured and with a known schema (there’s a good reason for not using Data Connection). My download function jus…

---

## [Python incremental transform - current vs previous for output](https://community.palantir.com/t/python-incremental-transform-current-vs-previous-for-output/898)

<div class="topic-metadata">

**Author:** [@abudaniel](https://community.palantir.com/u/abudaniel)\
**Replies:** 1\
**Last updated:** [August 1, 2024, 10:15am UTC](https://community.palantir.com/t/python-incremental-transform-current-vs-previous-for-output/898 "2024-08-01T10:15:28Z")

</div>

In an incremental python transform, if I want to find the current state of the output before all the processing has started, should I be using mode=‘previous’ or mode=‘current’. What would be the difference between thes…

---

## [How can I pass a base64 encoded image as part of AIP Orchestrator to call a Vision model?](https://community.palantir.com/t/how-can-i-pass-a-base64-encoded-image-as-part-of-aip-orchestrator-to-call-a-vision-model/478)

<div class="topic-metadata">

**Author:** [@VincentF](https://community.palantir.com/u/VincentF)\
**Replies:** 3\
**Last updated:** [July 30, 2024, 2:05pm UTC](https://community.palantir.com/t/how-can-i-pass-a-base64-encoded-image-as-part-of-aip-orchestrator-to-call-a-vision-model/478 "2024-07-30T14:05:58Z")

</div>

In this example of the documentation, I can use an AIP orchestrator to call at high pace a GPT Vision model, for instance to query pictures that are stored in a mediaset. However, I don’t have a mediaset in my case, but…

---

## [Deleting Files in historic transactions without breaking the incremental](https://community.palantir.com/t/deleting-files-in-historic-transactions-without-breaking-the-incremental/824)

<div class="topic-metadata">

**Author:** [@nicornk](https://community.palantir.com/u/nicornk)\
**Replies:** 5\
**Last updated:** [July 31, 2024, 12:01pm UTC](https://community.palantir.com/t/deleting-files-in-historic-transactions-without-breaking-the-incremental/824 "2024-07-31T12:01:25Z")

</div>

I have an incremental transform writing out daily partitioned parquet files based on an incremental ingestion. (\>10TB) Due to Business Requirements there might be corrections for past dates. Therefore, I would like to …

---

## [Get file contents contained in a repository programatically in Python](https://community.palantir.com/t/get-file-contents-contained-in-a-repository-programatically-in-python/7122)

<div class="topic-metadata">

**Author:** [@anjaligarg](https://community.palantir.com/u/anjaligarg)\
**Replies:** 0\
**Last updated:** [August 19, 2026, 10:13am UTC](https://community.palantir.com/t/get-file-contents-contained-in-a-repository-programatically-in-python/7122 "2026-08-19T10:13:27Z")

</div>

I have marketplace product name, fetched using REST endpoint in python. I want to fetch package.json file contents from code repository using marketplace product name programmatically in python. is there any REST endpoi…

---

## [Access/Read package.json file from code repository using marketplace product id/blockset\_id](https://community.palantir.com/t/access-read-package-json-file-from-code-repository-using-marketplace-product-id-blockset-id/7096)

<div class="topic-metadata">

**Author:** [@anjaligarg](https://community.palantir.com/u/anjaligarg)\
**Replies:** 0\
**Last updated:** [August 11, 2026, 1:55pm UTC](https://community.palantir.com/t/access-read-package-json-file-from-code-repository-using-marketplace-product-id-blockset-id/7096 "2026-08-11T13:55:14Z")

</div>

Need to read contents of package.json(trypescript repo) and conda-versions.run.linux-64.lock file(python repo) from code repository using marketplace packagename/productid/blocksetid. Basically need to access repository…

---

## [NetSuite SuiteQL Connector – Schema Discovery Failures & Custom Ingestion Workaround](https://community.palantir.com/t/netsuite-suiteql-connector-schema-discovery-failures-custom-ingestion-workaround/6874)

<div class="topic-metadata">

**Author:** [@manassesj](https://community.palantir.com/u/manassesj)\
**Replies:** 1\
**Last updated:** [July 2, 2026, 10:50am UTC](https://community.palantir.com/t/netsuite-suiteql-connector-schema-discovery-failures-custom-ingestion-workaround/6874 "2026-07-02T10:50:10Z")

</div>

Problem While integrating NetSuite data into Palantir Foundry, we encountered persistent failures when attempting to preview or ingest certain tables using the native SuiteQL connector. The connector’s metadata/schema di…

---

## [renderShrinkwrap fails with AccessOutsideProjectDenied ](https://community.palantir.com/t/rendershrinkwrap-fails-with-accessoutsideprojectdenied/6868)

<div class="topic-metadata">

**Author:** [@alfonso.gi](https://community.palantir.com/u/alfonso.gi)\
**Replies:** 0\
**Last updated:** [June 30, 2026, 8:12am UTC](https://community.palantir.com/t/rendershrinkwrap-fails-with-accessoutsideprojectdenied/6868 "2026-06-30T08:12:19Z")

</div>

Hi all, I’m troubleshooting a Foundry Code Repository CI issue and would appreciate guidance from anyone who has seen something similar. We have a Python Transforms repository in Project A. The repo produces multiple o…

---

## [Custom Model Evaluator missing GLIB library with Modeling Objective evaluation](https://community.palantir.com/t/custom-model-evaluator-missing-glib-library-with-modeling-objective-evaluation/631)

<div class="topic-metadata">

**Author:** [@ROC5COR](https://community.palantir.com/u/ROC5COR)\
**Replies:** 2\
**Last updated:** [July 10, 2024, 6:31am UTC](https://community.palantir.com/t/custom-model-evaluator-missing-glib-library-with-modeling-objective-evaluation/631 "2024-07-10T06:31:03Z")

</div>

Hello, I am using a custom model evaluator to compute metrics in a Modeling Objective. But when running the evaluation I am having the following error: "ImportError: /lib/x86\_64-linux-gnu/libstdc++.so.6: version \`GLIBC…

---

## [GPU resource allocation on sidecar transforms](https://community.palantir.com/t/gpu-resource-allocation-on-sidecar-transforms/6748)

<div class="topic-metadata">

**Author:** [@michaello](https://community.palantir.com/u/michaello)\
**Replies:** 2\
**Last updated:** [June 16, 2026, 4:54pm UTC](https://community.palantir.com/t/gpu-resource-allocation-on-sidecar-transforms/6748 "2026-06-16T16:54:55Z")

</div>

I have a sidecar transform that needs to use a GPU. How should I configure the resources? @sidecar(image="my-compute-module", tag="0.0.1", volumes=\[Volume("shared")\], resource\_profile="LARGE\_CPU\_LARGE\_MEMORY\_GPU\_ENABL…

---

## [How/When can I change the spark configuration of a build via conf.set?](https://community.palantir.com/t/how-when-can-i-change-the-spark-configuration-of-a-build-via-conf-set/4882)

<div class="topic-metadata">

**Author:** [@VincentF](https://community.palantir.com/u/VincentF)\
**Replies:** 1\
**Last updated:** [June 11, 2026, 4:19am UTC](https://community.palantir.com/t/how-when-can-i-change-the-spark-configuration-of-a-build-via-conf-set/4882 "2026-06-11T04:19:39Z")

</div>

I have a pipeline, defined in code repository, and I need to apply particular configuration to make the run successful (for example if I want to change pivotMaxValues, or the skew configuration, etc.). Given that I can …

---

## [How can I create a dataset from a parsing of a file loaded as a dataset?](https://community.palantir.com/t/how-can-i-create-a-dataset-from-a-parsing-of-a-file-loaded-as-a-dataset/636)

<div class="topic-metadata">

**Author:** [@lucapwc](https://community.palantir.com/u/lucapwc)\
**Replies:** 1\
**Last updated:** [July 5, 2024, 6:08pm UTC](https://community.palantir.com/t/how-can-i-create-a-dataset-from-a-parsing-of-a-file-loaded-as-a-dataset/636 "2024-07-05T18:08:53Z")

</div>

I have an mf4 file that with the use of the mdfreader library can be converted into a pandas dataframe and then pyspark. I loaded the file as an unstructured dataset and wrote a transformation that reads the file from fi…

---

## [Which is the better configuration to parallelly run docker containers with transform?](https://community.palantir.com/t/which-is-the-better-configuration-to-parallelly-run-docker-containers-with-transform/485)

<div class="topic-metadata">

**Author:** [@ChengYangUmich](https://community.palantir.com/u/ChengYangUmich)\
**Replies:** 0\
**Last updated:** [June 21, 2024, 7:41am UTC](https://community.palantir.com/t/which-is-the-better-configuration-to-parallelly-run-docker-containers-with-transform/485 "2024-06-21T07:41:19Z")

</div>

Problem Statement: A project requires to optimize certain inputs of a model to achieve the wanted output targets. However, the model is wrapped in a docker container and therefore, it is not possible to directly put the…

---

## [Append/incremental file management](https://community.palantir.com/t/append-incremental-file-management/6481)

<div class="topic-metadata">

**Author:** [@ckucera](https://community.palantir.com/u/ckucera)\
**Replies:** 3\
**Last updated:** [June 3, 2026, 1:42pm UTC](https://community.palantir.com/t/append-incremental-file-management/6481 "2026-06-03T13:42:34Z")

</div>

I’m frequently running an incremental transform/append sync and the numbers of files in my dataset is growing too large. How can I manage this?

---

## [Resource Managment fails to attribute usage to resources, while the data sits there in Dataset API](https://community.palantir.com/t/resource-managment-fails-to-attribute-usage-to-resources-while-the-data-sits-there-in-dataset-api/6608)

<div class="topic-metadata">

**Author:** [@ZygD](https://community.palantir.com/u/ZygD)\
**Replies:** 7\
**Last updated:** [May 27, 2026, 2:44pm UTC](https://community.palantir.com/t/resource-managment-fails-to-attribute-usage-to-resources-while-the-data-sits-there-in-dataset-api/6608 "2026-05-27T14:44:23Z")

</div>

Resource Management is a great app to see usage, but this may be a bug. I see these false claims in the Resource Management app: “Some usage cannot be attributed to individual resources”; “Top resources” H…

---

## [Spark 4 support in Transforms timeline](https://community.palantir.com/t/spark-4-support-in-transforms-timeline/6547)

<div class="topic-metadata">

**Author:** [@jchlapek](https://community.palantir.com/u/jchlapek)\
**Replies:** 0\
**Last updated:** [May 5, 2026, 2:18pm UTC](https://community.palantir.com/t/spark-4-support-in-transforms-timeline/6547 "2026-05-05T14:18:53Z")

</div>

Hi everyone, what is the current timeline on Spark 4 support for transforms? The topic was picked up here https://community.palantir.com/t/spark-4-support-in-transforms/4080, where it was mentioned we can expect it arou…

---

## [Using Views to read an IncrementalOutput](https://community.palantir.com/t/using-views-to-read-an-incrementaloutput/551)

<div class="topic-metadata">

**Author:** [@nholloway](https://community.palantir.com/u/nholloway)\
**Replies:** 0\
**Last updated:** [June 27, 2024, 5:57pm UTC](https://community.palantir.com/t/using-views-to-read-an-incrementaloutput/551 "2024-06-27T17:57:51Z")

</div>

Hi all, I’m trying to write an incremental transform that reads in the previous version of the dataset. I’ve followed the guidance in “maintaining pipeline performance” and added projections to improve the speed of reads…

---

## [Best approach for incremental processing when S3 source provides full snapshots](https://community.palantir.com/t/best-approach-for-incremental-processing-when-s3-source-provides-full-snapshots/6451)

<div class="topic-metadata">

**Author:** [@Vardhan](https://community.palantir.com/u/Vardhan)\
**Replies:** 5\
**Last updated:** [April 21, 2026, 2:55pm UTC](https://community.palantir.com/t/best-approach-for-incremental-processing-when-s3-source-provides-full-snapshots/6451 "2026-04-21T14:55:06Z")

</div>

We’re ingesting data from an S3 bucket where an upstream Spark job writes a full snapshot (single file, fully replaced) on a scheduled basis. Each snapshot contains all rows — both unchanged and new/modified records. Our…

---

## [Can I delete a file in the FileSystem?](https://community.palantir.com/t/can-i-delete-a-file-in-the-filesystem/1693)

<div class="topic-metadata">

**Author:** [@jkane](https://community.palantir.com/u/jkane)\
**Replies:** 4\
**Last updated:** [April 20, 2026, 8:28am UTC](https://community.palantir.com/t/can-i-delete-a-file-in-the-filesystem/1693 "2026-04-20T08:28:59Z")

</div>

I can write a file in the FileSystem like so: with output.filesystem().open(“example.txt”, “wb”) as new\_file: new\_file.write(“example”) Is it possible to DELETE a file in the output filesystem within an incremental tr…

---

## [Marketplace Products with Code Repos](https://community.palantir.com/t/marketplace-products-with-code-repos/408)

<div class="topic-metadata">

**Author:** [@starng-4a](https://community.palantir.com/u/starng-4a)\
**Replies:** 3\
**Last updated:** [June 18, 2024, 8:36am UTC](https://community.palantir.com/t/marketplace-products-with-code-repos/408 "2024-06-18T08:36:09Z")

</div>

Hi, We are looking deploy code repos (python transforms) out of a marketplace product. However, our repos reference external libraries (within Foundry). Have noticed that creating the product fails to acknowledge these …

---

## [How can I get an external host’s IP address in Python External Transforms?](https://community.palantir.com/t/how-can-i-get-an-external-host-s-ip-address-in-python-external-transforms/6341)

<div class="topic-metadata">

**Author:** [@stephenlo](https://community.palantir.com/u/stephenlo)\
**Replies:** 0\
**Last updated:** [April 7, 2026, 8:19am UTC](https://community.palantir.com/t/how-can-i-get-an-external-host-s-ip-address-in-python-external-transforms/6341 "2026-04-07T08:19:17Z")

</div>

I am working in a Python transforms repository in Foundry. I need to resolve external hostnames (e.g., www.google.com) within my transform code using Python’s socket library. Currently, only internal hostnames (e.g., kub…

---

## [Dynamically Generating Transform](https://community.palantir.com/t/dynamically-generating-transform/6285)

<div class="topic-metadata">

**Author:** [@bbd5be9832271c105aa7](https://community.palantir.com/u/bbd5be9832271c105aa7)\
**Replies:** 4\
**Last updated:** [March 30, 2026, 8:53pm UTC](https://community.palantir.com/t/dynamically-generating-transform/6285 "2026-03-30T20:53:47Z")

</div>

Hi, I hope you are well. I am trying to dynamically make multiple output datasets based on a list of files paths I have. I managed to do this when I have the list of file paths written out manually. However i want to d…

---

## [list\_media\_items\_by\_path\_with\_media\_reference returns None despite correct setup](https://community.palantir.com/t/list-media-items-by-path-with-media-reference-returns-none-despite-correct-setup/6202)

<div class="topic-metadata">

**Author:** [@kpglemon](https://community.palantir.com/u/kpglemon)\
**Replies:** 0\
**Last updated:** [March 16, 2026, 7:27am UTC](https://community.palantir.com/t/list-media-items-by-path-with-media-reference-returns-none-despite-correct-setup/6202 "2026-03-16T07:27:19Z")

</div>

I am running list\_media\_items\_by\_path\_with\_media\_reference(ctx) in a Foundry Code Repository (not local), with transforms-media installed and listed in meta.yaml. The MediaSet path is correct, files exist in the MediaSet…

---

## [VELOX vs GLUTEN](https://community.palantir.com/t/velox-vs-gluten/6160)

<div class="topic-metadata">

**Author:** [@ZygD](https://community.palantir.com/u/ZygD)\
**Replies:** 1\
**Last updated:** [March 9, 2026, 8:20pm UTC](https://community.palantir.com/t/velox-vs-gluten/6160 "2026-03-09T20:20:01Z")

</div>

So, in Foundry, we can explicitly use BOTH references (GLUTEN as profile and VELOX as backend): @configure( \["EXECUTOR\_MEMORY\_MEDIUM", "EXECUTOR\_MEMORY\_OFFHEAP\_FRACTION\_HIGH", "GLUTEN"\], backend=ComputeBackend.VEL…

---

## [Bypassing Cyclical Dependencies In Python Transforms](https://community.palantir.com/t/bypassing-cyclical-dependencies-in-python-transforms/1874)

<div class="topic-metadata">

**Author:** [@ramverma](https://community.palantir.com/u/ramverma)\
**Replies:** 3\
**Last updated:** [November 28, 2024, 10:08am UTC](https://community.palantir.com/t/bypassing-cyclical-dependencies-in-python-transforms/1874 "2024-11-28T10:08:57Z")

</div>

I have: An output dataset which backs an Object Tyoe The Object Type has a materialized dataset I want to add the materialized dataset back as an input to my output dataset in (1). This works fine on a branch, but fai…

---

## [Polars / DuckDB - read and write column descriptions](https://community.palantir.com/t/polars-duckdb-read-and-write-column-descriptions/1975)

<div class="topic-metadata">

**Author:** [@ZygD](https://community.palantir.com/u/ZygD)\
**Replies:** 1\
**Last updated:** [May 6, 2025, 11:15am UTC](https://community.palantir.com/t/polars-duckdb-read-and-write-column-descriptions/1975 "2025-05-06T11:15:50Z")

</div>

We always use column descriptions. We read them from input datasets and write them to the output. In Python repositories, when using Spark, Below I provide our currently used functions for reading and writing the column…

[Next page](https://community.palantir.com/tag/transforms-python/84.md?match_all_tags=true&page=1&tags%5B%5D=transforms-python)
