Subject: Issues with Media References in Unit Testing in Palantir
Hello Palantir Community,
I am currently facing a challenge while working on unit testing within Palantir. My testing involves a dataset that includes indirect references to a media set, but I have encountered a limitation I hope to get assistance with.
Problem Description:
I understand that ideally, I could join the original media set to access the media items in this testing dataset. However, in my unit tests, I find there is no option to import or access any media sets. This limitation is preventing me from fully utilizing the media references present in my dataset.
Workarounds: Are there any known workarounds to leverage media references within unit tests in Palantir?
Feature Limitations: Is this a known limitation within the Palantir platform, and are there plans for future updates that might address this?
Best Practices: What are the best practices for handling datasets with media references in unit testing scenarios?
I would greatly appreciate any insights, suggestions, or examples from those who have navigated similar challenges. Thank you in advance for your help!
When this unit testing framework was built, it was an explicit decision to keep the input and output data within the pipeline itself. Having this ensures that the unit tests can’t be inadvertently broken unless explicitly changed in the pipeline. If we were to reference outside resources in unit tests, extraneous changes to those resources could break unit tests and block pipeline proposals.
There is a way to fit media into that philosophy, it just means a good amount of engineering effort to host media alongside the pipeline, rather than in the global, Foundry media context. That’d likely look like uploading media into the unit test itself rather than referencing media elsewhere in Foundry.
We’re currently decomping an evals framework (similar to Logic’s) that will need to reference outside resources, and doing so should allow you to bring in media to evaluate an LLM node. This work is realistically months away from reality though.
In the interim, we could allow you to hardcode media references at the risk of your unit tests failing if anyone else in your Foundry instance updates or deletes those media items. Is this a risk you’d tolerate?
