# Running PySpark Tests in Code Workspaces

**URL:** <https://community.palantir.com/t/running-pyspark-tests-in-code-workspaces/6529>\
**Category:** Ask the Community\
**Tags:** spark, vscode, code-workspaces, code-repositories\
**Created:** [April 30, 2026, 9:53am UTC](https://community.palantir.com/t/running-pyspark-tests-in-code-workspaces/6529 "2026-04-30T09:53:51Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![green](https://yyz1.discourse-cdn.com/palantir/user_avatar/community.palantir.com/green/32/2539_2.png) [@green](https://community.palantir.com/u/green)\
**Post date:** [April 30, 2026, 9:53am UTC](https://community.palantir.com/t/running-pyspark-tests-in-code-workspaces/6529/1 "2026-04-30T09:53:51Z")

</div>

What’s the preferred pattern to run Spark DataFrame based tests using Code-Workspaces please?

We have a few code respositories that run tests using the pattern described in the [Docs DataFrame Unit Test Example](https://www.palantir.com/docs/foundry/transforms-python/unit-tests#dataframe-test-example), e.g.

```python
def test_dataframe(spark_session):
    df = spark_session.createDataFrame([['a', 1], ['b', 2]], ['letter', 'number'])
    assert df.schema.names == ['letter', 'number']

```

But this doesn’t work in Code-Workspaces, the tests fail because there is no `spark_session` available.

I can find the existing fixture for this which is at `site-packages/foundry_pyls/fixtures.py` (code not posted because of copyright notice) and add that into my own conftest.py file as a PyTest Fixture, but it doesn’t work.

It additionally requires installing the ‘Pyspark’ Python package, but:

1. That doesn’t seem to work, even with Pyspark installed it can’t find Spark
2. Adding Pyspark directly like this seems to conflict with the Foundry Transforms imports, causing other issues.

How can we run Spark DataFrame tests in Code Workspaces please? Currently we’re stuck using Code Repositories for these parts.