# Partition\_cols in pipeline builder? **URL:** \ **Category:** Ask the Community\ **Tags:** pipeline-builder\ **Created:** [September 29, 2024, 1:05pm UTC](https://community.palantir.com/t/partition-cols-in-pipeline-builder/1394 "2024-09-29T13:05:25Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![Marcos](https://avatars.discourse-cdn.com/v4/letter/m/9de0a6/32.png) [@Marcos](https://community.palantir.com/u/Marcos)\ **Post date:** [September 29, 2024, 1:05pm UTC](https://community.palantir.com/t/partition-cols-in-pipeline-builder/1394/1 "2024-09-29T13:05:25Z")
Hi all, how to partition outputs by specific columns in pipeline builder? I would like to do something similar to this code repository sentence: output.write\_dataframe(df, partition\_cols=[‘col’]) Thanks! ---
**Author:** ![bkaplan](https://avatars.discourse-cdn.com/v4/letter/b/ea666f/32.png) [@bkaplan](https://community.palantir.com/u/bkaplan)\ **Post date:** [September 29, 2024, 4:28pm UTC](https://community.palantir.com/t/partition-cols-in-pipeline-builder/1394/2 "2024-09-29T16:28:11Z")
There is a transform called [Repartition Data](https://www.palantir.com/docs/foundry/pb-functions-transform/repartitionV1/#repartition-data) that can be used here. ---
**Author:** ![Marcos](https://avatars.discourse-cdn.com/v4/letter/m/9de0a6/32.png) [@Marcos](https://community.palantir.com/u/Marcos)\ **Post date:** [September 29, 2024, 8:36pm UTC](https://community.palantir.com/t/partition-cols-in-pipeline-builder/1394/3 "2024-09-29T20:36:33Z")
Hi bkaplan, I’m aware of repartition() and partitionBy() in pyspark, I’m looking for the 2nd one, in code repository with the @transform() decorator using output.write\_dataframe(df, partition\_cols=[“year”]) the files I get in the Details of the dataset are something like this: spark/year=2023/part-00000-372287f2-c241-4198-a0b0-47002fdf7c4e.c000.snappy.parquet spark/year=2024/part-00000-372287f2-c241-4198-a0b0-47002fdf7c4e.c000.snappy.parquet With the repartition data transform in pipeline builder I dont get files with the same pattern, it seems a shuffle of the data to optimize the pipeline but not to write the output dataset. Right? Thank you ---
**Author:** ![sandpiper](https://yyz1.discourse-cdn.com/palantir/user_avatar/community.palantir.com/sandpiper/32/273_2.png) [@sandpiper](https://community.palantir.com/u/sandpiper)\ **Post date:** [September 30, 2024, 2:20am UTC](https://community.palantir.com/t/partition-cols-in-pipeline-builder/1394/4 "2024-09-30T02:20:28Z")
Unfortunately, hive partitioning in Pipeline Builder outputs is not currently supported. You can make a Projection on the outputs, however, which may be sufficient for your use-case. See [my earlier answer](https://community.palantir.com/t/pipeline-optimization-dataset-projection-how-do-they-work-and-how-they-relate-to-hive-partitioning-bucketing/375/2) for a discussion of Projections vs hive partitioning. ---
**Author:** ![Marcos](https://avatars.discourse-cdn.com/v4/letter/m/9de0a6/32.png) [@Marcos](https://community.palantir.com/u/Marcos)\ **Post date:** [September 30, 2024, 2:02pm UTC](https://community.palantir.com/t/partition-cols-in-pipeline-builder/1394/5 "2024-09-30T14:02:16Z")
Thank you for the clarifications Regards ---
**Author:** ![system](https://yyz1.discourse-cdn.com/palantir/user_avatar/community.palantir.com/system/32/3_2.png) [@system](https://community.palantir.com/u/system)\ **Post date:** [March 3, 2025, 12:43pm UTC](https://community.palantir.com/t/partition-cols-in-pipeline-builder/1394/6 "2025-03-03T12:43:03Z")
---
**Author:** ![system](https://yyz1.discourse-cdn.com/palantir/user_avatar/community.palantir.com/system/32/3_2.png) [@system](https://community.palantir.com/u/system)\ **Post date:** [October 8, 2025, 7:49pm UTC](https://community.palantir.com/t/partition-cols-in-pipeline-builder/1394/7 "2025-10-08T19:49:29Z")