# Incremental Pipeline Builder with Media Set

**URL:** <https://community.palantir.com/t/incremental-pipeline-builder-with-media-set/3327>\
**Category:** Ask the Community\
**Tags:** pipeline-builder, workshop\
**Created:** [March 28, 2025, 9:21am UTC](https://community.palantir.com/t/incremental-pipeline-builder-with-media-set/3327 "2025-03-28T09:21:43Z")\
**Posts on this page:** 1\
**Showing post:** 2

<div class="post-metadata">

**Author:** ![sandpiper](https://yyz1.discourse-cdn.com/palantir/user_avatar/community.palantir.com/sandpiper/32/273_2.png) [@sandpiper](https://community.palantir.com/u/sandpiper)\
**Post date:** [March 29, 2025, 1:13am UTC](https://community.palantir.com/t/incremental-pipeline-builder-with-media-set/3327/2 "2025-03-29T01:13:57Z")

</div>

To confirm, is the idea here that the first pipeline is using the “append only new rows” write mode so that the second pipeline can be incremental?

You’re probably hitting the same error as [https://community.palantir.com/t/permissions-error-when-using-pipeline-builder-to-convert-media-set-into-dataframes/796/9](https://community.palantir.com/t/permissions-error-when-using-pipeline-builder-to-convert-media-set-into-dataframes/796/9). I believe that you should be able to resolve this by adding the original media set as another input (with the SNAPSHOT read mode, since it’s a media set) to the second pipeline. In the second pipeline, you’d _also_ convert the media set to table rows and then inner join it with the append-only input. You don’t need to actually include any columns from the media set input as part of the join - the only reason for the join is to make sure that the media set is indeed treated as an input, allowing the job token to read the media items.

---

_[View the full topic](https://community.palantir.com/t/incremental-pipeline-builder-with-media-set/3327)._
