Multiple pipelines in pipeline builder

Hello,

What are the performance implications of chaining two dependent pipelines in Pipeline Builder? The second pipeline uses the output of the first. What is the impact on overall build time, and what are the best practices for optimizing performance with this dependency?

do I need to separate them in difference builders?

for reference:

Hello Herson,

The jobs of each dataset will be independent, regardless of whether their
pipelines are in the same builder or different builders.

Hence, there is no impact in the overall build time. In this case, the decision is primarily a matter of clarity.

I hope you find this helpful!

1 Like