Conditionally Building Every Minute (Streaming)

Z3882_csx
Wed, 23 Oct 2024 19:42:49 +0000
Ask the Community

I have a project with multiple streaming pipelines. In order to ensure high availability, I have a schedule on each pipeline to build every minute - when one streaming build inevitably fails after some number of days or weeks, another build is immediately kicked off. I would like the ability to start and stop (pause) all these schedules at once, thereby starting or stopping the entire project.

I know that I can add a condition to a schedule using the “When multiple time or event conditions are met” but the extra conditions I can add all detect file changes, rather than value… ideally I’d like a build condition such as “Every minute AND start.txt exists” or maybe “Every minute AND start.txt has value ‘yes’”.

Has anyone done something like this before? Thank you

7 posts - 4 participants