How much time does it take usually for a pipeline to be deployed for a new dataset? There are only 2 tables, I used 1 LLM logic for the left table and then crossed joined the result to the right table and then created a new dataset as an output from the result. However, it is taking me a long time for the pipeline to be deployed.
Hi, there’s a lot of factors at play here – the size of your datasets, the complexity of your LLM logic, etc – so it’s hard to give a definitive answer for “how long it usually takes”. That said, bumping the compute profile in the build settings is a good idea if your pipeline is not getting enough resources for computing the output.
1 Like