How can I check if my pipeline suffers from LLM throttling?

Ask the Community
1

Hi Pipeline Builder team,

Is there a way to check if my pipeline is suffering from throttling?
I would like to increase the efficiency of my pipeline which has many LLM boards and someone suggested looking into throttling.

Thanks a lot,
Thor

2

Hey! Are there any nulls coming back as outputs of your LLM? Or are the outputs as expected but the build is slower than what you’re expecting?

This could also be helpful to look at if you want to see the specific AIP usage + limits for the models you’re using: https://www.palantir.com/docs/foundry/aip/llm-capacity-management/