How to see how many LLM tokens were used in a dataset build in Pipeline Builder?

[tgobindram]

Hey Pipeline Builder team, is there a way to see how many tokens were used by a dataset build that involved using UseLLM nodes?

[helenq]

Hi! We’re currently working on getting build metrics (which will include token count estimates) but for now you can see the estimates on previews and estimate how many tokens the entire build would take using that as a proxy

[helenq]

You can also use the trial runs if you want to go more granular or pick a specific row that you know is more computationally heavy and get an estimate from that