How can I call LLM from unit test in Typescript function?

Ask the Community
1

I’m having a Typescript function (FoO) that calls an LLM as part of its logic.
I need and want to test this function like the others, via unit test of the typescript function.

However, it seems I can’t call LLM when the unit tests are running.

How can I call an LLM as part of my unit tests in Typescript ?

2

Hi @VincentF,

Foundry Evals is not available yet for Typescript functions. However, you could call the typescript function from AIP Logic and set up the evaluation suite on this Logic file, which would then enable you to test your LLM function with different evaluators.

3

Thanks for the update. I would say many developers are going to want direct access to models in unit tests as well as the evals framework. In other words these things are not mutually exclusive. Hopefully with the release of evals for TypeScript functions we also get the ability to call models directly.

1 Like