- Give context on the use case / what you’re trying to achieve - Being able to use LLMs effectively in FoOs. Required for many use cases linked to workshop and AIP Logic,
- Why you can’t do it today / work-arounds - There appears to be some kind of memory overflow that happens in the ObjectSet API when too many records are returned. You eventually get an execution timeout error due to the overflow. This is common in semantic search applications where the number of vectors is huge. Even searching a few hundred pages will get you to this error. I’ve done a lot to try and limit the number of records but there is no support for projections (meaning I could select specific columns to avoid including vectors until I’ve widdled the object set down). We need longer timeouts, a lot more memory, and an improved API that has better support for projections and push-down predicates (which I don’t even think exists today).
- Why will this make a huge difference for you or your business? - We can deliver AIP use cases on real-world documents corpuses
- Bonus points if you want to provide a demo video to aid in your pitch!
5 Likes
Does FoOs mean “failure of observability” - meaning your timing out before your able to determine error stack? Or something else
FoO = Functions on Objects
1 Like