Hello, I had some general questions on the new AIP Threads and the AIP Agent Studio apps:
Is there a limitation on the number of documents that can be RAGed simultaneously?
Asking for a use-case where we’d be looking to query ~10,000 documents from ~1,000 contracts to ideally ask questions like “list all contracts that meet criteria X.”
Is there a limitation on the number of chunks that can be retrieved for a particular query?
Other models that I’ve utilized limit me to only utilizing 10 chunks for a query, which leans into the previous question as well.
What functionality do AIP Agents bring to the table? Is the current customizability limited to only temperature?
Is there any functionality to fine-tune existing models with additional training?
Querying the AIP Threads directly has provided some unclear responses and understandably since these tools are still in Beta the documentation is somewhat limited, so curious if anyone is able to help or has similar questions.
Thanks!
We do set a large max K value of 1,000 objects to load, but we could increase that value if there is a limit you are running into. Most likely, you’ll hit context window limits with the LLM you are using before you hit the K value maximum (and if this is not true please let us know and we will up that number).
AIP Agents make it easy to pull in the Ontology, documents, and tools to build enterprise-specific agents, including platform integrations into the AIP Interactive Widget & AIP Threads, and OSDK support. You should find it fairly easy to get started and build a valuable agent. You’ll get session storage and secure indefinite session retention for free by using an AIP Agent. And you can also publish your agent as a Function and pull it into Workshop, AIP Logic, or other Function consumers in Foundry, and AIP Agents can be bundled into a Marketplace package.
We just updated our documentation for AIP Threads & AIP Agent Studio because AIP Agent Studio is now GA. Please take a look at the updated docs and let us know if you have a specific question on querying in AIP Threads.
2 Likes