I made an AIP Agent within AIP-Agent-Studio. I have an Ontology Object Type with text and vector embedding properties. These objects are generated from a pipeline running from a media set full of PDF and DOCX files. I tried using the ontology objects as context for my AIP Agent, but I keep getting an error that my I’ve outrun the AIP Agent session tokens. I only have five objects in the set and I am unable to get a response to any one query.
Has anyone seen this issue before?
This is just speculation, but looking briefly at the Agent Context configuration for an object set, I don’t see that there is the ability to choose what properties are provided to the agent when serializing the objects to text.
If that’s the case, I imagine even with only 5 objects, if you have the full text and the document embedding vector as object properties, then you might be blowing through the context window?
To test this hypothesis, you could try toggling to one of the other Models (though what you have available may differ from the screenshot) and choose one with a higher context window.
Another approach might be that you store the full text and the text embedding vector on one object type and you have a linked version of the text, perhaps with a “summary” property generated with the Use LLM widget in Pipeline Builder, and the additional metadata in a linked object type. You could then provide this linked object type as context for the Agent.
It’d be helpful to know a few more details about the size of the text that you’re passing in with those 5 objects - how long are the source documents?
Agree with @lrhyne 's hypothesis here that it is worth checking the size of the content you are passing in on each of your document objects.
In terms of configuration options, assuming you are passing these using the Ontology Context for your Agent, you should be able to configure the properties of the object type which are passed to the Agent (see attached example from the Ontology Context documentation).
You can also configure the number of relevant objects which are returned to the Agent from the retrieval (the default is 5).
If the document content is too large for the model’s context window to pass as a single object, you will need to update your document pipeline to implement chunking and embedding of the document content and use this chunk object type in your Ontology context to only retrieve relevant chunks of content to the current user query (see related answer here)
This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.
