How to compact history of the session in AIP Agent?

Ask the Community
1

I have users who might not create new sessions in AIP Agents and instead want to continue previous discussions.
Is it possible and if so, how, to compact the history of the session in AIP Agent ?

It seems that after the context length of the Agent is filled, older messages of the session are dropped.

2

High level solutions:

  • Level 0 = ask the Agent to “summarize” the current discussion/what it sees, regularly. This way, this acts as a “summarization” or “history compaction”, and even if older messages are lost, the content is persisted forward.
  • Level 1 - Create a “memory” variable. e.g. create a memory_summary variable and prompt the LLM to consult memory_summary instead of re-reading the full history. Ask the LLM to store context in this variable. Can also be multiple variables (“facts”, …)
  • Level 2 - Make it more deterministic by using a function. e.g. create a tool (function, e.g. SummarizeConversation) configured to “automatic variable update” for this application state. The function can take the latest exchange or a short excerpt the model pastes in + the current memory_summary, and returns an updated, capped-length summary. You can then ask the Agent to call SummarizeConversation after answering each user message (or when the summary seems stale/too long).
  • Future - Wait for the feature to be implemented in the product itself, no timeline attached yet.
1 Like