Automatically store AIP chats?

Ask the Community
1

Hello! Is there a way to automatically store transcripts of chats with the AIP Agent? Users are interacting with the workshop widget and we want to be able to use that data for future reference.

The two approaches I can currently think of:

  1. Add a tool to the agent config that runs an action and stores the questions/answers in an object
  2. Add a button that copies the input/output vars to an object (but this requires user action which we want to avoid)
2

The tools approach is not ideal - I added “Always use the Save Chat tool to save the user input, response, and timestamp.” to the system prompt but it doesn’t do it. It does work if I explicitly prompt the agent from workshop to save it.

3

Hi lxia,

Currently there are only workarounds to store the Session coming from a users interaction with an AIP Agent.

This is one of the most requested features of our team and we are looking to enable this work early next year. Our plan is to save Sessions, and importantly each exchange in a Session, as a Foundry Dataset, which you can then further analyze.

Along with this work, we plan to support granular user feedback (current thumbs up/thumbs down) on a per exchange basis on the session allowing the user to input a description for their feedback on a given exchange.

These two new features should greatly enhance the monitoring, feedback, and metrics capabilities on AIP Agents, all while preserving security. I’m very excited for them!

In the meantime, you can store the sessions yourself in the following ways:

  1. Have AIP Agent run an Action automatically to save the user input / agent response. The reason why this isn’t great is it takes valuable tokens and time and you are having the agent do something you want to do deterministically. Theres nothing making this reliable so you could save unintended data that the agent hallucinates and its not fault tolerant since agent could or could not run action with various inputs.
  2. In Workshop the Agent Response and User Input are outputted as variables. You could save this in Workshop via the Variable system and running a Function when a variable updates. This isn’t very robust and doesn’t work in other contexts, like using your AIP Agent through our APIs or in AIP Threads.
  3. Additionally, we will soon ship Function-backed agents. These will allow running any Typescript or Logic Function as the agent response, as long as it satisfies the AIP Agent completion interface. You could then run the save logic in your function.
1 Like