Is anyone else having consistent issues with Foundry’s AIP Assist and its hallucination of features, settings, and tools that simply don’t exist in the platform? I’ve seen these issues in Pipeline Builder, Workshop, Quiver, etc. The most reliable use case for AIP Assist has been code generation when I need a function, but otherwise I have had to stop relying on it.
Does anyone have best practices on how to get more consistent value and accuracy from AIP Assist?
Yes, I’ve personally switched to using either the MCP or AI-FDE for anything I’d like to do that requires AI. AI-FDE has been significantly more reliable than the AIP Assist in my experience
hey @100hamilton @kurtsace, thanks for posting! We’ve just finished rewriting the entire retrieval engine for AIP Assist, which makes it faster and more accurate. It’s still in beta. We’d love to get it out to you early. If you’re up for that, would you mind us reaching out via email? No need to share your email address on this thread, I can follow up separately.
On hallucinations, do you have examples of questions that cause hallucinations?
@cardamom yes please do reach out. I have many examples that I am happy to walk you through. Thanks!
@cardamom latest example of hallucinations today. AIP Assist answered a question about Object Explorer by linking me to documentation for Workhop:
https://www.palantir.com/docs/foundry/workshop/object-set-filter-variables#supported-starting-filters
Have reached out separately!
Your latest example with the Object Explorer question resulting in a Workshop docs citation is exactly the kind of hallucination the new work will allow us to address: specifically, we’re going to investigate parametrising the retrieval by the app you’re asking about.
Ran into an example of this today. It told me that the option to create a static list of numbers to filter on is possible using the Dropdown widget rather than creating a new object. It then gave me instructions on how to create and configure the Dropdown widget. I couldn’t find this widget in Workshop, so I replied that it doesn’t exist. AIP Assist then told me I am correct, the widget doesn’t exist and Workshop currently does not support static lists…
