How to trace ontology edits back to AIP Logic Function runs for debugging?

Ask the Community
,
1

How to trace ontology edits back to AIP Logic Function runs for debugging?

Context

I’ve built an AIP Logic function that makes ontology edits (creates, deletes, and modifications to ontology objects). The function is published and connected to an automation rule that triggers both on ontology object modifications and via cron schedule.

Current Observability Capabilities

The existing observability features in Palantir are excellent:

In Ontology Manager:

  • Selecting the function and navigating to Observability shows execution metrics including success/failure rates and P95 latency

In Workflow Lineage (Run History):

  • Detailed execution traces showing:

    • Every LLM call with request/response and latency

    • LLM tool usage and function calls

    • The specific model used

    • Ontology objects that were edited

This audit trail is comprehensive and invaluable for understanding what happened during each function execution.

The Challenge: Reverse Tracing from Ontology Edits

My operational debugging workflow needs to work in reverse:

  1. Starting Point: An ontology object that was created or edited by my AIP Logic function

  2. Desired Path: Navigate from this specific object edit → to the automation/function run → to view the complete execution details

  3. Debug Information Needed:

    • The LLM prompt that led to this specific edit

    • The LLM response and reasoning (chain-of-thought)

    • The tools/functions the LLM invoked

    • The input parameters that triggered this execution

Questions

  1. Object → Run Traceability: How can I select an ontology object and navigate to the specific automation run or function execution that created/modified it?

  2. Execution Replay: Once I’ve identified the problematic run:

    • Can I save the input parameters from that execution?

    • Can I replay those inputs against an updated function version after making changes to the function, tools, or prompts?

  3. Best Practices: What’s the recommended operational workflow for quickly debugging issues by tracing from an unexpected ontology edit back to its originating Logic function run?

What I’ve Explored

  • Edit History Widget: Provides audit trail of changes but doesn’t seem to link back to the Logic function execution

  • Action Log: Maps to Action types but unclear if this extends to automated Logic function runs

  • Workflow Lineage: Great for forward tracing (function → edits) but not clear on reverse direction (edit → function run)

Use Case

This is critical for production debugging scenarios where:

  • An object has incorrect values and I need to understand which execution and LLM reasoning led to those values

  • I need to reproduce the issue by replaying the exact inputs that caused the problem

  • I want to verify my fix works against the original problematic inputs

Has anyone implemented a workflow for this type of reverse tracing? Are there features in AIP Observability or Workflow Lineage that support this that I might have missed?

Additional Context:

  • Using AIP Logic with “Apply actions” tool in Use LLM blocks

  • Functions are published and consumed via automation rules

  • Object Storage V2 enabled with Track user edit history enabled