Why we built it: Solution Designer & AIP Architect

If you’re interested in learning more about our product development behind the scenes, this post is for you. Today, we’ll share the inside story of “why we built it” for Solution Designer & AIP Architect. This is hopefully the first of a new series of posts.

Palantir’s platforms can be intimidating for beginners because of the sheer range of use cases that can be built in them, the pathways for both no-code and pro-code, and the sometimes unorthodox mental models with which they were designed. (More can be written on that!)

One of the most common challenges we were hearing from users new to the platform is “how do I build use case X in Foundry?”. Knowing how to leverage the right components for the problem at hand is something our FDEs are well-versed in, but takes practice for new users to learn:

  • There are an endless number of pathways to take in Foundry, that cannot just be put on a decision chart. Details of what you’ll need (e.g. build frequency, incrementality, types of data, when compute happens) do matter.
  • You shouldn’t have to read up about all the concepts (ontology? actions? functions?) in advance before you can get started.
  • You shouldn’t need to know how different Foundry concepts interact and complement each other (that health checks are applied to datasets, actions can be backed by functions, and so on).

The Solution Designer application was built to solve this problem: leverage a library of components and implementation patterns to compose a solution architecture, in an easy graph interface. It was started, like many of our products, by a few FDEs who ran into the same problem over and over: seeing engineers trying to whiteboard solutions, but not speaking the same language as the Foundry platform (or each other). While whiteboards afford endless flexibility, they also invite a little too much hand-waving, and you don’t necessarily come out with a clean design or mutual understanding. They decided that Foundry should have its own whiteboard, and that the whiteboard should know what’s possible! As one does at Palantir, they built it and started testing it with users.

Solution Designer is used daily by hundreds of users now: by developers to explain their ideas or implementations, PMs to write project proposals, and architects to explain the high-level architecture of solutions. We’ve seen that it helps people grasp what is or will be built much faster, weigh alternatives and validate that Foundry can provide a working solution to a given problem, before actually building it.

More recently, we saw an opportunity for making it even easier to learn Foundry concepts and methodology. AIP Architect is a new capability within Solution Designer that uses LLMs as translators to help you get started. The idea is to give developers an AI thought partner that already knows all Foundry concepts to think through the problem space, ask targeted questions and point them in the right direction. It is still in beta so you may not have heard of it before, but it’s available to anyone!(*)

Have you tried it? Did it actually help you get a new use case off the ground? How and when would you like to use it? Please comment below. We’d love to hear feedback, additional questions and see examples of the graphs you’re most proud of!

To finish off, we’ll answer a few frequently asked questions…

:thinking: What has surprised you the most? We have people coming to us with Solution Designer diagrams all the time, to ask for feedback. And they’re often really impressive, with dozens or hundreds of nodes and annotations! We expected that it would be most useful to brand new users, and that the more repetitions users get at pre-designing a solution, the less they’d need Solution Designer. However, it appears that even very advanced users like it as a way of validating their thinking or just pitching it to a colleague.

:exploding_head: What’s a cool feature I might not know about? Did you know you can import diagrams from Monocle? If you want to evolve an existing use case, you can import it from your existing implementation and edit it in Solution Designer to plan out your changes.

:construction: What features are you thinking about doing next? We did some experimentation with including upstream systems or external building blocks into diagrams, to expand the architecture beyond just Foundry, though we’re not sure yet if we’ll invest further in this. If you have strong demand for this, please let us know in the comments.

Note: you need AIP and GPT-4o turned on in order to use AIP Architect.

15 Likes

(idk if this is new but I like this post style)

Below is a vague/messy description of what I seem to want out of solution designer:

I have an app :iphone:

  • I have 2 “features” I want to add, I am contemplating which one to do first
  • I imported the app’s data lineage into solution designer
  • I want to ask someone (either a Palanterian, the Developer form, an AI agent) which feature to add first

Currently I have 3 solution designer files :three:

  • Base app architecture
  • Base app architecture + new feature 1 architecture
  • Base app architecture + new feature 2 architecture

I think I would love :heart:

  • 1 solution designer “file” where can toggle between next steps
  • I want to be able to toggle through and “present” different paths
  • I want to be able for each path be able to click next so I can step by step show how I am building my proposed solution
  • I want to use Solution designer as a presentation / communication method (powerpoint-ish)
4 Likes

Ooo I really like this idea of being able to use a “presentation mode” in Solution Design and have different “frames” representing the evolution (or potential evolution of the project).

This is actually a feature (often overlooked) in Data Lineage, where you can create a layout, save it as a frame, and then make more changes, etc. Then go through them one-by-one. I use this frequently when I’m creating a curated Data Lineage asset as part of documenting a project, normally starting with a high level view with lots of grouped nodes, then a frame that focuses on the data ingestion and transforms logic, then another that focuses on the ontology and link types, and a final slide that shows the applications and actions.

I’d love to keep seeing us invest in having interoperability and some feature parity between the two tools as, in my mind, they’re flip sides of the same coin (what’s been built - as represented by actual Foundry resources and their dependencies) and what’s been planned or envisioned. A way to “present” or “document” a curated set of these views that bridges both would be really nice!

p.s. if you don’t see the presentation tools in Data Lineage, you have to toggle on the Simple Presentation Mode in the little user settings gear wheel :gear: in the upper right by the branch selector.

1 Like

Data Lineage frames were dope / easy / cool
Thanks! :pray:

I liked solution designer because it got me familiar with different components (Data connection, AIP Logic, Pipeline, workshop, etc). In my opinion Solution designer stoped being useful once I started building:

  • Because I couldn’t find a nice consistent step by step intuitive way to anchor back my solution designer diagram to my existing ontology
  • (My Best bad guess was to keep importing from data lineage which rewrites entire solution designer file)

I love your description of different classes of apps (What’s been built vs what’s been planned). I see the breakdown like this:

  • What’s been built aka Ground truth of ontology (Data Lineage, Vertex)
  • What’s been planned aka what I want to add next (solution designer)

I see there existing a 2 way interplay between these classes of apps:

  • What I want —> ontology
    • I think I would love the ability to bit by bit to impose solution designer steps onto ontology
    • Aka from solution designer be able to create an object with properties
  • Ontology —> what I want
    • I think I would love the ability to do work outside of solution designer, than be able to to compare that work to my current solution designer architecture

I feel like solution designer is the most versatile largest scope view of any given project. (Maybe I just don’t understand Data Lineage enough :face_with_spiral_eyes:)

I’d love to be able to go into the weeds / details :female_detective: and build some things and then have those changes be represented globally :earth_americas: in solution designer.

I would love to hear how Palantir-ians think about the interplay between and build with the 3 apps (data linage / vertex / solution designer)

1 Like