I think you are missing the point. The advantage of using built-in AIP models is not the abstraction offered by the model catalog. It’s the fact that these are private endpoints provisioned by the hyperscaler in partnership with Palantir. They tend to offer better rate limits and better security than anything I could provision on my own. I’m totally familiar with bring your own model, etc. In the early days of AIP, Palantir provided us with API keys and the URLs of these private endpoints. I actually used to make raw API requests in my transforms to these endpoints. Then this concept of abstraction was introduced. I can see how this might be valuable in providing an interface that allows features in modules like AIP logic to interoperate with different models. That’s not a problem that most developers building AI systems close to the foundation layer have. For example, we made a focused decision to use the Gemini family of models for planning due to their cost and performance characteristics. We use OpenAI for coding as they tend to make fewer mistakes. As such, we are choosing to couple these models to the solution tightly. A simple example of how the current abstraction in Foundry makes this difficult is the inability to use structured outputs that pass a schema. Both Gemini and OpenAI support structured outputs, but within Foundry, I don’t see a structured outputs option in the type. This feature is critical for driving error rates in generated output to zero. Below is a sample request that uses structured outputs and OpenAI with gpt-5-mini
const response = await fetch('https://api.openai.com/v1/responses', {
method: 'POST',
headers: {
'Content-Type': 'application/json',
Authorization: `Bearer ${process.env.OPEN_AI_KEY}`,
},
body: JSON.stringify({
model: 'gpt-5-mini',
input: [
{ role: 'system', content: [{ type: 'input_text', text: system }] },
{ role: 'user', content: [{ type: 'input_text', text: user }] },
],
reasoning: { effort: 'low' },
// Optional: keep or remove web_search; it isn't needed if you fully inline the spec + code
tools: [
{
type: 'web_search',
user_location: { type: 'approximate', country: 'US' },
},
],
text: {
format: {
type: 'json_schema',
name: 'EditPlanV0',
schema,
strict: true,
},
verbosity: 'low',
},
store: true,
}),
});
Additionally, tools like WebSearch don’t appear to be supported. Additionally, the outputs may lack the details I need. For example, are the input/output tokens included in the responses from AIP? Ideally, developers could have direct access to the model endpoint and could use native tooling from OpenAI, Google, Vercel, or make a raw fetch request.
