What models are supported with LLM proxies?

Ask the Community
,
1

When calling OpenAI LLM proxies the only model I can get to work so far is the gpt 4 series. IE gpt-4.1, gpt-4.1-mini etc. Is the GPT 5 series supported? How am I susppoed to know which models are supported?

import {
  SupportedFoundryClients,
  type OpenAIService,
} from '@codestrap/developer-foundations-types';
import OpenAI from 'openai';
import { foundryClientFactory } from '../factory/foundryClientFactory';
import type { ChatCompletionCreateParamsStreaming } from 'openai/resources/chat';
import type { RequestOptions } from 'openai/core';
import type { ResponseCreateParamsStreaming } from 'openai/resources/responses/responses';

// ADd tpe definitions for the OpenAI response here, or in a separate file and import them in, to ensure type safety when working with the API response data.
export function makeOpenAIService(): OpenAIService {
  const { getToken, url, ontologyRid } = foundryClientFactory(
    process.env.FOUNDRY_CLIENT_TYPE || SupportedFoundryClients.PRIVATE,
    undefined,
  );

  return {
    // TODO code out all methods using OSDK API calls
    completions: async (
      body: ChatCompletionCreateParamsStreaming,
      options?: RequestOptions,
    ) => {
      const token = await getToken();
      const client = new OpenAI({
        baseURL: `${url}/api/v2/llm/proxy/openai/v1`,
        apiKey: process.env.FOUNDRY_TOKEN,
      });

      const stream = await client.chat.completions.create(body, options);

      let text = '';
      for await (const chatCompletionChunk of stream) {
        text += chatCompletionChunk.choices[0]?.delta?.content || '';
      }
      return text;
    },
    responses: async (
      body: ResponseCreateParamsStreaming,
      options?: RequestOptions,
    ) => {
      const token = await getToken();
      const client = new OpenAI({
        baseURL: `${url}/api/v2/llm/proxy/openai/v1`,
        apiKey: process.env.FOUNDRY_TOKEN,
      });

      // Responses API streaming emits semantic events (delta, completed, error, etc.)
      const stream = await client.responses.create(
        { ...body, stream: true },
        options,
      );

      let text = '';

      for await (const event of stream) {
        if (event.type === 'error') {
          throw new Error(`OpenAI API error: ${event.code} - ${event.message}`);
        }

        if (event.type === 'response.output_text.delta') {
          text += event.delta ?? '';
        }
      }

      return text;
    },
  };
}
2

Hey @CodeStrap, we’re working on exposing this information directly in Model Catalog, with code/usage examples. In the meantime, the currently supported models/endpoints for the proxy [1] are:

  • Anthropic “Text Completion” models support the proxied Anthropic Messages API.
  • OpenAI “Text Completion” models (generally) support both the proxied OpenAI Chat Completions and Responses APIs. (OpenAI itself doesn’t support Chat Completions for all models, e.g. gpt-5.3-codex only supports Responses.)
  • OpenAI “Embeddings” models support the OpenAI embeddings API.

The above should be the case for any models exposed in Model Catalog.

Additionally, we’re starting to roll out in beta xAI Chat Completions, xAI Responses, Google generateContent, and Google streamGenerateContent APIs. There might be inconsistencies between the Foundry API and provider’s specs which we are actively working to remove as we GA the endpoints. If you encounter any issues with those, please let us know.

You can use the model RID from Model Catalog (e.g. ri.language-model-service..language-model.gpt-5 for GPT-5) as the model name within requests. We also support model aliases within requests (e.g. gpt-5 for GPT-5) and are working to expose the supported aliases within Model Catalog.

[1] https://www.palantir.com/docs/foundry/aip/llm-provider-compatible-apis