Unified SDKs & AI gateway layer

A "unified SDK" is one SDK that speaks to many vendors (Vercel AI SDK, LiteLLM, Portkey, …). They relate to the gateway in two layers:

1. Treat the gateway as "just another OpenAI-compatible backend" — point the SDK's base_url at the gateway and set model to the gateway model name (the body of this section).
2. The gateway and these SDKs are separate layers — the gateway manages upstream aggregation/keys/billing, the SDK manages client-side orchestration; they can stack (e.g. the Vercel AI SDK provider points at the gateway, which in turn fronts multiple upstreams).

Vercel AI SDK (TypeScript)

npm i ai @ai-sdk/openai-compatible @ai-sdk/anthropic

Use createOpenAICompatible to treat the gateway as an OpenAI-compatible backend; use createAnthropic to call an Anthropic upstream.

OpenAI-compatible provider

import { createOpenAICompatible } from "@ai-sdk/openai-compatible";
import { generateText } from "ai";

const provider = createOpenAICompatible({
  name: "gatellm",
  baseURL: "http://localhost:7890/v1",
  apiKey: "<your access key>",
});

const { text } = await generateText({
  model: provider("gpt-5.4-mini"),
  prompt: "Introduce yourself in one sentence",
});
console.log(text);

Anthropic provider

import { createAnthropic } from "@ai-sdk/anthropic";
import { generateText } from "ai";

const anthropic = createAnthropic({
  baseURL: "http://localhost:7890/v1",   // includes /v1; the SDK appends /messages
  apiKey: "<your access key>",
});

const { text } = await generateText({
  model: anthropic("claude-haiku-4-5"),
  prompt: "Introduce yourself in one sentence",
});
console.log(text);

Note the base URL convention is the reverse of the official SDK: @ai-sdk/anthropic's baseURL includes /v1 (it appends /messages itself), whereas the official Anthropic SDK takes the root base_url (it appends /v1/messages itself). Getting it wrong adds or drops the /v1 segment.

Streaming + structured output + tool calling:

import { streamText, generateObject, tool } from "ai";
import { z } from "zod";

// Streaming
const { textStream } = await streamText({
  model: provider("gpt-5.4-mini"),
  prompt: "Count to 5",
});
for await (const t of textStream) process.stdout.write(t);

// Structured output (OpenAI-compatible providers use json_object mode; the prompt must mention "json")
const { object } = await generateObject({
  model: provider("gpt-5.4-mini"),
  schema: z.object({ name: z.string(), age: z.number() }),
  prompt: "Generate a fictional character in JSON format",
});
console.log(object);

// Tool calling
const { text } = await generateText({
  model: provider("gpt-5.4-mini"),
  prompt: "Weather in Beijing?",
  tools: {
    get_weather: tool({
      description: "Query the weather of a city",
      parameters: z.object({ city: z.string() }),
    }),
  },
});

Next.js route handler (app/api/chat/route.ts):

import { createOpenAICompatible } from "@ai-sdk/openai-compatible";
import { streamText } from "ai";

const provider = createOpenAICompatible({
  name: "gatellm",
  baseURL: process.env.GATELLM_BASE_URL ?? "http://localhost:7890/v1",
  apiKey: process.env.GATELLM_API_KEY ?? "",
});

export async function POST(req: Request) {
  const { messages } = await req.json();
  const result = await streamText({ model: provider("gpt-5.4-mini"), messages });
  return result.toTextStreamResponse();
}

LiteLLM (Python)

LiteLLM is itself a gateway; using it as a client SDK, set api_base to the gateway:

from litellm import completion

resp = completion(
    model="openai/gpt-5.4-mini",   # openai/ prefix routes to the OpenAI-compatible provider
    messages=[{"role": "user", "content": "Introduce yourself in one sentence"}],
    api_base="http://localhost:7890/v1",
    api_key="<your access key>",
)
print(resp.choices[0].message.content)

LiteLLM proxy (config.yaml) with the gateway as upstream:

model_list:
  - model_name: gpt-5.4-mini
    litellm_params:
      model: openai/gpt-5.4-mini
      api_base: http://localhost:7890/v1
      api_key: <your access key>

The openai/ prefix is LiteLLM's provider routing marker; api_base overrides the default api.openai.com and points at your gateway.

Portkey

Portkey is an AI gateway with routing/caching/fallback. To put the gateway behind Portkey, register a custom OpenAI provider in the Portkey dashboard with baseURL = http://localhost:7890/v1 and apiKey = your gateway access key; the client keeps using Portkey's own SDK with model set to the virtual model name configured in Portkey. The wiring is the same shape; details vary with Portkey's version — see the Portkey docs.

Braintrust Gateway / Langbase

Both are hosted "unified API + many models" gateways; wiring is the same as Portkey: register the gateway as an OpenAI-compatible "custom provider / model provider" with baseURL → http://localhost:7890/v1 and apiKey = gateway access key. This layer stacks with the gateway — only worth it if you also need their routing/eval capabilities.

Semantic Kernel (Python / .NET)

pip install semantic-kernel

Treat the gateway as an OpenAI-compatible backend:

import asyncio
from semantic_kernel import Kernel
from semantic_kernel.connectors.ai.open_ai import OpenAIChatCompletion

kernel = Kernel()
service = OpenAIChatCompletion(
    service_id="gatellm",
    ai_model_id="gpt-5.4-mini",
    endpoint="http://localhost:7890/v1",
    api_key="<your access key>",
)
kernel.add_service(service)

async def main():
    resp = await kernel.invoke_prompt("Introduce yourself in one sentence")
    print(resp)

asyncio.run(main())

Parameter names vary with the Semantic Kernel version (endpoint / base_url / api_key) — follow the Semantic Kernel docs for your installed version. This section is written per the official docs and not run in this environment.

Spring AI (Java)

Spring AI's OpenAI starter speaks the OpenAI-compatible protocol. Configure application.yml:

spring:
  ai:
    openai:
      base-url: http://localhost:7890
      api-key: <your access key>
      chat:
        options:
          model: gpt-5.4-mini

Spring AI's base-url does not include /v1 (it appends it). The Java example follows the Spring AI docs and is not run in this environment.

Next

• Orchestration frameworks (LangChain / LlamaIndex / CrewAI…): see Orchestration frameworks.
• Full capability examples for the official vendor SDKs: see Vendor SDKs.