> Raw Markdown twin (generated at build time from the source Markdown). Rendered page: https://docs.gatellm.io/en/sdk/unified-sdks · Doc index: https://docs.gatellm.io/en/llms.txt


# Unified SDKs & AI gateway layer

A "unified SDK" is one SDK that speaks to many vendors (Vercel AI SDK, LiteLLM, Portkey, …). They relate to the gateway in two layers:

1. **Treat the gateway as "just another OpenAI-compatible backend"** — point the SDK's base_url at the gateway and set `model` to the gateway model name (the body of this section).
2. **The gateway and these SDKs are separate layers** — the gateway manages upstream aggregation/keys/billing, the SDK manages client-side orchestration; they can stack (e.g. the Vercel AI SDK provider points at the gateway, which in turn fronts multiple upstreams).

## Vercel AI SDK (TypeScript) {#vercel-ai-sdk}

```bash
npm i ai @ai-sdk/openai-compatible @ai-sdk/anthropic
```

Use `createOpenAICompatible` to treat the gateway as an OpenAI-compatible backend; use `createAnthropic` to call an Anthropic upstream.

::: code-group

```ts [OpenAI-compatible provider]
import { createOpenAICompatible } from "@ai-sdk/openai-compatible";
import { generateText } from "ai";

const provider = createOpenAICompatible({
  name: "gatellm",
  baseURL: "http://localhost:7890/v1",
  apiKey: "<your access key>",
});

const { text } = await generateText({
  model: provider("gpt-5.4-mini"),
  prompt: "Introduce yourself in one sentence",
});
console.log(text);
```

```ts [Anthropic provider]
import { createAnthropic } from "@ai-sdk/anthropic";
import { generateText } from "ai";

const anthropic = createAnthropic({
  baseURL: "http://localhost:7890/v1",   // includes /v1; the SDK appends /messages
  apiKey: "<your access key>",
});

const { text } = await generateText({
  model: anthropic("claude-haiku-4-5"),
  prompt: "Introduce yourself in one sentence",
});
console.log(text);
```

:::

> Note the base URL convention is the reverse of the official SDK: `@ai-sdk/anthropic`'s `baseURL` **includes `/v1`** (it appends `/messages` itself), whereas the [official Anthropic SDK](/en/sdk/vendor-sdks.md#anthropic) takes the root `base_url` (it appends `/v1/messages` itself). Getting it wrong adds or drops the `/v1` segment.

**Streaming + structured output + tool calling**:

```ts
import { streamText, generateObject, tool } from "ai";
import { z } from "zod";

// Streaming
const { textStream } = await streamText({
  model: provider("gpt-5.4-mini"),
  prompt: "Count to 5",
});
for await (const t of textStream) process.stdout.write(t);

// Structured output (OpenAI-compatible providers use json_object mode; the prompt must mention "json")
const { object } = await generateObject({
  model: provider("gpt-5.4-mini"),
  schema: z.object({ name: z.string(), age: z.number() }),
  prompt: "Generate a fictional character in JSON format",
});
console.log(object);

// Tool calling
const { text } = await generateText({
  model: provider("gpt-5.4-mini"),
  prompt: "Weather in Beijing?",
  tools: {
    get_weather: tool({
      description: "Query the weather of a city",
      parameters: z.object({ city: z.string() }),
    }),
  },
});
```

**Next.js route handler** (`app/api/chat/route.ts`):

```ts
import { createOpenAICompatible } from "@ai-sdk/openai-compatible";
import { streamText } from "ai";

const provider = createOpenAICompatible({
  name: "gatellm",
  baseURL: process.env.GATELLM_BASE_URL ?? "http://localhost:7890/v1",
  apiKey: process.env.GATELLM_API_KEY ?? "",
});

export async function POST(req: Request) {
  const { messages } = await req.json();
  const result = await streamText({ model: provider("gpt-5.4-mini"), messages });
  return result.toTextStreamResponse();
}
```

## LiteLLM (Python) {#litellm}

LiteLLM is itself a gateway; using it as a **client SDK**, set `api_base` to the gateway:

```python
from litellm import completion

resp = completion(
    model="openai/gpt-5.4-mini",   # openai/ prefix routes to the OpenAI-compatible provider
    messages=[{"role": "user", "content": "Introduce yourself in one sentence"}],
    api_base="http://localhost:7890/v1",
    api_key="<your access key>",
)
print(resp.choices[0].message.content)
```

LiteLLM **proxy** (`config.yaml`) with the gateway as upstream:

```yaml
model_list:
  - model_name: gpt-5.4-mini
    litellm_params:
      model: openai/gpt-5.4-mini
      api_base: http://localhost:7890/v1
      api_key: <your access key>
```

> The `openai/` prefix is LiteLLM's provider routing marker; `api_base` overrides the default `api.openai.com` and points at your gateway.

## Portkey {#portkey}

Portkey is an AI gateway with routing/caching/fallback. To put the gateway behind Portkey, register a **custom OpenAI provider** in the Portkey dashboard with `baseURL` = `http://localhost:7890/v1` and `apiKey` = your gateway access key; the client keeps using Portkey's own SDK with `model` set to the virtual model name configured in Portkey. The wiring is the same shape; details vary with Portkey's version — see the [Portkey docs](https://portkey.ai/docs).

## Braintrust Gateway / Langbase {#braintrust-langbase}

Both are hosted "unified API + many models" gateways; wiring is the same as Portkey: register the gateway as an OpenAI-compatible "custom provider / model provider" with `baseURL` → `http://localhost:7890/v1` and `apiKey` = gateway access key. This layer **stacks** with the gateway — only worth it if you also need their routing/eval capabilities.

## Semantic Kernel (Python / .NET) {#semantic-kernel}

```bash
pip install semantic-kernel
```

Treat the gateway as an OpenAI-compatible backend:

```python
import asyncio
from semantic_kernel import Kernel
from semantic_kernel.connectors.ai.open_ai import OpenAIChatCompletion

kernel = Kernel()
service = OpenAIChatCompletion(
    service_id="gatellm",
    ai_model_id="gpt-5.4-mini",
    endpoint="http://localhost:7890/v1",
    api_key="<your access key>",
)
kernel.add_service(service)

async def main():
    resp = await kernel.invoke_prompt("Introduce yourself in one sentence")
    print(resp)

asyncio.run(main())
```

> Parameter names vary with the Semantic Kernel version (`endpoint` / `base_url` / `api_key`) — follow the [Semantic Kernel docs](https://learn.microsoft.com/semantic-kernel/overview) for your installed version. This section is written per the official docs and not run in this environment.

## Spring AI (Java) {#spring-ai}

Spring AI's OpenAI starter speaks the OpenAI-compatible protocol. Configure `application.yml`:

```yaml
spring:
  ai:
    openai:
      base-url: http://localhost:7890
      api-key: <your access key>
      chat:
        options:
          model: gpt-5.4-mini
```

> Spring AI's `base-url` does not include `/v1` (it appends it). The Java example follows the [Spring AI docs](https://docs.spring.io/spring-ai/reference/) and is not run in this environment.

## Next {#next}

- Orchestration frameworks (LangChain / LlamaIndex / CrewAI…): see [Orchestration frameworks](/en/sdk/orchestration-frameworks.md).
- Full capability examples for the official vendor SDKs: see [Vendor SDKs](/en/sdk/vendor-sdks.md).