Official vendor SDKs

The official SDKs are each vendor's "native language". Using them through the gateway differs from calling the vendor directly only by the three universal steps: change base_url, swap in the access key, use the gateway model name. This section gives Python and Node examples per vendor, covering text, image, embeddings, and speech (realtime is covered separately in Realtime & audio).

Model names in the examples are real names under some gateway config; use whatever your console defines. The key is the placeholder <your access key>.

OpenAI SDK

pip install openai            # Python
npm i openai                  # Node

base_url points to /v1. Both Python and Node can call text, image, embeddings, and speech; the rerank endpoint (/v1/rerank) has no official SDK method, so use raw HTTP.

Text (Chat Completions):

Python

from openai import OpenAI

client = OpenAI(base_url="http://localhost:7890/v1", api_key="<your access key>")

resp = client.chat.completions.create(
    model="gpt-5.4-mini",
    messages=[{"role": "user", "content": "Introduce yourself in one sentence"}],
)
print(resp.choices[0].message.content)

TypeScript

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "http://localhost:7890/v1",
  apiKey: "<your access key>",
});

const resp = await client.chat.completions.create({
  model: "gpt-5.4-mini",
  messages: [{ role: "user", content: "Introduce yourself in one sentence" }],
});
console.log(resp.choices[0].message.content);

Responses API (usable when the upstream model is openai_response protocol):

Python

resp = client.responses.create(
    model="gpt-5.6-sol",
    input="Introduce yourself in one sentence",
    max_output_tokens=64,
)
print(resp.output_text)

TypeScript

const resp = await client.responses.create({
  model: "gpt-5.6-sol",
  input: "Introduce yourself in one sentence",
  max_output_tokens: 64,
});
console.log(resp.output_text);

Cross-protocol fields are translated and constrained by the target protocol: if your gateway's gpt-5.6-sol upstream is a Responses protocol, a max_tokens passed via a Chat Completions client is translated to max_output_tokens, and the upstream may require a lower bound (e.g. ≥16) — too small → 400. Using the Responses client (above) keeps the field aligned.

Streaming (Chat Completions):

stream = client.chat.completions.create(
    model="gpt-5.4-mini",
    messages=[{"role": "user", "content": "Count to 5"}],
    stream=True,
)
for chunk in stream:
    delta = chunk.choices[0].delta.content
    if delta:
        print(delta, end="")

Tool calling:

tools = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Query the weather of a city",
        "parameters": {
            "type": "object",
            "properties": {"city": {"type": "string"}},
            "required": ["city"],
        },
    },
}]
resp = client.chat.completions.create(
    model="gpt-5.4-mini",
    messages=[{"role": "user", "content": "What's the weather in Beijing?"}],
    tools=tools,
)
print(resp.choices[0].message.tool_calls)

Embeddings:

r = client.embeddings.create(
    model="text-embedding-3-small",
    input=["hello", "world"],
)
print(len(r.data[0].embedding))  # embedding dimension

Image generation:

img = client.images.generate(
    model="gpt-image-2",
    prompt="A cat swimming in an ocean of code, cyberpunk style",
    size="1024x1024",
)
print(img.data[0].url)

Speech-to-text (ASR):

audio = open("speech.mp3", "rb")
t = client.audio.transcriptions.create(model="whisper-1", file=audio)
print(t.text)

Text-to-speech (TTS):

speech = client.audio.speech.create(model="tts-1", voice="alloy", input="Hello, world")
speech.write_to_file("out.mp3")

Rerank (raw HTTP):

import requests

r = requests.post(
    "http://localhost:7890/v1/rerank",
    headers={"Authorization": "Bearer <your access key>"},
    json={"model": "bge-m3-reranker", "query": "cat", "documents": ["a cat", "a car"]},
)
print([res["relevance_score"] for res in r.json()["results"]])

Anthropic SDK

pip install anthropic          # Python
npm i @anthropic-ai/sdk        # Node

base_url points to the root (the SDK appends /v1/messages). Anthropic is a language protocol with image input understanding — no image/video generation or realtime endpoints.

Python

import anthropic

client = anthropic.Anthropic(base_url="http://localhost:7890", api_key="<your access key>")

msg = client.messages.create(
    model="claude-haiku-4-5",
    max_tokens=256,
    messages=[{"role": "user", "content": "Introduce yourself in one sentence"}],
)
# content is an array; thinking models may return type=="thinking" before type=="text"
text = "".join(b.text for b in msg.content if b.type == "text")
print(text)

TypeScript

import Anthropic from "@anthropic-ai/sdk";

const client = new Anthropic({
  baseURL: "http://localhost:7890",
  apiKey: "<your access key>",
});

const msg = await client.messages.create({
  model: "claude-haiku-4-5",
  max_tokens: 256,
  messages: [{ role: "user", content: "Introduce yourself in one sentence" }],
});
const text = msg.content.filter((b) => b.type === "text").map((b) => b.text).join("");
console.log(text);

Thinking blocks: Claude thinking models (e.g. claude-opus-5, claude-sonnet-5) may return a type == "thinking" content block before a type == "text" block. Filter by type when extracting text — don't assume content[0] is text.

Via protocol translation this SDK can also call non-Claude models — set model to any model name configured in the gateway, see Protocol interop matrix.

Google GenAI SDK

pip install google-genai        # Python
npm i @google/genai             # Node

base_url points to the root, via http_options.base_url (not the base_url parameter).

Python

from google import genai

client = genai.Client(
    api_key="<your access key>",
    http_options={"base_url": "http://localhost:7890"},
)

resp = client.models.generate_content(
    model="gemini-3.5-flash",
    contents="Introduce yourself in one sentence",
)
print(resp.text)

TypeScript

import { GoogleGenAI } from "@google/genai";

const client = new GoogleGenAI({
  apiKey: "<your access key>",
  httpOptions: { baseUrl: "http://localhost:7890" },
});

const resp = await client.models.generateContent({
  model: "gemini-3.5-flash",
  contents: "Introduce yourself in one sentence",
});
console.log(resp.text);

Image generation (inline image output via generateContent, interoperable with openai_images):

from google.genai import types

resp = client.models.generate_content(
    model="gemini-3.1-flash-image",
    contents="Draw a cat swimming in an ocean of code, cyberpunk style",
    config=types.GenerateContentConfig(response_modalities=["IMAGE", "TEXT"]),
)
for part in resp.candidates[0].content.parts:
    if part.inline_data:
        with open("cat.png", "wb") as f:
            f.write(part.inline_data.data)

OpenAI-compatible third parties

DeepSeek, Moonshot Kimi, Zhipu GLM, xAI Grok, MiniMax, ByteDance Doubao, Groq, Together, Mistral, Cohere, OpenRouter, Azure OpenAI, Cerebras — and self-hosted services like vLLM / Ollama — all expose OpenAI-compatible endpoints: on the client side you keep using the OpenAI SDK unchanged, changing only base_url and model.

from openai import OpenAI

client = OpenAI(base_url="http://localhost:7890/v1", api_key="<your access key>")

# model is the third-party model name configured in the gateway
resp = client.chat.completions.create(
    model="deepseek-v4-pro",   # or glm-5.2 / kimi-k3 / grok-4.5, etc.
    messages=[{"role": "user", "content": "Introduce yourself in one sentence"}],
)
print(resp.choices[0].message.content)

The only difference is the "upstream real endpoint": these vendors' upstreams are still their own OpenAI-compatible addresses, with the gateway translating or passing through in between. From your (client) side, the code is identical to calling OpenAI. Which capabilities each vendor opens (image/embeddings/rerank) depends on the upstream itself, see Supported providers & models.

Vendor	Example model name (gateway config)	Notes
DeepSeek	deepseek-v4-pro / deepseek-v4-flash	text/code only
Zhipu GLM	glm-5.2 / glm-5.1	text only
Moonshot Kimi	kimi-k3 / kimi-k2.7-code	text/code
xAI Grok	grok-4.5	text only
Azure OpenAI	your deployment name	base_url → https://<resource>.openai.azure.com, gateway side uses openai compatible

AWS Bedrock (upstream-side protocol)

Bedrock is not a client protocol — the gateway exposes no Bedrock-shaped client endpoint. To call Bedrock-hosted models, use the OpenAI / Anthropic / Gemini SDKs on the client via protocol translation, or go through /v3/{model}/{*rest} passthrough. See Endpoints · About AWS Bedrock and passthrough and Connect an AWS Bedrock upstream.

Next

• Frontend / Next.js unified SDKs: see Unified SDKs & gateways.
• Realtime voice sessions: see Realtime & audio.
• DashScope native SDK and async video: see DashScope native & async tasks.