> Raw Markdown twin (generated at build time from the source Markdown). Rendered page: https://docs.gatellm.io/en/reference/supported-models-and-sdks · Doc index: https://docs.gatellm.io/en/llms.txt


# Supported providers, models & SDKs

The gateway is **protocol-oriented**: there is no built-in provider or model allowlist — any upstream that speaks a supported protocol can be connected, and whatever model name you configure in the console is what the client can call. This page lists three "what's supported" in one place: providers, models, and SDKs.

## Supported model providers

### Connected by protocol (static API key / credentials)

| Provider | Description | Upstream protocol |
|--------|------|---------|
| OpenAI | The full GPT / o family, GPT-Image, embedding, audio, realtime | `openai` / `openai_response` / `openai_images` / `openai_embeddings` / `openai_audio` / `openai_realtime` |
| OpenAI-compatible providers | Azure OpenAI, DeepSeek, Moonshot Kimi, Zhipu GLM, xAI Grok, MiniMax, ByteDance Doubao, Groq, Together, Mistral, OpenRouter, Cerebras, and self-hosted OpenAI-compatible services like vLLM / Ollama | the `openai` family |
| Anthropic | Claude official API | `anthropic` |
| Google | Gemini (AI Studio / Vertex AI's Gemini API) | `google` |
| AWS Bedrock | Bedrock-hosted Claude / Llama / Nova / Mistral and other models | `aws_converse` / `aws_invoke` |
| Kiro (CodeWhisperer / Q Developer family) | AWS's AI IDE channel, logged in via SSO credentials (see the table below) | `aws_codewhisperer_streaming` |
| Alibaba Cloud Bailian DashScope | Qwen language / multimodal, wan image, video generation, embeddings, rerank, ASR, realtime | `dashscope` / `dashscope_realtime` |
| Rerank providers | Jina / Cohere / vLLM rerank endpoints | `openai_rerank` |
| Any HTTP upstream | No protocol translation, forwarded as-is | `passthrough` |

> Providers not in the table: as long as they provide a **compatible endpoint** of any of the above protocols, they can be connected by that protocol, in exactly the same way as in-table providers.

### SSO credential login (OAuth / machine identity, no static API key)

Some upstreams don't accept static API keys and require an OAuth login to obtain a refresh credential. The gateway bundles a `login` subcommand and credential custody — logging in auto-creates the upstream:

| Provider | Login method | Status |
|--------|---------|------|
| Kiro | Portal OAuth (Builder ID / Google / GitHub / IAM Identity Center) | **Experimental** (currently the only `enabled=true`) |
| Google Service Account | Service-account JWT (machine identity) | Registered, not enabled |
| Alibaba Cloud Bailian Aliyun Bailian | client_credentials (machine identity) | Registered, not enabled |
| Qoder / Qoder (China site) | Manual PAT / OAuth token paste | Registered, not enabled |
| Z.ai / Zhipu BigModel | OAuth + manual paste | Registered, not enabled |
| Anthropic (WIF) / OpenAI (WIF) | Workload identity federation (token exchange) | Registered, not enabled |

Steps are in [Upstream SSO credential login](/en/howto/upstream-sso-login.md). **Status is authoritative in the `sso.toml` directory**: Kiro is currently the only `enabled=true` vendor (and itself `experimental=true`, its login endpoint behavior may change, and on failure you can fall back to **manual paste** in the console); the other eight have profiles registered in the directory, but their `enabled` gate (default false, fail-closed) is not open — the console doesn't show them and credential import is rejected; wait until their flow is adapted and the switch is turned on before logging in.

## Supported model list

First the mechanism: **the gateway does not restrict calls by model name**. Model names are defined by the admin in the console, and clients fill in the name configured in the gateway. The gateway additionally ships a set of **model metadata recognized by wildcard patterns** (tokenizer, context/output caps, reference prices); new models of major families automatically fall into the family rules as soon as they're released. Models outside the tables below can also be connected — they just use default capabilities, and pricing can be filled in manually (see [Pricing and billing](/en/howto/setup-pricing-and-billing.md)).

### Language (chat / reasoning / code)

| Provider | Representative models (newest first) | Connection protocol |
|--------|--------------------|---------|
| OpenAI | GPT-5.6 Sol / Terra / Luna, GPT-5.5 family, GPT-5.4 family, o-series reasoning, GPT-5.3 Codex | `openai` / `openai_response` |
| Anthropic | Claude Opus 5, Claude Sonnet 5, Claude Haiku 4.5, Fable 5 | `anthropic` |
| Google | Gemini 3.1 Pro, Gemini 3.5 Flash, Gemini 2.5 Pro / Flash | `google` |
| Alibaba Cloud Bailian | Qwen3.8-Max, Qwen3.7-Max, Qwen3.7 / 3.6 / 3.5-Plus, Qwen3-Coder, QwQ | `dashscope` |
| DeepSeek | DeepSeek V4-Pro, V4-Flash, R1 | `openai` compatible |
| Zhipu | GLM-5.2, GLM-5.1, GLM-5 | `openai` compatible |
| Moonshot | Kimi K3, K2.7, K2.6 | `openai` compatible |
| xAI | Grok 4.5 | `openai` compatible |
| MiniMax | MiniMax M3 | `openai` compatible |
| ByteDance Doubao | Doubao Seed 2.1 Pro / Turbo | `openai` compatible |
| AWS Bedrock | Bedrock-hosted models (e.g. `anthropic.claude-opus-5*`) | `aws_converse` / `aws_invoke` |
| Gemma | Gemma 4 (Cerebras-hosted) | `openai` compatible |

### Image (generation / editing)

| Provider | Representative models | Connection protocol |
|--------|---------|---------|
| OpenAI | GPT-Image-2 (`/v1/images/generations`, `/v1/images/edits`) | `openai_images` |
| Google | gemini-3.1-flash-image family (inline image output in `generateContent`, inter-translated with `openai_images`) | `google` |
| Alibaba Cloud Bailian | wan2.6-image, wanx family | `dashscope` |

### Video (generation)

| Provider | Representative models | Connection method |
|--------|---------|---------|
| Alibaba Cloud Bailian | wanx2.1-t2v, happyhorse-1.0 / 1.1 (t2v / i2v / r2v) | `dashscope` **async task** (`direct_path` + `async_mode`, see [DashScope quick deploy](/en/quickstart/dashscope.md#video-generation)) |
| Volcano Engine | Doubao Seedance 2.x | `openai` compatible / `passthrough` |
| Google | Gemini Omni family (preview, video output) | `google` |

### Realtime (Realtime voice sessions)

| Provider | Representative models | Upstream protocol |
|--------|---------|---------|
| OpenAI | gpt-realtime family | `openai_realtime` |
| Alibaba Cloud Bailian | Qwen3.5-Omni-Plus Realtime | `dashscope_realtime` |

The two realtime upstreams share one client entry `GET /v1/realtime` (OpenAI Realtime client protocol), see [Realtime sessions](/en/reference/realtime.md).

### Other capabilities (embeddings / rerank / speech)

| Capability | Representative models | Connection protocol |
|------|---------|---------|
| Embeddings | text-embedding-3 family, text-embedding-v4, BGE-M3 | `openai_embeddings` / `dashscope` |
| Rerank | gte-rerank, Qwen3-VL-Rerank, BGE Reranker v2-m3, Jina / Cohere compatible endpoints | `openai_rerank` / `dashscope` |
| Speech | whisper family, fun-asr, paraformer (ASR); TTS | `openai_audio` / `dashscope` |

## Supported SDKs and integration examples

General rule (same for all SDKs):

1. Point `base_url` at the gateway (default `http://localhost:7890`);
2. Set the API key to **the gateway-issued access key** (not the upstream's `sk-...`);
3. Set `model` to **the model name configured in the gateway** — the examples below use each vendor's current latest model; the actual value is whatever you configured.

Capability coverage at a glance:

| SDK | Language | Image | Video | Realtime |
|-----|------|------|------|------|
| OpenAI SDK | ✅ | ✅ | — | ✅ (WebSocket) |
| Anthropic SDK | ✅ | — (image-input understanding only) | — | — |
| Google GenAI SDK | ✅ | ✅ | — | — |
| DashScope SDK / HTTP | ✅ | ✅ | ✅ (async task) | ✅¹ |
| boto3 (Bedrock) | no client endpoint² | — | — | — |

> ¹ DashScope's realtime models connect via the gateway's OpenAI Realtime client protocol (see the realtime example below; `model` is the Qwen3.5-Omni-Plus Realtime config name in the gateway).
> ² Bedrock is an **upstream-side** protocol: clients use the OpenAI / Anthropic / Gemini SDKs via protocol translation to access Bedrock models, see [Endpoints · About AWS Bedrock and passthrough](/en/reference/endpoints.md#aws-bedrock-passthrough).

### OpenAI SDK (Python / Node) — language, image, realtime

```bash
pip install openai websockets
```

**Language** (latest flagship GPT-5.6 Sol):

```python
from openai import OpenAI

client = OpenAI(base_url="http://localhost:7890/v1", api_key="<your access key>")

resp = client.chat.completions.create(
    model="gpt-5.6-sol",
    messages=[{"role": "user", "content": "用一句话介绍你自己"}],
)
print(resp.choices[0].message.content)
```

**Image** (latest GPT-Image-2):

```python
img = client.images.generate(
    model="gpt-image-2",
    prompt="一只在代码海洋里游泳的猫，赛博朋克风",
    size="1024x1024",
)
print(img.data[0].url)
```

**Realtime** (latest gpt-realtime, WebSocket session):

```python
import asyncio, json, websockets

async def main():
    async with websockets.connect(
        "ws://localhost:7890/v1/realtime?model=gpt-realtime",
        additional_headers={
            "Authorization": "Bearer <your access key>",
            "OpenAI-Beta": "realtime=v1",
        },
    ) as ws:
        await ws.send(json.dumps({
            "type": "response.create",
            "response": {"modalities": ["text"],
                         "instructions": "用中文说一句问候"},
        }))
        async for raw in ws:
            ev = json.loads(raw)
            if ev["type"] == "response.output_text.delta":
                print(ev["delta"], end="")
            elif ev["type"] == "response.done":
                break

asyncio.run(main())
```

> The audio modality (voice conversation) opens `modalities: ["audio"]` on the same connection and pushes base64 PCM frames; OpenAI Realtime input/output sample rates are both 24 kHz. Embeddings / speech / rerank follow `client.embeddings` / `client.audio` / POST `/v1/rerank` the same way.

### Anthropic SDK — language

```bash
pip install anthropic
```

```python
import anthropic

# base_url goes to the root (SDK appends /v1/messages)
client = anthropic.Anthropic(base_url="http://localhost:7890", api_key="<your access key>")

msg = client.messages.create(
    model="claude-opus-5",
    max_tokens=1024,
    messages=[{"role": "user", "content": "用一句话介绍你自己"}],
)
print(msg.content[0].text)
```

> The Anthropic protocol is a language protocol (supports image **input understanding**, no image/video generation or realtime session endpoints). Via protocol translation, this SDK can also call non-Claude models — set `model` to any model name configured in the gateway, see [Protocol interop matrix](/en/reference/protocol-matrix.md).

### Google GenAI SDK — language, image

```bash
pip install google-genai
```

**Language** (latest Gemini 3.1 Pro):

```python
from google import genai

client = genai.Client(
    api_key="<your access key>",
    http_options={"base_url": "http://localhost:7890"},
)

resp = client.models.generate_content(
    model="gemini-3.1-pro",
    contents="用一句话介绍你自己",
)
print(resp.text)
```

**Image** (latest gemini-3.1-flash-image):

```python
from google.genai import types

resp = client.models.generate_content(
    model="gemini-3.1-flash-image",
    contents="画一只在代码海洋里游泳的猫，赛博朋克风",
    config=types.GenerateContentConfig(response_modalities=["IMAGE", "TEXT"]),
)
for part in resp.candidates[0].content.parts:
    if part.inline_data:
        with open("cat.png", "wb") as f:
            f.write(part.inline_data.data)
```

### DashScope SDK / HTTP — language, image, video, realtime

DashScope goes through the gateway's passthrough entry `/v1/services/{*rest}`; the SDK only needs to change `base_http_api_url`:

```bash
pip install dashscope requests
```

```python
import dashscope

dashscope.base_http_api_url = "http://localhost:7890/v1"
dashscope.api_key = "<your access key>"
```

**Language** (latest Qwen3.8-Max):

```python
from dashscope import Generation

resp = Generation.call(
    model="qwen3.8-max",
    messages=[{"role": "user", "content": "用一句话介绍你自己"}],
)
print(resp.output.choices[0].message.content)
```

**Image** (latest wan2.6-image, message-based synchronous API):

```python
from dashscope.aigc.image_generation import ImageGeneration

resp = ImageGeneration.call(
    model="wan2.6-image",
    messages=[{"role": "user", "content": [{"text": "一只在代码海洋里游泳的猫，赛博朋克风"}]}],
)
# The synchronous API returns the result in one shot; the image is in message.content
# (not results[0].url of the legacy prompt API)
print(resp.output.choices[0].message.content)
```

**Video** (latest wanx2.1-t2v, async task): video generation is a two-stage "submit task → poll result" flow. The SDK's `fetch` polling path differs from the gateway's task-query path (`/v1/services/{model}/tasks/{task_id}`), so here HTTP is used to demonstrate the full flow:

```python
import time, requests

BASE = "http://localhost:7890/v1/services"
H = {"Authorization": "Bearer <your access key>", "Content-Type": "application/json"}

# 1) Submit the async video task (the model must be configured with direct_path + async_mode,
#    see the DashScope quick deploy)
r = requests.post(f"{BASE}/aigc/video-generation/video-synthesis", headers=H, json={
    "model": "wanx2.1-t2v",
    "input": {"prompt": "一只在代码海洋里游泳的猫，赛博朋克风"},
    "parameters": {"size": "1280*720"},
})
task_id = r.json()["output"]["task_id"]

# 2) Poll the task status and take the video URL on success
while True:
    s = requests.get(f"{BASE}/wanx2.1-t2v/tasks/{task_id}", headers=H).json()
    status = s["output"]["task_status"]
    if status == "SUCCEEDED":
        print(s["output"]["video_url"])
        break
    if status in ("FAILED", "CANCELED"):
        raise RuntimeError(s)
    time.sleep(5)
```

**Realtime** (latest Qwen3.5-Omni-Plus Realtime): reuse the OpenAI SDK realtime example above, replacing the model in the URL with its gateway config name:

```text
ws://localhost:7890/v1/realtime?model=qwen3.5-omni-plus-realtime
```

> The client side always speaks the OpenAI Realtime protocol; when the upstream is `dashscope_realtime`, the gateway bridges to a DashScope realtime session, see [Realtime sessions](/en/reference/realtime.md). Note that DashScope realtime sample rates are asymmetric: input 16 kHz, output 24 kHz (OpenAI is 24 kHz for both input and output). When playing back, don't decode the output stream with the input rate (otherwise you get 1.5× slow playback with lowered pitch) — this is a vendor-side convention; the gateway bridge passes frames through without resampling.

## FAQ

**Q: My model / provider isn't in the tables above — can I still use it?**
Yes. The gateway connects by protocol, not by allowlist: as long as the provider offers a compatible endpoint, it can be connected; the model name is freely defined by you in the console. Models outside the tables use default metadata, and pricing can be configured manually.

**Q: The vendor released a newer-generation model — do I need to upgrade the gateway to use it?**
No. Configure the new model in the console and it's callable; built-in metadata is recognized by family wildcard patterns (e.g. `gpt-5*`, `claude*`), so a new model automatically inherits the family's tokenizer and context rules; exact prices can be filled in manually or arrive with image updates.

**Q: Can one SDK call another vendor's model?**
Yes, this is exactly protocol translation: the OpenAI SDK can call Claude / Gemini / Qwen, and the Anthropic SDK can also call GPT — set `model` to the model name configured in the gateway, and the gateway translates automatically. Supported combinations are in [Protocol interop matrix](/en/reference/protocol-matrix.md).

**Q: Can video / realtime models also go through load balancing and billing?**
Yes. Video goes through the async task path ("billing per second" only holds for models that declare a `per_second` billing mode such as `happyhorse-1.0/1.1` and ASR; `wanx2.1-t2v` has no pricing entry in `meta.toml`, and is billed per your actual config — see the model-list section); realtime counts each turn of the session into the same statistics / billing pipeline as HTTP, see [Realtime sessions](/en/reference/realtime.md).

**Next**: [Endpoints · auth · protocol interop](/en/reference/endpoints.md) for the full endpoint list; [Protocol interop matrix](/en/reference/protocol-matrix.md) for inter-translation combinations; [Upstream and model fields](/en/reference/upstreams-models-fields.md) for how to configure a model; [Client integration and gateway differences](/en/reference/clients-and-gateway-diffs.md) for behavioral differences through the gateway.