> Raw Markdown twin (generated at build time from the source Markdown). Rendered page: https://docs.gatellm.io/en/reference/upstreams-models-fields · Doc index: https://docs.gatellm.io/en/llms.txt


# Upstream and model fields

This chapter covers how to configure upstream services (model providers) and models in the console. This is the prerequisite for the gateway to forward requests.

Entry: Console → **Upstream services** (admin only). Each upstream has several models under it.

## Upstream service management

### Create an upstream

Console → Upstream services → New, fill in the following fields:

| Field | Description | Default | Example |
|------|------|--------|------|
| Name | Unique upstream identifier; models reference it | (required) | `openai` |
| Protocol | Determines the executor and request schema (15 in total) | (required) | `openai` / `openai_response` / `openai_images` / `openai_embeddings` / `openai_audio` / `openai_rerank` / `openai_realtime` / `anthropic` / `google` / `aws_converse` / `aws_invoke` / `aws_codewhisperer_streaming` / `dashscope` / `dashscope_realtime` / `passthrough` |
| Base URL | The upstream's real address, no trailing `/` | (required) | `https://api.openai.com/v1` |
| API Key | Upstream key; multiple can be added; each entry = key (masked on edit) + weight (`0` = standby) + remove; deterministically weighted distribution, the same caller sticks to the same key | (required) | `sk-...` |
| Enabled | Disabled means it doesn't participate in routing | `true` | ✓ |
| Custom headers | Extra request headers; supports `{{var}}` and `{{alt1\|alt2}}` fallback chains | empty | |
| User-Agent | Overrides the global UA; supports templates | uses the global UA | |
| Proxy | Upstream outbound proxy | none | |
| DNS | per-upstream DNS (DoH/DoT/UDP auto-detected) + TTL | uses system DNS | |
| Request body rules | Request-body rewrite rules (path/value/mode) | empty | |
| Request/response script | JS script transform (see [Scripting API reference](/en/reference/scripting-api.md)) | none | |
| Script error mode | `log-and-continue` / `log-and-reject` | `log-and-continue` | |
| Script position | `after` (after rules) / `before` | `after` | |
| Token counting | `upstreamApi` / `tiktoken` | `upstreamApi` | |
| Rate limit | Upstream-level rate limit: `rpm` / `max_concurrency` (concurrency) / `tpm` / `tps_min_interval_ms` (minimum interval between adjacent requests, ms) / `wait_timeout_secs` (wait-for-permit timeout, default 10s) / `initial_tokens` (initial token-bucket fill percentage 0–100, default 100) | unlimited | |

### Fetch available models (probe)

The **"Fetch available models"** button in the upstream form makes a real model-list probe to that upstream (e.g. `GET /models` for an OpenAI-compatible upstream), opening the "Available model probe" panel: it shows the parsed model-ID count (including the upstream's raw return count and source) and the model-ID list (truncated to the first several when too long), with a one-click **copy all**. Use it to confirm which model IDs the upstream actually offers before creating a model, avoiding typos.

### Multi-key and weights {#multi-key-weights}

An upstream can have multiple API keys. The selection algorithm is **deterministic weighted distribution + sticky binding**, not round-robin:

- First selection hashes `(caller access_key, upstream name)` with FNV-1a, and lands in a cumulative weight interval by each key's `weight` as a proportion of the total weight. Different callers spread across different keys by weight ratio — a `weight=2` key gets twice the traffic of a `weight=1` key.
- The same caller access_key **sticks to the same key** (the binding is stored in an in-memory session-key binding table, memory-only and not shared across nodes — each node maintains its own local prompt-cache stickiness), guaranteeing upstream prompt-cache and signature consistency. The binding has an **idle TTL** (`KEY_BINDING_TTL_SECS`, default 1800 seconds / 30 minutes, `0` = permanent): if the caller makes no new request within the TTL it expires and is re-selected next time.
- **Config reload no longer wipes all bindings**; invalidation is **fingerprint-driven**: each binding records the fingerprint of the upstream's candidate key set (computed from each key's position and weight). When reading a binding, if the fingerprint doesn't match the current candidate set it's treated as invalid and re-selected. Therefore — **rotating a key's value in place** (same position, new value) doesn't break stickiness; whereas **adding/removing, reordering keys, or changing weights** changes the fingerprint and triggers re-selection. This matches the minimum-disruption semantics of industrial gateways (Envoy / AWS ALB).
- `weight = 0` is a **standby key**: only enabled after all `weight > 0` keys have been tried and failed, and no binding is stored.
- When a key fails, the gateway rotates within that upstream to the next untried key and retries, but does **not** override the first sticky binding — subsequent requests from that caller still start from the originally bound key.

> This is the multi-key selection mechanism **within a single upstream**, and it is a separate mechanism from **load-balancer entry selection** (Smooth WRR smooth weighted round-robin, see [Load-balancing fields](/en/reference/load-balancing-fields.md)): LB entry selection happens at the handler layer, across upstreams; within-upstream multi-key selection happens at the dispatch layer, within the same upstream. The two are orthogonal and stack (LB selects an entry → that entry's upstream then selects a key stickily per this section).

**Viewing live bindings in the console**: upstream list → the **Monitor** icon on a model row of that upstream → the "Model monitor" panel. The **Access Key Bindings** tab lists, row by row, each access key that can reach this model via group ACL, and which upstream key it is **currently** stuck to: source `binding` means an active sticky binding exists; `prediction` means there's no active binding right now and the shown upstream key is the deterministic weighted prediction computed from **this model's candidate set** (different models of the same upstream may predict differently, since each model computes its own candidate set). Bindings whose fingerprint doesn't match the current candidate set are marked **stale** — still shown, but the next request re-selects (consistent with dispatch truth). The **upstream keys** roster above acts as a legend; hovering a binding row highlights its corresponding upstream key. This view is **single-node** (in both storage modes the binding table is in memory and not shared across nodes), and the panel bottom carries the instance identifier. The panel auto-refreshes every 3 seconds by default (can be turned off). The rate-limit gauge is on the **Rate Limit** tab; when the model has no rate limit configured that tab shows an empty state, which doesn't affect the binding view. The full selection/switch semantics are as described above (deterministic weighted hash + fingerprint invalidation + idle TTL).

### Delete an upstream

If the upstream still has models referencing it, deletion returns **422** (delete or migrate the models first, then delete the upstream).

### Custom headers and User-Agent

Attach custom headers to all the upstream's requests, supporting template variables. The full variable list and per-config-scope availability are in [Template variable reference](/en/reference/template-variables.md); variables available in this context:

| Variable | Source |
|------|------|
| `{{request_access_key_name}}` | The matched access key name |
| `{{request_access_key_group}}` | The access key group name |
| `{{request_id}}` | The client's `z-request-id` header |
| `{{access_key_hash}}` | First 16 bytes of the access-key-name SHA-256 hash, base64 |
| `{{system_prompt_hash}}` | SHA-256 of the system prompt content (protocol-agnostic) |
| `{{header:Key}}` | The value of a client request header (case-insensitive) |

**Fallback-chain syntax**: `{{var1|var2}}` takes the first available variable in order; if all are empty, that header is omitted (an empty string is not sent).

**Example — include the access key name in the User-Agent for upstream log correlation (Console → Upstream services → edit that upstream):**

1. Set the **User-Agent** field to `gatellm/key-{{request_access_key_name}}`.
2. Add two entries in the **Custom headers** area:
   - Header name `X-Request-Access-Key`, value `{{request_access_key_name}}`.
   - Header name `X-Request-Group`, value `{{request_access_key_group}}`.

### Per-upstream DNS resolver

Each upstream can have its own DNS servers, bypassing container/OS-level DNS caching, to solve persistent 503/504 caused by stale cached IPs after an upstream ALB IP rotation:

| Scheme | Protocol | Port |
|--------|------|------|
| (none) | UDP standard DNS | 53 |
| `udp://` | UDP + TCP fallback | 53 |
| `tls://` | DNS-over-TLS | 853 |
| `https://` | DNS-over-HTTPS | 443 |

In Console → Upstream services → edit that upstream → the **DNS** field, enter a server list (scheme determines protocol, see the table above):

- Use Alibaba DNS (UDP): `223.5.5.5`, `223.6.6.6`.
- Use Cloudflare DNS-over-TLS: `tls://1.1.1.1`, `tls://1.0.0.1`.

## Model management

Click **expand** on an upstream row in the upstream list to see that upstream's model subtable (columns: Name / Upstream model ID / Protocol / Aliases / Hidden primary name). Each model has 5 actions: **Test model**, **Model monitor**, **Copy config** (quickly create a new one using this model's config as a template), **Edit**, **Delete**.

### Create a model

| Field | Description | Default | Example |
|------|------|--------|------|
| Name | The model name exposed to clients, customizable | (required) | `gpt-4o` |
| Upstream | The owning upstream | (required) | `openai` |
| Upstream model ID | The real model name sent to the upstream | (required) | `gpt-4o` |
| Aliases | Extra usable names (share a globally unique space with the model name) | empty | `gpt4o` |
| Hide primary name | When `hide_name=true`, the primary name is inaccessible (404); only aliases or load-balancer access work | `false` | |
| Anthropic Signature | Preserve reasoning context across multi-turn conversations (signature passthrough of thinking blocks) | off | |
| Protocol override | If empty, uses the upstream protocol; if set, this model uses a different protocol | inherit upstream | |
| Base URL override | If empty, uses the upstream base_url; with `direct_path` this value is treated as the full endpoint URL | inherit upstream | |
| Type kind | Variant identifier for protocols like DashScope | none | |
| inference_profile | Bedrock's inference configuration ARN | none | |
| Request body rules | Model-level request-body rewrite rules (override upstream-level) | empty | |
| Request/response script | Model-level script (overrides upstream-level) | none | |
| Rate limit override | `inherit` (inherit upstream) or concrete values of the same shape as the upstream (fields in the upstream "Rate limit" row) | `inherit` | |
| Token counting override | Overrides upstream-level | inherit upstream | |
| Enabled | | `true` | ✓ |

> **Name vs upstream model ID**: the client calls the "Name"; the gateway sends the "upstream model ID" to the upstream. When they differ you get model aliasing/renaming. `aliases` are extra usable names.

### Test model

Model list → that model → **Test model**, opens the test drawer to make one real call verifying the chain (`POST /console/api/models/test`):

- **Client protocol**: choose `openai` / `anthropic` / `openai_response` / `google`; after choosing, the drawer shows the "→ upstream protocol" translation mapping in real time — used to verify cross-protocol translation (e.g. Anthropic client protocol calling an OpenAI upstream).
- **Request body**: a JSON editor pre-filled by the chosen client protocol, freely editable before clicking **Send test**, returning the real upstream response (success / failure and error details).
- **realtime protocol** (`openai_realtime` / `dashscope_realtime`) models have a separate realtime session test: it opens a WebSocket session to the resolved upstream for end-to-end verification (start session / end session, showing handshake and session-establishment status).

## Model aliases and hidden names

### Aliases

Define multiple accessible names for a model. Console → Model → New/Edit `gpt-4o-2024-05-13`, fill in:

- **Upstream** select `openai`.
- **Upstream model ID** fill `gpt-4o-2024-05-13`.
- **Aliases** fill `gpt-4o`, `gpt4o` (two, sharing the globally unique space with the primary name).

Clients can reach the same model with either `gpt-4o` or `gpt4o`.

> **Note**: a key group's model ACL only matches the **canonical name** (the `name` field), not aliases. If the model's canonical name is `gpt-4o-2024-05-13`, the key group's model list must list `gpt-4o-2024-05-13`, not an alias.

### Hide primary name (hide_name)

When `hide_name = true` is set, clients **cannot** access the model by its canonical name (404) — only by alias. Console → Model → New/Edit `claude-sonnet-4-5`, fill in:

- **Upstream** select `anthropic`.
- **Upstream model ID** fill `claude-sonnet-4-5`.
- **Aliases** fill `writing-assistant` (a custom public-facing name, fictitious in this example).
- Check **Hide primary name** (`hide_name`).

Effects:
- Clients can only access it via `writing-assistant`
- The `/v1/models` list only lists the alias, not `claude-sonnet-4-5`
- Load balancers can reference `claude-sonnet-4-5` (a hidden model can be an LB entry)
- Statistics, logs, and credential filtering still use the canonical name

**Use cases**:
- **White-label**: expose a model under a brand name, hiding the real upstream model ID
- **Migration**: expose a stable alias, change the underlying model name
- **LB-only**: hide the underlying model, expose it only through an LB name

> When `hide_name = true`, `aliases` may be empty. The model is then invisible to external APIs but can still be referenced by a load balancer.

## Request body modification (Request Payload)

Modify the JSON request body before it's sent to the upstream. Rules apply **after protocol translation**, affecting the final upstream request body.

### Rule format

Each rule contains:

| Field | Type | Description |
|------|------|------|
| `path` | String | Dot-separated JSON path (e.g. `temperature`, `messages.0.content`) |
| `value` | JSON | The value to set, append, or remove |
| `mode` | String | **Required**, see the mode table below |
| `condition` | Object | Conditions required by some modes |

### Path syntax

| Syntax | Example | Description |
|------|------|------|
| Object key | `temperature` | Top-level field |
| Nested key | `response_format.type` | Drill into nested objects |
| Array index | `messages.0.content` | Access the element at index 0 |
| Wildcard | `messages.*.content` | Match **all** elements of the array |

> ⚠️ `messages[].content` and `messages[*].content` are **illegal**. Use `messages.*.content` for wildcard and `messages.0.content` for a specific index.

### Mode overview

| Mode | Description | Example |
|------|------|------|
| `overwrite` | Replace the value at the target path | `{"path": "temperature", "value": 0.7, "mode": "overwrite"}` |
| `remove` | Delete the key at the target path | `{"path": "temperature", "value": null, "mode": "remove"}` |
| `add-if-absent` | Set the value only when the key doesn't exist | `{"path": "top_p", "value": 0.9, "mode": "add-if-absent"}` |
| `append` | Unconditionally append to an array | `{"path": "tools", "value": {...}, "mode": "append"}` |
| `append-if-missing` | Conditional append (see below) | |
| `remove-matching` | Conditionally remove array elements (see below) | |
| `strip-lines` | Remove matching lines from a string | |
| `filter-content-types` | Remove blocks of specified types from a content array | |
| `filter-tools` | Filter a tools array | |
| `inject-system-prompt` | Inject a system prompt at the protocol-appropriate position (`value` is the prompt text) | `{"value": "Be concise.", "mode": "inject-system-prompt"}` |
| `switch-route` | **Does not modify the body**; evaluates `when` against the **client** shape **before** protocol translation, and when any predicate holds, re-dispatches this request to `use_model` (immediate switch, **supports cross-protocol**). Zero script overhead; predicates don't read binaries, large-body disk spooling stays on | see below |

### Common examples

**Simple parameter overwrite** — Console → model detail (or the upstream's "default request body rules") → **Request body rules** → **+ Add rule**, add three:

| Path | Mode | Value |
|------|------|----|
| `temperature` | `overwrite` | `0.3` |
| `max_tokens` | `overwrite` | `4096` |
| `response_format.type` | `overwrite` | `json_schema` |

**Append a system message (if absent)** — Console → request body rules → **+ Add rule**, fill in:

- **Mode** select `append-if-missing`.
- **Path** fill `messages`.
- **Value** fill `{"role": "system", "content": "You are a helpful assistant."}`.
- **Condition**: scope fill `messages`; add a where — path `role`, equal `system` (i.e. "skip when a role=system element already exists in messages, otherwise append").

**Append a thinking block (if absent)** — Console → request body rules → **+ Add rule**, fill in:

- **Mode** select `append-if-missing`.
- **Path** fill `messages`.
- **Value** fill `{"type": "thinking", "thinking": {"budget_tokens": 10000}}`.
- **Condition**: scope fill `messages`; add a where — path `type`, equal `thinking`.

**Remove a thinking block** — Console → request body rules → **+ Add rule**, fill in:

- **Mode** select `remove-matching`.
- **Path** fill `messages`.
- **Value** leave empty.
- **Condition**: scope fill `messages`; add a where — path `type`, equal `thinking` (i.e. "delete elements with type=thinking from messages").

**Detect image/web-search → switch to a multimodal model (`switch-route`, zero script overhead):**

Does not read/write the body; evaluates `when` against the **client** shape **before** protocol translation, and when a predicate holds, re-dispatches this request to `use_model` (immediate switch before translation, **supports cross-protocol**). Predicates only do structural/small-value checks and don't read binary content, so a large request body containing base64 stays on disk as placeholder + temp file rather than being read whole into the script engine's memory and OOMing like an after-slot script. This is the native replacement for "detect image → switch model" scripts.

**How to configure in the console (no JSON needed):** model detail (or the upstream's "default request body rules") → **Request body rules** → **+ Add rule** → in the mode dropdown select **`switch-route`** → the row expands two controls: in "**Switch to model (use_model)**" select the target model (same-protocol or cross-protocol); in "**Hit condition when**" add predicates one by one — each fills a **field path** (written in the **client** protocol shape) and an **equal**; leaving equal empty means "hit when the field exists", filling a value means "hit when values are equal", and multiple predicates are OR-ed.

Example: select `qwen-vl-max` in use_model, add three predicates —

- path `messages.*.content.*.image_url`, leave "equal" empty (OpenAI Chat client image block, existence)
- path `messages.*.content.*.type`, fill "equal" with `image` (Anthropic client image block)
- path `tools.*.type`, fill "equal" with `web_search_20250305` (web-search tool)

> ⚠️ An Anthropic client's image block may also be nested one level inside a `tool_result` block (`messages.*.content.*.content.*.type` equals `image`); configuring only top-level paths misses images in tool history. Full config steps and per-client-protocol condition paths are in [Route image-bearing requests to a vision model](/en/howto/route-image-requests-to-vision-model.md).

`when` is OR semantics: omitting `eq` is a field-existence check, giving `eq` is an equality check; `*` wildcards take existence semantics. `switch-route` evaluates against the **client** shape, so paths must be written per the caller's protocol — the example above lists both OpenAI Chat (`messages.*.content.*.image_url`) and Anthropic (`messages.*.content.*.type` equals `"image"`) client shapes, OR-ed, either hit switches; to support more client protocols, add the corresponding shapes in `when` (DashScope clients are `input.messages.*.content.*.image`). `use_model` can be same-protocol or cross-protocol with the current route (the immediate switch before translation re-translates the body). The switched-to target is treated as an **admin-specified internal route**: the gateway does **not** re-validate the caller's group model ACL for the `use_model` target (the model ACL is checked only once at request entry against the client's original request model). The full retirement case is in [Configuration reference](/en/reference/configuration.md).

### Writing dynamic values with expressions (cache key)

By default, rules write fixed values into `value`. If you want the value to vary with the request — e.g. injecting "same key for the same session" into an upstream's private cache field — write `value` as an expression: string values containing `{{...}}` are resolved as templates, with `|` connecting a fallback chain that takes the first non-empty result. For example "session header first, system hash as fallback":

Console → model detail → request body rules → **+ Add rule**, fill in: mode `overwrite`, path `prompt_cache_key`, value `{{header:x-claude-code-session-id|system_prompt_hash}}`.

Resolution: ① the request header `x-claude-code-session-id` exists and is non-empty → use the header value (multiple requests of the same session carry the same header value, getting the same cache key and hitting the upstream cache); ② the header is missing or empty → fall to `system_prompt_hash`, i.e. the sha256 hash of the system prompt content (the same system prompt yields the same hash, also hitting the cache; protocol-agnostic, compatible with Anthropic `system`, OpenAI `messages[role=system]`, DashScope `input.messages[role=system]`, Google `system_instruction`); ③ header missing and no system prompt → `system_prompt_hash` degrades to the fixed `sha256("")` value; if you want to not inject at all in this case, don't put the hash in the chain — when the whole chain is empty, this write is omitted. Header names are case-insensitive; leading/trailing whitespace in the header value is treated as missing. Other variables available in this context: `request_id`, `request_access_key_name`, `request_access_key_group` (`access_key_hash` is empty here; use `system_prompt_hash` for cache-key fallback). Strings without `{{` are literals, non-string values are written as-is, and old rules are unaffected. This rule is usually **attached to the model** (which model needs a cache key is a model-level policy); rules apply after protocol translation, so the written upstream-private field won't be dropped by translation. In the console form, write the expression directly in the rule row's "value" input box (the placeholder hints the syntax). The full variable list and per-config-scope availability are in [Template variable reference](/en/reference/template-variables.md).

### Filter content types

Remove content blocks of specified types (e.g. `image`, `video`) from a message content array. Console → request body rules → **+ Add rule**, fill in: mode `filter-content-types`, path `messages`, value `image`.

**Remove multiple types from all known paths (auto-expand)** — Console → request body rules → **+ Add rule**, fill in: mode `filter-content-types`, path empty (i.e. all known content paths), value fill `image`, `video` (an array).

When image blocks are removed, a placeholder text is auto-injected to tell the model the user attached an image; customize it via the `hint` field directly in the database.

### Model-level rules

Model-level rules have higher priority than upstream-level. Upstream rules apply first, then model rules overwrite or extend. Console → Model → New/Edit `qwen3.6-plus-strict`, **Upstream** select `openai`, add two rules in **Request body rules** (model-level):

| Path | Mode | Value |
|------|------|----|
| `temperature` | `overwrite` | `0` |
| `response_format.type` | `overwrite` | `json_schema` |

## Model-level base_url override

Set `base_url` on a model to override the upstream's default. Console → Model → New/Edit `custom-model`, fill in: **Upstream** `openai`, **Upstream model ID** `qwen3.6-plus`, **Base URL override** `https://custom-proxy.example.com/v1`.

Credentials (API key) still come from the upstream config; only the request's target endpoint changes. Use it to route a specific model to a separate proxy, a vLLM instance, or a custom endpoint.

### direct_path mode

When `direct_path = true` is set, the model's `base_url` is used as the **full endpoint URL**, with no path appended. Console → Model → New/Edit `my-model`, fill in:

- **Upstream** select `azure-openai`.
- **Protocol override** select `openai_images`.
- Check **direct_path**.
- **Base URL override** fill `https://{resource}.openai.azure.com/openai/deployments/my-model/images/generations?api-version=2025-04-01-preview`.

## Model-level protocol override

A model can override the upstream's protocol via the `protocol` field, letting one set of credentials serve models of different API formats. Console → Model → New/Edit `glm-5`, fill in: **Upstream** `aws-global`, **Upstream model ID** `zai.glm-5`, **Protocol override** `openai`, **Base URL override** `https://bedrock-runtime.us-west-2.amazonaws.com/openai/v1/`.

## Upstream Inference Profile

`upstream_inference_profile` overrides the model ID sent to the upstream, sourced from AWS Bedrock's application inference configuration, usable as a generic model-ID override. Console → Model → New/Edit `glm-5`, fill in: **Upstream** `aws-global`, **Upstream model ID** `zai.glm-5`, **inference_profile** `arn:aws:.../abcd1234`.

Once set, this value is used as the upstream model ID in all requests, with `upstream_model_id` as the fallback.

## DashScope model configuration

DashScope models of different types have different endpoints, selected via the `kind` field:

| kind | Endpoint | Applicable models |
|------|------|---------|
| (default) | `/api/v1/services/aigc/text-generation/generation` | qwen-turbo, qwen-plus, qwen-max |
| `dashscope-multimodal` | `/api/v1/services/aigc/multimodal-generation/generation` | qwen-vl-chat-v1, qwen-vl-plus, qwen-audio, fun-asr-flash |
| `dashscope-text-embedding` | `/api/v1/services/embeddings/text-embedding/text-embedding` | text-embedding-v4 |
| `dashscope-rerank` | Text rerank endpoint | gte-rerank |
| `dashscope-audio-asr` | Async speech ASR | fun-asr |

### Text generation

Console → Model → New/Edit `qwen-turbo`, fill in: **Upstream** `dashscope`, **Upstream model ID** `qwen-turbo`, **Protocol override** `dashscope`.

### Video generation (direct_path)

Async models like video generation need `direct_path = true`. Console → Model → New/Edit `wanx2.1-t2v`, fill in:

- **Upstream** select `dashscope`.
- **Upstream model ID** fill `wanx2.1-t2v`.
- **Protocol override** select `dashscope`.
- Check **direct_path**.
- **Base URL override** fill `https://dashscope.aliyuncs.com/api/v1/services/aigc/video-generation/video-synthesis`.

Async models also need `extra_config` set to: `{"dashscope": {"async_mode": true}}`

### Text embeddings (kind selects endpoint)

Text-embedding models use the Bailian MaaS domain and need a separate upstream. Console → Model → New/Edit `text-embedding-v4`, fill in: **Upstream** `dashscope-maas`, **Upstream model ID** `text-embedding-v4`, **Protocol override** `dashscope`, **Type kind** `dashscope-text-embedding`.

Clients send requests with the standard OpenAI `/v1/embeddings` protocol; the gateway translates them into DashScope format automatically.

### Rerank

Two paths:

**DashScope translation** (Bailian models) — Console → Model → New/Edit `gte-rerank`, fill in: **Upstream** `dashscope-maas`, **Upstream model ID** `gte-rerank`, **Protocol override** `dashscope`, **Type kind** `dashscope-rerank`.

**OpenAI-compatible upstream** (vLLM/Cohere/Jina) — Console → Model → New/Edit `bge-reranker-v2-m3`, fill in: **Upstream** `vllm-rerank`, **Upstream model ID** `BAAI/bge-reranker-v2-m3`, **Protocol override** `openai_rerank`.

### Speech transcription (OpenAI compatible)

Synchronous audio models like `qwen-audio` and `fun-asr-flash` are configured with `kind = "dashscope-multimodal"`, supporting the OpenAI `/v1/audio/transcriptions` interface. Console → Model → New/Edit `qwen-audio`, fill in: **Upstream** `dashscope`, **Upstream model ID** `qwen-audio`, **Protocol override** `dashscope`, **Type kind** `dashscope-multimodal`.

Clients upload audio via standard OpenAI multipart; the gateway translates to DashScope format automatically.

### Async speech ASR

Async ASR needs `kind = "dashscope-audio-asr"` + `extra_config.dashscope.async_mode = true`. Console → Model → New/Edit `fun-asr`, fill in: **Upstream** `dashscope-maas`, **Upstream model ID** `fun-asr`, **Protocol override** `dashscope`, **Type kind** `dashscope-audio-asr`, **extra_config** fill `{"dashscope": {"async_mode": true}}`.

Clients submit to `POST /v1/services/{*rest}` and poll `GET /v1/services/{model}/tasks/{task_id}`.

## OpenAI Images model configuration

Configure image generation/editing via the `openai_images` protocol + `direct_path = true`. Console → Model → New/Edit `dall-e-3`, fill in:

- **Upstream** select `openai-images`.
- **Upstream model ID** fill `dall-e-3`.
- **Protocol override** select `openai_images`.
- Check **direct_path**.
- **Base URL override** fill `https://api.openai.com/v1/images/generations`.

Gemini image models use `protocol = "google"` (not `openai_images`); the gateway translates automatically. Console → Model → New/Edit `gemini-3-pro-image-preview`, fill in: **Upstream** `google-gemini`, **Upstream model ID** `gemini-3-pro-image-preview`, **Protocol override** `google`.

## Model script configuration

Each upstream and model can have JavaScript scripts to custom-transform request/response bodies:

| Field | Description | Default |
|------|------|--------|
| `request_transform_before` | Pre-translation request-body script | none |
| `request_transform_after` | Post-translation request-body script | none |
| `response_transform` | Response-body script | none |
| `script_error_mode` | `log-and-continue` / `log-and-reject` | `log-and-continue` |

**Inline script** — Console → model detail (or upstream) → **Request/response script** → paste into the **post-translation request body** (`request_transform_after`) slot:

```javascript
function transform(body, context) { body.model = context.upstreamModel; return body; }
```

**File script** — the console's script editor only accepts inline content. Referencing an external `.js` file (e.g. `{"file": "scripts/inject-thinking.js"}`) can only be set via the Console API; the UI preserves an already-set file reference as-is until you type inline content into that slot to override it. See [Write your first script transform](/en/howto/write-script-transform.md).

**Script levels**:

| Level | Scope | Use case |
|------|---------|---------|
| Upstream | All models passing through this upstream | Protocol adaptation, common field injection |
| Model | This model only | Model-specific adjustments, parameter tuning |

Model-level scripts override upstream-level. Full syntax, built-in functions, and execution order are in [Scripting API reference](/en/reference/scripting-api.md).

**Common script example — inject a system prompt:**

```javascript
function transform(body, context) {
    if (body.messages.length === 0 || body.messages[0].role !== "system") {
        const systemMsg = { role: "system", content: "You are a helpful assistant." };
        body.messages = [systemMsg, ...body.messages];
    }
    return body;
}
```

**Conditional logic by access key group:**

```javascript
function transform(body, context) {
    if (context.accessKeyGroup === "premium") {
        body.max_tokens = 8192;
    } else {
        body.max_tokens = 4096;
    }
    return body;
}
```

## FAQ

**Q: After setting hide_name on a model, calling the primary name returns 404?**
Yes, by design. A `hide_name=true` model can only be accessed by alias or via a load balancer; the primary name doesn't appear in the `/v1/models` list.

**Q: I want one model configured across multiple upstreams (failover).**
Use a load balancer, see [Load-balancing fields](/en/reference/load-balancing-fields.md). Add the multiple upstream+model pairs as entries.

**Q: The upstream key is about to expire — how do I rotate it?**
Edit the upstream → change the API Key (rotation); the old value becomes invalid immediately. Or add a new key and delete the old one.

**Q: How do I configure a DashScope model?**
Set the protocol to `dashscope` and the base URL to the DashScope address. Text-generation models don't need `direct_path`; async models like video/image need `direct_path = true` + full URL + `extra_config.dashscope.async_mode = true`. DashScope clients can use `/v1/services/{*rest}` (passthrough) or `/v1/chat/completions` (translation mode).

**Q: request_payload vs script (request_transform_before/after) — how to choose?**
Use `request_payload` for simple field add/remove/modify (declarative rules, rewritten directly inside the gateway without the script engine's serialization, best performance); use scripts for complex logic, conditional branches, cross-field computation. For **pure routing decisions** like "detect image/web-search → switch model", use `request_payload`'s `switch-route` (zero script overhead, predicates don't read binaries, large bodies aren't read into memory, same-protocol or cross-protocol — it evaluates against the client shape before translation and switches immediately; the script path no longer fully materializes the body due to lazy projection, but pure routing decisions don't need a script); use scripts only when you need complex judgments like cross-field computation. See [Scripting API reference](/en/reference/scripting-api.md).

**Q: What if model-level and upstream-level request_payload conflict?**
Upstream rules apply first, then model rules. Model rules can override or extend the upstream's same-path rules.

**Next**: this chapter suffices when one upstream is enough; for multi-upstream failover/load balancing see [Load-balancing fields](/en/reference/load-balancing-fields.md); for script transforms see [Scripting API reference](/en/reference/scripting-api.md); for template variables see [Template variable reference](/en/reference/template-variables.md).