Upstream and model fields

This chapter covers how to configure upstream services (model providers) and models in the console. This is the prerequisite for the gateway to forward requests.

Entry: Console → Upstream services (admin only). Each upstream has several models under it.

Upstream service management

Create an upstream

Console → Upstream services → New, fill in the following fields:

Field	Description	Default	Example
Name	Unique upstream identifier; models reference it	(required)	openai
Protocol	Determines the executor and request schema (15 in total)	(required)	openai / openai_response / openai_images / openai_embeddings / openai_audio / openai_rerank / openai_realtime / anthropic / google / aws_converse / aws_invoke / aws_codewhisperer_streaming / dashscope / dashscope_realtime / passthrough
Base URL	The upstream's real address, no trailing /	(required)	https://api.openai.com/v1
API Key	Upstream key; multiple can be added; each entry = key (masked on edit) + weight (0 = standby) + remove; deterministically weighted distribution, the same caller sticks to the same key	(required)	sk-...
Enabled	Disabled means it doesn't participate in routing	true	✓
Custom headers	Extra request headers; supports {{var}} and {{alt1|alt2}} fallback chains	empty
User-Agent	Overrides the global UA; supports templates	uses the global UA
Proxy	Upstream outbound proxy	none
DNS	per-upstream DNS (DoH/DoT/UDP auto-detected) + TTL	uses system DNS
Request body rules	Request-body rewrite rules (path/value/mode)	empty
Request/response script	JS script transform (see Scripting API reference)	none
Script error mode	log-and-continue / log-and-reject	log-and-continue
Script position	after (after rules) / before	after
Token counting	upstreamApi / tiktoken	upstreamApi
Rate limit	Upstream-level rate limit: rpm / max_concurrency (concurrency) / tpm / tps_min_interval_ms (minimum interval between adjacent requests, ms) / wait_timeout_secs (wait-for-permit timeout, default 10s) / initial_tokens (initial token-bucket fill percentage 0–100, default 100)	unlimited

Fetch available models (probe)

The "Fetch available models" button in the upstream form makes a real model-list probe to that upstream (e.g. GET /models for an OpenAI-compatible upstream), opening the "Available model probe" panel: it shows the parsed model-ID count (including the upstream's raw return count and source) and the model-ID list (truncated to the first several when too long), with a one-click copy all. Use it to confirm which model IDs the upstream actually offers before creating a model, avoiding typos.

Multi-key and weights

An upstream can have multiple API keys. The selection algorithm is deterministic weighted distribution + sticky binding, not round-robin:

• First selection hashes (caller access_key, upstream name) with FNV-1a, and lands in a cumulative weight interval by each key's weight as a proportion of the total weight. Different callers spread across different keys by weight ratio — a weight=2 key gets twice the traffic of a weight=1 key.
• The same caller access_key sticks to the same key (the binding is stored in an in-memory session-key binding table, memory-only and not shared across nodes — each node maintains its own local prompt-cache stickiness), guaranteeing upstream prompt-cache and signature consistency. The binding has an idle TTL (KEY_BINDING_TTL_SECS, default 1800 seconds / 30 minutes, 0 = permanent): if the caller makes no new request within the TTL it expires and is re-selected next time.
• Config reload no longer wipes all bindings; invalidation is fingerprint-driven: each binding records the fingerprint of the upstream's candidate key set (computed from each key's position and weight). When reading a binding, if the fingerprint doesn't match the current candidate set it's treated as invalid and re-selected. Therefore — rotating a key's value in place (same position, new value) doesn't break stickiness; whereas adding/removing, reordering keys, or changing weights changes the fingerprint and triggers re-selection. This matches the minimum-disruption semantics of industrial gateways (Envoy / AWS ALB).
• weight = 0 is a standby key: only enabled after all weight > 0 keys have been tried and failed, and no binding is stored.
• When a key fails, the gateway rotates within that upstream to the next untried key and retries, but does not override the first sticky binding — subsequent requests from that caller still start from the originally bound key.

This is the multi-key selection mechanism within a single upstream, and it is a separate mechanism from load-balancer entry selection (Smooth WRR smooth weighted round-robin, see Load-balancing fields): LB entry selection happens at the handler layer, across upstreams; within-upstream multi-key selection happens at the dispatch layer, within the same upstream. The two are orthogonal and stack (LB selects an entry → that entry's upstream then selects a key stickily per this section).

Viewing live bindings in the console: upstream list → the Monitor icon on a model row of that upstream → the "Model monitor" panel. The Access Key Bindings tab lists, row by row, each access key that can reach this model via group ACL, and which upstream key it is currently stuck to: source binding means an active sticky binding exists; prediction means there's no active binding right now and the shown upstream key is the deterministic weighted prediction computed from this model's candidate set (different models of the same upstream may predict differently, since each model computes its own candidate set). Bindings whose fingerprint doesn't match the current candidate set are marked stale — still shown, but the next request re-selects (consistent with dispatch truth). The upstream keys roster above acts as a legend; hovering a binding row highlights its corresponding upstream key. This view is single-node (in both storage modes the binding table is in memory and not shared across nodes), and the panel bottom carries the instance identifier. The panel auto-refreshes every 3 seconds by default (can be turned off). The rate-limit gauge is on the Rate Limit tab; when the model has no rate limit configured that tab shows an empty state, which doesn't affect the binding view. The full selection/switch semantics are as described above (deterministic weighted hash + fingerprint invalidation + idle TTL).

Delete an upstream

If the upstream still has models referencing it, deletion returns 422 (delete or migrate the models first, then delete the upstream).

Custom headers and User-Agent

Attach custom headers to all the upstream's requests, supporting template variables. The full variable list and per-config-scope availability are in Template variable reference; variables available in this context:

Variable	Source
{{request_access_key_name}}	The matched access key name
{{request_access_key_group}}	The access key group name
{{request_id}}	The client's z-request-id header
{{access_key_hash}}	First 16 bytes of the access-key-name SHA-256 hash, base64
{{system_prompt_hash}}	SHA-256 of the system prompt content (protocol-agnostic)
{{header:Key}}	The value of a client request header (case-insensitive)

Fallback-chain syntax: {{var1|var2}} takes the first available variable in order; if all are empty, that header is omitted (an empty string is not sent).

Example — include the access key name in the User-Agent for upstream log correlation (Console → Upstream services → edit that upstream):

1. Set the User-Agent field to gatellm/key-{{request_access_key_name}}.
2. Add two entries in the Custom headers area:
   • Header name X-Request-Access-Key, value {{request_access_key_name}}.
   • Header name X-Request-Group, value {{request_access_key_group}}.

Per-upstream DNS resolver

Each upstream can have its own DNS servers, bypassing container/OS-level DNS caching, to solve persistent 503/504 caused by stale cached IPs after an upstream ALB IP rotation:

Scheme	Protocol	Port
(none)	UDP standard DNS	53
udp://	UDP + TCP fallback	53
tls://	DNS-over-TLS	853
https://	DNS-over-HTTPS	443

In Console → Upstream services → edit that upstream → the DNS field, enter a server list (scheme determines protocol, see the table above):

• Use Alibaba DNS (UDP): 223.5.5.5, 223.6.6.6.
• Use Cloudflare DNS-over-TLS: tls://1.1.1.1, tls://1.0.0.1.

Model management

Click expand on an upstream row in the upstream list to see that upstream's model subtable (columns: Name / Upstream model ID / Protocol / Aliases / Hidden primary name). Each model has 5 actions: Test model, Model monitor, Copy config (quickly create a new one using this model's config as a template), Edit, Delete.

Create a model

Field	Description	Default	Example
Name	The model name exposed to clients, customizable	(required)	gpt-4o
Upstream	The owning upstream	(required)	openai
Upstream model ID	The real model name sent to the upstream	(required)	gpt-4o
Aliases	Extra usable names (share a globally unique space with the model name)	empty	gpt4o
Hide primary name	When hide_name=true, the primary name is inaccessible (404); only aliases or load-balancer access work	false
Anthropic Signature	Preserve reasoning context across multi-turn conversations (signature passthrough of thinking blocks)	off
Protocol override	If empty, uses the upstream protocol; if set, this model uses a different protocol	inherit upstream
Base URL override	If empty, uses the upstream base_url; with direct_path this value is treated as the full endpoint URL	inherit upstream
Type kind	Variant identifier for protocols like DashScope	none
inference_profile	Bedrock's inference configuration ARN	none
Request body rules	Model-level request-body rewrite rules (override upstream-level)	empty
Request/response script	Model-level script (overrides upstream-level)	none
Rate limit override	inherit (inherit upstream) or concrete values of the same shape as the upstream (fields in the upstream "Rate limit" row)	inherit
Token counting override	Overrides upstream-level	inherit upstream
Enabled		true	✓

Name vs upstream model ID: the client calls the "Name"; the gateway sends the "upstream model ID" to the upstream. When they differ you get model aliasing/renaming. aliases are extra usable names.

Test model

Model list → that model → Test model, opens the test drawer to make one real call verifying the chain (POST /console/api/models/test):

• Client protocol: choose openai / anthropic / openai_response / google; after choosing, the drawer shows the "→ upstream protocol" translation mapping in real time — used to verify cross-protocol translation (e.g. Anthropic client protocol calling an OpenAI upstream).
• Request body: a JSON editor pre-filled by the chosen client protocol, freely editable before clicking Send test, returning the real upstream response (success / failure and error details).
• realtime protocol (openai_realtime / dashscope_realtime) models have a separate realtime session test: it opens a WebSocket session to the resolved upstream for end-to-end verification (start session / end session, showing handshake and session-establishment status).

Model aliases and hidden names

Aliases

Define multiple accessible names for a model. Console → Model → New/Edit gpt-4o-2024-05-13, fill in:

• Upstream select openai.
• Upstream model ID fill gpt-4o-2024-05-13.
• Aliases fill gpt-4o, gpt4o (two, sharing the globally unique space with the primary name).

Clients can reach the same model with either gpt-4o or gpt4o.

Note: a key group's model ACL only matches the canonical name (the name field), not aliases. If the model's canonical name is gpt-4o-2024-05-13, the key group's model list must list gpt-4o-2024-05-13, not an alias.

Hide primary name (hide_name)

When hide_name = true is set, clients cannot access the model by its canonical name (404) — only by alias. Console → Model → New/Edit claude-sonnet-4-5, fill in:

• Upstream select anthropic.
• Upstream model ID fill claude-sonnet-4-5.
• Aliases fill writing-assistant (a custom public-facing name, fictitious in this example).
• Check Hide primary name (hide_name).

Effects:

• Clients can only access it via writing-assistant
• The /v1/models list only lists the alias, not claude-sonnet-4-5
• Load balancers can reference claude-sonnet-4-5 (a hidden model can be an LB entry)
• Statistics, logs, and credential filtering still use the canonical name

Use cases:

• White-label: expose a model under a brand name, hiding the real upstream model ID
• Migration: expose a stable alias, change the underlying model name
• LB-only: hide the underlying model, expose it only through an LB name

When hide_name = true, aliases may be empty. The model is then invisible to external APIs but can still be referenced by a load balancer.

Request body modification (Request Payload)

Modify the JSON request body before it's sent to the upstream. Rules apply after protocol translation, affecting the final upstream request body.

Rule format

Each rule contains:

Field	Type	Description
path	String	Dot-separated JSON path (e.g. temperature, messages.0.content)
value	JSON	The value to set, append, or remove
mode	String	Required, see the mode table below
condition	Object	Conditions required by some modes

Path syntax

Syntax	Example	Description
Object key	temperature	Top-level field
Nested key	response_format.type	Drill into nested objects
Array index	messages.0.content	Access the element at index 0
Wildcard	messages.*.content	Match all elements of the array

⚠️ messages[].content and messages[*].content are illegal. Use messages.*.content for wildcard and messages.0.content for a specific index.

Mode overview

Mode	Description	Example
overwrite	Replace the value at the target path	{"path": "temperature", "value": 0.7, "mode": "overwrite"}
remove	Delete the key at the target path	{"path": "temperature", "value": null, "mode": "remove"}
add-if-absent	Set the value only when the key doesn't exist	{"path": "top_p", "value": 0.9, "mode": "add-if-absent"}
append	Unconditionally append to an array	{"path": "tools", "value": {...}, "mode": "append"}
append-if-missing	Conditional append (see below)
remove-matching	Conditionally remove array elements (see below)
strip-lines	Remove matching lines from a string
filter-content-types	Remove blocks of specified types from a content array
filter-tools	Filter a tools array
inject-system-prompt	Inject a system prompt at the protocol-appropriate position (value is the prompt text)	{"value": "Be concise.", "mode": "inject-system-prompt"}
switch-route	Does not modify the body; evaluates when against the client shape before protocol translation, and when any predicate holds, re-dispatches this request to use_model (immediate switch, supports cross-protocol). Zero script overhead; predicates don't read binaries, large-body disk spooling stays on	see below

Common examples

Simple parameter overwrite — Console → model detail (or the upstream's "default request body rules") → Request body rules → + Add rule, add three:

Path	Mode	Value
temperature	overwrite	0.3
max_tokens	overwrite	4096
response_format.type	overwrite	json_schema

Append a system message (if absent) — Console → request body rules → + Add rule, fill in:

• Mode select append-if-missing.
• Path fill messages.
• Value fill {"role": "system", "content": "You are a helpful assistant."}.
• Condition: scope fill messages; add a where — path role, equal system (i.e. "skip when a role=system element already exists in messages, otherwise append").

Append a thinking block (if absent) — Console → request body rules → + Add rule, fill in:

• Mode select append-if-missing.
• Path fill messages.
• Value fill {"type": "thinking", "thinking": {"budget_tokens": 10000}}.
• Condition: scope fill messages; add a where — path type, equal thinking.

Remove a thinking block — Console → request body rules → + Add rule, fill in:

• Mode select remove-matching.
• Path fill messages.
• Value leave empty.
• Condition: scope fill messages; add a where — path type, equal thinking (i.e. "delete elements with type=thinking from messages").

Detect image/web-search → switch to a multimodal model (switch-route, zero script overhead):

Does not read/write the body; evaluates when against the client shape before protocol translation, and when a predicate holds, re-dispatches this request to use_model (immediate switch before translation, supports cross-protocol). Predicates only do structural/small-value checks and don't read binary content, so a large request body containing base64 stays on disk as placeholder + temp file rather than being read whole into the script engine's memory and OOMing like an after-slot script. This is the native replacement for "detect image → switch model" scripts.

How to configure in the console (no JSON needed): model detail (or the upstream's "default request body rules") → Request body rules → + Add rule → in the mode dropdown select switch-route → the row expands two controls: in "Switch to model (use_model)" select the target model (same-protocol or cross-protocol); in "Hit condition when" add predicates one by one — each fills a field path (written in the client protocol shape) and an equal; leaving equal empty means "hit when the field exists", filling a value means "hit when values are equal", and multiple predicates are OR-ed.

Example: select qwen-vl-max in use_model, add three predicates —

• path messages.*.content.*.image_url, leave "equal" empty (OpenAI Chat client image block, existence)
• path messages.*.content.*.type, fill "equal" with image (Anthropic client image block)
• path tools.*.type, fill "equal" with web_search_20250305 (web-search tool)

⚠️ An Anthropic client's image block may also be nested one level inside a tool_result block (messages.*.content.*.content.*.type equals image); configuring only top-level paths misses images in tool history. Full config steps and per-client-protocol condition paths are in Route image-bearing requests to a vision model.

when is OR semantics: omitting eq is a field-existence check, giving eq is an equality check; * wildcards take existence semantics. switch-route evaluates against the client shape, so paths must be written per the caller's protocol — the example above lists both OpenAI Chat (messages.*.content.*.image_url) and Anthropic (messages.*.content.*.type equals "image") client shapes, OR-ed, either hit switches; to support more client protocols, add the corresponding shapes in when (DashScope clients are input.messages.*.content.*.image). use_model can be same-protocol or cross-protocol with the current route (the immediate switch before translation re-translates the body). The switched-to target is treated as an admin-specified internal route: the gateway does not re-validate the caller's group model ACL for the use_model target (the model ACL is checked only once at request entry against the client's original request model). The full retirement case is in Configuration reference.

Writing dynamic values with expressions (cache key)

By default, rules write fixed values into value. If you want the value to vary with the request — e.g. injecting "same key for the same session" into an upstream's private cache field — write value as an expression: string values containing {{...}} are resolved as templates, with | connecting a fallback chain that takes the first non-empty result. For example "session header first, system hash as fallback":

Console → model detail → request body rules → + Add rule, fill in: mode overwrite, path prompt_cache_key, value {{header:x-claude-code-session-id|system_prompt_hash}}.

Resolution: ① the request header x-claude-code-session-id exists and is non-empty → use the header value (multiple requests of the same session carry the same header value, getting the same cache key and hitting the upstream cache); ② the header is missing or empty → fall to system_prompt_hash, i.e. the sha256 hash of the system prompt content (the same system prompt yields the same hash, also hitting the cache; protocol-agnostic, compatible with Anthropic system, OpenAI messages[role=system], DashScope input.messages[role=system], Google system_instruction); ③ header missing and no system prompt → system_prompt_hash degrades to the fixed sha256("") value; if you want to not inject at all in this case, don't put the hash in the chain — when the whole chain is empty, this write is omitted. Header names are case-insensitive; leading/trailing whitespace in the header value is treated as missing. Other variables available in this context: request_id, request_access_key_name, request_access_key_group (access_key_hash is empty here; use system_prompt_hash for cache-key fallback). Strings without {{ are literals, non-string values are written as-is, and old rules are unaffected. This rule is usually attached to the model (which model needs a cache key is a model-level policy); rules apply after protocol translation, so the written upstream-private field won't be dropped by translation. In the console form, write the expression directly in the rule row's "value" input box (the placeholder hints the syntax). The full variable list and per-config-scope availability are in Template variable reference.

Filter content types

Remove content blocks of specified types (e.g. image, video) from a message content array. Console → request body rules → + Add rule, fill in: mode filter-content-types, path messages, value image.

Remove multiple types from all known paths (auto-expand) — Console → request body rules → + Add rule, fill in: mode filter-content-types, path empty (i.e. all known content paths), value fill image, video (an array).

When image blocks are removed, a placeholder text is auto-injected to tell the model the user attached an image; customize it via the hint field directly in the database.

Model-level rules

Model-level rules have higher priority than upstream-level. Upstream rules apply first, then model rules overwrite or extend. Console → Model → New/Edit qwen3.6-plus-strict, Upstream select openai, add two rules in Request body rules (model-level):

Path	Mode	Value
temperature	overwrite	0
response_format.type	overwrite	json_schema

Model-level base_url override

Set base_url on a model to override the upstream's default. Console → Model → New/Edit custom-model, fill in: Upstream openai, Upstream model ID qwen3.6-plus, Base URL override https://custom-proxy.example.com/v1.

Credentials (API key) still come from the upstream config; only the request's target endpoint changes. Use it to route a specific model to a separate proxy, a vLLM instance, or a custom endpoint.

direct_path mode

When direct_path = true is set, the model's base_url is used as the full endpoint URL, with no path appended. Console → Model → New/Edit my-model, fill in:

• Upstream select azure-openai.
• Protocol override select openai_images.
• Check direct_path.
• Base URL override fill https://{resource}.openai.azure.com/openai/deployments/my-model/images/generations?api-version=2025-04-01-preview.

Model-level protocol override

A model can override the upstream's protocol via the protocol field, letting one set of credentials serve models of different API formats. Console → Model → New/Edit glm-5, fill in: Upstream aws-global, Upstream model ID zai.glm-5, Protocol override openai, Base URL override https://bedrock-runtime.us-west-2.amazonaws.com/openai/v1/.

Upstream Inference Profile

upstream_inference_profile overrides the model ID sent to the upstream, sourced from AWS Bedrock's application inference configuration, usable as a generic model-ID override. Console → Model → New/Edit glm-5, fill in: Upstream aws-global, Upstream model ID zai.glm-5, inference_profile arn:aws:.../abcd1234.

Once set, this value is used as the upstream model ID in all requests, with upstream_model_id as the fallback.

DashScope model configuration

DashScope models of different types have different endpoints, selected via the kind field:

kind	Endpoint	Applicable models
(default)	/api/v1/services/aigc/text-generation/generation	qwen-turbo, qwen-plus, qwen-max
dashscope-multimodal	/api/v1/services/aigc/multimodal-generation/generation	qwen-vl-chat-v1, qwen-vl-plus, qwen-audio, fun-asr-flash
dashscope-text-embedding	/api/v1/services/embeddings/text-embedding/text-embedding	text-embedding-v4
dashscope-rerank	Text rerank endpoint	gte-rerank
dashscope-audio-asr	Async speech ASR	fun-asr

Text generation

Console → Model → New/Edit qwen-turbo, fill in: Upstream dashscope, Upstream model ID qwen-turbo, Protocol override dashscope.

Video generation (direct_path)

Async models like video generation need direct_path = true. Console → Model → New/Edit wanx2.1-t2v, fill in:

• Upstream select dashscope.
• Upstream model ID fill wanx2.1-t2v.
• Protocol override select dashscope.
• Check direct_path.
• Base URL override fill https://dashscope.aliyuncs.com/api/v1/services/aigc/video-generation/video-synthesis.

Async models also need extra_config set to: {"dashscope": {"async_mode": true}}

Text embeddings (kind selects endpoint)

Text-embedding models use the Bailian MaaS domain and need a separate upstream. Console → Model → New/Edit text-embedding-v4, fill in: Upstream dashscope-maas, Upstream model ID text-embedding-v4, Protocol override dashscope, Type kind dashscope-text-embedding.

Clients send requests with the standard OpenAI /v1/embeddings protocol; the gateway translates them into DashScope format automatically.

Rerank

Two paths:

DashScope translation (Bailian models) — Console → Model → New/Edit gte-rerank, fill in: Upstream dashscope-maas, Upstream model ID gte-rerank, Protocol override dashscope, Type kind dashscope-rerank.

OpenAI-compatible upstream (vLLM/Cohere/Jina) — Console → Model → New/Edit bge-reranker-v2-m3, fill in: Upstream vllm-rerank, Upstream model ID BAAI/bge-reranker-v2-m3, Protocol override openai_rerank.

Speech transcription (OpenAI compatible)

Synchronous audio models like qwen-audio and fun-asr-flash are configured with kind = "dashscope-multimodal", supporting the OpenAI /v1/audio/transcriptions interface. Console → Model → New/Edit qwen-audio, fill in: Upstream dashscope, Upstream model ID qwen-audio, Protocol override dashscope, Type kind dashscope-multimodal.

Clients upload audio via standard OpenAI multipart; the gateway translates to DashScope format automatically.

Async speech ASR

Async ASR needs kind = "dashscope-audio-asr" + extra_config.dashscope.async_mode = true. Console → Model → New/Edit fun-asr, fill in: Upstream dashscope-maas, Upstream model ID fun-asr, Protocol override dashscope, Type kind dashscope-audio-asr, extra_config fill {"dashscope": {"async_mode": true}}.

Clients submit to POST /v1/services/{*rest} and poll GET /v1/services/{model}/tasks/{task_id}.

OpenAI Images model configuration

Configure image generation/editing via the openai_images protocol + direct_path = true. Console → Model → New/Edit dall-e-3, fill in:

• Upstream select openai-images.
• Upstream model ID fill dall-e-3.
• Protocol override select openai_images.
• Check direct_path.
• Base URL override fill https://api.openai.com/v1/images/generations.

Gemini image models use protocol = "google" (not openai_images); the gateway translates automatically. Console → Model → New/Edit gemini-3-pro-image-preview, fill in: Upstream google-gemini, Upstream model ID gemini-3-pro-image-preview, Protocol override google.

Model script configuration

Each upstream and model can have JavaScript scripts to custom-transform request/response bodies:

Field	Description	Default
request_transform_before	Pre-translation request-body script	none
request_transform_after	Post-translation request-body script	none
response_transform	Response-body script	none
script_error_mode	log-and-continue / log-and-reject	log-and-continue

Inline script — Console → model detail (or upstream) → Request/response script → paste into the post-translation request body (request_transform_after) slot:

function transform(body, context) { body.model = context.upstreamModel; return body; }

File script — the console's script editor only accepts inline content. Referencing an external .js file (e.g. {"file": "scripts/inject-thinking.js"}) can only be set via the Console API; the UI preserves an already-set file reference as-is until you type inline content into that slot to override it. See Write your first script transform.

Script levels:

Level	Scope	Use case
Upstream	All models passing through this upstream	Protocol adaptation, common field injection
Model	This model only	Model-specific adjustments, parameter tuning

Model-level scripts override upstream-level. Full syntax, built-in functions, and execution order are in Scripting API reference.

Common script example — inject a system prompt:

function transform(body, context) {
    if (body.messages.length === 0 || body.messages[0].role !== "system") {
        const systemMsg = { role: "system", content: "You are a helpful assistant." };
        body.messages = [systemMsg, ...body.messages];
    }
    return body;
}

Conditional logic by access key group:

function transform(body, context) {
    if (context.accessKeyGroup === "premium") {
        body.max_tokens = 8192;
    } else {
        body.max_tokens = 4096;
    }
    return body;
}

FAQ

Q: After setting hide_name on a model, calling the primary name returns 404? Yes, by design. A hide_name=true model can only be accessed by alias or via a load balancer; the primary name doesn't appear in the /v1/models list.

Q: I want one model configured across multiple upstreams (failover). Use a load balancer, see Load-balancing fields. Add the multiple upstream+model pairs as entries.

Q: The upstream key is about to expire — how do I rotate it? Edit the upstream → change the API Key (rotation); the old value becomes invalid immediately. Or add a new key and delete the old one.

Q: How do I configure a DashScope model? Set the protocol to dashscope and the base URL to the DashScope address. Text-generation models don't need direct_path; async models like video/image need direct_path = true + full URL + extra_config.dashscope.async_mode = true. DashScope clients can use /v1/services/{*rest} (passthrough) or /v1/chat/completions (translation mode).

Q: request_payload vs script (request_transform_before/after) — how to choose? Use request_payload for simple field add/remove/modify (declarative rules, rewritten directly inside the gateway without the script engine's serialization, best performance); use scripts for complex logic, conditional branches, cross-field computation. For pure routing decisions like "detect image/web-search → switch model", use request_payload's switch-route (zero script overhead, predicates don't read binaries, large bodies aren't read into memory, same-protocol or cross-protocol — it evaluates against the client shape before translation and switches immediately; the script path no longer fully materializes the body due to lazy projection, but pure routing decisions don't need a script); use scripts only when you need complex judgments like cross-field computation. See Scripting API reference.

Q: What if model-level and upstream-level request_payload conflict? Upstream rules apply first, then model rules. Model rules can override or extend the upstream's same-path rules.

Next: this chapter suffices when one upstream is enough; for multi-upstream failover/load balancing see Load-balancing fields; for script transforms see Scripting API reference; for template variables see Template variable reference.