Enable sticky sessions and static binding

When enabled, the same client's requests keep routing to the same upstream node, suitable for scenarios that need session context (such as stateful conversations) and for reducing cross-node cache invalidation.

Prerequisite: create the LB first by following Create a load balancer.

Enable sticky sessions

Console → Load Balancers → edit the target LB → check "Sticky session" (corresponds to extra_config.sticky_session_enabled, which is true by default; checking it is an explicit confirmation), then fill in:

Field	Value	Description
Sticky TTL	`600` (example, in seconds)	`binding_ttl_secs`, the binding lifetime; `0` = permanent binding
Binding key template	`{{header:X-Session-Id}}`	`binding_key_template`, a custom binding key

json

{
  "extra_config": { "sticky_session_enabled": true },
  "binding_ttl_secs": 600,
  "binding_key_template": "{{header:X-Session-Id}}"
}

ℹ️ sticky_session_enabled is nested under the extra_config object (not a top-level field) and defaults to true. binding_ttl_secs, binding_key_template, and static_bindings are top-level fields. To disable sticky sessions (so every request reselects by weight), set "extra_config": { "sticky_session_enabled": false }.

Save. From now on, requests with the same X-Session-Id keep routing to the same node until the TTL expires or the node becomes unavailable.

Binding key

The binding key determines "which requests count as the same client". The binding key template supports {{...}} expressions and fallback chains; the variables available in this context (full list and available scopes per config location in Template variable reference):

Template variable	Source
`{{request_access_key_name}}`	Access key name
`{{request_access_key_group}}`	Access key group name
`{{request_id}}`	The client's `z-request-id` header
`{{access_key_hash}}`	Hash of the access key (default)
`{{header:X-Session-Id}}`	A client custom request header

ℹ️ Note that {{system_prompt_hash}} is unavailable (always empty) in the binding key context: binding key resolution happens at an earlier stage than request-body availability, so the system prompt cannot be read. Sticky sessions typically use access_key_hash or a session header. When the chain is entirely empty or contains unresolved variables, it falls back to the raw access key as the binding key.

Fallback chain example

json

{
  "binding_key_template": "{{header:X-Session-Id|access_key_hash}}"
}

Step-by-step degradation: has a session header → use the header (same session, same node); no header but authenticated → use the access key hash (same user, same node); both empty → fall back to the raw access key as the binding key (guarantees uniqueness per session, avoiding all requests piling into the same dynamic binding session).

Static binding

Admins preset fixed bindings so that a specific key or key group always routes to a designated node.

Configuration entry point: static_bindings is a top-level field, and the console's LB edit form has no bulk-edit entry for it — configure the full array below through the Console API (PUT load balancer). To manage it in the console UI, open the LB's Monitoring panel, where you can operate on active sessions one by one: pin a dynamic session as a static binding (Pin), change its target node, or delete a static binding.

json

{
  "static_bindings": [
    {
      "binding_key": "vip-user-hash",
      "upstream": "openai-us",
      "model": "gpt-4o",
      "access_key": "alice-key"
    },
    {
      "binding_key": "team-a-hash",
      "upstream": "openai-eu",
      "model": "gpt-4o",
      "access_key_group": "team-a"
    }
  ]
}

binding_key: a manually specified binding key value (not a template; write the actual value directly)
upstream + model: the fixed routing target
access_key / access_key_group: restrict to a specific key or key group (optional)

Useful for pinning a node for VIP users or teams (e.g. team A always goes through the Europe node to meet compliance requirements).

Verifying sticky sessions

Send two requests in a row with the same X-Session-Id and confirm they land on the same upstream node:

bash

for i in 1 2; do
  curl -s http://localhost:7890/v1/chat/completions \
    -H "Authorization: Bearer <your access key>" \
    -H "X-Session-Id: demo-session-1" \
    -H "Content-Type: application/json" \
    -d '{"model":"gpt-4o-ha","messages":[{"role":"user","content":"hi"}]}' > /dev/null
done

Then go to Console → Logs and look at the "Upstream" column for these two requests — they should land on the same upstream node. If they are spread across different nodes, check whether sticky sessions are enabled and whether the binding key template resolves both requests to the same key (in this example the binding key is {{header:X-Session-Id}}).

FAQ

Q: Which variables does the binding key template support?request_access_key_name / request_access_key_group / request_id / access_key_hash / {{header:Key}}. system_prompt_hash is always empty, because the binding key resolution stage has not yet read the request body. See Template variable reference for details.

Q: What happens when the chain is entirely empty? It falls back to the raw access key as the binding key, guaranteeing uniqueness per session and avoiding all requests piling into the same dynamic binding session. A template containing an unresolved {{...}} also falls back the same way.

Q: Can sticky-session bindings be cleared manually? Bindings have an expiration (binding_ttl_secs) and are cleared automatically on expiry. To clear immediately, restart the gateway (in-memory bindings are lost) or wait for the TTL to expire.

Q: What happens when sticky sessions and static bindings conflict? Static binding takes precedence over dynamic binding. A request that matches a static binding goes directly to the static binding's node; only non-matching requests go through dynamic binding.

Q: Does a sticky binding migrate when a node fails? Yes. When the bound node is unavailable, it automatically switches to the next available node; after the node recovers, the next request returns to the originally bound node (if the TTL has not expired).

Next: Create a load balancer for the full LB creation steps; Load-balancing fields for the full field tables and retry configuration; Template variable reference for the full variable list.

Enable sticky sessions and static binding ​