Realtime sessions

Beyond "one-question-one-answer" HTTP traffic, the gateway also supports realtime multimodal sessions: a long-lived, bidirectional, stateful WebSocket connection between the client and the upstream, continuously exchanging audio and events (the typical scenario is voice conversation). This path runs parallel to and independently of the HTTP path.

Entry: GET /v1/realtime (WebSocket upgrade), speaking the OpenAI Realtime client protocol.

Configuring a Realtime model

Configure the upstream + model in the console just like an ordinary model, only choosing a realtime protocol variant:

• Upstream protocol select openai_realtime (or dashscope_realtime), and set the base URL to the upstream's realtime WebSocket endpoint.
• Model attaches to that upstream, fill the upstream model ID.

Clients then connect with an access key to ws(s)://<gateway>/v1/realtime?model=<model-name> to establish a session.

Auth and admission

/v1/realtime uses the same auth as the HTTP data plane: access key (Authorization: Bearer) + Header ACL rules apply as usual. The key group must grant that model for the session to establish.

Session scope (vs HTTP)

A realtime connection is a session, not a single request, so it does not go through the per-request middleware (per-request timeout, concurrency, disk write, per-request statistics). Session-level governance (idle/lifetime limits, per-turn metering) is implemented independently. Key points:

• One bridge task per session: after the gateway connects to the upstream WebSocket, it pumps frames bidirectionally. The current implementation is passthrough — frames are forwarded as-is, no protocol translation; the gateway observes upstream control events out-of-band (session.created / response.create / response.done) to track session state.
• Failover only before session establishment: during the connection-establishment phase it can switch among multiple credential candidates; once the upstream confirms the session (session.created), the upstream holds session state that can't be migrated, and switching nodes is no longer allowed.
• Graceful shutdown: on gateway drain/shutdown, it sends a close frame to each session, giving a sufficient close-handshake time budget.

Billing and logging (per turn)

Within a session, each turn (one response.create → response.done round-trip) is roughly one request: the gateway records it into the same persistent statistics and logging pipeline as HTTP requests.

• Statistics / billing: each turn enters request_stats; the billing page joins pricing by model — realtime usage and HTTP usage are aggregated by model in the same statistics/billing.
• Logs: each turn produces one log event, entering the Logs page and the Live SSE stream, viewable by access key/model.

Therefore realtime cost is accumulated per turn, with the same billing caliber as HTTP (priced by the target model), see Pricing and billing.

FAQ

Q: Can realtime use a load balancer? Realtime routing resolves a model to a single upstream session; multi-node / multi-credential failover manifests as credential-candidate switching in the session-establishment phase, not the HTTP-style per-request node switching.

Q: Will a realtime session be cut off by a request timeout? The HTTP per-request timeout doesn't apply — realtime is session-scoped, governed by session idle/lifetime limits.

Q: Does realtime support protocol translation? The current bridge is passthrough (frames forwarded as-is). The client side speaks the OpenAI Realtime protocol; the upstream must be a same-family realtime protocol (openai_realtime / dashscope_realtime).

Next: Endpoints · auth · protocol interop for the full endpoint list; Protocol interop matrix for the protocol list; Pricing and billing for the billing caliber.