Multi-tenant isolation
The goal of multi-tenant isolation is to let each tenant reach only authorized models, see only its own usage and cost, and be subject to its own rate limits. The gateway implements this with a five-layer combination of "key group + access key + Header ACL + model mapping + rate limiting".
Five dimensions of isolation
| Dimension | Mechanism | Configured at |
|---|---|---|
| Model reachability | The key group's models list | Console → Key Groups |
| Load balancer reachability | The key group's load_balancers list | Console → Key Groups |
| Client identity | Access key + Header ACL rules | Console → Access Keys / Header ACL |
| Model name rewrite | The access key/key group's model_mappings (from→to, can branch by when request-header conditions) | Console → Access Keys / Key Groups |
| Usage constraint | Per-key rate limit (RPM / concurrency / TPM) | RATE_LIMIT_* + Console |
Split models and load balancers with key groups
One key group per tenant; models lists the model names that tenant can call, and load_balancers lists the LB names it can call. The tenant's access keys are attached to the corresponding key group. See Access keys and key groups fields for fields.
LB is an independent authorization dimension
A key group's models and load_balancers are two independent lists. "All models" (*) does not include LB permission, and vice versa — the two dimensions must be checked separately. This is the most common misconfiguration in multi-tenant scenarios. See "Relationship with key groups" in Create a load balancer.
Rewrite model names per tenant (identity model mapping)
To route different tenants requesting the same client model name to different upstreams, use identity-scoped model mapping: declare from → to rules on the access key or key group, and the gateway auto-rewrites the model name after the ingress gate; you can also add a when request-header condition to branch by tenant (e.g. X-Tenant: vip routes to a premium line). The mapping target still has to pass that identity's permission allowlist — configuring a mapping does not grant permission. See Configure identity-scoped model mapping for full semantics, wildcards, priority, and operational contracts.
A load balancer is an authorization unit (easy to get wrong)
The gateway validates only once, at request ingress, whether the LB name is in the key group's load_balancers allowlist, and does not then validate each entry inside that LB against the models list. In other words, once a key group allows an LB, that group's keys can reach every entry in that LB.
Don't use an LB for strict isolation
To strictly restrict a tenant to only one model, configure it as an ordinary model and add it to that tenant key group's models list — don't put it into an LB that's already authorized for multiple tenants, because an LB's entry set is the reachable union of all keys holding that permission.
Distinguish clients with request-header rules
To further branch within the same key group by client, use Header ACL rules to match request headers (e.g. X-Tenant) for allowlist/blocklist. See Header ACL rule fields for rule fields, and the allowlist mode of Configure Header ACL for a configuration example.
Per-tenant rate limit and quota
- The per-access-key RPM / concurrency / TPM caps are configured in the
RATE_LIMIT_*environment variables (see Audit and security configuration). - In multi-instance deployments rate limiting is shared via Redis; otherwise each instance is independent and thresholds are scattered. See Achieve high availability.
Billing granularity
After configuring a price snapshot in Pricing and billing, the monthly bill aggregates by model + access-key dimension. Each tenant uses its own key group and access key, so billing naturally splits by tenant.
FAQ
Q: Can one access key be used by multiple tenants? No. An access key is attached to one key group, and permission is decided by that group. Multi-tenant requires a separate key group and access key per tenant.
Q: Can the key group's models list use an alias? No. The key group's model ACL only matches the canonical name (the name field), not aliases. See "Model aliases and hidden names" in Upstream and model fields.
Q: How do I keep one tenant's rate limits from interfering with another's? Give each tenant its own access key, and rate limiting is counted per key. Configure Redis in multi-instance so rate limits are shared across instances.
Next: Configure identity-scoped model mapping for per-tenant model-name rewrite; Meet compliance and audit requirements for compliance; Access keys and key groups fields for fields; Access control design for design principles.