Connect an AWS Bedrock upstream

AWS Bedrock is an upstream-side protocol — the gateway does not expose a Bedrock-shaped client endpoint. Clients access Bedrock models through OpenAI / Anthropic / Gemini protocol endpoints (translated by the gateway), or pass through via /v3/{model}/{*rest}. This chapter teaches you how to create an AWS Bedrock upstream + model and call it with the OpenAI SDK.

Prerequisites

• The gateway is running (http://localhost:7890) and you can log in to the Console
• ENCRYPTION_KEY is set — the upstream API key and access key you're about to save are encrypted before storage; if unset, saving reports encryption_key not set in config. See Docker single node → Prerequisites
• A Bedrock API key (issued in the Amazon Bedrock Console → "API keys"; long- or short-term both work)
• Know the Bedrock model ID you want to call (e.g. anthropic.claude-3-5-sonnet-..., zai.glm-5, etc.)
• Know the region (e.g. us-west-2, us-east-1)

1. Create an AWS Bedrock upstream

Console → Upstreams → New:

Field	Value	Description
Name	aws-global	Unique upstream identifier
Protocol	aws_converse	Bedrock Converse API; use aws_invoke for the InvokeModel path
Base URL	https://bedrock-runtime.us-west-2.amazonaws.com	Bedrock runtime endpoint
API Key	The Bedrock API key verbatim	Paste a single Bedrock API key value. Do not fill in access_key_id / secret_access_key, and don't build a combined string like id:secret:region
Enabled	✓

Save.

The gateway accesses the Bedrock runtime with Authorization: Bearer <API Key>, so what you fill in here is the Bedrock API key, not the IAM access key ID / secret. The region is determined by the base URL (https://bedrock-runtime.<region>.amazonaws.com); to change region, change the upstream's base URL.

2. Create a Bedrock model

In the aws-global upstream row, click Expand → model sub-table → New model:

Field	Value	Description
Name	glm-5	The model name exposed to clients
Upstream	aws-global	Select the upstream you just created
Upstream model ID	zai.glm-5	Bedrock's real model ID
Enabled	✓

Save.

Override the model ID with inference_profile

If Bedrock routes traffic using an application inference profile, fill the inference_profile field with the ARN to override upstream_model_id as the real ID sent to the upstream:

{
  "name": "glm-5",
  "upstream": ["aws-global"],
  "upstream_model_id": "zai.glm-5",
  "upstream_inference_profile": "arn:aws:bedrock:us-west-2:123456789012:inference-profile/abcd1234"
}

Once set, this value is used as the upstream model ID in all requests, with upstream_model_id as the fallback.

Protocol override (OpenAI compatibility)

If you want to access the Bedrock model via the OpenAI protocol endpoint, keep the upstream protocol as aws_converse but override the model-level protocol to openai:

{
  "name": "glm-5",
  "upstream": ["aws-global"],
  "upstream_model_id": "zai.glm-5",
  "protocol": "openai",
  "base_url": "https://bedrock-runtime.us-west-2.amazonaws.com/openai/v1"
}

Credentials still come from the upstream config; only the target endpoint and protocol format of the request change. base_url has no trailing / (consistent with the site-wide convention; even if it does, the gateway strips it before concatenation).

3. Configure a key group to allow it

Console → Access Keys → Key Groups tab → New:

Field	Value	Description
Name	default	Group name
Models	Select glm-5, or * (all)	Determines which models this group's keys can call
Enabled	✓

4. Issue an access key

Console → Access Keys → New:

Field	Value	Description
Name	my-app-key	The key's identifier, used for management and auditing
API Key	click "Generate"	Auto-generates a string; the credential the client carries when calling
Group	default	Determines which models this key can access
Enabled	✓

5. Call with the OpenAI SDK

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:7890/v1",
    api_key="<your access key>",
)

resp = client.chat.completions.create(
    model="glm-5",
    messages=[{"role": "user", "content": "hello"}],
)
print(resp.choices[0].message.content)

The gateway receives the OpenAI-format request, translates it to the Bedrock protocol (aws_converse or aws_invoke) per the model config, sends it to the Bedrock runtime, and translates the response back to OpenAI format.

6. Or use pass-through

When the client protocol matches the model's configured target protocol, you can pass through via /v3/{model}/{*rest}, with the request body forwarded verbatim. {*rest} is appended to the upstream base_url nginx-reverse-proxy style — for Bedrock Converse, {*rest} is /model/<upstream model ID>/converse. A minimal converse request:

curl http://localhost:7890/v3/glm-5/model/zai.glm-5/converse \
  -H "Authorization: Bearer <your access key>" \
  -H "Content-Type: application/json" \
  -d '{
    "messages": [
      {"role": "user", "content": [{"text": "hello"}]}
    ]
  }'

The gateway appends model/zai.glm-5/converse to the upstream https://bedrock-runtime.us-west-2.amazonaws.com, and forwards the request body (Bedrock's native converse format) verbatim. For streaming, use .../converse-stream.

Cross-protocol details

• AWS Bedrock is an upstream-side protocol: clients access it via OpenAI / Anthropic / Gemini protocol endpoints, and the gateway performs protocol translation
• When the client protocol matches the upstream protocol, it takes the identity path (request body forwarded verbatim); among these, Anthropic→Anthropic is a same-protocol non-pass-through translator
• For the full interop matrix, see Protocol interop matrix
• In cross-protocol tool calls, tool_call_id must keep its original value across the round trip

FAQ

Q: Client reports 400 unsupported_feature? There's no translator for that "client protocol → upstream protocol" pair. Check the Protocol interop matrix.

Q: Client reports 403?

• 403 model_access_denied: the key group's "models" list doesn't include glm-5
• 403 ip_banned: your IP is banned (triggered by repeated login failures)

Q: Client reports 502 bad_gateway? The upstream is unreachable. Check the Bedrock runtime URL, API key, and region.

Q: Can I use IAM access key / secret or an EC2 IAM Role? No. The gateway only carries the Bedrock API key in Authorization: Bearer; it does not do SigV4 signing and does not read EC2 instance metadata. Issue an API key in the Bedrock Console and fill it into the upstream's API Key field.

Next: Upstream and model fields for the full upstream/model fields; Endpoints · auth · protocol interop for the full endpoint list; Protocol interop matrix for Bedrock's interop with other protocols.