# MCP Gateway

[English](mcp.md) | [简体中文](mcp.zh-CN.md)

**Related**: [Configuration](configuration.md) | [Architecture](architecture.md) | [Console API](console-api.md) | [ADR-008](adr/008-mcp-gateway-architecture.md)

## Overview

Protoflux includes a built-in **MCP (Model Context Protocol) Gateway** that enables LLM clients to discover and invoke tools from external MCP servers. The gateway acts as a unified proxy, aggregating multiple upstream MCP servers and exposing their tools through a single endpoint with fine-grained access control and production-grade reliability features.

```mermaid
graph LR
    Client["LLM Client<br/>(Claude, Cursor)"]
    GW["Protoflux MCP Gateway<br/>• ACL filtering<br/>• Circuit breaker<br/>• Rate limiting<br/>• PII masking<br/>• Audit logging<br/>• BM25 search<br/>• SSE notifications"]
    SrvA["MCP Server A<br/>(GitHub)"]
    SrvB["MCP Server B<br/>(Slack)"]

    Client -- "MCP" --> GW
    GW -- "MCP" --> SrvA
    GW -- "MCP" --> SrvB
```

### Use Cases

- **Tool aggregation**: Connect multiple MCP servers (GitHub, Slack, databases) and expose a unified tool set
- **Access control**: Restrict which tools each user or group can access via ACL rules
- **Reliability**: Per-server circuit breakers, concurrency limits, and RPS rate limiting protect upstream services
- **Smart search**: Built-in `search_tools` meta-tool with BM25 ranking helps LLMs discover relevant tools autonomously
- **Compliance**: PII masking on tool arguments and structured audit logging for all tool invocations
- **Real-time updates**: SSE notification stream pushes `notifications/tools/list_changed` events when tools are added or removed

## Client Endpoints

The gateway exposes two endpoints:

| Method | Path | Description |
|--------|------|-------------|
| `POST` | `/mcp` | JSON-RPC 2.0 request/response (Streamable HTTP transport) |
| `GET` | `/mcp/sse` | Server-Sent Events stream for real-time notifications |

### Authentication

All requests require a valid Protoflux access key in the `Authorization` header:

```bash
Authorization: Bearer your-api-key-here
```

Keys are validated against an in-memory SHA-256 index (O(1) lookup). Disabled keys are excluded at index build time and rejected as invalid.

### Supported JSON-RPC Methods

| Method | Description |
|--------|-------------|
| `initialize` | Returns gateway capabilities and protocol version (`2025-03-26`) |
| `tools/list` | Returns ACL-filtered tool list plus the built-in `search_tools` meta-tool |
| `tools/call` | Invokes a tool on an upstream MCP server (or the built-in `search_tools`) |
| `tools/search` | Programmatic BM25 search across all registered tools |

### Request Format (JSON-RPC 2.0)

**Initialize**:

```json
{
  "jsonrpc": "2.0",
  "id": 1,
  "method": "initialize",
  "params": {}
}
```

Response includes `"protocolVersion": "2025-03-26"` (client-facing). Upstream servers use `"2024-11-05"` — the gateway intentionally translates between versions.

**List tools**:

```json
{
  "jsonrpc": "2.0",
  "id": 2,
  "method": "tools/list",
  "params": {}
}
```

Returns all ACL-allowed tools plus the built-in `search_tools` meta-tool.

**Call a tool**:

```json
{
  "jsonrpc": "2.0",
  "id": 3,
  "method": "tools/call",
  "params": {
    "name": "github.create_issue",
    "arguments": {
      "title": "Bug report",
      "body": "Description here"
    }
  }
}
```

Tool names follow the `{server_name}.{tool_name}` convention. The gateway routes the call to the appropriate upstream server.

**Search tools** (programmatic API):

```json
{
  "jsonrpc": "2.0",
  "id": 4,
  "method": "tools/search",
  "params": {
    "query": "create github issue",
    "limit": 5
  }
}
```

### Example: curl

```bash
curl -X POST http://localhost:7890/mcp \
  -H "Authorization: Bearer your-api-key" \
  -H "Content-Type: application/json" \
  -d '{
    "jsonrpc": "2.0",
    "id": 1,
    "method": "tools/list",
    "params": {}
  }'
```

### Example: Claude Code

Add to your `.mcp.json` or project settings:

```json
{
  "mcpServers": {
    "protoflux": {
      "type": "sse",
      "url": "http://localhost:7890/mcp/sse",
      "headers": {
        "Authorization": "Bearer your-api-key"
      }
    }
  }
}
```

Or for Streamable HTTP (if supported by the client):

```json
{
  "mcpServers": {
    "protoflux": {
      "type": "streamable-http",
      "url": "http://localhost:7890/mcp",
      "headers": {
        "Authorization": "Bearer your-api-key"
      }
    }
  }
}
```

### Example: Claude Desktop

Add to your `claude_desktop_config.json`:

```json
{
  "mcpServers": {
    "protoflux": {
      "url": "http://localhost:7890/mcp/sse",
      "headers": {
        "Authorization": "Bearer your-api-key"
      }
    }
  }
}
```

## SSE Notifications

The `GET /mcp/sse` endpoint provides a real-time event stream following the MCP SSE transport specification:

1. Client connects with `Authorization: Bearer` header
2. Gateway sends an `endpoint` event indicating the POST URL (`/mcp`)
3. Gateway streams `message` events containing `notifications/tools/list_changed` JSON-RPC notifications whenever tools are added, removed, or updated
4. Keepalive pings (`: ping`) are sent every 30 seconds to maintain the connection through proxies

Notifications are distributed via the unified `BroadcastBackend<McpNotification>` abstraction. In single-instance mode, an in-memory broadcast channel is used. In multi-instance mode (Redis), notifications propagate across instances via Redis Pub/Sub with an anti-loop mechanism: remote messages trigger a local index rebuild that notifies local SSE clients via `publish_local()` without re-publishing back to Redis. See [Deployment → Unified Broadcast Architecture](deployment.md#unified-broadcast-architecture-redis-pubsub) for details.

If the broadcast channel overflows (client too slow), missed events are logged and the client self-heals on its next `tools/list` call.

## Configuration

MCP servers are managed via the Web Console or Console API. Business configuration (MCP servers, access keys, ACLs) is stored in the database and can be modified at runtime without restarting.

### Web Console

Navigate to **MCP Servers** in the sidebar to add, edit, or delete MCP server configurations.

### Console API

| Method | Endpoint | Description |
|--------|----------|-------------|
| `GET` | `/console/api/mcp-servers` | List all MCP servers |
| `POST` | `/console/api/mcp-servers` | Create an MCP server |
| `PUT` | `/console/api/mcp-servers/{name}` | Update an MCP server |
| `DELETE` | `/console/api/mcp-servers/{name}` | Delete an MCP server |

### Server Configuration Fields

| Field | Type | Required | Description |
|-------|------|----------|-------------|
| `name` | string | Yes | Unique server identifier (used in tool name prefix) |
| `description` | string | No | Human-readable description |
| `transport` | enum | Yes | Wire protocol to upstream: `streamable_http` or `sse` |
| `endpoint` | string | Yes | Upstream MCP server URL |
| `auth_header` | string | No | Authorization header value for upstream |
| `extra_headers` | object | No | Additional HTTP headers sent to upstream |
| `defer_loading` | boolean | No | Skip tool discovery until first `tools/call`; reduces startup overhead |
| `tags` | array | No | Tags for categorization and ACL tag-based filtering |
| `priority` | integer | No | Priority for context budget (higher = preferred, default: 100) |
| `connection_type` | enum | Yes | Session model: `stateful`, `stateless`, or `rest_bridge` |
| `idle_timeout_secs` | integer | No | Connection idle timeout (default: 300) |
| `max_concurrency` | integer | No | Maximum concurrent requests to this server (enforced via semaphore) |
| `rate_limit` | object | No | Rate limiting: `{ "rps": <requests_per_second> }` |
| `extra_config` | object | No | Arbitrary JSON passed to the transport adapter |

### Transport vs Connection Type

These are independent dimensions:

- **Transport** (`streamable_http` / `sse`): The wire protocol used to communicate with the upstream MCP server.
- **Connection Type** (`stateful` / `stateless` / `rest_bridge`): The session model governing client isolation and connection pooling.

For example, a server can use `sse` transport with `stateless` connection type — meaning it uses SSE wire protocol but shares connections across clients.

### Example: Add a GitHub MCP Server

In the Console, open **MCP Servers** → click **Add MCP Server** and fill in:

- **Name**: `github`
- **Description**: `GitHub MCP server for repository management`
- **Transport**: `Streamable HTTP`
- **Connection Type**: `Stateful`
- **Endpoint**: `https://mcp.github.com`
- **Auth Header**: `Bearer ghp_xxxxxxxxxxxx`
- **Tags**: `development, vcs`
- **Priority**: `100`

> **Note**: `max_concurrency` (value `10`) and `rate_limit` (value `{ "rps": 50 }`) have no Console UI field — they are Console-API-only. Use the Console API (`POST /console/api/mcp-servers`) to set them if you need per-server concurrency caps or RPS limiting.

## Transport Protocols

### Streamable HTTP

Standard HTTP transport with JSON-RPC payloads. Suitable for most MCP servers.

- **Stateful**: Maintains a persistent session per client. Each API key gets an isolated connection pool with its own cookie jar.
- **Stateless**: No session state. Connections are pooled and shared across clients.

### SSE (Server-Sent Events)

For MCP servers that use SSE for server-to-client streaming. The gateway:

1. Opens a GET SSE connection to the server's SSE endpoint
2. Receives the `endpoint` event with the POST URL
3. Sends JSON-RPC requests via POST
4. Receives responses via the SSE stream

### REST Bridge

Exposes REST APIs as MCP tools using an OpenAPI specification. Useful for integrating legacy APIs without MCP support.

## Access Control (ACL)

MCP tool access is controlled at the **Access Key Group** level via the `mcp_tool_acl` field. Each group can specify three filter dimensions:

| Field | Description |
|-------|-------------|
| `allowed_tools` | Whitelist of tool name patterns (supports wildcards); empty means no restriction |
| `denied_tools` | Blacklist of tool name patterns; **always takes highest priority** |
| `allowed_tags` | Server tag whitelist; when non-empty, tools must belong to a server with at least one matching tag |

### Wildcard Syntax

| Pattern | Matching Rule | Example |
|---------|---------------|---------|
| `*` | Matches all tools | — |
| `server.*` | Matches `server` itself and all `server.<tool>` names | `github.*` → `github`, `github.create_issue`, `github.list_repos` |
| Exact name | Matches only that specific tool | `slack.send_message` |

### Evaluation Priority

```
denied_tools (highest) → allowed_tools → allowed_tags → default allow
```

1. If the tool name matches any pattern in `denied_tools` → **DENY**
2. If `allowed_tools` is non-empty and the tool name matches none of its patterns → **DENY**
3. If `allowed_tags` is non-empty and the tool's server has no matching tag → **DENY**
4. Otherwise → **ALLOW**

When a tool is denied by ACL, the gateway records an audit log entry (`AclViolation` event) and increments the `protoflux_mcp_acl_filtered_total` metric. ACL snapshots are cached for 300 seconds and automatically invalidated on config hot-reload.

### Console UI Configuration

In the Web Console: **Access Keys → Key Groups → Edit Group → MCP Access Control**

- **Allowed MCP Tools**: Check `*` (all) or per-server `server.*` patterns
- **Denied MCP Tools**: Check per-server `server.*` patterns to deny
- **Allowed MCP Tags**: Enter comma-separated tag list

> ⚠️ The MCP ACL section is only visible when at least one MCP server exists. Add servers first via the **MCP Servers** page.

### API Example

```bash
curl -X PUT http://localhost:7890/console/api/access-key-groups \
  -H "Authorization: Bearer console-token" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "developers",
    "models": ["gpt-4", "claude-3-opus"],
    "mcp_tool_acl": {
      "allowed_tools": ["github.*", "slack.send_message"],
      "denied_tools": ["github.delete_repository"],
      "allowed_tags": ["development"]
    }
  }'
```

## Schema Sanitization

Tool schemas from upstream MCP servers and LLM clients pass through a **two-boundary defense** to prevent provider-specific validation errors (e.g., Moonshot rejecting non-string `description` fields).

### Boundary 1: Ingestion Normalization

When tools are received from upstream MCP servers (during periodic sync), schemas are normalized to canonical JSON Schema form before caching:

- Non-string `description` → coerced to string or removed
- Non-object property schemas (e.g., `"[REDACTED]"`) → replaced with `{"type": "string"}`
- Invalid `type` values → removed
- Depth limit (32 levels) prevents stack overflow from maliciously deep schemas

This ensures all cached tool schemas are structurally valid regardless of upstream quality. Normalization is idempotent and only modifies malformed schemas — well-formed schemas pass through unchanged.

### Boundary 2: Egress Adaptation

When translating requests to a specific LLM provider, provider-specific schema profiles adapt the normalized schema to match each provider's supported JSON Schema subset:

| Provider | Key Adaptations |
|----------|----------------|
| **Google** | `const`→`enum`, empty enum filtering, `anyOf`/`oneOf`→first variant, `allOf`→deep merge, `examples`→`example`, strips ~20 unsupported keys |
| **DashScope** | Strips `$`-prefixed keys (`$schema`, `$ref`, `$defs`, etc.) and `additionalProperties` |
| **OpenAI** | Strips `$`-prefixed metadata keys and `definitions` |
| **AWS Converse** | Strips `$`-prefixed metadata keys and `definitions` |
| **Anthropic** | Passthrough (richest schema support); converts singular `example`→`examples` |

Profiles are implemented via the Strategy pattern (`SchemaProfile` trait). Adding a new provider requires only a new profile file and one match arm — zero changes to existing code.

> ⚠️ Profile adaptations use best-effort transformations for unsupported features (e.g., flattening `anyOf` to first variant for Google). This is preferable to API rejection but may narrow schema semantics. Profiles should be updated when providers expand their schema support.

## Production Guards

Every `tools/call` request passes through a series of production guards before reaching the upstream server. These are applied in order:

### 1. Circuit Breaker (Per-Server)

Each MCP server has an independent circuit breaker with three states:

- **Closed** (normal): Requests pass through. Consecutive failures are counted.
- **Open** (tripped): Requests are rejected immediately. After `reset_timeout` (default: 30s), transitions to HalfOpen.
- **HalfOpen** (probing): Allows one request through. Success closes the circuit; failure reopens it.

Default thresholds: 5 consecutive failures to trip, 2 consecutive successes to recover.

The circuit breaker uses an async-friendly API (`pre_check()` / `record_success()` / `record_failure()`) that separates the state check from the async operation, avoiding holding a mutex across await points.

Circuit breaker state transitions are reported to Prometheus via `protoflux_mcp_circuit_breaker_state` gauge (0=Closed, 1=Open, 2=HalfOpen).

### 2. Concurrency Limit (Per-Server)

When `max_concurrency` is configured, a semaphore enforces the limit. Excess requests receive an immediate error response without contacting the upstream server.

### 3. RPS Rate Limiter (Per-Server)

When `rate_limit.rps` is configured, an atomic token bucket limits requests per second. Uses `tokio::time::Instant` (monotonic clock, immune to system clock skew) with CAS-loop refill for lock-free concurrent access.

### 4. PII Masking

When enabled, tool call arguments are recursively scanned for PII patterns (emails, phone numbers, SSNs, credit cards, IP addresses). Only string leaf values are masked — JSON structure (key order, nesting, non-string types) is preserved.

### 5. Metrics Recording

After each tool call (success or failure):
- Duration histogram: `protoflux_mcp_tool_call_duration_seconds`
- Success/failure counters: `protoflux_mcp_tool_calls_total`
- Circuit breaker state gauge update

### 6. Audit Logging

All tool calls (success and failure) and ACL violations are logged via the structured audit logger with: access key, server name, tool name, outcome, and duration.

## Tool Search

The gateway provides two ways to search for tools:

### Built-in `search_tools` Meta-Tool

Injected into every `tools/list` response as a regular tool. LLMs call it autonomously when they need to discover tools:

```json
{
  "jsonrpc": "2.0",
  "id": 1,
  "method": "tools/call",
  "params": {
    "name": "search_tools",
    "arguments": {
      "query": "create github issue",
      "limit": 5
    }
  }
}
```

Results are ACL-filtered so restricted keys don't see disallowed tools.

### Programmatic `tools/search` Method

Direct JSON-RPC method for clients that want search results as structured data:

```json
{
  "jsonrpc": "2.0",
  "id": 1,
  "method": "tools/search",
  "params": {
    "query": "create github issue",
    "limit": 5
  }
}
```

Search latency is recorded in `protoflux_mcp_search_duration_seconds`.

### BM25 Ranking Parameters

- **Name boost**: Tool names are weighted 2× higher than descriptions
- **Tag matching**: Tools whose server tags match query terms receive a bonus
- **Top-K limiting**: Returns only the most relevant results up to the specified limit

## Background Sync

The gateway periodically synchronizes tool definitions from upstream MCP servers:

- **Sync interval**: Configurable (default: 60 seconds)
- **Deferred loading**: Servers with `defer_loading: true` skip initial tool discovery; tools are loaded on first `tools/call` to reduce startup overhead
- **Change notification**: After each sync that changes the tool set, a `notifications/tools/list_changed` event is broadcast to all connected SSE clients

## Metrics

MCP Gateway metrics are exposed via the `/metrics` endpoint alongside core gateway metrics:

| Metric | Type | Description |
|--------|------|-------------|
| `protoflux_mcp_connections_total` | Counter | Total MCP connections established |
| `protoflux_mcp_connections_active` | Gauge | Currently active connections |
| `protoflux_mcp_tool_calls_total` | Counter | Total tool invocations (labels: server, tool, status) |
| `protoflux_mcp_tool_call_duration_seconds` | Histogram | Tool call latency |
| `protoflux_mcp_context_budget_used` | Gauge | Tokens used by context |
| `protoflux_mcp_context_tools_loaded` | Gauge | Number of tools in context |
| `protoflux_mcp_context_lru_evictions_total` | Counter | LRU eviction count |
| `protoflux_mcp_search_queries_total` | Counter | Search queries processed |
| `protoflux_mcp_search_duration_seconds` | Histogram | Search latency |
| `protoflux_mcp_acl_filtered_total` | Counter | Tools filtered by ACL |
| `protoflux_mcp_circuit_breaker_state` | Gauge | Circuit breaker state per server (0=Closed, 1=Open, 2=HalfOpen) |
| `protoflux_mcp_errors_total` | Counter | Total errors |

## Architecture

See [ADR-008: MCP Gateway Architecture](adr/008-mcp-gateway-architecture.md) for detailed design decisions.

### Request Flow (tools/call)

```mermaid
flowchart TD
    A["Client POST /mcp"] --> B["Authentication<br/>Bearer token → SHA-256 index lookup"]
    B --> C["Load Context (per API key)<br/>• Check ACL snapshot TTL<br/>• Refresh ACL if expired<br/>• Load/evict tools based on budget"]
    C --> D["Parse JSON-RPC Request"]
    D --> E["ACL Filter<br/>• allowed_tools / denied_tools<br/>• allowed_tags / denied_tags<br/>• Audit + metrics on violation"]
    E --> F["Production Guards<br/>1. Circuit breaker pre_check<br/>2. Concurrency semaphore<br/>3. RPS token bucket<br/>4. PII argument masking"]
    F --> G{"Route to Upstream"}
    G --> H1["HttpMcpServer<br/>(Streamable HTTP)"]
    G --> H2["SseMcpServer<br/>(SSE)"]
    G --> H3["RestBridgeServer<br/>(REST Bridge)"]
    H1 --> I["Post-call Processing<br/>• CB record_success/failure<br/>• Metrics recording<br/>• Audit logging"]
    H2 --> I
    H3 --> I
    I --> J["Return JSON-RPC Response"]
```

### Connection Isolation

For stateful servers, the gateway maintains per-client isolation:

```mermaid
graph LR
    KA["API Key A"] --> CA["Client A<br/>• Isolated HTTP client<br/>• Isolated cookie jar<br/>• Isolated connection pool<br/>• Isolated context state"]
    KB["API Key B"] --> CB["Client B<br/>• Isolated HTTP client<br/>• Isolated cookie jar<br/>• Isolated connection pool<br/>• Isolated context state"]
    CA --> Srv["MCP Server"]
    CB --> Srv
```

## Limitations

- **Protocol scope**: Currently supports tools only. Resources, prompts, and sampling extensions are not implemented.
- **REST Bridge**: Requires an OpenAPI 3.x specification. Swagger 2.0 is not supported.
- **Context budget**: Token estimation is approximate (based on tiktoken-rs). Actual usage may vary.
- **Multi-instance**: Context state is local to each instance. Sticky sessions or Redis-backed state recommended for multi-instance deployments.
- **Rate limiting granularity**: Per-server RPS and max_concurrency only. Per-key or global rate limits are not yet implemented.

## Troubleshooting

### "Tool not found" error

- Check if the tool is allowed by the access key group's ACL
- Verify the tool name format: `{server_name}.{tool_name}`
- Use the `search_tools` meta-tool to discover available tools
- Check server tags if using tag-based ACL

### "Circuit breaker open" error

- The upstream server has exceeded the failure threshold (default: 5 consecutive failures)
- Wait for the reset timeout (default: 30s) — the circuit will automatically probe
- Check upstream server health and network connectivity
- Monitor `protoflux_mcp_circuit_breaker_state` metric for state transitions

### "Concurrency limit exceeded" / "Rate limit exceeded"

- Increase `max_concurrency` or `rate_limit.rps` in the server configuration
- Distribute load across multiple server instances
- Check if a single client is monopolizing capacity

### Connection timeout

- Increase `idle_timeout_secs` in server configuration
- Check upstream server health
- Verify network connectivity to the upstream endpoint

### Context budget exceeded

- Increase the global context budget in gateway configuration
- Mark low-priority tools with `defer_loading: true`
- Reduce the number of configured MCP servers