> Raw Markdown twin (generated at build time from the source Markdown). Rendered page: https://docs.gatellm.io/en/console/search · Doc index: https://docs.gatellm.io/en/llms.txt


# Web search

Many models cannot search the web themselves, but clients (such as Claude Code) declare a web-search tool in their requests. When forwarded as-is, such models either ignore the search tool or error out directly.

**Gateway search** solves this: when the routed target model does not support native search, the gateway performs the search on the model's behalf, organizes the results into reference material for the model to answer with, then wraps the search results in the native "the model searched itself" shape and returns them to the client. After the gateway performs the search, it **always calls the upstream model once** — the search results (including the vendor's own synthesized answer) are injected into the user message as clearly-labeled reference material, and the model organizes its answer based on that material; the gateway never returns the vendor's text directly as a model response. The client is completely unaware.

Entry: Console → **Settings** → **Search** (admin only).

## Quick start

1. **Add a search engine**: click "Add search engine", choose type `serper` (the first supported vendor, [serper.dev](https://serper.dev)), paste the API Key, and save.
2. **Connectivity test**: click "Test" in the list row; the gateway issues a real query (short timeout) and returns the result count and elapsed time.
3. **Set default engine** (optional): choose a default search engine in the system configuration area; if not set, it is selected automatically by priority.
4. Afterwards, any request where the client declares a search tool and the routed model has no native search capability automatically goes through gateway search.

## Hijack switch: three states and priority chain

Each model can set the "Gateway search" three-state switch in the model form (written to `extra_config.search.enabled`):

| Value | Behavior |
|------|------|
| Default · follow capability | Model has no native search → gateway hijacks and searches on its behalf; has native search → yield to the native path |
| Force on | Even if the model has native search, the gateway hijacks and searches on its behalf (the search tool is stripped, so the upstream won't double-search) |
| Force off | Never hijack under any circumstances |

The decision is resolved via a **priority chain**: **model config > upstream config > capability declaration**. The upstream can also configure the switch in `extra_config.search`; model-level config fully overrides upstream-level config; only when neither is configured does it fall back to the metadata capability declaration (whether the model natively supports search).

> **Common question**: models like qwen declare native search capability in their metadata (e.g. DashScope `enable_search`); in default mode these models take the native path and are **not** hijacked by the gateway. To have the gateway search on their behalf uniformly, set that model's switch to "Force on".

## Engine selection: three-level fallback

Which engine a hijack uses is chosen via three levels:

1. The engine name specified by the model/upstream in `extra_config.search.provider`;
2. The system default engine (Settings → Search → default search engine);
3. The highest-priority enabled engine.

## System configuration and tuning

The lower half of the Settings → Search page provides global tuning parameters (saving applies them immediately):

| Parameter | Default | Meaning |
|------|------|------|
| Pages per query | 2 | How many pages of results to fetch from the vendor per hijack (about 10 items per page) |
| Max characters per snippet | 500 | Truncation length of each snippet injected to the model |
| Total injected character cap | 16384 | Total cap of reference-material text; excess is dropped from the tail |
| Max query extraction characters | 400 | Cap on the query length extracted from the user message |
| Per-engine timeout | 10000 ms | Query timeout for a single engine |
| Total hijack budget | 15000 ms | Overall time budget including degraded retries |
| Concurrency gate capacity | 64 | Cap on simultaneous searches |
| Gate queue timeout | 2000 ms | Time limit for waiting in the gate queue; shedding on timeout |
| Max response bytes | 512 KB | Hard cap on vendor response; connection cut if exceeded |

**Memory ceiling**: resident usage ≈ gate capacity × pages per query × max response bytes (default 64 × 2 × 512KB ≈ 64MB). Tuning these three knobs together matches the instance's memory budget.

## Degradation behavior (search failure does not affect the request)

Search is a best-effort side capability; any failure **does not block the request**, and **does not strip the client-declared search tool**:

- Engine timeout / HTTP error / parse failure / no results → the request body is forwarded as-is (the web_search tool is kept), and the response carries a degradation notice;
- High memory pressure or concurrency gate queue timeout → shed load, the request body is likewise forwarded as-is;
- No engine configured → no hijack, the request is forwarded as-is.

When the search fails, the retained web_search tool is handed to the upstream: routes with native search capability (such as qwen's `enable_search`) let the upstream model continue searching as a fallback; routes without native capability are handled by the protocol conversion layer's existing rules (possibly silently dropped, the model cannot search this turn — consistent with the behavior before the search failure). The search tool is stripped only after the search succeeds and the results are successfully injected into the user message, avoiding the situation of "search failed but the model's search capability was also taken away".

Client perspective during degradation: the turn's response where the search failed has no gateway search block (equivalent to "the model searched itself or did not search"), and the answer is still returned normally. On the operations side, the degradation reason can be seen in the request log's translation notices.

## Complete data flow of a hijacked request

1. The gateway extracts the query from the user's last message;
2. Issues a search to the engine (paginated fetch of titles/links/snippets, without crawling the body); if the vendor returns its own synthesized answer text, it is retained as well;
3. The results are rendered into numbered reference material and appended to the user's last message. The injected text clearly begins with "This is the search result retrieved by the gateway on your behalf; cite the source URLs when answering" — making it clear to the model that this is search material rather than user input; the vendor's synthesized answer is placed as a separate "search vendor summary" paragraph before the result list, also clearly labeled;
4. Only **after the search succeeds and the injection succeeds** does the gateway strip the client-declared web_search tool (to avoid the upstream double-searching); if injection fails, the tool is retained and the request body is forwarded as-is (see degradation behavior);
5. The upstream model only sees "question + clearly-labeled reference material" and answers normally (**always one call**; the gateway never answers on the model's behalf);
6. The gateway synthesizes the saved search results into native search blocks per the client protocol and appends them to the response:
   - Anthropic → `server_tool_use` + `web_search_tool_result` content blocks;
   - Chat Completions → `url_citation` citation annotation;
   - Responses → `web_search_call` output item.

## Known costs: upstream prefix cache and mandatory upstream call

After the gateway performs the search, it **always calls the upstream model once** — this is an invariant. Even if the search vendor returns a ready-made synthesized answer, the gateway only injects it as material and lets the model organize its answer based on it, never returning the vendor's text directly as a model response.

The injected reference material is visible only to the upstream; the client's next-turn request does not carry it. For stateless protocols (Chat Completions / Anthropic), the upstream prefix cache's hit range shrinks to before the last hijacked turn; stateful protocols (Responses + store) are unaffected. This is the inherent cost of "client-unaware". Each hijacked turn also consumes 2 vendor request quotas (2 pages by default).

## Billing scope

Hijacked searches are **not counted in usage or billing**: the response body's `usage.server_tool_use.web_search_requests` is only protocol disguise (identical in shape to a real vendor response), and the internal usage ledger does not record search counts. Search cost is the gateway operator's procurement spend with the vendor, separate from the upstream billing system.