# Does a context gate for search agents actually work?
> Testing context gating on BrowseComp-Plus. A cheap label-free gate cuts a search agent's input tokens 1.4x, gate costs included, at baseline accuracy.
Published: 2026-07-17
Author: Nikhil Kasukurthi
Tags: LLM, Agents, RAG, Evaluation
Canonical: https://bluenotebook.io/blog/context-gating-browsecomp/
---
At Berlin Buzzwords 2026, Lester Solbakken gave a talk titled "Better retrieval makes agents worse" [+buzzwords].
These two slides matched my experience building search for agents.
Distractors degrade accuracy nonlinearlyControl context admission, not just top-k
Lester's argument was that agentic retrieval is **context admission control**.
An agent retrieves context to act. A false positive is not a wasted result on a page: it enters the context, gets re-read at every later step, and shapes the next action.
Classic retrieval optimizes recall. A search tool inside an agent has to optimize precision.
The talk inspired me to build a context gate inside the search tool: a second model call that decides, per retrieved document, what enters the agent's context.
BrowseComp-Plus [+browsecomp] was the test-bed.
It ships with labeled evidence documents for every query, which makes the idea measurable.
The rest of this post measures it: six agent-and-gate configurations over the same 180 queries.
Counting its own tokens, a cheap gate cuts input 1.4×, and accuracy holds.
But on DeepSeek's cache pricing the gate loses money, and it doubles per-query latency.
## Why context accumulates
An agent with a search tool issues several search calls for a single question, and every result stays in the conversation until the final answer.
By search 8, the agent is re-reading the distractors from search 1 on every step.
Massive context windows make context stuffing the easy way out: leave everything in and trust the model to figure out the answer.
It mostly works, and it pays for that in tokens and latency at every step.
Prior work has measured what it also costs in accuracy.
The first-drop-of-ink paper Lester cites is worth a closer read.
From Gao et al.
Left: accuracy against the share of hard distractors in that context.
Right: the shaded first 10%, zoomed in.
Of the 25 points the model loses in total, 14.5 are gone before the context is even a tenth distractors.
Gao et al. [+firstink] pin the damage on hard distractors, documents close enough to the topic to pass for evidence.
The first such documents to enter the context do most of the harm.
In my ungated baseline run on BrowseComp-Plus, a single question reads just over a million input tokens this way, 1,035K on average.
It writes roughly 1/100 of that.
An idealized picture of what the model re-reads at each step. Hover a row for the breakdown. Blue blocks are the evidence the answer needs. The bar grows every search, but the evidence share doesn't: by search 18, 78% of the prompt is dead weight. The model has read ~0.9M tokens to obtain three useful documents.
## The context gate
The context gate is another agent/LLM call inside the `search` tool, one batched model call per search.
Conditioned on the question, the gate assigns each retrieved document one of three admissions.
1. **Full document.** The complete document, truncated at a maximum token boundary.
2. **Span.** The gate returns only the snippets relevant to the question.
3. **Withhold.** The gate judges the document a distractor and returns only its title, with a note that it withheld the rest.
Per-document admissions. A withheld doc keeps its title and a get_document hook, so the agent can overrule the gate. Over a full run the admissions compound: net of the gate's own reads, the gated run uses 1.4× fewer input tokens and lands within one query of the ungated baseline.
The gate is write-once. It decides what enters at retrieval time and never edits the conversation afterwards, because pruning earlier messages invalidates the prefix cache.
What reaches the agent is only what the gate judged relevant at that moment.
## Experiment setup
All the experiments are on BrowseComp-Plus: a fixed corpus of 100K passages, with queries labeled for gold documents (contain the answer) and evidence documents (needed to connect the hops).
The agent is DeepSeek (v4-flash or v4-pro depending on the arm) with a `search` and a `get_document` tool, capped at 30 iterations.
I am not building a better retriever here. The baseline retriever from the BrowseComp-Plus paper is hybrid BM25 + Qwen3-0.6B embeddings, at k=10.
It surfaces a labeled document for about a quarter of the annotated queries, 180 in all. I call these the *firing* queries.
The gate only decides what happens after retrieval surfaces something, so the experiments run on the queries where it has ground truth to act on.
The exact run configuration
Selection arithmetic: of the benchmark's 830 queries, 743 had span annotations I could validate for the oracle. Hybrid retrieval surfaces a labeled document for 199 of those (27%). Removing pilot-run queries and one query that deterministically blows the context window in the oracle arm leaves 180.
Retrieval is hybrid BM25 + dense with reciprocal-rank fusion at k=10.
An iteration can issue several tool calls, so search counts per query run higher than 30.
The same judge model (deepseek-v4-flash) scores all arms, one trajectory per query per arm, sampling temperature left at the API default.
I audited by hand the runs that ended in a raw tool-call fragment instead of a final answer, and scored judge false positives as incorrect.
```bash
uv run python -m search_agent.deepseek_client \
--model deepseek-v4-pro --query runs/hybrid/newfiring182.tsv \
--get-document --query-template QUERY_TEMPLATE \
--searcher-type hybrid --bm25-index-path indexes/bm25 \
--dense-index-path 'indexes/qwen3-embedding-0.6b/corpus.shard*_of_4.pkl' \
--model-name Qwen/Qwen3-Embedding-0.6B --attn-implementation sdpa \
--snippet-max-tokens 512 --k 10 --max-iterations 30 \
--num-threads 16 --max_tokens 64000 \
--gate model --gate-model deepseek-v4-flash \
--gate-withhold soft --gate-spans-only \
--output-dir runs/hybrid/newfiring182_proagent_flashgate_v3
```
Drop the four `--gate*` flags for the ungated baselines. The gate itself is a
system prompt plus a shim in the search tool, about 200 lines.
## The experiments
Before building a real gate, I tested the idea with an oracle that reads the answer key.
The oracle admits only the labeled documents, with pre-annotated answer spans where they exist, and withholds everything else the search returns.
Its score is the ceiling for any real gate.
With `deepseek-v4-flash` as the agent, the oracle gate scores 80% to the vanilla searcher's 79%.
And it uses 2.5× fewer tokens.
For judging the final answer, I re-use the BrowseComp-Plus judge mechanism, with deepseek-v4-flash as the judge model.
The deployable version has no answer key to look up.
It reads each question–document pair and predicts the admission from the text alone.
Two of its design decisions came out of watching early runs fail.
### 1. Withholding must be soft.
The first version replaced withheld documents with a bare `[withheld]` marker.
The agent treated every marker as a mystery box and called `get_document` on nearly all of them, pushing expansions and tokens *above* the ungated baseline.
Showing the title fixed it. The agent has only one question about a withheld document, whether it is worth fetching, and the title answers it in a few tokens.
The agent's reaction to withheld documents. Hiding everything triggers an expansion storm. Showing the title keeps expansions rare.
### 2. Spans beat full documents.
The best variant forbids admitting any document whole. Kept documents are compressed to the verbatim spans that connect them to the question.
That bounds the context no matter how long the agent runs.
One query in my set blows the model's 1M-token context window when the gate admits whole evidence documents. Under spans-only it completes.
## Results
Net of the tokens the gate itself reads, gating cuts the pro agent's input 1.4× while matching the baseline's accuracy.
The ungated pro agent scores 148/180 (82.2%). With `deepseek-v4-pro` as both agent and gate, it scores 147/180 (81.7%), one query apart.
All six experiments on the same 180 queries, same judge. Hover or tap a point for that arm's full numbers.
Gating moves every arm left (fewer tokens). None moves up (no accuracy gain).
The agent's own reads fall from 1,035K to 589K tokens per query. Adding back what the gate reads nets out at the 1.4×.
Swapping the pro gate for a flash gate keeps nearly all of that reduction and costs one more query, 146/180.
Under the weaker flash agent the compression is deepest. Accuracy also drops six queries, from 143/180 (79.4%) to 137/180 (76.1%).
The failure happens when one needed document is hidden and the agent wanders, issuing excess search calls.
## What gating costs
The API costs come from each arm's measured usage and DeepSeek's price sheet [+pricing].
### API costs
| arm | accuracy | agent $/query | gate $/query | total $/query | total, 180 queries |
|---|---|---|---|---|---|
| pro, ungated | 148/180 (82.2%) | $0.063 | n/a | $0.063 | $11.31 |
| pro + flash gate | 146/180 (81.1%) | $0.044 | $0.030 | $0.074 | $13.36 |
| pro + pro gate | 147/180 (81.7%) | $0.042 | $0.093 | $0.135 | $24.34 |
| flash, ungated | 143/180 (79.4%) | $0.027 | n/a | $0.027 | $4.80 |
| flash + flash gate | 137/180 (76.1%) | $0.015 | $0.026 | $0.041 | $7.31 |
The ungated baseline is the cheapest arm for its accuracy, because of context caching.
The ungated agent re-reads a long, stable prefix on every step, and almost all of those reads are billed as cache hits.
The gate reads each document once, in a fresh prompt, so its tokens are billed at cache-miss prices.
Despite the token reduction, gating costs more per query.
89% of the baseline's input tokens are cache hits.
Cache-hit tokens are billed at 1/120th of the miss price. The 165M tokens of stale re-reading come to about $0.60.
The tokens the gate removes were nearly free.
At cache-miss prices throughout, the ungated arm would cost $82.65 and the gated one $56.06.
That is a 1.47× saving.
Interpolating between the measured run and that no-cache hypothetical puts the crossover at roughly an 83% cache-hit rate. This run sat at 89%.
On a well-cached agent with a 99% cache discount, stale context is almost free to re-read, and a gate has to justify itself some other way.
### Latency
I did not instrument detailed timing during the runs. I did keep the agent traces, with timestamps, and each gate's call counters.
Every latency number below is a median, because API dropouts contaminate individual timings.
On serial single-thread runs, the median query takes 83 seconds ungated and 165 seconds gated, twice as long.
Per-query wall-clock on serial single-thread runs (flash agent, 14 baseline and 16 gated samples), reconstructed from run persist timestamps. Hover a band for the numbers. The flash-gate tail runs well past its band, topping out at 727 s.
The oracle arm applies the same gating semantics through a zero-cost label lookup, and it is *faster* than the baseline, a median of 66 seconds per query to the baseline's 83.
Admitting gold spans early ends queries in fewer searches.
The mechanism is one blocking model call per search, on the critical path by construction. The agent's next turn conditions on the gated results, so there is no async escape.
The gate memoizes decisions per document, but reformulated searches mostly surface new documents. In practice it fires 17–22 times per query, at 7–10 seconds a call.
Those seconds are mostly decode. Spans-only means the gate writes the evidence out verbatim.
That is about 800 completion tokens per call for the flash gate, 1,900 for pro.
Spans-only bounded the context window, and spans-only decode is most of the added latency.
### Cutting the overhead
Three levers:
1. **Emit character offsets, not verbatim spans.** The shim already holds the document text. Let the gate return `start:end` and slice locally. That cuts the decode from ~800 tokens to under 100. Per call, an estimated 8 seconds drops to 1–2. This needs a gate model competent at emitting exact offsets. Smaller models struggle with it.
2. **Gate documents in parallel.** One batched call decodes every span serially, while k single-document calls decode only as long as the slowest one. Composes with the first lever.
3. **A purpose-built small gate model.** Distill the gate's decisions into a 0.6–8B cross-encoder-style model served next to the agent: sub-second per search, no API round-trip. The agent×gate grid already showed gate capability is not the binding constraint. The flash gate ties the pro gate under a pro agent. Ding et al. see the same shape: robustness to retrieval noise rises with agent strength [+robustrag].
## Where the losses come from
I read the trajectory of every query a gated arm lost against the baseline, with the gate decisions alongside.
(One direction only: the queries the gated arms *won* have not had the same close read.)
Losses against the ungated baseline, classified from the trajectories and gate decisions. Hover a bar for the query IDs.
The failure mode specific to gating is **trajectory steering**.
The gate withholds a document early, and the agent's later query reformulations drift away from the gold documents.
On one query the baseline retrieved all three gold documents. The gated agent issued 56 search calls across its 30 iterations and retrieved none of them.
A per-search gate cannot know its decision will derail the search three steps later.
This affects 8 of 180 queries, 4.4%.
I had been preparing the training side of this idea: distill the gate into a small open-weights model, then RL with trajectory-level reward to fix this steering.
The census shelved the RL half. Steering affects 4.4% of queries, so at best RL recovers half an accuracy point. That gain sits entirely inside the ±6-point noise band.
Distillation is still worth doing, but only for the 7–10 seconds per gate call.
The census also showed the largest failure bucket has nothing to do with gating.
The agent holds the right documents, hits the 30-iteration cap without committing to an answer, and emits a raw tool-call fragment.
On the oracle arm this accounts for about 9 of its 15 losses.
For accuracy on this benchmark, termination handling is worth more than any gate improvement.
## One benchmark, one retriever, one judge
Everything above is measured on one setup. BrowseComp-Plus queries are multi-hop questions over a fixed 100K-passage corpus, and the 180 firing queries are the subset where the stock hybrid retriever surfaces labeled evidence. deepseek-v4-flash judged every arm, one trajectory per query, and at this sample size the error bars are about ±6 points, wide enough to swallow every accuracy difference between arms. I did not vary the gate prompt, so its sensitivity to wording is unmeasured. And nothing here says how the gate behaves on an unlabeled corpus, where no annotation ever checks its withhold decisions.
## Takeaways
The gate holds up as compression, 1.4× fewer input tokens with no training.
The ink paper reaches the same verdict from a controlled direction. In its filtering experiments, the gain came from the shorter context rather than from which documents were removed [+firstink].
It loses on cost here because DeepSeek's cache discount makes stale context nearly free to re-read.
The dollar case needs a setting where caching is weak or absent.
And as built, it costs about 2× wall-clock latency, most of which the levers from the latency section would remove. I have not built them yet.
### When to use a gate
- **Cache-hit rate under roughly 83%.** Below that crossover the gate saves money as well as tokens. Multi-turn products that interleave user messages between searches, and serving stacks without prefix caching, sit well under it.
- **The context window binds before the budget does.** Spans-only admission caps what each search can add, so the context stays bounded however long the agent runs.
- **A few extra seconds per search are acceptable.** Each gate call blocks for 7–10 seconds as built. Character offsets and parallel calls remove most of that, but only after you build them.
- **Never for accuracy.** No configuration beat the ungated baseline, and on this benchmark termination handling is worth more than any gate improvement.
[+buzzwords]: Lester Solbakken. "When better retrieval makes agents worse." Berlin Buzzwords 2026. [Talk recording](https://youtu.be/07kARdSIjVI). Lester builds [Hornet.dev](https://hornet.dev).
[+firstink]: Gao, Chen, and Huang. 2026. "The First Drop of Ink: Nonlinear Impact of Misleading Information in Long-Context Reasoning." The paper behind the talk's ink slide: a small fraction of hard distractors causes most of the degradation, and filtering gains come mainly from context-length reduction rather than distractor removal. https://arxiv.org/abs/2605.10828
[+browsecomp]: Chen et al. 2025. "BrowseComp-Plus: A More Fair and Transparent Evaluation Benchmark of Deep-Research Agent." ACL 2026. OpenAI's BrowseComp queries (Wei et al. 2025, https://arxiv.org/abs/2504.12516) rehosted over a fixed 100K-document corpus with labeled gold and evidence documents, indexed with BM25 and Qwen3 embeddings. https://arxiv.org/abs/2508.06600 · [dataset](https://huggingface.co/datasets/Tevatron/browsecomp-plus-corpus)
[+robustrag]: Ding et al. 2025. "On the Diminishing Returns of Complex Robust RAG Training." SIGIR-AP 2025. The robustness benefit of defending an LLM against noisy retrieval shrinks substantially as model capacity grows. https://arxiv.org/abs/2502.11400
[+pricing]: DeepSeek [price sheet](https://api-docs.deepseek.com/quick_start/pricing) as of July 2026, per 1M tokens. v4-flash: $0.14 input (cache miss), $0.0028 input (cache hit), $0.28 output. v4-pro: $0.435 / $0.003625 / $0.87. Arm costs computed from each run's measured cached/uncached/output token counts. Gate tokens are priced as cache misses, which makes the gate's cost a slight upper bound.