Agentic Search Lost to a Vector Index

The direction of travel for the last year has been to replace retrieval with agents. Rather than embed a corpus and query it, you give an agent a planner, some subagents and a set of tools, and let it go and look. RAG in plain english covered the older approach here; the newer one has been widely assumed to supersede it.

An empirical study published 2 August 2026 tested that assumption on repository-level code question answering and found the opposite.

The result

*Deep Agentic Search for Repository-Level Code Question Answering: An Empirical Study* (arXiv 2608.01507) compared the two approaches on the SWE-QA benchmark.

| approach | correct answers | | --- | --- | | Semantic search | 65.2% | | Deep agentic search | 46.2% |

Semantic search was also correct at less than half the cost.

That is not a narrow win. Nineteen points of accuracy and better than a two-to-one cost advantage, in favour of the approach the field has been treating as legacy.

Where the agent lost

The failure taxonomy is more useful than the headline. The largest single failure category, 41.8 percent, was not bad retrieval, not a missing file, not a context limit. It was the handoff between the planner and the subagent.

And the characteristic failure mode was not a crash or a refusal. The paper describes it as producing a fluent and confident answer that was wrong.

This should be familiar to anyone who has run a multi-agent pipeline. Each hop is a lossy re-encoding. The planner compresses the question into an instruction, the subagent interprets that instruction without the original context, and returns something that answers the instruction rather than the question. Nothing errors. The result reads well. It is about the wrong thing.

The authors are careful, and their conclusion is worth stating precisely: deep agentic search addresses a genuine problem with context pollution, but for read-only questions over a repository that can be indexed, retrieval was the stronger and cheaper option.

Read the conditions. Read-only. Indexable. Those are the boundaries of the result, and they happen to describe an enormous share of what people are currently building agents to do.

The rule this suggests

If the corpus can be indexed and the question is read-only, index it. Reach for an agent when the work requires acting, when the corpus cannot be indexed ahead of time, or when the question cannot be answered from any fixed set of documents.

Agentic search is not a strictly better version of retrieval. It is a different tool with a higher floor cost and a failure mode that does not announce itself.

Retrieval did not stand still either

Part of why the older approach won is that it kept improving while attention was elsewhere.

voyage-context-4 (29 June 2026) does contextualised chunk embeddings with automatic chunking. Each chunk is embedded with awareness of the document around it, which addresses the oldest complaint about naive chunking: that a chunk stripped of its surroundings loses the thing that made it findable. Pricing dropped from 0.18 to 0.12 dollars per million tokens.

jina-reranker-v3.5 (3 August 2026) is a 0.6 billion parameter listwise reranker matching a 4 billion parameter model. Listwise matters: it scores candidates against each other rather than one at a time, which is what you actually want when you are picking the top few from a shortlist. At that size it is cheap enough to sit in a hot path.

Neither is exciting. Both make the boring option better, which is exactly what the empirical result rewards.

The related finding about tools

There is a version of this same lesson on the tool side, from *Hybrid Semantic Tool Discovery for Enterprise MCP Gateway* (arXiv 2608.23992, 25 August 2026).

Loading every tool definition into context consumed 140,200 tokens, about 70.1 percent of the available context. Retrieving only the relevant tool definitions brought that to 1,300 tokens.

Same principle. Do not walk the agent through everything you have; index it and fetch what is needed. Note also that this is exactly the pressure behind the cacheable list results in the 2026-07-28 MCP specification. The protocol and the research arrived at the same problem from opposite ends.

And a warning about agent memory

If the answer to retrieval-versus-agents is often "index it," the next question is what happens to what the agent remembers. Two papers this month say: less than you would hope.

arXiv 2609.08258 (8 September 2026) tested five agent memory systems and found that none of them enforce revocation markers by default. If you tell an agent to forget something, the marker is recorded and, in the default configuration, not honoured on read. Treat agent memory as append-only unless you have verified otherwise in your own stack, and do not put anything in it you might later be obliged to remove.

arXiv 2609.05339 (4 September 2026) found that fixed-schema memory survives model upgrades while compressed free-text notes are highly model-coupled. Notes summarised by one model degrade when a different model reads them. If your memory layer stores prose written by whatever model was current, a model upgrade quietly damages it, and nothing in your monitoring will tell you.

The structured, boring option wins again. That is three for three in this post, which is probably the actual lesson.