Information Retrieval Misses the Mark for LLM Agents
SSRN preprint (position paper), 2026 · DOI 10.2139/ssrn.6903579
Abstract
Current widely deployed LLM agents often retrieve information using simple rule-based tools, such as grep, rather than learned encoders, vector indices, or rerankers; industry commentary has captured this shift with the slogan "RAG is dead, agents just need grep." Although we reject the claim that retrieval is obsolete, we argue that existing information retrieval (IR) research cannot yet comprehensively satisfy the information needs of LLM agents. The retrieval substrate of the agent era is real and deployed, yet it is poorly captured by standard IR formulations and benchmarks. This misalignment is structural across five dimensions: today's IR stack assumes the wrong corpus, the wrong input, the wrong objective, the wrong episode, and the wrong retrievable for agents that plan, browse, call tools, manage memory, and decide whether to continue searching. We argue that the productive next step is to reformulate retrieval for agents as a state-conditioned evidence-acquisition policy. We organize existing agentic RAG and reasoning-search systems under a taxonomy and a formal model, run a direct test of the misalignment by holding the agent constant and varying only the retrieval substrate (BM25, vector, grep, hybrid, closed-book) on HotpotQA-distractor and 2WikiMultihopQA over a 100K Wikipedia corpus, and propose a five-direction research agenda with dual-track evaluation.
Full text: https://www.dukesun99.link/papers/ir-misses-the-mark.pdf (PDF)