RAG 检索瓶颈转移：从“找得到”走向“选得出”
英文标题：🔶 Retrieval's bottleneck is moving from finding to selecting
随着 Embedding 模型演化、查询重写以及 Agent 循环检索等技术的普及，RAG（检索增强生成）系统在海量语料中召回候选内容的“查找（Finding）”能力已大幅提升。Kapa.ai 针对企业知识库基准的实测表明，当前检索失败的原因有一半发生在搜索之后，在搜索已经成功定位的“黄金切片”（Gold Chunks）中，有高达 34% 在重排（Rerank）筛选环节被错误遗弃。一味堆砌切片换取召回率会导致显著的上下文腐化与成本激增，这意味着 RAG 与 Agent 系统的核心竞争点已从单纯的前端召回，转向高效、精准的上下文切片筛选。
106.7 KB
2.5 KB
118.5 KB
107.8 KB
575 B
5.6 KB
300 B
71.6 KB
中文
对于给定的查询，检索器（Retriever）的任务是从语料库的文档中找到最相关的切片（Chunks）。现代检索器通常至少执行两步操作：首先，从语料库中**查找（Find）出潜在候选池并进行召回；然后，从候选池中筛选（Select）**出最相关的切片。例如，经典 RAG 中的常见做法是通过混合搜索（BM25 + 向量搜索）召回候选切片，随后借助专用的重排器（Reranker）精细化筛选，仅返回 top-k 个结果。
直到最近，检索中最主要的失败模式始终是“找不到正确的候选”。换言之，检索器召回的候选往往根本不包含下游任务所需的正确切片。在过去一年左右的时间里，这一差距已被大幅弥合：Embedding 模型显著增强，LLM 如今极擅长在搜索前进行查询重写与分解，且 Agent 检索器（Agentic Retrievers）能够迭代搜索直至收集到充分的信息。
这引发了一个新假说：检索的瓶颈正从“查找”转向“筛选”。 搜索现在已经能够浮现出查询所需的大部分内容，而信息损失发生在后续阶段——即系统管线决定将哪些已召回切片呈交给模型之时。本文通过追踪我们 企业知识评测基准（Company Knowledge Bench） 问题中每一个“黄金切片”（Gold Chunk，即完整、正确回答所必需的切片）在生产级检索管线中的流转过程，对这一假说进行了验证。这种转变已然发生：搜索虽然仍会遗漏部分黄金切片，但在它确实召回的内容中，有近三分之一最终从未展示给模型。
查找能力大幅提升
查找能力进步最清晰的印证莫过于代码搜索。模型在使用 grep 方面已经变得非常熟练，以至于语义搜索路径对编程 Agent 而言已不再带来显著助益。在一个高能力模型驱动下的词法匹配工具，如今已足以在大型代码库中自如穿梭。
然而，grep 本身不进行任何筛选。它会返回所有匹配项，因此所有的过滤都转移到了模型的上下文窗口之内。筛选问题并未消失，只是被转移到了模型的上下文之中。此外，代码是一个特例：它语法严格，标识符特征明显且处处拼写一致，无需考虑拼写错误或同义词，因此高能力模型可以不断微调精确模式，直到定位到正确文件。这也是为什么 grep 对 Kapa 而言远远不够——Kapa 为从开发者工具到半导体等众多领域的科技公司提供服务。企业知识分散在文档、工单、论坛和发布说明中，各自拥有独立的话语体系，而一个问题可能同时触及其中多个来源。在庞大且异构的语料库中，字面匹配返回的内容可能会远远超出模型的上下文承载能力，且同一个概念可能会以任何模式都无法预料的方式被表述或拼错。
黄金切片都流向了哪里
在我们的企业知识基准中，每个测试用例均由一个用户查询以及完整回答所必需的切片构成——即黄金切片（Gold Chunks）。跟踪黄金切片在检索管线中的流动，可以清晰揭示检索失败发生的位置。
在本次分析中，我们考虑一个包含两个模型的简化管线：一个是负责搜索语料库并挑选保留哪些切片的检索器（Retriever）；另一个是依据最终切片集合撰写回复的回答模型（Answer Model）。这里我们参考了类似 重排器评测分析 中的检索模型设置，使用 GPT-6 Luna 进行查询分解，并使用 Voyage-3 对召回的切片进行排序。对于每个查询，我们检索 50 个切片，然后将最多 100 个切片输入重排器，由其返回 top 15。该配置仅用于演示目的，后续我们将讨论更贴近实际的 Agent 检索器。正如我们在 博客文章 中所解释的，某些黄金切片属于可互相替代的备选项，为简便起见我们在此予以忽略。
黄金切片未能传递给回答模型的原因通常有两个：
- 检索器从未找到该黄金切片（即重排器根本没有机会将其选为 top 切片）。
- 重排器看到了该黄金切片，但判定其相关性不足，未能将其排入 top-k 切片中。
第一类错误意味着检索器在“查找”黄金切片时遇到困难；第二类错误则意味着它在“筛选”黄金切片时遇到了困难。从历史上看，查找往往远难于筛选。大约 18 个月前，约 80% 的失败评测案例是因为黄金切片从未进入重排器，仅有 20% 是由重排器自身导致的。而在今天，失败测试用例中约有一半归咎于搜索；另一半则发生在搜索之后——重排器为回答模型挑选了错误的切片。从这项分析中我们可以得出一个明确的结论：检索的难点正从“查找”转移到“筛选”。
以切片数量来统计，这一现象更为鲜明：在搜索找到的所有黄金切片中，有 34% 从未展现给 Agent 检索器。本文后续将这一比例称为筛选差距（Selection Gap）。
被隐藏的黄金切片并不一定深埋在重排候选池的底部。有些确实如此，但另一些在搜索结果中排名其实很靠前，只是排序还不足以挤进最终选拔线。这促使人们很容易想到一个显而易见的解决办法：直接向 Agent 检索器展示更多内容。
你可以用 Token 购买召回率
向回答模型呈现更多切片显然是有效的。例如，每次搜索返回的结果从 top 5 增加到 top 10，能让回答模型接收到黄金切片的问题比例提升 14%。然而，其边际收益递减非常迅速。遗漏的黄金切片分布在 k 较小时较为密集，但越往下越收缩为长尾；与此同时，排名每往下一步，都会向回答模型宝贵的上下文窗口注入相同体量的文本。精确率（Precision，即返回切片中回答实际所需的比例）下降的速度与召回率（Recall）上升的速度一样快，而每一个额外的切片都是回答模型不得不阅读的文本负担。
“直接让回答模型忽略噪声”
一种简单的思路是直接让回答模型去处理这一切。SID 在其 技术报告 中给出了返回更多内容的论据：能力强劲的回答模型可以轻松应对少量无关文档，却无法凭空恢复缺失的文档，因此“过度召回”比“召回不足”更安全。缺失切片确实比多余切片更致命，但召回优先绝不意味着不惜一切代价追求召回：
- 每个切片都需要付出成本。 在 Kapa 的助手中，召回切片占单次查询成本的约三分之二（详见我们的 上下文剪枝文章）。
- 模型并非拥有无穷能力。 随着输入体量增加，准确率会出现下滑；当多余文本与模型寻找的内容高度相似时，下降速度更快（Liu et al., 2023；Chroma, 2025）。
- 最具破坏性的噪声往往看起来极其相关。 过时的页面或另一款产品中的相似特性，恰恰最容易被重排器推向顶端。
因此，当今的检索器往往被迫在召回率与精确率之间寻求权衡。这种权衡是信息检索领域最古老的发现之一，可以追溯到 1950 年代后期。当时，Robert Fairthorne 在《分类的委派》（Delegation of Classification）中就已经指出了两个极端：系统可以返回“包含全部但不限于”（all but not only）相关文档，也可以返回“仅包含但不涵盖全部”（only but not all）相关文档。做到“全面且精准”（all and only）才是真正的难点，而单纯检查黄金切片是否被召回的指标根本无法区分这两者。
尽管如此，现实中的检索器往往遵循 SID 的逻辑，倾向于滑向第一种极端。例如，在企业知识基准上，使用 grep 进行搜索的 Agent 在召回率方面表现与 Kapa 的深度检索器相当。然而对于 grep 来说，其返回的切片中大约只有二十五分之一是真正必需的；相比之下，Kapa 的深度检索器返回的文本 Token 数量要少 8 倍（详见我们的 博客文章）。
这就是为什么在关注召回率的同时必须考察精确率。单纯的召回率分数奖励的是“返回更多内容”，任何忽视与黄金切片一同携带了多少冗余的对比，本质上都在偏袒返回内容最臃肿的检索器。
Agent 检索器只是部分解决方案
到目前为止，我们仅考察了单次检索管线。多数现代管线使用的是更为复杂的 Agent 检索器，它们能够迭代微调搜索结果。然而，这仅仅是将问题从回答模型的上下文转移到了 Agent 检索器自身的上下文之中，相同的局限性依然成立。使筛选变得困难的本质是一个逻辑闭环：Agent 检索器判断某个切片是否必需的唯一可靠方式是去阅读它，而阅读所付出的正是筛选本身想要节省的上下文。因此，典型的 Agent 检索管线会在 Agent 检索器之前放置一个重排器，在切片进入其上下文之前进行过滤。结果便是：关于 Agent 检索器该读什么的决定，是在它阅读任何内容之前做出的。重排器的架构并非为此类决策设计：它们逐一孤立评估每个切片，而相关性往往属于一个集合——单个切片单独看可能毫无用处，与另一个切片组合在一起时却是必不可少的（参见我们的 上下文剪枝文章）。同时，每个问题所需的正确切片数量也各不相同。于是，筛选需要消耗它本打算节省的上下文，我们陷入了循环论证。
已有方法试图化解这一局限。列表式重排器（List-wise Rerankers）会同时通读候选切片而非单独打分，从而能够识别冗余切片或仅在组合时才生效的切片，但其代价是将阅读开销转嫁给了一个同样有上下文上限的廉价模型。分层方法（Hierarchical Approaches）则选择向 Agent 检索器展示每个切片的简短预览（如标题、摘要或前几行），由其决定完整阅读哪些。此时检索器自行做出裁决，但代价是必须通读所有切片的预览，且每展开一个切片都要增加一次步骤。在此方案下，问题又转移为了如何撰写能够保留切片核心价值的预览。
尽管 Agent 检索器并不完美，但它们在架构设计上提供了极高的自由度，目前在大多数检索基准中名列前茅。Kapa 当前的 Agent 检索方案试图在搜索工具以及向下游模型返回切片的方式上，同时应对从“查找”向“筛选”的重心迁移。大多数检索器返回固定数量的切片，或者采用随 Agent 持续搜索而只增不减（append-only）的集合。相反，Kapa 的深度检索器能够根据问题需要动态返回切片，并剔除事实证明不再需要的切片。在企业知识基准上，该深度检索器的得分高于 grep Agent，同时返回的文本量约为后者的八分之一，精确度高出约六倍（基准评测结果）。然而，它并没有完全消除这种权衡，依然留下了未解的问题：筛选应当归属于检索器还是回答模型？为了更纯净的上下文，放弃多少召回率是值得的？当多套不同切片组合都能回答同一问题时，又该如何对“最小化”建立评测标准？
可以明确的是瓶颈所在维度的演变。搜索现在已经能够定位问题所需的大部分信息。搜索之后遗失的黄金切片，是这一难题中增长最快的部分，也是基于召回率的指标无法捕捉的部分。自此往后的技术演进，将完全取决于高效的筛选机制：保留真金，而不必为其周围的一切沙砾买单。
改进之前，始于度量。构建企业知识基准是迈出的第一步，因为它让追踪管线中的黄金切片成为可能。而接下来的挑战，就是牢牢留住它们。
English / 原文
For a given query, the task of a retriever is to find the most relevant chunks from documents in a corpus. Modern retrievers usually perform at least two steps. First, they find a pool of likely candidates which they retrieve from the corpus. Then, they select the most relevant chunks from the candidates. For example, a common approach in classic RAG is to retrieve candidate chunks via hybrid search (BM25 + vector search) and then refine the selection via a specialized reranker, returning only the top-k.
Until recently, the biggest failure mode in retrieval was finding the right candidates. That is, the retriever would often end up with candidates that made it impossible to select the right chunks for the downstream task. Over the past year or so, much of this gap has closed: embedding models have improved, LLMs are now good at rewriting and decomposing queries before searching, and agentic retrievers search iteratively until they have gathered enough.
This suggests a hypothesis: the bottleneck in retrieval is moving from finding to selecting. Search now surfaces most of what a query needs, and the loss happens afterwards, when the pipeline decides which retrieved chunks the model gets to see. This post tests that hypothesis by tracing every gold chunk (a chunk that a complete, correct answer actually needs) through a production retrieval pipeline on questions from our Company Knowledge Bench. The shift is already under way: search still misses some of the gold chunks, but of the ones it did surface, almost a third were never shown to the model.
Finding got much better
The clearest indication of how far finding has come is code search. Models have become good enough at using grep that the semantic search path no longer helps coding agents in a meaningful way. A lexical tool, driven by a capable model, is now sufficient to navigate large codebases.
Grep, however, does no selection. It returns every match, so all filtering happens inside the model's context window. The selection problem doesn't go away; it just moves into the model's context. Further, code is a special case: it has a strict syntax, identifiers are distinctive and spelled the same way everywhere, with no typos or synonyms to account for, so a capable model can refine exact patterns until it finds the right file. This is also why grep isn't enough for Kapa, which serves technical companies across many fields from dev tools to semiconductors. Company knowledge is spread across docs, tickets, forums, and release notes, each with its own vocabulary, and one question can touch many of them. In a large, varied corpus, a literal match can return more than a model can take in its context, and the same concept may be phrased, or misspelled, in ways no pattern anticipates.
Where the gold goes
In our Company Knowledge Bench each test case consists of a user query and the chunks a complete answer requires: its gold chunks. Following the gold through a retrieval pipeline shows where retrieval failures happen.
For our analysis, we consider a simplified pipeline which involves two models: a retriever which searches the corpus and picks which chunks to keep, and an answer model which writes the response from that final set. Here we consider a retriever model similar to the one in our reranker analysis, which uses GPT-6 Luna for query decomposition and Voyage-3 for ranking the retrieved chunks. For each query, we retrieve 50 chunks and then feed up to 100 chunks into the reranker which returns the top 15. This setup is only for illustrative purposes; we will talk about more realistic agentic retrievers later. As we explain in our blog post, some gold chunks are interchangeable alternatives; we will ignore this here for the sake of simplicity.
There can be two reasons why a gold chunk didn't make it to the answer model:
-
The retriever never found the gold, i.e. the reranker could not select it as a top chunk.
-
The reranker saw the gold but decided it was not relevant enough to make it into the top-k chunks.
The first error means the retriever had trouble finding the gold, the second error means it had trouble selecting the gold. Historically, finding was always much harder than selecting. Roughly 18 months ago, about 80% of the failed eval cases were caused by the gold never entering the reranker, and only 20% by the reranker itself. Today, about half of the failed test cases are caused by search. The other half fail after search, when the reranker selected the wrong chunks for the answer model. The insight we can get from this analysis is simple: the difficulty in retrieval is shifting from finding to selecting.
Counted by chunk, the picture is sharper. Of all the gold chunks search found, 34% were never shown to the agentic retriever. The rest of this post calls this share the selection gap.
The hidden gold chunks are not necessarily buried deep in the reranked pool. Some are, but others ranked well in the search results, just not well enough to make the cut. That makes the obvious remedy tempting: show the agentic retriever more.
You can buy recall with tokens
Showing more chunks to the answer model obviously works. For example, going from the top 5 to the top 10 chunks per search increases the share of questions where the answer model receives the gold chunks by 14%. However, the returns shrink quickly. The distribution of missed gold chunks may be dense for low k but thins into a long tail further down, while every step down the ranking adds the same amount of text to the answer model's valuable context window. Precision, the share of returned chunks the answer actually needs, falls as fast as recall rises, and every extra chunk is text the answer model has to read.
âJust let the answer model ignore the noiseâ
A simple solution would be to just let the answer model deal with it. A case for returning more comes from SID: their technical report argues that a capable answer model can easily cope with a few irrelevant documents but cannot recover a missing one, so over-reporting is safer than under-reporting. A missing chunk is indeed worse than an extra one. But recall first is not recall at any price:
-
Every chunk is paid for. Retrieved chunks make up about two-thirds of the cost of a query in Kapa's assistants (see our pruning post).
-
Models are not infinitely powerful. Accuracy drops as inputs grow, and faster when the extra text resembles what the model is looking for (Liu et al., 2023; Chroma, 2025).
-
The worst noise looks relevant. An outdated page or a similar feature in another product is exactly what a ranker places near the top.
Therefore, today's retrievers are often forced to balance recall and precision. This trade-off is one of the oldest findings in information retrieval, dating back to the late 1950s. Back then, in "Delegation of Classification", Robert Fairthorne had already named the two ends: a system can return all but not only the relevant documents, or only but not all of them. Getting all and only is the hard part, and a metric that only checks whether the gold was retrieved cannot tell the two apart.
Nevertheless, real retrievers often follow SID's argument and tend to drift toward the first case. For example, on Company Knowledge Bench, an agent that searches with grep performs comparably to Kapa's deep retriever when it comes to recall. However, for grep, only about one in twenty-five of the chunks it returns is needed, while Kapa's deep retriever returns eight times fewer text tokens (see our blog post).
This is why one needs to look at precision next to recall. A recall score on its own rewards returning more, so any comparison that ignores what came along with the gold favors the retriever that returns the most.
Agentic retrievers are only a partial solution
So far we have only looked at a simple one-shot retriever. Most modern pipelines use sophisticated agentic retrievers, which iteratively refine search results. However, this merely shifts the problem from the answer model's context to the agentic retriever's context and the same limitations apply. What makes selection hard is a loop. The only reliable way for the agentic retriever to judge whether a chunk is needed is to read it, and reading costs exactly the context that selection is meant to save. For this reason, typical agentic retriever pipelines put a reranker in front of the agentic retriever that filters the chunks before they enter its context. As a result, the decision about what the agentic retriever reads is made before it has read anything. Rerankers are not built for that decision. They judge each chunk on its own, while relevance often belongs to a set: a chunk can be useless alone and essential next to another (see our pruning post). And the right number of chunks differs for every question. So the selection needs the context that it is meant to save, and we have completed the circular argument.
There are methods that try to address this limitation. List-wise rerankers read the candidates together rather than scoring each one alone, so they can account for chunks that are redundant or only useful in combination, but they do so by handing the reading to a cheaper model with its own context limit. Hierarchical approaches instead show the agentic retriever a short preview of each chunk, such as a title, a summary or its first lines, and let it decide which ones to read in full. The retriever then judges for itself, at the cost of reading a preview of everything and an extra step for every chunk it opens. Here the problem shifts to writing previews that keep what makes a chunk useful.
Even though agentic retrievers aren't perfect, they offer a lot of freedom in their design and currently top most retrieval benchmarks. Kapa's current approach to agentic retrieval tries to address this shift from finding to selecting both in its search tools and in how it returns the chunks to the downstream model. Most retrievers return a fixed number of chunks, or a set that only grows as the agent keeps searching (append-only). Kapa's deep retriever instead returns as many chunks as a question needs, and can drop chunks that turn out not to be needed. On Company Knowledge Bench, the deep retriever scores higher than the grep agent while returning about an eighth of the text, at about six times the precision (benchmark results). However, it does not fully remove the trade-off, and open questions remain: does selection belong to the retriever or to the answer model? How much recall is worth giving up for a cleaner context? How can minimality be benchmarked when several sets of chunks can answer the same question?
What is clear is where the gap has moved. Search now finds most of what a question needs. The gold lost after search is the part of the problem that is growing fastest, and the part that recall-based metrics cannot see. Progress from here depends on efficient selection: keeping the gold without paying for everything around it.
Before improvement comes measurement. Building Company Knowledge Bench was the first step, because it made it possible to trace the gold through the pipeline. What's left is keeping it.
Follow the blog for more findings as the work continues, or reach out with questions.