当学习式上下文规划未能胜过强检索:针对长上下文问答中规划、路由与重排序的对照研究
英文标题:When Learned Context Planning Fails to Beat Strong Retrieval: A Controlled Study of Planning, Routing, and Reranking for Long-Context QA
该研究考察了学习式上下文规划在强检索、路由、预算选择器和重排序等严格对照条件下,是否能提升长上下文多项选择问答表现。基于 LongBench-v2 的全部 503 道 MCQ 题目和 Qwen2.5-7B-Instruct,在 1.8 万字符预算下,锚定混合检索准确率为 36.18%,BM25 为 35.98%,最佳直接规划引导方法仅为 34.19%。在未接触的 152 题测试集上,锚定混合检索仍以 42.11% 对 36.84% 领先。研究认为,在该实验设置下,学习式规划只能提供较弱的相关性信号,尚不能替代强大的检索系统。
640 B
612 B
380 B
9.1 KB
32.7 KB
22.8 KB
24.2 KB
34.7 KB
中文
[提交于 2026 年 9 月 22 日]
标题:当学习式上下文规划未能胜过强检索:针对长上下文问答中规划、路由与重排序的对照研究
查看 Yingrui Li 及其他 1 位作者撰写的论文《当学习式上下文规划未能胜过强检索:针对长上下文问答中规划、路由与重排序的对照研究》的 PDF
摘要:学习式上下文规划会在回答模型进行推理之前选择证据原子。我们测试了在纳入强检索、路由、预算选择器和重排序对照后,这种学习式选择是否能够提升长上下文多项选择问答(QA)的表现。我们的主要诊断使用了 LongBench-v2 的全部 503 道 MCQ 题目以及 Qwen2.5-7B-Instruct。规划器基于从 140 道训练题和 28 道开发题中按结果筛选的轨迹进行监督微调(SFT)训练;由于这 503 题分析包含了这些题目,因此该分析部分具有传导式(transductive)特征。在 1.8 万字符预算下,锚定混合检索的准确率达到 36.18%,BM25 达到 35.98%,而最佳的直接规划引导方法仅达到 34.19%。在未接触的 152 道测试集上,锚定混合检索依然更高(42.11% 对 36.84%)。在防止信息泄漏的条件下,路由器无法将较大的预言机(oracle)差距转化为实际收益。在紧预算下,最佳规划器仅在 6000 字符预算时领先 0.40 个百分点,并在 9000 字符预算时落后;规划器引导的重排序在 6000 字符预算下估计有 +1.79 个百分点的提升,但其配对区间跨越零点,并且在 9000 字符预算下与对照方法持平。打包顺序和分数平坦度分析均未发现稳定的作用机制。在这一实验设置下,学习式规划是一种较弱的相关性信号,而非强检索的替代方案。
提交历史
发件人:Han Chen [
]
[v1]
2026 年 9 月 22 日,星期二,19:12:16 UTC(11 KB)
English / 原文
[Submitted on 22 Sep 2026]
Title:When Learned Context Planning Fails to Beat Strong Retrieval: A Controlled Study of Planning, Routing, and Reranking for Long-Context QA
View a PDF of the paper titled When Learned Context Planning Fails to Beat Strong Retrieval: A Controlled Study of Planning, Routing, and Reranking for Long-Context QA, by Yingrui Li and 1 other authors
Abstract:Learned context planning selects evidence atoms before an answer model reasons over them. We test whether this learned selection improves long-context multiple-choice QA after strong retrieval, routing, budgeted-selector, and reranking controls. Our primary diagnostic uses all 503 LongBench-v2 MCQ questions with Qwen2.5-7B-Instruct. The planner is SFT-trained on outcome-selected traces from 140 training and 28 development questions; because the 503-question analysis includes those questions, it is partly transductive. At an 18k-character budget, anchored hybrid retrieval reaches 36.18% accuracy and BM25 reaches 35.98%, while the best direct planner-guided method reaches 34.19%. On the untouched 152-question test split, anchored hybrid remains higher (42.11% versus 36.84%). Leakage-safe routers cannot convert a large oracle gap. Under tight budgets, the best planner is ahead by only 0.40 points at 6k and loses at 9k; planner-guided reranking has a +1.79-point estimate at 6k with a paired interval crossing zero and ties the control at 9k. Packing-order and score-flatness analyses did not identify a stable mechanism. Under this setup, learned planning is a weak relevance signal rather than a replacement for strong retrieval.
Submission history
From: Han Chen [
]
[v1]
Tue, 22 Sep 2026 19:12:16 UTC (11 KB)