AI 智能体并非孤立失败:上下文首先失效
英文标题:AI Agents Do Not Fail Alone:The Context Fails First
上下文工程是构建可靠 AI 智能体的核心,但其质量长期缺乏量化测量。本研究通过 ProofAgent-Harness 开源评估框架,采用多陪审员共识评分,从角色清晰度、护栏覆盖等七项标准独立测量上下文质量,并隔离行为指标进行非循环验证。在固定前沿 LLM 智能体、仅改变操作上下文的对照实验中,上下文质量标准可一致预测抗幻觉、抗操纵、指令遵循及工具使用等行为结果。该工作将上下文测量确立为智能体可靠性的预检信号,并将上下文工程定位为可审计的评估与治理层,对开发者构建安全可靠的智能体系统具有重要指导意义。
640 B
612 B
380 B
9.1 KB
32.7 KB
22.8 KB
24.2 KB
34.7 KB
中文
[提交于 2026 年 7 月 15 日]
标题:AI 智能体并非孤立失败:上下文首先失效
查看论文 PDF:AI 智能体并非孤立失败:上下文首先失效,作者 Fouad Bousetouane
摘要:上下文工程已成为构建可靠 AI 智能体的核心,但其质量在很大程度上仍未得到测量。智能体并非孤立地失败:它们的行为由指令、工具、记忆、检索到的知识、护栏以及在其上下文中积累的不可信输入所塑造。当这种上下文薄弱时,智能体会发生漂移、产生幻觉、误用工具、忽视约束、容易受到注入攻击并浪费令牌。本文验证了上下文工程的质量可作为智能体可靠性的独立先行指标。我们在 ProofAgent-Harness 中实现了这一测量,这是一个用于 AI 智能体评估的开源基础设施,采用多陪审员、共识驱动的评分方式。该测试框架从七个标准评估上下文:角色清晰度、护栏覆盖、指令一致性、工具模式质量、事实依据充分性、注入防护和令牌效率。至关重要的是,上下文评分与行为指标及发布决策相隔离,从而实现了非循环验证。通过在受监管的智能体领域开展的一项受控上下文质量研究,我们固定前沿 LLM 智能体,仅改变其操作上下文,结果表明上下文质量的标准能够一致地预测其相应的行为结果。事实依据充分性可预测抗幻觉能力,护栏覆盖可预测抗操纵能力,指令一致性可预测指令遵循能力,工具模式质量可预测工具使用能力。这些发现将上下文测量确立为智能体可靠性经过验证的预检信号,并将上下文工程定位为智能体评估与治理中一个可审计的层面。
提交历史
来自:Fouad Bousetouane [
]
[v1]
2026 年 7 月 15 日星期三 18:33:02 UTC (1,097 KB)
English / 原文
[Submitted on 15 Jul 2026]
Title:AI Agents Do Not Fail Alone:The Context Fails First
View a PDF of the paper titled AI Agents Do Not Fail Alone:The Context Fails First, by Fouad Bousetouane
Abstract:Context engineering has become central to building reliable AI agents, yet it remains largely unmeasured. Agents do not fail in isolation: their behavior is shaped by the instructions, tools, memory, retrieved knowledge, guardrails, and untrusted inputs accumulated in their context. When this context is weak, agents drift, hallucinate, misuse tools, ignore constraints, become vulnerable to injection, and waste tokens. This paper validates context-engineering quality as an independent leading indicator of agent reliability. We implement the measurement in ProofAgent-Harness, an open-source infrastructure for AI agent evaluation that uses multi-juror, consensus-based scoring. The harness assesses context across seven criteria: role clarity, guardrail coverage, instruction consistency, tool schema quality, grounding sufficiency, injection hardening, and token efficiency. Crucially, the context score is isolated from behavioral metrics and release decisions, enabling a non-circular validation. Through a controlled context-quality study across regulated agent domains, holding frontier LLM agents fixed and varying only their operating context, we show that context-quality criteria consistently predict their corresponding behavioral outcomes. Grounding sufficiency predicts hallucination resistance, guardrail coverage predicts manipulation resistance, instruction consistency predicts instruction following, and tool-schema quality predicts tool use. These findings establish context measurement as a validated preflight signal for agent reliability and position context engineering as an auditable layer of agent evaluation and governance.
Submission history
From: Fouad Bousetouane [
]
[v1]
Wed, 15 Jul 2026 18:33:02 UTC (1,097 KB)