不仅要事实正确，更要来源正确：面向 MCP Agent 的来源感知验证

使用工具的 LLM Agent 已不再只读取单一检索段落。通过 Model Context Protocol（MCP），Agent 可以调用搜索工具、查看结构化患者或账户记录、查询数据库并提取元数据，随后将这些信息编织成一个答案。这使得通常关于事实性的判断比表面上更加微妙。大多数用于检查 LLM 回答的系统——从 RAGAS faithfulness 到 MiniCheck、AlignScore 和 SummaC 等细粒度检查器——都会询问：当可用证据被汇集后，某项声明是否得到支持。以其通常形式而言，它们并不能告诉我们，哪一个 MCP 工具输出支持每项声明，或该输出是否正是答案所指明的来源。

我们最新的论文 ProvenanceGuard: Source-Aware Factuality Verification for MCP-Based LLM Agents（可在 Hugging Face 阅读，暂时也可在 arXiv 阅读）旨在弥补这一缺口。我们关注的失败模式称为跨来源混淆（cross-source conflation）：一项声明在证据中的某处确实为真，但被归因于错误的来源。来源盲验证器可能会放行它，因为该事实的确存在于证据池中；来源感知验证器则不应如此。

问题：在某处得到支持，不等于由正确来源支持

设想一个客服 Agent 回答：“根据账户记录，该套餐包含 30 天退款期。”退款期本身可能完全真实，但它写在政策文档中，而非答案所指向的账户记录中。将两者汇集起来，这项声明看起来有证据支持；将它们分开看，归因就是错误的。在数据敏感场景中，错误归因造成的损害可能与事实错误同样严重。相同模式也会出现在临床 Agent 中：一项取自患者病史工具的患者特定用药信息，一旦答案将其表述为医学文献的发现，就会产生误导。

一项声明可能得到某个 MCP 来源的支持，但答案却将其归因于另一个来源。来源盲评分会在汇集后的证据中看到支持并予以放行；ProvenanceGuard 则会单独检查支持性来源是否与答案明确陈述或隐含指向的来源一致。来源：论文图 1。

这正是为什么，尽管 faithfulness（忠实度）评分很有用，但对于 MCP Agent 来说仍不够。一个答案携带着来源信息，有时是明确的（“根据账户记录”），有时则是隐含的。ProvenanceGuard 让声明与来源之间的这种联系始终可供检查。

ProvenanceGuard 的作用

ProvenanceGuard 是位于黑盒 MCP Agent 之上的生成后验证层。它在 Agent 生成答案后运行，绝不会将证据压缩为一个匿名上下文。相反，它在整个管线中持续保留来源身份。它读取捕获的 MCP 轨迹，包括工具输出及其来源 ID，无需重新训练 Agent。随后，它依次执行五项操作：将答案拆分为具体声明；为每项声明找到最相关的来源；检查该来源是否确实支持该声明；将该来源与答案明确指明或隐含指向的来源进行比较；最后输出逐声明的来源判定，以及全局的、答案级别的放行或拦截决定。

验证流程。来源身份会贯穿分解、路由、支持性评分、归因检查和修复环节，而不是被汇集起来。被拦截的答案可经过 RARR 风格的修复并再次验证。来源：论文图 2。

其中一些设计选择值得特别说明。在论文实验中，我们使用本地模型，以便在受控的离线环境中处理捕获的轨迹：MiniLM 用于帮助寻找相关来源；DeBERTa NLI 验证器模型 检查该来源是否支持声明；本地语言模型则帮助将答案拆分为声明。验证器还会严格检查字面值：如果数字、日期或标识符未出现在来源中，仅因句子听起来合理并不能通过验证。经过校准的决策步骤会综合这些信号。如果答案被拦截，RARR 风格的修复步骤可以尝试进行基于来源的修订，或给出安全的回退答复，之后验证器会再次进行检查。

上述具名模型是我们评估时采用的配置，而非 ProvenanceGuard 的硬性要求。相同的声明、来源和决策步骤也可以适配团队更倾向使用云服务的托管模型；新的配置需要经过其自身的测试和校准。我们报告的结果来自本地配置。其保守的决策策略适合数据敏感型审查场景：在这类场景中，来源正确比尽快生成答案更重要。

结果

我们在一个医疗 Agent 生成的答案上测试了 ProvenanceGuard，该 Agent 使用了患者记录、研究论文和其他工具。这为我们提供了 281 条真实轨迹进行研究。医学是一个有用的测试场景，因为来自患者记录的事实和来自通用研究的事实不能被视为同一来源。当 Agent 保留工具输出及来源 ID 的记录时，该方法也可用于其他领域。在主要测试中，人类专家审查了 40 个从系统开发数据中留出的答案中的 361 项声明。

最直接的结果如下：专家认为 139 项声明不应被放行，而 ProvenanceGuard 捕获了其中 138 项，仅放过了 1 项。它还拦截了 67 项被专家认为有证据支持的声明，并将其送交审查或修复。这体现了我们所测试的谨慎设置：它宁可让一些获得支持的声明接受二次检查，也不愿让缺乏支持的声明通过。对于具有可识别来源的声明，在该测试中，它还以约 86% 的准确率选出了正确来源。

我们在相同声明上运行了另外四种支持性检查器。ProvenanceGuard 在论文用于衡量系统捕获应被拦截的声明、同时避免不必要拦截能力的指标上得分最高。此次比较中的其他检查器并不会告诉我们哪一个工具输出支持每项声明。ProvenanceGuard 会记录这种对应关系，因此审查人员可以查看每项声明检查的是哪个来源，以及由此产生的决策。

在相同留出声明数据包上的二元支持性指标。ProvenanceGuard 在拦截表现上与来源盲基线相当或更优，同时还能生成逐声明的来源判定。来源：论文摘要和表 III。

当来源看起来相似时检查声明

在另一项包含多个相似来源、难度更高的测试中，ProvenanceGuard 在决定应拦截哪些声明方面取得了 0.846 的 F1，但仅在 50.3% 的声明中正确识别出精确来源。如何区分相似来源仍然是一个重要的改进方向。

我们还进行了一个聚焦于错误归因的受控测试：在 50 个案例中，我们修改了被点名的来源，同时保持支持性证据不变。ProvenanceGuard 捕获了全部 50 次替换。这表明它能够检测清晰的来源错误；而更困难的测试则表明，在众多看似合理的来源中进行选择仍具挑战。

修复被拦截的答案

只有在能够处理被拦截答案的情况下，拦截才有意义。接入 RARR 风格的修复循环后，完整轨迹运行解决了全部 173 个被拦截答案，但其中 144 个最终以回退文本结束，而非实质性改写；这意味着系统选择避免生成无法验证的答案，而不是虚构一个答案。在重建的多来源测试轨迹上，一次新的修复运行解决了全部 59 个初始被拦截的答案，最终仅有两个回退答案。作为离线门控机制，在报告的本地配置上，其额外开销较小：每个答案约半秒，而 NLI 和路由调用本身只需数十毫秒。

为什么这适合 Multiverse Computing

随着 Agent 从单段落 RAG 迁移到多工具 MCP 架构，“事实实际来自哪个来源”的问题不再只是脚注，而成为事实性定义的一部分。ProvenanceGuard 逐项声明地让这种来源关联变得可见。对于 Multiverse Computing 而言，这意味着可在需要时将敏感轨迹保留在受控环境中，同时对现有 Agent 进行检查。医疗研究只是一个用例；只要 Agent 的轨迹保留了工具及其来源，这种方法就可以进行适配。

这种适配已经体现在 NVIDIA NVFlow 中：该项目为其金融 Agent 合并了一个可选的事实依据验证阶段。它会将完成的答案与 Agent 检索到的 SEC 摘录进行核对，并在不改变原始 rollout 或训练数据的情况下保存独立决策。NVFlow 的贡献采用了 ProvenanceGuard 的来源感知验证方法；上文讨论的修复循环属于更广泛的研究系统。

ProvenanceGuard 还曾以海报形式在 UC Berkeley 举办的 Agentic AI Summit 2026 上展示。

想了解完整技术细节，包括路由和 NLI 推导、校准消融实验、多来源压力测试切片以及完整结果表吗？请在 Hugging Face 阅读完整论文，或联系我们的团队，讨论如何将来源感知验证应用于你自己的 Agent。

Tool-using LLM agents no longer read from a single retrieved passage. Through the

Model Context Protocol (MCP)

, an agent can call a search tool, inspect a structured patient or account record, query a database, and pull metadata, then weave all of it into one answer. That m aakes the usual question of factuality more subtle than it looks. Most of the systems built to check LLM answers, from

RAGAS faithfulness

to fine-grained checkers like MiniCheck, AlignScore, and SummaC, ask whether a claim is supported by the available evidence once that evidence has been pooled together. In their usual form, they do not tell us which MCP tool output supports each claim, or whether that is the source the answer names.

Our latest paper, ProvenanceGuard: Source-Aware Factuality Verification for MCP-Based LLM Agents (read it on Hugging Face, or on arXiv in the meantime), targets that gap. The failure mode we care about is one we call cross-source conflation: a claim that is true somewhere in the evidence, but attributed to the wrong source. A source-blind verifier may pass it, because the fact does exist in the pool. A source-aware verifier should not.

The problem: supported somewhere is not the same as supported by the right source

Consider a customer support agent that answers, "According to the account record, this plan includes a 30-day refund window." The refund window may be perfectly real, but stated in a policy document, not in the account record the answer points to. Pool the two together and the claim looks supported. Keep them separate and the attribution is wrong, and in a data-sensitive setting a wrong attribution can be as damaging as a wrong fact. The same pattern shows up in a clinical agent, where a patient-specific medication detail taken from a patient-history tool becomes misleading the moment the answer presents it as a finding from the medical literature.

A claim can be supported by one MCP source while the answer attributes it to another. Source-blind scoring sees support in the pooled evidence and passes it; ProvenanceGuard separately checks whether the supporting source matches the one the answer states or implies. Source: paper Figure 1.

This is why faithfulness scores, useful as they are, are not enough for MCP agents. An answer carries provenance, sometimes explicitly ("according to the account record") and sometimes implicitly. ProvenanceGuard keeps that connection between claim and source available for inspection.

What ProvenanceGuard does

ProvenanceGuard is a post-generation verification layer that sits on top of a black-box MCP agent. It runs after an agent produces an answer, and never collapses the evidence into one anonymous context. Instead it carries the source identity all the way through the pipeline. It reads the captured MCP trace, including the tool outputs and their source IDs, without retraining the agent. Then it does five things in sequence: it breaks the answer into specific claims, finds the source most relevant to each one, checks whether that source actually supports it, compares the source with the one the answer names or implies, and finally emits both a per-claim source verdict and a global, answer-level allow or block decision.

The verification flow. Source identity is preserved through decomposition, routing, support scoring, attribution checking, and repair, rather than being pooled. Blocked answers can go through RARR-style repair and be re-verified. Source: paper Figure 2.

A few of the design choices are worth calling out. For the experiments in our paper, we used local models so the captured traces could be processed in a controlled, offline setup: MiniLM helps find the relevant source, a DeBERTa NLI verifier model checks whether that source supports the claim, and a local language model helps break answers into claims. The verifier also checks literal values closely: a number, date, or identifier absent from the source cannot pass merely because the sentence sounds plausible. A calibrated decision step combines these signals. If an answer is blocked, a RARR-style repair step can try a source-grounded revision or a safe fallback, which the verifier then checks again.

Those named models are the setup we evaluated, not a requirement of ProvenanceGuard. The same claim, source, and decision steps can be adapted to hosted models where a team prefers cloud services; a new setup would need its own testing and calibration. Our reported results come from the local configuration. Its conservative decision policy suits data-sensitive review, where getting the source right matters more than producing the fastest possible answer.

Results

We tested ProvenanceGuard on answers from a medical agent that had used patient records, research articles, and other tools. This gave us 281 real traces to study. Medicine is a useful test because a fact from a patient's record and a fact from general research cannot be treated as the same source. The method can also be used in other fields when an agent keeps a record of its tool outputs and source IDs. For the main test, human experts checked 361 claims from 40 answers set aside from the data used to develop the system.

The most direct result is this: experts said 139 claims should not pass, and ProvenanceGuard caught 138 of them. It let one through. It also held 67 claims that the experts considered supported, sending them for review or repair. This reflects the cautious setting we tested: it favors a second look at some supported claims over letting unsupported ones through. For claims with an identifiable source, it also picked the right source about 86% of the time in this test.

We ran four other support checkers on the same claims. ProvenanceGuard scored highest on the paper's measure of how well a system catches claims that should be blocked while avoiding unnecessary blocks. The other checkers in this comparison did not tell us which tool output supported each claim. ProvenanceGuard records that connection, so a reviewer can see the source checked for each claim and the decision it produced.

Binary support metrics on the same held-out claim packet. ProvenanceGuard matches or beats the source-blind baselines on blocking while also producing per-claim source verdicts. Source: paper abstract and Table III.

Checking claims when sources look similar

In a separate, harder test with several similar sources, ProvenanceGuard scored 0.846 F1 for deciding which claims to block, but identified the exact source correctly in 50.3% of claims. Telling similar sources apart remains an important area for improvement.

We also ran a controlled test focused on wrong attribution: we changed the named source in 50 cases while leaving the supporting evidence intact. ProvenanceGuard caught all 50 swaps. This shows it can detect a clear source error, while the harder test shows the challenge of choosing among many plausible sources.

Repairing blocked answers

Blocking is only useful if there is something to do with a blocked answer. Wired to the RARR-style repair loop, the full-trace run resolved all 173 blocked answers, though 144 of them ended in fallback text rather than a substantive rewrite, which is the system choosing to avoid an unverifiable answer rather than manufacture one. On reconstructed multi-source test traces, a fresh repair run resolved all 59 initially blocked answers with only two terminal fallbacks. As an offline gate the overhead is modest, roughly half a second per answer on the reported local configuration, with the NLI and routing calls themselves in the tens of milliseconds.

Why this fits Multiverse Computing

As agents move from single-passage RAG to multi-tool MCP setups, the question of which source a fact actually came from stops being a footnote and becomes part of what factuality means. ProvenanceGuard makes that source connection visible claim by claim. For Multiverse Computing, that means a way to check existing agents while keeping sensitive traces in a controlled environment when needed. The medical study is one use case; the same approach can be adapted wherever an agent's trace preserves its tools and sources.

That adaptation is already visible in NVIDIA NVFlow, which merged an optional grounding-verification stage for its finance agent. It checks completed answers against the SEC excerpts the agent retrieved and saves separate decisions without changing the original rollout or training data. The NVFlow contribution uses ProvenanceGuard's source-aware verification approach; the repair loop discussed above belongs to the broader research system.

ProvenanceGuard was also presented as a poster at the Agentic AI Summit 2026 at UC Berkeley.

Want the full technical details, including the routing and NLI derivations, the calibration ablations, the multi-source stress slices, and the complete results tables? Read the full paper on Hugging Face, or get in touch with our team to talk about applying source-aware verification to your own agents.