我们亟需实时监控 AI Agent 的行为

记录“Agent 如何做出决策”的数据记录更具挑战性。这一问题给模型提供商和研究人员带来了一系列开放性研究课题：模型所展示的逐步推理过程（即思维链，Chain of Thought），是否真正反映了模型做出决策的实际方式？我们是否有可靠的方法来评估 Agent 对该决策的置信度？在任何人针对如何使用此类可观测性（Observability）记录制定标准之前，这些问题都必须先得到解答。

没有任何单一机构或行业能够独自弥合可观测性方面的空白，但各方协同合作，便能构建起防止危险事故并保障安全所需的基础设施。

构建 Agent 框架的 AI 开发者（如 OpenAI、Anthropic、CrewAI 和 LangChain）必须集成遥测信号，以记录 Agent 的持久身份（persistent identity）、任何人类干预，以及对 Agent 权限和记忆的任何更改。与此同时，采用这些 Agent 框架的企业也发挥着至关重要的作用；在尚未建立标准的领域，银行和医院等受监管行业的企业可以向 Agent 开发者施加压力，促使其提供这些缺失的可观测性记录。

企业还必须制定策略、权限和角色，明确 Agent 可以安全执行的操作，因为仅靠遥测数据无法评估 Agent 是否违反了具体规则。最后，监管机构和 AI 标准机构必须明确高风险 AI 部署所需的可观测性记录要求，同时将技术细节留给产业界去实现。我们在此已经看到了初步的推进势头：美国的 NIST Agent 标准倡议（NIST’s agent standards initiative） 正在研究 Agent 身份机制，而这正是我们在研究中发现的最显著的空白之一。

没有哪家银行会允许一千名未佩戴工牌的员工随意接触客户账户，并在出事之后才寄希望于拼凑还原发生了什么。然而，这恰恰是我们目前部署 AI Agent 的方式。让它们的行为具备可观测性，是纠正这一现状的第一步。而通过建立能够根据所观察到的行为采取行动的监管与政策生态系统，我们才能对相关机构进行有效问责。在此之前，我们交给 Agent 的每一项任务，实质上也是在交出我们自己的主导权。

Madhulika Srikumar 是 Partnership on AI 的 AI 安全项目负责人。该机构是一个由领先行业机构、学术界和民间社会组织组成的全球非营利合作组织，致力于推动 AI 治理的最佳实践。

Claire Leibowicz 是 Partnership on AI 的“AI、信任与社会”部门主管，也是牛津大学的博士候选人。

Preserving “how agents decided” records is more challenging. The issue raises open research questions for model providers and researchers: Does the step-by-step reasoning a model displays, known as chain of thought, reflect how the model actually made a decision? And do we have reliable ways to measure how confident the agent is in that decision? These questions need answers before anyone sets standards for how such observability records are used.

No single organization or sector can close the observability gaps on their own, but together, they can support the infrastructure required to prevent dangerous incidents and support safety.

AI developers building agent frameworks, like OpenAI, Anthropic, CrewAI, and LangChain, must integrate telemetry signals recording an agent’s persistent identity, any human interventions, and any changes to an agent’s permissions and memory. Meanwhile, enterprises adopting these agent frameworks have an influential role to play; enterprises in regulated industries, like banks and hospitals, can put pressure on the agent developers to supply these missing observability records where standards do not yet exist.

Enterprises must also define policies, permissions, and roles that set out what agents can safely do, since telemetry alone cannot assess whether agents broke specific rules. Finally, regulators and AI standards institutes must specify the observability records required for high-stakes AI deployments while leaving the technical details to industry. We have already seen initial momentum here: NIST’s agent standards initiative in the US is working on agent identity, one of the most salient gaps we found in our research.

No bank would let a thousand unbadged employees loose on customer accounts and hope to piece together what happened when something goes wrong. Yet that’s how we’re deploying AI agents. Making their behavior observable is the first step toward fixing this. And by building the regulatory and policy ecosystem that makes it possible to act on what we see, we can hold organizations accountable. Until then, every task we hand to an agent hands over our agency, too.

Madhulika Srikumar is the Head of the AI Safety Program at the Partnership on AI, a global non-profit partnership of leading industry, academic, and civil society organizations, advancing best practices in AI governance.

Claire Leibowicz is the Director of AI, Trust, and Society at the Partnership on AI and a doctoral candidate at Oxford University.