🔶 AI Agent 治理:谁来守护护栏?
英文标题:🔶 AI Agents Governance: Who guards the guardrails?
文章指出,AI Agent 的策略提示、审核层和允许列表本身都是软件,其可信度取决于它们运行的位置。作者提出三层观测信任阶梯:Agent 自述语义最丰富但几乎不可信,Guest 内观测器可信度有限,而 VM 硬件边界外的观测记录最难被 Agent 篡改。通过将网络、块 I/O 和 VM 退出等边界证据“意图提升”为可读操作,并与 Agent 转录记录交叉验证,可将叙述与实际行为之间的偏差变成高价值安全信号。对开发者而言,这意味着 Agent 的生产审计不应只依赖日志或转录,而应以硬件隔离边界提供的证据为基础,再逐层验证。
1.7 KB
5.0 KB
95.2 KB
66.3 KB
92.6 KB
93.7 KB
中文
Quis custodiet ipsos custodes? 谁来守护守护者自身?
这个问题由来已久。柏拉图曾在《理想国》中提出:一旦任命守护者来保护城邦,又由谁来约束这些守护者?我们如今熟知的这句拉丁语格言后来才出现,出自罗马诗人尤维纳利斯。
对于在生产环境中运行的 AI Agent 而言,这已不再是一个修辞性问题。如今每个平台都在提供护栏:策略提示词、审核层、允许列表,以及“安全”旁车服务。它们全都是软件。软件总要运行在某个地方。而护栏_运行在哪里_,决定了你能否相信它告诉你的内容。
我们此前曾讨论过,为什么默认情况下应将 Agent 视为不可信,以及一个沙箱必须满足什么条件,才能成为真正的控制措施,而不只是一种感觉,参见沙箱宣言。这些文章主要关注执行:通过urunc实现硬件隔离的 microVM,并为每项任务分配一个 microVM。本文讨论的是问题的另一半——受到关注远少得多的一半。
不是_阻止_ Agent。
而是知道它做了什么。
护栏是软件,而软件总要运行在某个地方
这是我们关心的失效模式:一个 Agent 持有生产环境的凭据。某件事出了问题,或者审计人员只是问:这个 Agent 上周二究竟做了什么?
最直观的答案是查看 Agent 自己的转录记录:它的工具调用、日志和会话叙述。这份记录内容丰富、易于阅读,但作为证据,它一文不值。它由正在接受审计的对象写成,且处于攻击者可以注入内容的同一上下文中。如果 Agent 已被攻陷,或者只是出错,转录记录会是最先受害的对象。
接下来的做法,是在 Agent 身旁部署一个监控器,运行于同一个容器或同一个内核中。这更好一些,但监控器现在与它监管的工作负载共担命运。生活在爆炸半径之内的守卫,可能被其守护对象禁用、欺骗或通过竞争条件绕过。我们在宣言中说过,这里依然成立:与其监管的 Agent 运行在同一内核中的护栏,只是一种建议,而不是边界。
这不可避免地引出一个问题:是否存在一个 Agent _无法触及_的观测视角?
三种观测视角,三级信任
在我们看来,观察 Agent 的位置基本上只有三种,它们构成了一架阶梯。
**在 Agent 的上下文之内。**即自我报告:转录记录、工具日志,以及 Agent 对会话的自身描述。这里的语义最丰富,可信度却为零。任何 Agent 能读写的内容,攻击者同样可以读写。
**在 Guest 内部。**我们的执行层有意将 Guest 保持在最简状态:没有从 Guest 返回 Host 的控制通道,也不运行任何任务未要求运行的内容。当我们确实为了获得更丰富的遥测而在 microVM 内放置观测器时,它必须遵循严格规则:只可单向向外通信;我们将其视为证人,而不是裁判。它以良好的语义观察 OS 层面的事实——进程、文件和连接——但它仍处于爆炸半径之内。理论上,一个被完全攻陷的 Guest 可以对它说谎,或伪造与它有关的信息。
**在 VM 边界处。**这才是关键所在。Agent 发出的每个数据包、写入的每个数据块、触发的每次退出,都会穿过一个由 CPU 强制执行的边界;我们从 Host 侧记录这些信息,位于 Guest 可触及范围之外。Guest 无法压制这份记录,无法重写它,而且——这正是我们喜欢的部分——甚至无法知道自己被监视得有多严密。观测是被动且带外的;Guest 内没有任何可供发现的探针。
请注意其中的权衡:沿着阶梯向下,信任度提高,语义丰富度下降。转录记录为你讲述一个故事。边界则用一种没人愿意阅读的语言告诉你真相。
因此,设计并非“选择某一层”。而是:使用每一层,只在其观测视角所应得的范围内信任它,并向下交叉验证。Agent 的叙述是一项假设。Guest 内视图要么支持它,要么不支持它。边界记录则给出最终结论。
图 1:内省信任阶梯。Agent 的叙述和 Guest 内观测器都位于爆炸半径之内;边界记录器从 Host 侧捕获数据包、块 I/O 和 VM 退出,将其提升为具名操作;审计轨迹对三种记录进行交叉验证,并标记所有不一致之处。
这个分层问题正开始拥有一个名称。Uber 最近将Agentic Detection and Response(ADR,Agent 检测与响应)正式定义为一门实践:通过基于传感器的遥测,从 Agent 自己的本地存储中捕获提示词、推理链和工具调用。这是真正的进步,这个类别也会持续增长。
但请注意,他们的传感器位于这架阶梯的哪个位置:它解析 Agent 的缓存,这意味着它读取的是 Agent 选择写下的内容。这是第 1 级。我们关心的观测视角——那个 Agent 无法靠叙述摆脱的视角——仍然位于更下方。
这也是对尤维纳利斯问题的诚实回答。Guest 内观测器守护 Agent。硬件边界守护观测器。最后的守护者并不是任何软件可以与之交谈的对象;它就是隔离机制本身。
最有价值的信号是不一致
一旦你从多个观测视角看到同一次会话,就会免费获得一项很好的能力:偏差检测。
如果 Agent 声称它运行了测试套件,边界记录应当显示测试套件会产生的文件和网络活动。如果转录记录说“我只读取了配置”,但边界显示存在一次未被提及的出站连接,你不需要判断 Agent 是在说谎还是感到困惑。无论哪种情况,这都是一次不应被信任的会话,而且它被 Agent 无法伪造的证据标记了出来。
在经过交叉验证的审计轨迹中,叙述与边界现实之间的_缺口_不是噪声,而是你所拥有的最强信号。
弥合语义鸿沟
当然,没有人希望通过阅读原始边界事件来审计 Agent。一次到某个地址的 TCP 连接和一阵块 I/O 是真相,但它并不能回答“Agent 做了什么?”
这正是 VM 内省中经典的语义鸿沟,也是历史上大多数带外方法放弃的地方。我们的看法是:Agent 实际上是一个_友好_的案例,适合弥合这一鸿沟。与任意软件不同,Agent 通过一组出人意料地规律的操作开展工作——获取依赖、克隆仓库、运行构建、调用 API,等等——而这些操作中的每一种都会在边界留下特征性模式。
因此,我们基于真实 Agent 会话,构建了一套真实 Agent 实际执行操作的分类体系,并将边界证据提升至这一体系中。我们将其称为意图提升(intent lifting):将“这种连接和 I/O 模式”转化为“该 Agent 从这个软件源安装了一个包”,并为每项操作跟踪其置信度。最终得到的审计轨迹读起来像转录记录,但每一行都由边界证据支撑;无法被支撑的行也会被明确标记出来。
仅凭边界证据能够多好地还原意图?坦率地说,比我们刚开始时预期的更好。我们正在正式整理方法论和数据,它们值得单独写一篇文章,因此目前先点到为止。
成本如何
对此最常见的反对意见是开销,而这恰恰是我们最乐于回答的反对意见,因为观测视角本身已经替我们完成了大部分工作。
在边界进行的观测是被动的。Agent 不需要等待观测器;热路径上不存在插入式拦截,工作负载无需修改代码,Guest 中也不存在需要对抗的插桩。microVM 本身可以在毫秒级冷启动——我们此前已经讨论过这一点——而额外的内省开销足够小,因此我们在生产环境中默认对每项任务都启用它。完整测量数据是上文所述报告的一部分;简而言之,如果你承担得起沙箱,就承担得起监控它。
审计优先,执行其次
我们有意在这里先讨论可观测性,而不是执行控制,这并不是回避问题,而是一种顺序。
执行控制决定 Agent 可以做什么。我们已经讨论过这些控制应该部署在哪里,从隔离模型,到负责实施隔离的运行时。但执行控制的效果取决于其所依据的事实基础;而在事后,审计轨迹_就是_你的事实基础。当审计人员、事件复盘,或你自己的事后分析询问 Agent 做了什么时,“这是它告诉我们的内容”并不是答案。“这是穿过硬件边界的内容,被提升为操作,并与它告诉我们的内容进行交叉核验”才是答案。
本质上,我们希望对一次 Agent 会话的事后分析遵循与故障复盘相同的标准:证据优先,叙述其次。
这个问题的每一部分都还有更多可说之处——分类体系、意图提升准确率,以及各层在真实环境中出现分歧时会发生什么——我们将在未来几周继续讨论。如果你现在就想了解完整图景(威胁模型、隔离评分卡,以及每项主张背后的证据),相关内容都已收录在下方白皮书中。
Quis custodiet ipsos custodes? 边界会守护它们。这正是设置边界的意义所在。
English / 原文
Quis custodiet ipsos custodes? Who guards the guards themselves?
The problem is old. Plato raised it in the Republic: once you appoint guardians to protect the city, who keeps the guardians in check? The Latin tag we remember it by came later, from the Roman poet Juvenal.
For AI agents in production, the question is not rhetorical anymore. Every platform ships guardrails now: policy prompts, moderation layers, allowlists, "safety" sidecars. All of them are software. Software runs somewhere. And where a guardrail runs decides whether you can believe what it tells you.
We have written before about why the agent is untrusted by default and what a sandbox has to mean for that to be a real control rather than a vibe. Those posts were mostly about enforcement: hardware-isolated microVMs, one per task, via urunc. This post is about the other half of the problem, the one that gets far less attention.
Not stopping the agent. Knowing what it did.
Guardrails are software, and software runs somewhere
Here is the failure mode we care about. An agent holds credentials to your production environment. Something goes wrong, or an auditor simply asks: what exactly did this agent do last Tuesday?
The obvious answer is to read the agent's own transcript: its tool calls, its logs, its narration of the session. That record is rich and readable, and it is worth exactly nothing as evidence. It is written by the thing you are auditing, in the same context an attacker can inject into. If the agent was compromised, or just wrong, the transcript is the first casualty.
The next answer is to put a monitor next to the agent, in the same container or on the same kernel. Better, but the monitor now shares fate with the workload it polices. A guard that lives inside the blast radius can be disabled, deceived, or raced by the thing it guards. We said it in the manifesto and it holds here: a guardrail running in the same kernel as the agent it polices is a suggestion, not a boundary.
Which inevitably leads us to the question: is there a vantage point the agent cannot reach?
Three vantage points, three levels of trust
The way we think about it, there are essentially three places you can observe an agent from, and they form a ladder.
Inside the agent's context. Self-reporting: transcripts, tool logs, the agent's own account of the session. Richest semantics, zero trust. Anything the agent can read or write, an attacker can too.
Inside the guest. Our execution layer deliberately keeps the guest minimal: no control channel from the guest back into the host, nothing running that the task did not ask for. When we do place an observer inside the microVM for richer telemetry, it goes in under a strict rule: it talks one way, outward, and we treat it as a witness rather than a judge. It sees the OS-level truth (processes, files, connections) with good semantics, but it lives inside the blast radius. A fully compromised guest can, in principle, lie to it or about it.
At the VM boundary. This is the interesting one. Every packet the agent sends, every block it writes, every exit it takes crosses a boundary the CPU enforces, and we record it from the host side, outside anything the guest can touch. The guest cannot suppress this record, cannot rewrite it, and (this is the part we like) cannot even tell how closely it is being watched. Observation is passive and out-of-band; there is no probe inside the guest to find.
Notice the trade: as you move down the ladder, trust goes up and semantic richness goes down. The transcript tells you a story. The boundary tells you the truth, in a language nobody wants to read.
So the design is not "pick a layer". It is: use every layer, trust each one only as far as its vantage point warrants, and corroborate downward. The agent's story is a hypothesis. The in-guest view either supports it or it does not. The boundary record settles it.
Figure 1: the introspection trust ladder. The agent's story and the in-guest observer live inside the blast radius; the boundary recorder captures packets, block I/O and VM exits from the host side, lifts them into named operations, and the audit trail corroborates all three accounts, flagging whatever does not line up.
This layered problem is starting to get a name. Uber recently formalized Agentic Detection and Response (ADR) as a discipline: sensor-based telemetry that captures prompts, reasoning chains, and tool calls from the agent's own local stores. It is genuine progress, and the category will grow. But notice where their sensor sits on the ladder: it parses the agent's caches, which means it reads what the agent chose to write. That is Level 1. The vantage point we care about, the one the agent cannot narrate its way out of, is still below.
And that is the honest answer to Juvenal. The in-guest observer guards the agent. The hardware boundary guards the observer. The last guard is not software anyone can talk to; it is the isolation mechanism itself.
The most useful signal is disagreement
Once you have the same session observed from multiple vantage points, something nice falls out for free: divergence detection.
If the agent claims it ran a test suite, the boundary should show the file and network activity a test suite produces. If the transcript says "I only read the config", but the boundary shows an outbound connection nobody mentioned, you do not need to decide whether the agent is lying or confused. Either way, you have a session that should not be trusted, flagged by evidence the agent had no way to forge.
In a corroborated audit trail, a gap between the story and the boundary reality is not noise. It is the strongest signal you have.
Closing the semantic gap
Of course, nobody wants to audit an agent by reading raw boundary events. A TCP connection to some address and a burst of block I/O is the truth, but it is not an answer to "what did the agent do?".
This is the classic semantic gap of VM introspection, and it is where most out-of-band approaches historically gave up. Our take: agents are actually a friendly case for closing it. Unlike arbitrary software, an agent works through a surprisingly regular set of operations (fetch a dependency, clone a repo, run a build, call an API, and so on), and each of those leaves a characteristic shape at the boundary.
So we built a taxonomy of the operations real agents actually perform, learned from real agent sessions, and we lift boundary evidence into it. We call this intent lifting : turning "this pattern of connections and I/O" into "the agent installed a package from this registry", with the confidence tracked per operation. The result is an audit trail that reads like the transcript, except every line is backed by boundary evidence, and the lines that cannot be backed are marked as such.
How well does boundary-only evidence recover intent? Better than we expected when we started, honestly. We are writing up the methodology and numbers properly, and they deserve their own post, so we will leave it at that for now.
What it costs
The usual objection to all of this is overhead, and it is the objection we have the most fun answering, because the vantage point does the work for us.
Observation at the boundary is passive. The agent does not wait for the observer; there is no interposition on the hot path, no code changes in the workload, no instrumentation for the guest to fight with. The microVM itself cold-starts in milliseconds (we have covered that ground before), and the introspection tax on top is small enough that we simply run it on every task, by default, in production. Full measurements are part of the write-up above; the short version is that if you can afford the sandbox, you can afford to watch it.
Audit first, enforcement second
We are deliberately leading with observability here, not enforcement, and that is not a dodge. It is an ordering.
Enforcement decides what an agent may do, and we have written about where those controls have to live, from the isolation model down to the runtime that enforces it. But enforcement is only as good as the ground truth it acts on, and after the fact, the audit trail is your ground truth. When the auditor, the incident review, or your own postmortem asks what the agent did, "here is what it told us" is not an answer. "Here is what crossed the hardware boundary, lifted into operations, cross-checked against what it told us" is.
Essentially, we want the postmortem of an agent session to be held to the same standard as the postmortem of an outage: evidence first, narrative second.
There is more to say about every piece of this (the taxonomy, the lifting accuracy, what happens when the layers disagree in the wild), and we will say it in the coming weeks. If you want the full picture now (the threat model, the isolation scorecard, and the evidence behind each claim), it is all in the whitepaper below.
Quis custodiet ipsos custodes? The boundary does. That is rather the point of having one.