Show HN：Agent Chaperone——使用 Jev 筛查 AI Agent 的工具调用及其结果

面向 AI Agent 工具调用的校准型防火墙。

agent-chaperone 会在 AI Agent 的工具调用执行前对其进行筛查，也会在 Agent 读取调用返回的工具结果前对结果进行筛查。每项决策均以概率形式返回；每个阈值都写在策略文件中而非提示词中；每项判断都会连同产生该判断的数值写入本地日志。

它从 shadow 模式开始，该模式不会拦截任何内容。你应先阅读自己的日志，再决定哪些内容本应被阻止。

agentchaperone.dev 以网页形式提供相同材料，包括实测结果以及各类部署方式的配置指南。

请先阅读这一部分，再继续阅读其余内容。

• 不是沙箱。 它无法阻止服务器执行调用描述中未提及的行为。对此请配合操作系统级隔离机制。

• 不是客户端权限提示的替代品。 它提供了能看到内容的第二意见，而权限提示无法看到内容。它绝不会代表你自动批准任何操作。

• 不提供保证。 底层模型默认不会把输入视为敌对内容；针对本工具编写的自适应攻击会有部分绕过。下文数字报告了它能捕获和会漏掉的内容，包括漏检情况。

• 并非仅本地运行。 筛查会将待筛查的参数和结果发送给配置的模型后端，因此这些内容会离开机器。系统会先遮蔽看起来像秘密信息的字符串；对于必须保留在本地的内容，也可以按服务器关闭筛查。

  npm install -g agent-chaperone

会话中已经回答过的问题不会再次提问。Agent 重复读取同一个文件时，状态和判断组每次都相同，因此这相当于同一个请求；缓存的答案会记录其成本为零，而不会为一个实际未发送的请求计费。缓存仅在当前会话中存在，因为内容本身并不保存。

筛查需要环境变量中的 TYPESAFE_API_KEY。没有该变量时，确定性规则仍会执行，包括允许和拒绝列表以及秘密信息模式；每项判断也都会记录未向模型发起请求。

OPENROUTER_API_KEY 或 AI_GATEWAY_API_KEY 可改为通过通用模型进行筛查，具体模型由 AGENT_CHAPERONE_MODEL 选择。这些概率并未校准。 此处随附的所有阈值均依据 Jev 的实测数据选定；通用模型返回的数字看起来相同，含义却不同，因此下方结果并不适用于它。提供这一选项是为了让工具能在无法使用 Jev 的环境中运行。在让这些模型决定任何事情前，请先在 shadow 模式下阅读自己的日志，并据此调整配置。

只需修改一次客户端的 MCP 配置。-- 之后的所有内容都是原本就会运行的服务器：

{
  "mcpServers": {
    "filesystem": {
      "command": "npx",
      "args": ["-y", "agent-chaperone", "--", "npx", "-y", "@modelcontextprotocol/server-filesystem", "."]
    }
  }
}

agent-chaperone wrap <config> 可代你完成该编辑。它会打印将要修改的内容，只有加上 --write 才会写入文件；它还会在原文件旁保留原始副本，因为客户端缺少该文件就无法启动，而一次错误编辑会同时破坏所有服务器。--unwrap 可将其移除。无论包装还是解除包装，重复运行两次都不会造成额外改动。

筛查客户端自身的工具

代理只能看到 MCP 流量。它看不到客户端自行执行的 shell 命令、文件编辑或网页抓取，而在用户实际使用的客户端中，大部分风险恰恰存在于这些操作。两个命令会读取客户端的 hook 载荷，并依据同一策略文件和同一日志通过 stdout 返回结果。

{
  "hooks": {
    "PreToolUse": [
      {
        "matcher": "Bash|PowerShell|Edit|Write|WebFetch",
        "hooks": [{ "type": "command", "command": "agent-chaperone hook pre" }]
      }
    ],
    "PostToolUse": [
      {
        "matcher": "Bash|PowerShell|Read|WebFetch",
        "hooks": [{ "type": "command", "command": "agent-chaperone hook post" }]
      }
    ],
    "PostToolUseFailure": [
      {
        "matcher": "Bash|PowerShell|Read|WebFetch",
        "hooks": [{ "type": "command", "command": "agent-chaperone hook post" }]
      }
    ]
  }
}

完整配置示例、各命令返回的内容以及 hooks 看不到什么，详见 docs/hooks.md。

有一个问题不包含在任何工具调用中：某次调用是否与用户实际提出的请求有关。agent-chaperone task "fix the login redirect" 会为当前目录记录该任务，代理和 hooks 都会读取它。未设置任务时，这个问题不会被发送。任务会被信任 12 小时，因为过期任务会导致筛查器拿上周的意图来判断今天的调用。

模式	行为
shadow	筛查全部内容，但不拦截任何内容。每项判断都会被记录，包括它原本会采取的动作。这是默认模式。
enforce	对判断采取行动。它会暂停调用等待你的批准，阻止策略拒绝的内容，并扣留看似在指示 Agent 的结果。
strict	与 enforce 相同，此外，当筛查根本无法运行时，它会停止而非继续执行。

从 shadow 迁移至 enforce

先在 shadow 模式下运行一段时间，并阅读它原本会采取的动作：

14:03:11 call   forward  read_file  destructive 0.02 exfiltration 0.01 $0.000029
14:03:11 call   forward  write_file (would have held it)  destructive 0.91 severity 2.0 $0.000029
14:03:11 result WITHHELD fetch  instructs_reader 0.97 $0.000029

(would have held it) 一列正是 shadow 模式的核心：它列出了切换模式后会发生变化的事项。阅读这些记录；如果你不同意其中某项，请调整阈值而不是切换模式。当日志不再让你感到意外时，在策略文件中设置 mode: enforce。

agent-chaperone log --follow 可监看运行中的会话，agent-chaperone show <id> 会打印被暂停或被扣留的内容。

在 enforce 和 strict 模式中，跨过阈值的调用不会执行。系统会以 Agent 能够据此行动的方式告知它：

agent-chaperone held this call to write_file for the user to approve: it looks like
it changes something in a way that is hard to undo. The possible damage was rated
high. Nothing ran. The user can allow it by running: agent-chaperone approve b2c4e6a8f0

Agent 没有被告知要尝试别的操作，也没有被告知调用失败。它被告知，正由一名用户作出决定。

然后，你可以查看它实际想做什么；如你同意，则允许该调用：

agent-chaperone show b2c4e6a8f0
agent-chaperone approve b2c4e6a8f0

令牌标识的是那一次调用而非整个工具，它以服务器、工具和接收到的参数摘要为键。允许向某一路径写入，并不会放行向另一条路径写入。令牌在使用时立即失效，并且会过期。被策略明确拒绝的工具无法获得批准：那是你写下的一条长期规则，而非系统向你提出的问题；命令会说明这一点，并指向策略文件。

• 调用前。 在转发工具调用之前：它是否具有破坏性，是否会将私有数据或秘密发送到外部，是否违反以自然语言编写的策略，若执行会造成多大损害。
• 结果后。 在工具结果到达 Agent 之前：内容是否试图指示正在读取它的 AI，是哪一段内容触发了判断，是否暴露秘密信息，遵循它会造成多大危害。

每次筛查都会结合代码中的确定性规则，以及来自 Jev——TypeSafe 的 System One 模型——的一组小型、带类型且经过校准的判断。

此外，服务器声明的工具列表会与其首次声明的列表进行比较。被改写的描述、新出现的工具以及消失的工具都会被报告。该比较是本地摘要：不会向任何地方发送内容，也绝不会扣留工具列表，因为无法读取工具列表的客户端无法调用任何工具。agent-chaperone trust <server> 可在你查看后接受一次变更。

该比较能发现描述发生了变化，但不会读取描述内容。读取描述是一个独立开关：screen_tool_descriptions。它默认关闭，升级时也将保持关闭，因为开启后会把每条新增或变更的描述发送到模型后端。开启后，如果某段描述读起来更像是在指示 Agent，而非文档说明，系统会报告它；过长而无法判断的描述，或筛查器无法访问的描述，则会被报告为未读，而不会被直接放过。

这一问题不包含在下方数字中。基准测试没有发送这一类内容，因此其阈值是判断而非从曲线读出的数值，通知中也会说明这一点。

知道本工具存在的 Agent，比只能猜测的 Agent 更能妥善处理被阻止的调用。skills/agent-chaperone 中的技能文件会告诉它每种通知的含义：被暂停的调用由用户决定是否放行，而不是由 Agent 自己决定；被扣留的结果也不意味着可以换一种方式获取相同内容。

npx skills add agent-chaperone/agent-chaperone

该技能也随 npm 软件包发布，因此全局安装后它已存在于本地磁盘。

agent-chaperone [options] -- <command> [args...]   包装并筛查服务器
agent-chaperone log [--follow]                     读取当前会话的决策
agent-chaperone show <id>                          打印被暂停或被扣留的内容
agent-chaperone approve <id>                       一次性放行一项被暂停的调用
agent-chaperone trust <server>                     接受服务器当前声明的工具
agent-chaperone report                             汇总日志内容，以及启用强制模式将阻止什么
agent-chaperone replay [--policy <path>]           使用另一份策略，对已判断内容重新决策
agent-chaperone hook pre|post                      通过 hook 筛查客户端自身的工具

选项	含义
--policy <path>	策略文件。默认值为 ~/.config/agent-chaperone/policy.yaml
--server <name>	应用策略的配置段。默认值：命令名称
--no-store-content	记录判断，但不记录参数或结果

对于客户端以固定命令行启动的 hook 命令，AGENT_CHAPERONE_POLICY 可设置策略路径，AGENT_CHAPERONE_STORE_CONTENT=0 的作用等同于 --no-store-content。

筛查问题使用 jev-1.13.0 于 2026-09-21 针对公开提示词注入基准测试和一套人工标注的工具调用进行了评估，每个条目一次请求。发送的判断组与工具实际发送的完全一致，逐个问题对应。完整运行共计 1,947 次请求，按公开价格计算成本为 $0.062。

数据集	条目数	正例数	AUC	阈值 0.5 下的精确率 / 召回率	阈值 0.3 下的结果
InjecAgent 工具响应	1,394	1,054	0.976	0.989 / 0.805	0.956 / 0.949
BIPIA 邮件	250	200	1.000	1.000 / 0.825	1.000 / 0.850
讨论注入但实际无害	68	0	不适用	8 个误报	13 个误报
人工标注的工具调用	100	51	0.993	0.980 / 0.961	0.909 / 0.980

每次被筛查的调用会产生一次模型请求，平均使用 754 个输入 token。由允许或拒绝列表直接判定的调用不会发送给模型。

工具提出的三个问题不包含在这些数字中。policy_violation 和 off_task 仅在配置了策略或任务时发送，而这里没有任何一行设置了二者之一。description_steers 询问的是工具描述而非调用或结果，基准测试没有可供测试的工具列表集合，因此上述数据并未衡量这三个问题中的任何一个。

人工标注数据集有意偏向内置工具场景：100 个已评分调用中有 37 个是 shell 命令，51 个危险调用中有 21 个是 shell 命令。在上述客户端中，这些属于内置工具而不是 MCP 流量，因此会通过 hooks 适配器进入筛查流程。

方法论、各阈值对应的表格、漏检案例以及原始记录的模型响应，详见 bench/。

• docs/design.md：架构、筛查机制及其问题、策略文件、审计日志、隐私和性能预算。
• docs/hooks.md：筛查客户端自身的工具。
• docs/adr/：设计背后的决策及原因。
• ROADMAP.md：各版本将交付的内容。
• [agentc

A calibrated firewall for AI agent tool calls.

agent-chaperone screens an AI agent's tool calls before they run, and the tool results those calls return before the agent reads them. Every decision comes back as a probability, every threshold lives in a policy file rather than a prompt, and every judgment is written to a local log with the numbers that produced it.

It starts in shadow mode, which blocks nothing. You read your own log first and decide what you would have wanted stopped.

agentchaperone.dev carries the same material as pages, including the measured results and a guide for each way people set this up.

Read this part before the rest.

• Not a sandbox. It cannot stop a server from doing something the call did not describe. Pair it with OS-level isolation for that.

• Not a replacement for the client's permission prompts. It is a second opinion that sees the content, which permission prompts do not. It never auto-approves anything on your behalf.

• Not a guarantee. The underlying model does not treat its input as hostile by default, and adaptive attacks written against this tool will get through. The numbers below report what it catches and what it misses, including the misses.

• Not local-only. Screening sends the arguments and results being screened to the configured model backend, so that content leaves the machine. Secret-shaped strings are redacted first, and screening can be turned off per server for content that has to stay local.

  npm install -g agent-chaperone

A question the session has already answered is not asked again. An agent rereading one file is the same state and the same battery every time, which is the same request, and a cached answer records that it cost nothing rather than charging for a request nobody sent. The cache lives for the session, because content does not.

Screening needs TYPESAFE_API_KEY in the environment. Without it the deterministic rules still run, which is the allow and deny lists and the secret patterns, and every judgment records that no model was asked.

OPENROUTER_API_KEY or AI_GATEWAY_API_KEY will screen through a general model instead, with AGENT_CHAPERONE_MODEL choosing which. Those probabilities are not calibrated. Every threshold that ships here was chosen against measured numbers from Jev, and a general model returns a number that looks the same and means something else, so the results below say nothing about it. It exists so this runs where Jev cannot. Read your own log in shadow mode and tune from it before letting one of those decide anything.

One change to the client's MCP configuration. Everything after -- is the server that would have run anyway:

{
  "mcpServers": {
    "filesystem": {
      "command": "npx",
      "args": ["-y", "agent-chaperone", "--", "npx", "-y", "@modelcontextprotocol/server-filesystem", "."]
    }
  }
}

agent-chaperone wrap <config> does that edit for you. It prints what it would change and writes nothing until you add --write, and it keeps the original beside the file, because a client will not start without this file and a bad edit breaks every server at once. --unwrap takes it back out. Running it twice changes nothing either way.

Screening a client's own tools

A proxy sees MCP traffic. It does not see the shell, the file edits or the web fetches a client runs itself, and on the clients people actually use those are where most of the damage lives. Two commands read a client's hook payload and answer on stdout, against the same policy file and the same log.

{
  "hooks": {
    "PreToolUse": [
      {
        "matcher": "Bash|PowerShell|Edit|Write|WebFetch",
        "hooks": [{ "type": "command", "command": "agent-chaperone hook pre" }]
      }
    ],
    "PostToolUse": [
      {
        "matcher": "Bash|PowerShell|Read|WebFetch",
        "hooks": [{ "type": "command", "command": "agent-chaperone hook post" }]
      }
    ],
    "PostToolUseFailure": [
      {
        "matcher": "Bash|PowerShell|Read|WebFetch",
        "hooks": [{ "type": "command", "command": "agent-chaperone hook post" }]
      }
    ]
  }
}

The worked configuration, what each command answers, and what the hooks do not see are in docs/hooks.md.

One question needs something no tool call contains: whether a call has anything to do with what you actually asked for. agent-chaperone task "fix the login redirect" records that for the directory you are in, and the proxy and the hooks both read it. Without one, the question is never sent. It is believed for twelve hours, because a stale task would have the screen judging today's calls against last week's intent.

Mode	What it does
shadow	Screens everything, blocks nothing. Every judgment is logged, including what it would have done. This is the default.
enforce	Acts on the judgments. Holds a call for your approval, blocks what the policy denies, and withholds a result that reads as an instruction.
strict	The same, and additionally stops rather than continuing when a screen could not run at all.

Moving from shadow to enforce

Run in shadow for a while and read what it would have done:

14:03:11 call   forward  read_file  destructive 0.02 exfiltration 0.01 $0.000029
14:03:11 call   forward  write_file (would have held it)  destructive 0.91 severity 2.0 $0.000029
14:03:11 result WITHHELD fetch  instructs_reader 0.97 $0.000029

The (would have held it) column is the whole point of shadow mode: it is the list of things that change when you switch. Read them, and if you disagree with one, move the threshold rather than the mode. When the log stops surprising you, set mode: enforce in the policy file.

agent-chaperone log --follow watches a running session, and agent-chaperone show <id> prints what was held or withheld.

In enforce and strict, a call that crosses a threshold does not run. The agent is told so, in terms it can act on:

agent-chaperone held this call to write_file for the user to approve: it looks like
it changes something in a way that is hard to undo. The possible damage was rated
high. Nothing ran. The user can allow it by running: agent-chaperone approve b2c4e6a8f0

The agent has not been told to try something else, and it has not been told the call failed. It has been told a person is deciding.

You then look at what it actually wanted to do, and allow it if you agree:

agent-chaperone show b2c4e6a8f0
agent-chaperone approve b2c4e6a8f0

The token names that one call rather than that tool, keyed by a digest of the server, the tool and the arguments as they arrived. Agreeing to a write to one path does not release a write to another. It is spent the moment it is used, and it expires. A tool the policy denies outright is not approvable: that is a standing rule you wrote, not a question you were asked, and the command says so and points at the policy file.

• Pre-call. Before a tool call is forwarded: is it destructive, does it send private data or secrets outside, does it break the policy written in plain English, how bad would it be if it ran.
• Post-result. Before a tool result reaches the agent: does the content try to instruct the AI reading it, which block does it, does it expose a secret, how much harm would following it cause.

Each screen combines deterministic rules in code with a small battery of typed, calibrated judgments from Jev, TypeSafe's System One model.

Alongside them, the tool list a server advertises is compared against the one it first advertised. A rewritten description, a tool that appeared, and a tool that vanished are each reported. That comparison is a local digest: it sends nothing anywhere, and it never withholds the list, because a client that cannot read the tool list cannot call anything. agent-chaperone trust <server> accepts a change once you have looked at it.

That comparison notices a description changed and never reads it. Reading them is a separate switch, screen_tool_descriptions, which is off by default and stays off when you upgrade, because turning it on sends every new or changed description to the model backend. With it on, a description that reads more like an instruction to the agent than like documentation is reported, and one too long to judge, or that the screen could not reach, is reported as unread rather than passed over.

That question is not covered by the numbers below. Nothing in the benchmark sends it, so its threshold is a judgement rather than a figure read off a curve, and the notice says so.

An agent that knows this tool exists handles a blocked call better than one that guesses. The skill in skills/agent-chaperone tells it what each notice means, that a held call is the user's to release and not its own, and that a withheld result is not an invitation to fetch the same thing another way.

npx skills add agent-chaperone/agent-chaperone

It ships in the npm package too, so a global install already has it on disk.

agent-chaperone [options] -- <command> [args...]   Wrap and screen a server
agent-chaperone log [--follow]                     Read this session's decisions
agent-chaperone show <id>                          Print what was held or withheld
agent-chaperone approve <id>                       Let one held call through, once
agent-chaperone trust <server>                     Accept the tools a server now advertises
agent-chaperone report                             What the log adds up to, and what enforcing would stop
agent-chaperone replay [--policy <path>]           Decide again with another policy, over what was judged
agent-chaperone hook pre|post                      Screen a client's own tools, from a hook

Option	Meaning
--policy <path>	Policy file. Default ~/.config/agent-chaperone/policy.yaml
--server <name>	Which section of the policy applies. Default: the command name
--no-store-content	Record the judgments and not the arguments or results

For the hook commands, which a client launches with a fixed command line, AGENT_CHAPERONE_POLICY sets the policy path and AGENT_CHAPERONE_STORE_CONTENT=0 does what --no-store-content does.

The screening questions were evaluated against public prompt-injection benchmarks and a hand-labeled set of tool calls, using jev-1.13.0 on 2026-09-21, one request per item. The batteries sent were the ones the tool sends, question for question. The full run is 1,947 requests and costs $0.062 at the published price.

Set	Items	Positives	AUC	Precision / recall at 0.5	at 0.3
InjecAgent tool responses	1,394	1,054	0.976	0.989 / 0.805	0.956 / 0.949
BIPIA email	250	200	1.000	1.000 / 0.825	1.000 / 0.850
Discusses injection, benign	68	0	n/a	8 false positives	13 false positives
Hand-labeled tool calls	100	51	0.993	0.980 / 0.961	0.909 / 0.980

Each screened call costs one model request, averaging 754 input tokens. A call the allow or deny list settles never reaches the model.

Three questions the tool asks are not in these numbers. policy_violation and off_task are sent only when a policy or a task is configured, and no row here has either. description_steers asks about a tool description rather than a call or a result, and the benchmark has no set of tool lists to ask it against, so nothing above measures any of the three.

The hand-labeled set leans toward the built-in tool case on purpose: 37 of the 100 scored calls are shell commands, and 21 of the 51 dangerous ones are. On the clients above, those are built-in tools rather than MCP traffic, so they reach the screens through the hooks adapter.

Methodology, per-threshold tables, the misses, and the raw recorded model responses are in bench/.

• docs/design.md: architecture, the screens and their questions, policy file, audit log, privacy, and performance budget.
• docs/hooks.md: screening a client's own tools.
• docs/adr/: the decisions behind the design and why.
• ROADMAP.md: what lands in which version.
• agentchaperone.dev/guides: putting a server behind the proxy, screening results for prompt injection, covering a client's own tools, and catching a secret on its way out.

See CONTRIBUTING.md. Agents and contributors can start from AGENTS.md and CURRENT_STATE.md.

Apache-2.0. See LICENSE and NOTICE.