AI News
商业产业 Hacker News AI 2026-10-09T13:58 正文 ok 801 words

🔶 用决策模型以自然语言为 Agent 设置安全护栏

英文标题：🔶 Safeguard agents in plain English with decision models

开源项目 guardrails-md 为具备 Bash 权限的编码 Agent 增加了一道执行前闸门：每条命令都会根据仓库中的 GUARDRAILS.md 进行评分，超过阈值即被拦截。该系统基于 berget/bev 的 System One 决策模型，以约 100 毫秒完成固定问题的评分，而非让第二个 LLM 生成审查意见。项目可拦截破坏性操作、凭证泄露、团队规则违规及部分提交内容，但官方承认其在红队命令上的准确率仅为 74%，不能替代沙箱、最小权限凭证和人工审查。开发者可将其接入 OpenCode、Pi 以及 Git pre-commit 流程，并通过 Pull Request 修改 GUARDRAILS.md 来调整长期策略。
打开原文相关 3 · 机会 1
media244.5 KB media530.5 KB media7.5 KB

中文

具备 Bash 访问权限的编码 Agent 即使完全遵从指令，也可能摧毁你的数据库、输出你的机密信息，或无视团队规则。今天以开源形式发布的 guardrails-md 是一道闸门：在任何命令运行前，它都会依据仓库中的 GUARDRAILS.md 为每条命令评分。如果某条命令超过阈值，调用将被拦截，Agent 会收到原因，从而选择另一条路径。整个判断过程约需 100 毫秒。

上方的两分钟讲解视频展示了完整流程。以下是简要说明。

流程

在会话启动时，闸门会读取一次 GUARDRAILS.md（仅前 2,000 个字符），然后将其冻结。Agent 之后仍可像编辑其他文件一样编辑该文件，但修改只会在下一次会话中生效，并且应像其他修改一样经过审查。

当 Agent 调用 Bash 时，钩子会在任何操作运行前进行拦截。它会构建一段状态文本：命令本身、该命令所指向的任何脚本文件内容，以及被冻结的护栏规则。随后，它会在一次请求中向决策模型提出四个固定问题：

  • destructive：该命令是否会不可逆地销毁数据、数据库或基础设施？
  • credentials：该命令是否包含、输出或发送机密信息、密钥或令牌？
  • guardrails_violation：该命令是否违反团队的 GUARDRAILS.md？
  • policy_exception：护栏文本是否明确将此命令列为允许执行？

每个回答都是介于 0 和 1 之间的分数，并与阈值进行比较（默认值为 0.7）。超过阈值时，调用会被拦截，原因将作为工具结果返回给 Agent：

SystemOne-gate: blocked command — destructive=0.98 > 0.7
  rm -rf ./important-data

为什么使用决策模型

基于模式的拒绝列表知道 rm -rf，但不知道你的政策。让第二个 LLM 审查每条命令确实可以理解政策，但它可以被说服或争辩，而且每条命令都要付出一次完整生成的成本。相反，这道闸门使用 berget/bev，即我们在发布 System One时介绍的 System One 决策模型：输入固定问题，输出分数，只需一次前向传播。它判断的是命令和规则，而不是促成这些命令的对话。无论驱动模型被说服相信了什么，闸门都会仅根据命令本身的优劣进行判断。

不可协商的部分

  • 破坏性操作和凭证是兜底防线。通过在 GUARDRAILS.md 的 MAY 部分列出命令，可以解除 guardrails_violation；但无论文件如何规定，破坏性操作和凭证相关限制都不能被解除。
  • 重试会付出代价。每次被拦截后，下一次判断前的等待时间都会翻倍：起初是几秒，到第 20 次拦截时则达到一个半小时。同一命令的变体也无济于事。
  • 闸门默认失败关闭。失败关闭是有意设计：如果端点不可访问，命令会被拦截，直至其恢复响应。交互式用户可以通过 SYSTEMONE_FAIL_OPEN=1 选择退出。不存在快速通道、前缀允许列表，也不存在可由 Agent 写入的覆盖文件。
  • 某些文件受到直接保护。文件编辑工具拒绝修改 GUARDRAILS.md 和 harness 配置。该拒绝是确定性的：不调用模型、不比较阈值、也没有冷却时间。

覆盖权属于人类

要修改政策，请通过 Pull Request 编辑 GUARDRAILS.md；这是解决误报的持久性方案。要调整闸门，可提高阈值。要在单个会话中不启用它，可使用 SYSTEMONE_GATE=off 重启 harness。这三项操作都应由人完成；重启还会重新读取政策并重置冷却时间。

同样适用于提交时

同样的判断机制也可作为 git pre-commit hook 使用：每一项已暂存的 diff 在进入仓库前都会被评分。个人数据（GDPR）和机密信息会被拦截；而团队成员自己的姓名出现在署名和作者字段中则可以通过。

坦诚的局限性

这道闸门是经过训练的模型，而不是规则引擎。它在我们的留出测试中评分约为 96%，但在红队命令上的得分为 74%；它偶尔会在两个方向上作出错误判断，即拦截安全命令，或放过有风险的命令。它不会解码经过编码的载荷。应将其视为受限凭证、沙箱和人工审查之外的一层防护，而不是一个沙箱。完整评估表和方法论位于仓库中。

试用方法

三个步骤，约一分钟。

  1. 在你的仓库中添加一个 GUARDRAILS.md，说明团队允许哪些行为：

    ## The agent MUST NOT

- Push directly to main
- Run irreversible operations against shared systems

## The agent MAY

- Run tests, lint, and builds locally

  2. 在你的 harness 中安装闸门：

    • OpenCode： 在 opencode.json 中添加 "plugin": ["@bergetai/guardrails-md"]
    • Pi： 在终端中运行 pi install npm:@bergetai/guardrails-md

  3. 如果你已在 OpenCode 或 Pi 中登录 Berget AI，则无需密钥。否则，请设置 BERGET_API_KEY 并重启 harness。

完整的安装、配置和 pre-commit hook 说明请参阅 README。命令由 Berget AI API 进行判断，所有被评分的数据均不会被存储（零数据保留）。闸门调用的开销很小，5 欧元免费套餐额度可以使用很长时间。

English / 原文

A coding agent with Bash access can destroy your database, print your secrets, or ignore your team’s rules, all while following instructions. guardrails-md, released today as open source, is a gate that scores every command against your repo’s GUARDRAILS.md before anything runs. If a command tips over the threshold, the call is blocked and the agent is told why, so it can pick another route. The judgement takes about 100 ms.

The two-minute explainer above shows the whole pipeline. The short version follows.

The pipeline

At session start the gate reads GUARDRAILS.md once (just the first 2,000 characters) and freezes it. The agent may edit the file later like any other file, but the change only takes effect next session, having gone through review like any other edit.

When the agent calls Bash, the hook intercepts before anything runs. It builds a state text: the command itself, the contents of any script files it points at, and the frozen guardrails. Then it asks the decision model four fixed questions, in one request:

  • destructive. Does the command irreversibly destroy data, databases, or infrastructure?
  • credentials. Does it contain, print, or send secrets, keys, or tokens?
  • guardrails_violation. Does it violate the team’s GUARDRAILS.md?
  • policy_exception. Does the guardrails text explicitly name this command as allowed?

Each answer is a score between 0 and 1, compared against a threshold (0.7 by default). Above the threshold, the call is blocked and the reason returns to the agent as the tool result:

SystemOne-gate: blocked command — destructive=0.98 > 0.7
  rm -rf ./important-data

Why a decision model

A pattern deny-list knows rm -rf; it doesn’t know your policy. Asking a second LLM to review each command does know the policy, but it can be argued with, and it costs a full generation per command. The gate instead uses berget/bev, the System One decision model we described when we launched System One: fixed questions in, scores out, one forward pass. It judges the command and the rules, never the conversation that led to them. Whatever the driving model has been persuaded of, the gate sees the command on its own merits.

The parts that do not negotiate

  • Destructive and credentials are the backstop. A guardrails_violation can be lifted by naming the command in the MAY section of GUARDRAILS.md; destructive and credentials can’t be lifted, whatever the file says.
  • Retrying costs. Every block doubles the wait before the next judgement: a few seconds at first, an hour and a half by the 20th block. Variants of the same command don’t help.
  • The gate fails closed. Failing closed is deliberate: if the endpoint is unreachable, commands are blocked until it responds again. Interactive users can opt out with SYSTEMONE_FAIL_OPEN=1. No fast path, no prefix allowlist, no override file the agent could write.
  • Some files are protected outright. The file-editing tools refuse to touch GUARDRAILS.md and the harness configuration. The refusal is deterministic: no model call, no threshold, no cooldown.

Overrides belong to the human

To change the policy, edit GUARDRAILS.md in a pull request; that is the durable fix for a false positive. To tune the gate, raise the threshold. To run one session without it, restart the harness with SYSTEMONE_GATE=off. All three are things a person does, and restarting re-reads the policy and resets the cooldown.

Also at commit time

The same judgement works as a git pre-commit hook: every staged diff is scored before it enters the repository. Personal data (GDPR) and secrets are blocked; your team’s own names in bylines and author fields pass.

The honest limits

The gate is a trained model, not a rule engine. It scores around 96% on our held-out tests, but 74% on red-team commands, and it will occasionally judge a command wrong in both directions, blocking something safe or letting something risky through. It doesn’t decode encoded payloads. Treat it as one layer among scoped credentials, sandboxes, and human review, not a sandbox. The full evaluation table and methodology live in the repo.

Try it

Three steps, about a minute.

  1. Add a GUARDRAILS.md to your repo that says what your team allows:

    ## The agent MUST NOT

- Push directly to main
- Run irreversible operations against shared systems

## The agent MAY

- Run tests, lint, and builds locally

  2. Install the gate in your harness:

    • OpenCode: Add "plugin": ["@bergetai/guardrails-md"] to opencode.json
    • Pi: Run pi install npm:@bergetai/guardrails-md in your terminal

  3. If you’re already logged in to Berget AI in OpenCode or Pi, no key is needed. Otherwise set BERGET_API_KEY and restart the harness.

Full setup, configuration, and the pre-commit hook are in the README. Commands are judged by the Berget AI API, where nothing scored is stored (zero data retention). Gate calls are small enough that the €5 free-tier credit lasts a long time.