🔶 ProofRun：面向 AI 编程 Agent 的本地验证凭据
英文标题：🔶 ProofRun – a local verification receipt for AI coding agents
ProofRun 是一款本地验证工具，用于以密码学方式证明测试或构建检查确实在当前精确代码状态上运行过，而非仅相信 AI Agent 的口头声明。它将检查结果绑定到 Git `HEAD`、未提交改动及未跟踪文件的哈希，代码发生任意单字节变化后，原有结果会自动变为 `STALE`。该工具不调用 LLM、完全离线，并通过 `PASS`、`FAIL`、`STALE`、`NOT RUN` 四种状态向开发者提供可核验结果。项目还提供 GitHub Action，可在 PR 的精确 head commit 上重新执行检查，但当前无法阻止 PR 同时弱化 `.proofrun.yml` 中的检查配置。
1.2 KB
3.5 KB
2.2 KB
2.2 KB
1.2 KB
146.3 KB
88.3 KB
中文
ProofRun 不判断你的代码是否正确。它以密码学方式证明——而不是通过礼貌地询问——哪些检查确实针对你此刻拥有的精确代码运行过。
一名 AI 编程 Agent 说“所有测试均已通过”。这是真的吗？
也许是真的。上一次 Agent 实际运行测试时，这句话确实为真。但那可能已经是三次编辑之前的事了。Agent 甚至可能不记得运行过测试——它或许只是推断“这个改动看起来没问题，测试大概仍会通过”。仅从这些话本身，你无法区分“我运行了测试且它通过了”和“我相当确定它会通过”。
ProofRun 弥合了这一差距。它不是让 Agent 变得更诚实，而是让这一声明本身变得可检查。
$ proofrun run test -- pytest
...
test: pass (exit 0, 1841ms)
$ proofrun status
test PASS (exit 0, 1841ms)
# 此后的代码发生改动——无论来自 Agent 还是人类，都无关紧要
$ proofrun status
test STALE (last run: pass, exit 0 — code changed since)
每个检查结果都绑定到精确代码状态的指纹：Git commit，加上所有未提交内容的哈希——无论其是否已暂存、是否已跟踪。只要改动一个字节，结果就会自动变为 STALE。没有人需要记得询问“这个 PASS 是否依然有效？”。
curl -L https://github.com/yebiguo/proofrun/releases/download/v0.2.0/proofrun_linux_amd64.tar.gz | tar xz
# 其他平台：https://github.com/yebiguo/proofrun/releases
或者从源码构建：
go install github.com/yebiguo/proofrun/cmd/proofrun@latest
proofrun init # 写入 .proofrun.yml
proofrun run test -- pytest # 实际运行 pytest，并绑定结果
proofrun status --strict # 如有任何检查不是 PASS，则以非零状态码退出
为什么不只是信任 Agent
- 不在任何地方调用 LLM。 ProofRun 不使用 AI 来验证 AI。它会启动真实子进程并读取其真实退出码——这就是完整机制。
- 四种状态，绝不猜测。
PASS、FAIL、STALE、NOT RUN——每一种状态均来自被观测到的执行，或有文档记录的未执行状态。不存在第五种“大概没问题”。 - 完全离线。 零网络请求、零遥测、零账户。
- 精确比对 argv，而非字符串匹配。 声明为
pytest -k "foo bar"的检查，不能由一条仅在扁平化为文本后看起来相似的命令来满足——ProofRun 比较真实参数数组，而非字符串。
ProofRun 有意不做的事
它不解析测试输出，不评判代码质量，也不自动修复任何问题。完整边界请参阅 AGENTS.md。
由两个 AI Agent 构建，但不单独信任任何一个
ProofRun 由 AI 编程 Agent（Claude Code）在人类指导下编写。每一项改动——不只是首次发布——在合并前都会经由第二个 AI Agent（Codex）进行独立、只读的对抗性审查；人类负责指导工作并作出实际合并决定，但任何一个 Agent 自己声称“这能工作”都不会被单独视为充分依据。
这一审查流程并非形式主义。在首次发布前，它发现 ProofRun 自身的命令比较机制可以被欺骗：一个错误引用的 shell 参数会让检查在静默运行零个测试后仍报告 PASS。完整复现、精确修复方案，以及为何简单补丁不足以解决问题，见 → docs/case-study.md。随着项目成长，同一流程仍在发现真实的严重关键漏洞——最近一次是在 v0.3 中，它在代码发布前发现：符号链接形式的 .proofrun/ 目录可能使攻击者植入的签名密钥被静默采纳为可信密钥。
每一项修复在被接受前，都会针对真实复现进行验证——不仅是审查其表面合理性，也不仅限于发布时。一个旨在让 AI Agent 对其行为负责的工具，如果无法持续经受同样施加于自身的审查，而只是把它当作一次性关卡，就不该存在。
proofrun init # 生成 .proofrun.yml
proofrun run <check-name> -- <cmd> # 实际运行 <cmd>，将退出码和耗时绑定到当前 Git 状态
proofrun run-all [--only <name>] # 运行每项已声明检查，并在每项完成后保存结果
proofrun status [--strict] # 每项检查显示 PASS / FAIL / STALE / NOT RUN；若必需检查不是 PASS，--strict 将以非零状态码退出
proofrun report [--json] # 完整报告，人类或机器均可读取
checks:
test:
command: [pytest]
required: true
build:
command: [npm, run, build]
required: true
lint:
command: [ruff, check, .]
required: false
command 是 argv 列表，而不是 shell 字符串——ProofRun 从不通过 shell 运行命令，并且必须逐元素精确比较实际运行的命令与声明内容。required: true 会使某项检查阻止 status --strict 成功通过，这正是你会接入 pre-commit hook 或 CI gate 的机制。
指纹如何工作
每个结果都绑定到当前 Git HEAD，以及 git diff HEAD 与所有未跟踪、未忽略文件内容合并后计算的 SHA-256 哈希。proofrun status 每次都会重新计算该指纹，并与本地存储的内容比较——任何不匹配，即使只是改动一个空格或新增一个文件，都会报告 STALE。
每个存储的结果均通过首次使用时生成的随机密钥进行签名（HMAC-SHA256）；密钥保存在 .proofrun/secret。该文件会自动创建，并通过仓库本地的 .git/info/exclude 尽力避免被 Git 跟踪（即使你自己的 .gitignore 从未提及 .proofrun/）；如果密钥最终仍被 Git 跟踪，ProofRun 会拒绝信任它，而不是使用任何克隆仓库的人都已知的密钥进行签名）。ProofRun 自身绝不会将该密钥传输到任何地方。手动编辑过的 receipt.json——即使其中的指纹完全匹配——也将无法再通过验证。对于 .proofrun.yml 中声明的检查，这会显示为 NOT RUN；对于曾以 proofrun run 单次运行、但从未在那里声明的检查，不可信条目会被完全丢弃，该检查也会从 status 输出中消失——它绝不会显示虚假的 PASS，只是不会被计入结果。
这项机制保证什么，又不保证什么：
-
可发现篡改，而非防篡改。 它可以发现朴素的手动编辑（或不了解签名机制、试图伪造结果的 AI Agent）——无法防御已经能够读取
.proofrun/secret并伪造对应签名的复杂攻击者。这是任何纯本地完整性方案的固有局限。 -
机器本地证据，而非可移植证据。 在没有
.proofrun/secret的情况下，将receipt.json复制到另一台机器无法通过验证。它从来就不是要交给他人作为证明；GitHub Action 的独立重新运行才会产出第三方应当信任的证据。 -
不防御回滚或重放。 此前一次真实运行生成的、确实已签名的凭据，如果之后被恢复到具有相同指纹的工作树中（例如代码被还原），仍会正确通过验证——签名证明的是“这台机器曾在某一时刻生成过此结果”，而不是“这是最近一次运行”。
-
不迁移 v0.3 之前的凭据——它们会直接显示为
NOT RUN；请重新运行检查。 -
GitHub Action 不依赖上述任何机制——它绝不会信任已检出的
receipt.json（会在重新运行前清理.proofrun/），因此本地签名与 Action 输出何以可信无关。on: pull_request permissions: contents: read jobs: verify: runs-on: ubuntu-latest steps: - uses: yebiguo/proofrun@v1
它会自行检出精确的 PR head commit——绝不信任调用工作流已经检出的任何内容，因此 pull_request 触发器无法悄悄向它传入 GitHub 合成的合并预览 commit。随后，它会清除 PR 分支带入的所有 receipt.json，下载经校验和验证的 proofrun 二进制文件，并实际运行 proofrun run-all，然后以 proofrun status --strict 作为门禁。它从不信任提交到 PR 分支中的任何凭据——门禁看到的每项结果都由这次运行产生。
**已知限制：**它无法防止 .proofrun.yml 本身被与代码改动同一个 PR 弱化——PR 可以放宽或删除某项检查命令，Action 会如实重新运行这个更弱的版本。当 .proofrun.yml 与 PR 的基础分支不同时，它会发出警告（通过构建注释），但不会因此阻止合并；请像审查改动的任何其他部分一样审查这一 diff。
- v0.4 ——为常见测试运行器提供结构化输出支持（pytest、Jest、JUnit）
- 防止
.proofrun.yml在修改代码的同一 PR 内被弱化（目前仅发出警告，不会阻止——参见上文“已知限制”） - 比本地可发现篡改更强的机制（远程证明、公开密钥签名）尚未设计——有关 v0.3 做了什么以及有意不保证什么，请参阅上文“可发现篡改的凭据”
欢迎提交 Issue 和 PR。这是一个年轻的、pre-1.0 阶段的项目，范围刻意保持狭窄——在提出任何涉及 STALE 检测或凭据 schema 的建议前，请先阅读 AGENTS.md；这些正是该项目最不能出错的部分。
MIT
English / 原文
ProofRun doesn't judge whether your code is correct. It proves — cryptographically, not by asking nicely — which checks actually ran against the exact code you have right now.
An AI coding agent says "all tests pass." Is that true?
Maybe. It was true the last time the agent actually ran the tests. But that might have been three edits ago. The agent might not even remember running them — it might just be inferring "the change looks right, tests probably still pass." From the words alone, you have no way to tell "I ran it and it passed" apart from "I'm pretty sure it would pass."
ProofRun closes that gap. Not by making the agent more honest — by making the claim itself checkable.
$ proofrun run test -- pytest
...
test: pass (exit 0, 1841ms)
$ proofrun status
test PASS (exit 0, 1841ms)
# code changes after this point — agent or human, doesn't matter
$ proofrun status
test STALE (last run: pass, exit 0 — code changed since)
Every check result is bound to a fingerprint of your exact code state: the git commit, plus a hash of everything uncommitted — staged or not, tracked or not. Change a single byte, and the result flips to STALE automatically. Nobody has to remember to ask "does this PASS still count?"
curl -L https://github.com/yebiguo/proofrun/releases/download/v0.2.0/proofrun_linux_amd64.tar.gz | tar xz
# other platforms: https://github.com/yebiguo/proofrun/releases
Or build from source:
go install github.com/yebiguo/proofrun/cmd/proofrun@latest
proofrun init # writes .proofrun.yml
proofrun run test -- pytest # runs pytest for real, binds the result
proofrun status --strict # non-zero exit if anything isn't PASS
Why this, not just trusting the agent
- No LLM calls, anywhere. ProofRun doesn't use AI to verify AI. It starts a real subprocess and reads its real exit code — that's the entire mechanism.
- Four statuses, never a guess.
PASS,FAIL,STALE,NOT RUN— each one comes from an observed execution, or the documented absence of one. There's no fifth "probably fine." - Fully offline. Zero network calls, zero telemetry, zero accounts.
- Argv-exact, not string-matched. A check declared as
pytest -k "foo bar"can't be satisfied by a command that merely looks similar once flattened to text — ProofRun compares real argument arrays, not strings.
What ProofRun deliberately does not do
It does not parse test output, does not judge code quality, and does not auto-fix anything. See AGENTS.md for the complete boundary.
Built by two AI agents, neither of which is trusted alone
ProofRun is written by an AI coding agent (Claude Code) under human direction. Every change — not just the first release — goes through independent, read-only adversarial review by a second AI agent (Codex) before it merges; the human directs the work and makes the actual merge decisions, but neither agent's own claim of "this works" is treated as sufficient on its own.
That review process isn't a formality. Before the first release, it found that ProofRun's own command comparison could be tricked: a misquoted shell argument made a check silently run zero tests and still report PASS. Full repro, the exact fix, and why a simple patch wasn't enough → docs/case-study.md. The same process is still catching real, severity-critical bugs as the project grows — most recently in v0.3, where it caught a symlinked .proofrun/ directory letting an attacker-planted signing key get silently adopted as trustworthy, before that code ever shipped.
Every fix is verified against a real reproduction before being accepted — not just reviewed for plausibility, and not just at launch. A tool built to hold AI agents accountable has no business existing if it can't survive that same scrutiny applied to itself, continuously, not as a one-time gate.
proofrun init # generate .proofrun.yml
proofrun run <check-name> -- <cmd> # run <cmd> for real, bind exit code + duration to current git state
proofrun run-all [--only <name>] # run every declared check, saving a result after each one
proofrun status [--strict] # PASS / FAIL / STALE / NOT RUN per check; --strict exits non-zero if a required check isn't PASS
proofrun report [--json] # full report, human- or machine-readable
checks:
test:
command: [pytest]
required: true
build:
command: [npm, run, build]
required: true
lint:
command: [ruff, check, .]
required: false
command is an argv list, not a shell string — ProofRun never goes through a shell, and comparing what actually ran against what's declared has to be exact, element for element. required: true is what makes a check block status --strict, which is what you'd wire into a pre-commit hook or CI gate.
How the fingerprint works
Every result is bound to your current git HEAD plus a SHA-256 hash of git diff HEAD combined with the contents of any untracked, non-ignored files. proofrun status recomputes that fingerprint every time and compares it against what's stored locally — any mismatch, down to a single changed space or one new file, reports STALE.
Every stored result is signed (HMAC-SHA256) with a random key generated on first use and kept at .proofrun/secret — created automatically, kept out of git on a best-effort basis via the repository-local .git/info/exclude (even if your own .gitignore never mentions .proofrun/; if the key ever does end up git-tracked anyway, ProofRun refuses to trust it rather than signing with a key anyone who cloned the repo already knows). ProofRun itself never transmits the key anywhere. A hand-edited receipt.json — even one with a fingerprint that matches perfectly — no longer verifies. For a check declared in .proofrun.yml, that surfaces as NOT RUN; for a one-off check that was proofrun run without ever being declared there, the untrusted entry is discarded entirely and the check disappears from status output instead — it never shows a false PASS, it just won't be counted.
What this does and doesn't guarantee:
-
Tamper-evident, not tamper-proof. This catches a naive hand-edit (or an AI agent that doesn't know signing exists trying to fake a result) — not a sophisticated attacker who can already read
.proofrun/secretand forge a matching signature. That's an inherent limit of any local-only integrity scheme. -
Machine-local, not portable evidence. Copy
receipt.jsonto another machine without.proofrun/secretand it won't verify. This was never meant to be handed to someone else as proof — the GitHub Action's independent re-run is what produces evidence a third party should trust. -
Doesn't defend against rollback/replay. A genuinely signed receipt from an earlier real run, restored later against a working tree with the same fingerprint again (e.g. code reverted), verifies correctly — the signature proves "this machine produced this at some point," not "this is the latest run."
-
No migration for pre-v0.3 receipts — they simply read as
NOT RUN; re-run the check. -
The GitHub Action doesn't rely on any of this — it never trusts a checked-out
receipt.jsonin the first place (clears.proofrun/before re-running), so local signing has nothing to do with what makes the Action's output trustworthy.on: pull_request permissions: contents: read jobs: verify: runs-on: ubuntu-latest steps: - uses: yebiguo/proofrun@v1
This does its own checkout of the exact PR head commit — it never trusts whatever the calling workflow already checked out, so a pull_request trigger can't silently hand it GitHub's synthetic merge-preview commit instead. It then clears out any receipt.json that came in on the PR branch, downloads a checksum-verified proofrun binary, and runs proofrun run-all for real before gating on proofrun status --strict. Nothing about a receipt checked into the PR branch is ever trusted — every result the gate sees was produced by this run.
Known limitation: this does not protect .proofrun.yml itself from being weakened by the same PR that changes the code — a PR could loosen or remove a check's command and the Action would faithfully re-run the weaker version. It warns (via a build annotation) when .proofrun.yml differs from the PR's base branch, but it does not block on that; review that diff the same way you'd review any other part of the change.
- v0.4 — structured output support for common test runners (pytest, Jest, JUnit)
- Protecting
.proofrun.ymlitself from being weakened within the same PR that changes the code (currently only warned about, not blocked — see "Known limitation" above) - Anything stronger than local tamper-evidence (remote attestation, public-key signing) is not yet designed — see "Tamper-evident receipts" above for what v0.3 does and deliberately doesn't guarantee
Issues and PRs welcome. This is a young, pre-1.0 project with a narrow, deliberate scope — see AGENTS.md before proposing anything that touches STALE detection or the receipt schema; those are the parts this project can least afford to get wrong.
MIT