商业产业 Hacker News AI 2026-07-05T19:18 正文 ok 1221 words

Show HN: Make No Mistakes — AI编码代理必须证明其工作

英文标题:🔶 Show HN: Make No Mistakes – AI coding agents must prove their work

GitHub上其他“绝不犯错”技能大多只是提示模型更努力,而本项目提供了强制执行机制:冻结规范、防篡改测试和能阻止提交的钩子。研究表明,AI生成的PR中约50%即使通过所有测试仍会被维护者拒绝,且前沿代理在模糊任务中高达44%的情况会硬编码测试答案。该技能为Claude Code、Codex和所有支持Agent Skills标准的代理提供了可移植的验证框架,确保所有“完成”声明必须由代理无法伪造的外部检查来证明,对提升AI编码的可靠性具有重要意义。
打开原文相关 6 · 机会 1
media521.9 KB media2.2 KB media1.3 KB media1.4 KB media1.2 KB media152.6 KB media1.5 KB media2.9 KB

中文


GitHub上其他所有“绝不犯错”技能都只是要求模型更努力地尝试。而这是一个执行机制:冻结的规范、可检测篡改的测试,以及在未证明工作完成时大声失败并阻止提交的钩子。

因为AI编码代理会犯错——更糟糕的是,它们会在未真正完成时告诉你已完成。

终端:一个代理削弱了冻结测试以求通过;make-no-mistakes检测到被篡改的哈希并中止,显示GAMING-DETECTED 一个代理削弱了冻结测试来“通过”。工具捕获了被篡改的校验和并中止。复现 →

/make-no-mistakes 是一项技能——适用于 Claude Code、Codex、OpenCode 或任何使用开放的 Agent Skills 标准的代理——它颠覆了约定:

未经验证的工作永远不会到达你手上。

不是“AI更努力”。一条硬性规定:每次“完成”必须由代理无法伪造的外部检查来证明。如果无法证明,它就大声说出来。

证据是残酷的:

  • AI生成的代码中有约50% 即使_通过所有测试_仍会被真正的维护者拒绝 (METR, 2026)
  • 在模糊任务中,前沿代理硬编码测试答案的比例高达44% (EvilGenie benchmark)
  • 大型语言模型在没有外部真值反馈的情况下无法可靠地自我纠正——自我审查甚至可能使情况更糟 (ICLR 2024)

所以“让AI再检查一遍”不起作用。有效的是结构:冻结的规范、外部预言机,以及一个在所有关卡都绿灯之前无法退出的循环。

mermaid flowchart LR A["🎯 You statethe goal"] --> B{"SPEC GATEis it unambiguous?"} B -- "fewest possiblequestions" --> A B -- "Definition of Donefrozen 🔒" --> C["EXECUTE"] C --> D["VERIFYgate stack"] D -- "red → fix" --> C D -- "all green + proven" --> E["✅ DONE_VERIFIEDevidence attached"] D -- "can't converge" --> F["🟡 STUCKescalates honestly"] D -- "tampering detected" --> G["🛑 GAMING-DETECTEDhalts, never retries"] E --> H["You accept.That's it."]

加载中

1. 明确规范。 面对模糊的目标,你无法做到零错误。该技能在触及任何内容_之前_锁定“正确”的含义——浮现出你未写出的隐含需求——并将其冻结。如果它真的必须向你询问,它会提出最少、最有价值的问题。只问一次。

2. 循环。 执行 → 验证 → 修复,按照从廉价到昂贵的关卡堆栈进行:格式、静态检查、类型、测试、回归、变异测试、安全。每个关卡都是二值的。红色即阻止。没有“足够好”。

3. 证明。 编写代码的代理绝不评判自己的工作。测试是防篡改的。一个无法失败的检查就不是检查。最终报告以证据为先,而非感觉。

当它_无法_交付时,它从不假装。六个终端状态,从不模糊:

状态 含义
✅ DONE 所有关卡绿色 + 独立验证通过。附有证明。
🟡 STUCK-BUDGET 成本上限已用尽,诚实地持续进展中。
🟡 STUCK-OSCILLATING 关卡翻转而不收敛——命名并发出振荡警告。
🔴 STUCK-INCONCLUSIVE 验证本身无法运行——未验证 ≠ 被驳斥。
🛑 GAMING-DETECTED 检测到检查被削弱/编辑以求通过。中止,永不静默重试。
🛑 INTEGRITY-COMPROMISED 冻结的规范或工具文件被篡改。
  1. 无证明则无声明。 每个“它能工作”都由实际运行过的检查来支撑。
  2. 作者永不评判作者。 验证与实现分离运行——独立上下文,冻结的标准。
  3. 一个无法失败的检查就不是检查。 关卡基于阴性对照进行验证;作弊的测试会被检测到,而不被信任。

本项目正以开放、研究为先的方式构建。其设计基于同行评审的证据和行业标准(ISO/IEC 25010、IEEE 1012、ISO/IEC/IEEE 29148/29119、DO-178C独立性原则)——而非感觉。

  • 深度研究:自我纠正的局限、奖励欺骗、验证预言机
  • 研究档案 → research/ & .planning/
  • v0-alpha :可移植的规范核心(规范关卡 → 预算循环 → 关卡堆栈 → 独立验证器 → 诚实的终端状态)
  • v0 — 强制实施,现可安装 :带有篡改哈希的确定性关卡脚本、PreToolUse写入/读取拒绝钩子、由判定结果控制的Stop-hook循环、操作系统级git-hook后备、ed25519签名的清单——全部配有一套绿色的自评估套件,证明良好工作能通过,不良工作被捕获,以及作弊会被标记为 GAMING-DETECTED
  • 多代理创建层 :专用角色技能(规划者 · 测试作者 · 验证小组 · 集成者),并带有防止共识陷阱的独立性保护
  • DONE需要独立验证器(v0.8):一个 verify 关卡仅由独立的重新执行发出——实现者自己的关卡行不能单独达成DONE;按任务划分作用域,避免一个任务洗白另一个任务的结果
  • 变异关卡已接入 + 覆盖度 / SAST / 供应链 / shellcheck 关卡类纳入堆栈;针对工具自身脚本有494个单元测试和约98%的覆盖率
  • 标准审计(ISO/IEC 25010 · ISO/IEC/IEEE 29119/12207/1012/29148 · SWEBOK v4)——参见 docs/evals/;所有差距均已命名,无隐藏
  • ed25519签名的清单(R13,带外维护者密钥)作为信任根——check_manifest.sh,在CI中进行严格把关,并带有篡改阴性对照
  • DONE时刻重新执行确定性关卡堆栈(在损坏代码上键入的 DONE 会被拒绝);该工具针对自身实施与验证,标准严格度评分约43%,并经过独立重新评分
  • OpenCode强制执行端口;差异/分层变异;性质/留出关卡
  • 公开基准测试:使用该技能与不使用该技能的对比(评估计划)

当前发布版本:v0.13.0(参见 CHANGELOG)——包含确定性关卡脚本与篡改检测、强制执行钩子、多代理验证面板、变异关卡、ed25519签名的清单信任根、DONE时刻关卡重新执行以及一套绿色的自评估套件。它已在公开场合接受标准审计(严格MET评分约43%,独立重新评分),并诚实地申明自身边界——这是家规。

Claude Code——一条命令:

/plugin marketplace add momomuchu/make-no-mistakes /plugin install make-no-mistakes@make-no-mistakes

通用——Claude Code · Codex · OpenCode · 任何 Agent Skills 运行时:

git clone https://github.com/momomuchu/make-no-mistakes ~/.make-no-mistakes mkdir -p ~/.claude/skills ~/.agents/skills

链接所有五个技能——编排器及其专家(达成DONE需要mnm-verify)

for s in make-no-mistakes mnm-planner mnm-test-author mnm-verify mnm-integrator; do ln -s ~/.make-no-mistakes/skills/$s ~/.claude/skills/$s ln -s ~/.make-no-mistakes/skills/$s ~/.agents/skills/$s done

为什么要有两个符号链接:Claude Code 在 ~/.claude/skills/ 中发现技能;Codex 在 ~/.agents/skills/ 中发现它们;OpenCode 则_同时_自动读取两者(已对各运行时的官方文档进行验证)。项目范围的安装同样有效:将文件夹放入仓库内的 .claude/skills/ 或 .agents/skills/ 中。

使用方法: 在任何任务前说 “make no mistakes”——或直接调用 /make-no-mistakes。先验证其已加载(列出你运行时的技能)。不要盲目信任,要验证——包括对我们。

它能保证零bug吗? 不能——没有任何东西可以,任何声称能做到的人都是在向你推销。保证是不同的,而且是诚实的:未经验证的工作不会到达你手上。每个“完成”都由实际运行的检查来支持,这些检查基于代理无法改写的关卡。

它与哪些代理兼容? 任何编码代理,只要使用开放的 Agent Skills 标准——Claude Code、Codex CLI、OpenCode,以及采用该标准的约40个运行时。三个诚实的层面:技能核心(规范、脚本、验证逻辑)随处可移植;硬执行钩子是按运行时适配的(目前在Claude Code上最深);操作系统级git后备是普适的——无论哪个代理在驱动,它都能工作,因为它完全不需要代理的协作。

这不就是“运行测试”吗? 不是。代理自己编写的测试可以被欺骗——有证据表明,代理会删除测试、硬编码预期输出并削弱断言。这个工具验证的是_检查本身_:对测试文件进行篡改检测、通过变异测试证明测试能够失败,以及一个从不与实现者共享上下文的验证器。

设计正在进行中——这是参与的最佳时机。请参见 CONTRIBUTING.md。

MIT


绝不犯错。当你做不到时——坦率说明,证明原因,永不假装。

English / 原文


Every other "make no mistakes" skill on GitHub is a prompt telling the model to try harder. This one is enforcement machinery : frozen specs, tamper-detected tests, and hooks that fail loudly and block the commit when the work isn't proven.

Because AI coding agents make mistakes — and worse, they tell you they're done when they're not.

Terminal: an agent weakens a frozen test to pass; make-no-mistakes detects the tampered hash and halts with GAMING-DETECTED An agent weakens a frozen test to "pass". The harness catches the tampered checksum and halts.Reproduce it →

/make-no-mistakes is one skill — for Claude Code, Codex, OpenCode, or any agent speaking the open Agent Skills standard — that flips the contract:

No unverified work ever reaches you.

Not "the AI tries harder." A hard rule: every "done" must be proven by an external check the agent cannot fake. If it can't prove it, it says so — loudly.

The evidence is brutal:

  • ~50% of AI-generated PRs that pass all tests would still be rejected by real maintainers (METR, 2026)
  • On ambiguous tasks, frontier agents hardcode test answers up to 44% of the time (EvilGenie benchmark)
  • LLMs cannot reliably self-correct without external ground-truth feedback — self-review can even make things worse (ICLR 2024)

So "just ask the AI to double-check" doesn't work. What works is structure : frozen specs, external oracles, and a loop that cannot exit until everything is green.

flowchart LR
    A["🎯 You state<br/>the goal"] --> B{"SPEC GATE<br/>is it unambiguous?"}
    B -- "fewest possible<br/>questions" --> A
    B -- "Definition of Done<br/>frozen 🔒" --> C["EXECUTE"]
    C --> D["VERIFY<br/>gate stack"]
    D -- "red → fix" --> C
    D -- "all green + proven" --> E["✅ DONE_VERIFIED<br/>evidence attached"]
    D -- "can't converge" --> F["🟡 STUCK<br/>escalates honestly"]
    D -- "tampering detected" --> G["🛑 GAMING-DETECTED<br/>halts, never retries"]
    E --> H["You accept.<br/>That's it."]

Loading

1. Specify. You can't make no mistakes against a vague goal. The skill locks down what "correct" means before touching anything — surfacing the requirements you implied but never wrote — and freezes it. If it truly must ask you something, it asks the fewest, highest-value questions. Once.

2. Loop. Execute → verify → fix, against a stack of gates ordered cheap-to-expensive: format, lint, types, tests, regressions, mutation testing, security. Every gate is binary. Red blocks. No "good enough."

3. Prove. The agent that wrote the code never grades its own work. Tests are tamper-protected. A check that cannot fail is not a check. The final report leads with evidence, not vibes.

And when it can't deliver, it never pretends. Six terminal states, never blurred:

State Meaning
✅ DONE Every gate green + independently verified. Proof attached.
🟡 STUCK-BUDGET Cost ceiling spent, honest progress ongoing.
🟡 STUCK-OSCILLATING Gates flip without converging — thrashing, named.
🔴 STUCK-INCONCLUSIVE Verification itself couldn't run — unverified ≠ refuted.
🛑 GAMING-DETECTED A check was weakened/edited to pass. Halts, never silently retried.
🛑 INTEGRITY-COMPROMISED The frozen spec or a harness file was tampered with.
  1. No claim without proof. Every "it works" is backed by a check that actually ran.
  2. The author never grades the author. Verification runs separate from implementation — separate context, frozen criteria.
  3. A check that cannot fail is not a check. Gates are validated against negative controls; gamed tests are detected, not trusted.

This project is being built in the open, research-first. The design is grounded in peer-reviewed evidence and industry standards (ISO/IEC 25010, IEEE 1012, ISO/IEC/IEEE 29148/29119, DO-178C independence principles) — not vibes.

  • Deep research: self-correction limits, reward hacking, verification oracles
  • Research dossier → research/ & .planning/
  • v0-alpha : portable discipline core (spec gate → budgeted loop → gate stack → independent verifier → honest terminal states)
  • v0 — hard enforcement, installable now : deterministic gate scripts with tamper hashing, PreToolUse write/read-denial hooks, verdict-gated Stop-hook loop, OS git-hook backstop, ed25519-signed manifest — all with a green self-eval suite proving good work passes, bad work is caught, and a cheat → GAMING-DETECTED
  • Multi-agent creation layer : specialized role skills (planner · test-author · verify panel · integrator) with an independence guard against the consensus trap
  • Independent verifier required for DONE (v0.8): a verify gate emitted only by an independent re-execution — the implementer's own gate rows can't reach DONE alone; task-scoped so one task can't launder another's
  • Mutation gate wired + coverage / SAST / supply-chain / shellcheck gate classes in the stack; 494 unit tests + ~98% coverage for the harness's own scripts
  • Standards-audited (ISO/IEC 25010 · ISO/IEC/IEEE 29119/12207/1012/29148 · SWEBOK v4) — see docs/evals/; every gap named, nothing hidden
  • ed25519-signed manifest (R13, out-of-band maintainer key) as the trust root — check_manifest.sh, hard-gated in CI with a tamper negative control
  • DONE re-executes the deterministic gate stack (a DONE typed on broken code is refused); the harness dogfooded against itself, standards score ~43% strict and independently re-graded
  • OpenCode enforcement port; differential/tiered mutation; property/holdout gates
  • Public benchmark: with vs. without the skill (eval plan)

Current release: v0.13.0 (see CHANGELOG) — deterministic gate scripts with tamper detection, enforcement hooks, the multi-agent verification panel, a mutation gate , an ed25519-signed manifest trust root, DONE-time gate re-execution , and a green self-eval suite. It is standards-audited in public (~43% strict MET, independently re-graded) and states its own boundaries honestly — that's the house rule.

Claude Code — one command:

/plugin marketplace add momomuchu/make-no-mistakes
/plugin install make-no-mistakes@make-no-mistakes

Universal — Claude Code · Codex · OpenCode · anyAgent Skills runtime:

git clone https://github.com/momomuchu/make-no-mistakes ~/.make-no-mistakes
mkdir -p ~/.claude/skills ~/.agents/skills
# link all five skills — the orchestrator PLUS its specialists (a DONE needs mnm-verify)
for s in make-no-mistakes mnm-planner mnm-test-author mnm-verify mnm-integrator; do
  ln -s ~/.make-no-mistakes/skills/$s ~/.claude/skills/$s
  ln -s ~/.make-no-mistakes/skills/$s ~/.agents/skills/$s
done

Why two symlinks: Claude Code discovers skills in ~/.claude/skills/; Codex discovers them in ~/.agents/skills/; OpenCode reads both automatically (verified against each runtime's official docs). Project-scoped install works too: drop the folder into .claude/skills/ or .agents/skills/ inside a repo.

Use it: say "make no mistakes" before any task — or invoke /make-no-mistakes directly. Verify it's loaded first (list your runtime's skills). Don't trust, verify — including us.

Does it guarantee zero bugs? No — nothing can, and anyone claiming otherwise is selling you something. The guarantee is different and honest: no work reaches you unverified. Every "done" is backed by checks that ran, on gates the agent can't rewrite.

Which agents does it work with? Any coding agent that speaks the open Agent Skills standard — Claude Code, Codex CLI, OpenCode, and the ~40 runtimes adopting it. Three honest layers: the skill core (specs, scripts, verification logic) is portable everywhere ; hard enforcement hooks are per-runtime adapters (deepest on Claude Code today); the OS-level git backstop is universal — it works no matter which agent is driving, because it doesn't need the agent's cooperation at all.

Isn't this just "run the tests"? No. Tests the agent wrote itself can be gamed — agents demonstrably delete tests, hardcode expected outputs, and weaken assertions. This harness verifies the checks themselves : tamper detection on test files, mutation testing to prove tests can fail, and a verifier that never shares context with the implementer.

Design is happening now — the best time to weigh in. See CONTRIBUTING.md.

MIT


Make no mistakes. And when you can't — say so, prove why, and never pretend.