Agent 犯错如人类工程师：我是如何用工程评审流程约束 AI 的
英文标题：🔶 Agents fail like engineers, so I review them like engineers
作者在开发面向安全包注册表 Greenflagged 时发现，AI 编码 Agent 在代码库中犯错的模式与人类工程师惊人相似，往往源于盲目假设、逻辑疏漏或短视思维。为此，作者将成熟的持续交付与同行评审流程迁移至 Agent 工作流，设计了“怀疑者（The Skeptic）”、“逻辑学家（The Logician）”和“管家（The Steward）”三个专用 Subagent，分别负责质疑无事实依据的假设、验证逻辑严密性以及评估长期架构成本。这一实践表明，管理 Agent 的核心不在于单点提升模型能力，而在于建立与人类协作同等严谨的流程约束与多视角评审机制，为规模化落地 Agent 驱动的软件工程提供了高杠杆参考。
44.7 KB
中文
我一直对各类故障与事故分析（Incidents）情有独钟。只要看到技术大会上有关于事故排查或“复盘踩坑”的演讲，我都一定会去参加。从过去的错误中汲取教训并持续改进，是软件工程中最核心的一环。在处理过多次线上生产事故后，你就会发现系统崩溃往往遵循着相似的模式。根据我的经验，最常见的事故原因之一莫过于：“我们没有（或无法）严格遵循规范流程。”
最近几个月，我一直在开发专注于安全的包注册表 Greenflagged。在此期间，我注意到 Agent 在处理代码库时犯下了不少错误。而这些错误最耐人寻味之处在于——它们竟然显得如此“具有人性（human）”。
解决问题的方式
在 AI 编码工具普及之前，我通常会坐下来，对着试图用代码解决的问题反复“试探（poke）”。这往往涉及搭建测试环境、抽离代码以实现隔离，以及不断尝试新方案，直到找到自己满意的解法。
当我观察编码 Agent 的工作方式时，我发现它们基本上也遵循相同的循环。两者的区别在于，Agent 拥有近乎无限的工具知识库。Agent 既不需要专门去查 awk 的语法，在工具使用上也没有大片盲区。我曾亲眼看到它们调用我闻所未闻的 CLI 工具来验证思路，并随后将其融入到更大的解决方案中。
Claude 可以采用与我完全一致的工作模式，而且速度更快。我在优秀软件工程师身上观察到的最佳特质之一就是坚韧。当方案没有按计划奏效时，他们不会轻易放弃，而是迎难而上、另辟蹊径。但这种挫败感往往会影响人类的情绪并蒙蔽判断力；Agent 则完全不会受此干扰。它们乐此不疲地尝试替代方案，直到找到可行的办法为止。每次开启一个崭新的上下文（Fresh Context），就像人类美美睡上一觉后重新回到棘手难题面前一样充满活力。
调优你的工作流
如果你想实现长期可持续性，工作流与交付流程就是你最应该关注的核心。正如人类一样，Agent 的工作成果同样需要经过严格评审。我当前的代码变更评审流程，其构建方式与我以往搭建过的一些最成功的持续交付（CD）流水线高度相似。
在明确想要构建的功能之后（这可以另写一篇文章详细展开……），会由一个 Agent 规划实现方案，随后由多个 Agent 针对该方案进行评审与迭代（下文将详细展开）。在方案最终敲定后，由一个 Agent 编写实现并提交合并请求（Merge Request，MR）。接着，另一个新 Agent 会运行一个 change-review skill 尝试验证该变更。不同的评审者从不同维度审视该 MR 并提供反馈；完成修复后进入下一轮评审。CI 流水线则运行测试与校验，确保项目能够干净构建且测试全部通过。
这套流程与我早在 2020 年人类工程师纯手写所有代码时所使用的流程完全一致。作为当时背负大量评审请求的高级评审人（Senior Reviewer），我往往站在“该变更会对系统整体产生何种影响”的视角，而不是去逐行核对逻辑正确性；而其他更贴近业务需求的评审人则负责验证这部分变更。不同的人类各司其职，从不同视角审视变更。
我所见到的错误类型及应对策略
基于上述认知，我该如何应对？接下来我将重点介绍在我基于 Claude Code 的工作流中带来最高杠杆收益的三大改进。我附上了这些 Agent 定义的链接，但最好的做法或许是让你的 Agent 为你量身创建专属版本，以解决你所面临的实际问题。
假设盲区（Assumptions）
🙋🏻♂️ 我自己就经常对代码的运行机制做出错误假设并惨遭打脸。通常这种问题会在软件开发生命周期的某个阶段暴露出来，但我也确实曾把带有错误假设的代码直接推送到生产环境。
🤖 Agent 仅拥有你提供给它的上下文。凡是它未能自主发现、或你未曾告知的信息，全都会变成它的主观“假设”。
怀疑者（The Skeptic）
正是各种不实假设促使我创建了第一个子 Agent：The Skeptic。怀疑者的职责正如其名——对所有主张抱持怀疑态度。它负责确认各项假设；一旦找不到支持该假设的事实依据，就会提出质疑。
几个月前在尝试消除一些误报时，某个 Agent 声称它可以删除一段代码，理由是已有另一条代码路径覆盖了该逻辑。然而这个修改建议实际上会为 Greenflagged 的告警系统新增一个盲区。作为代码评审流程（是的，代码评审依然至关重要）的一部分，“怀疑者” Agent 识别出了这个新漏洞并及时阻止了合入。
逻辑错误（Logical errors）
🙋🏻♂️ 初学编程时，你总会遇到逻辑错误与语法错误。语法错误通常意味着代码拼写有误，机器通常能够自动格式化或提示修复。逻辑错误则隐蔽得多，尽管它有时源自糟糕的假设，但并非总是如此。
🤖 Greenflagged 中曾出现过一个真实问题：当我批准一个软件包后，AI 之后就不应再次将其拒绝。而在为该问题设计修复方案时，一个逻辑 Agent 发现了一个缺陷——该缺陷可能导致系统重新批准此前已被我拒绝的包。正是评审流程在任何代码付诸编写前就捕获了这一问题。
逻辑学家（The Logician）
由此引入 The Logician。它负责审查论证的结构，即结论是否能真正从前提推导出来。它的输出包括确认结论、标出需要进一步核实的问题漏洞，以及指出那些即使补充更多证据也无法成立的错误假设。
我在架构设计与评审阶段使用该 Agent。在对比不同模型执行软件包评估时，它曾对我提出过极其有价值的反驳。该 Agent 敏锐指出了我方案中的逻辑缺陷，并帮助梳理出了一个合乎逻辑的解决方案。
短视思维陷阱（The short-term thinking trap）
软件只需满足其实际应用诉求即可。一段仅运行一次用来整理文件的脚本，不需要追求极限性能；原型 demo 也不需要考虑扩展性。然而，长期维护运营软件则需要宏观且长远的眼光。将 100 个版本的软件包及其庞大的元数据 Blob 存入关系型数据库没有任何问题；但如果面对数百万个包版本，将极少访问的数据全都塞进关系型数据库，则会占用极其庞大的存储空间。
🙋🏻♂️ 为了尽可能快速迭代，我此前没有给予足够重视，未能将操作与请求数据同大体积数据分离开来。Agent 给出的方案将几乎所有内容都保存在 Postgres 中，而其中一部分本应卸载至对象存储。如果能在一开始对数据类型和访问模式进行分类，并针对对应规模选用合适存储，后续就能省去大量的迁移成本。
🤖 曾有一个微小的登录逻辑调整，差点导致生成数百万个会被搜索引擎保留数月的 URL。“管家”及时发现了该问题，最终修复方案仅需添加一个 HTML 属性（nofollow）。
管家（The Steward）
如果你在进行长期规划，The Steward 将是你 Agent 工作流中不可或缺的利器。它能够收窄视线，聚焦于提议变更带来的长期成本。我给它的指导原则是不要直接阻断流程，而是循序渐进地引导决策，助力系统平稳健康地演进。
严格遵循流程
某个 Agent 曾精准排查出了一个发生故障的 Redis 实例：Redis 当时正在将数据持久化到磁盘，在此期间 Sentinel 健康检查出现超时。该 Agent 合并了一个用于禁用持久化的变更，并假定该变更会自动生效。它验证了 ConfigMap 确实已正确更新，却漏掉了重启 Redis 这一步，导致 Redis 依然持续写入磁盘。Agent 跳过了原本可以捕获并修复该问题的生产验证环节。自此之后，我始终确保 Agent 明确掌握如何对其工作成果进行闭环验证。
这三个 Agent 是我迄今为止在工作流中所做的最高杠杆改进。缩短人类与 Agent 的反馈周期至关重要：如果某件事对人类工程师有益，那么它大概率也对 Agent 有益；反过来则未必成立。
这种类比在哪一步不再适用？
当然，Agent 的表现不可能完全与人类一致。它们会在会话与会话之间丢失上下文，而底层模型的更新也会改变其行为模式。即便仅仅是强约束它们严格执行既定流程，往往也是一件难事。
此外，Agent 不会感到挫败。正如我前面所说，它们会反复迭代，直到找到可行方案。机器人很乐意处理那些通常令人抓狂的用户体验问题。Greenflagged 将这一理念引入到了依赖管理中：它不仅能拦截通过供应链攻击传播的恶意软件和凭证盗窃，还能在发现安全问题时主动移除存在漏洞的依赖。当依赖拉取失败时，会导致 CI 构建中断——这对人类来说可能令人崩溃。而 Greenflagged 将工具和依赖建议开放给 Agent，只要将其纳入自动化开发循环，就能保障生产环境软件的稳固安全。
Greenflagged 目前支持 npm 包管理，个人使用完全免费。
English / 原文
I’ve long been fascinated with incidents. Any time I see a conference talk scheduled or a “What went wrong” I simply have to attend. Being able to learn from past mistakes and improve is one of the most important pieces of software engineering. After dealing with a handful of production incidents, you start to see similar patterns emerge in how things fail. One of the most common incident causes in my experience is “We didn’t (or couldn’t) follow our process.”
I’ve been building Greenflagged, a security-focused package registry, for a few months now, and I’ve noticed a handful of mistakes that agents have made while working on the codebase. The most striking thing about these errors is just how human the mistakes are.
Problem-solving
Prior to the widespread use of AI coding tools, I used to sit down with a problem I was trying to solve in code and “poke” at it for a while. This often involved setting up test environments, extracting pieces of code for isolation, and generally trying new things until finding a solution I liked.
As I’ve watched coding agents work, I noticed they generally fall into the same cycle. The difference is that they have a nearly endless knowledge of tools available to them. Agents don’t have to look up awk syntax or have large blind spots in their tool-use knowledge. I’ve watched them use CLI tools I’ve never heard of to validate ideas and then incorporate them into the larger solution.
Claude can work the same way that I work, but faster. One of the best traits I’ve noticed in software engineers is persistence. They don’t give up when a solution doesn’t work out as planned. They dig in their heels and figure out another way. This can be frustrating to humans and cloud judgment. Agents don’t care, however. They are happy to try an alternative method until they find something that works. A fresh context is like coming back to a tough problem after a good night’s rest.
Tuning your workflow
Your workflow and process are the most important things you can focus on when you want to create long-term sustainability. Just like humans, agents’ work needs to be reviewed. My current change review process is shaped very similarly to some of the most successful continuous delivery pipelines that I’ve built.
After defining the feature that I want to build (that’s another post…), an agent will plan the implementation, and then multiple agents review and iterate on the plan (read on to learn more about that). After the plan is finalized, an agent builds it and submits a Merge Request (MR). A new agent runs a change-review skill which attempts to validate the change. Different reviewers look at the MR from different perspectives and provide feedback. Fixes are made and another review round runs. CI runs tests and validations to make sure the project builds cleanly and the tests pass.
This is the same process I would use back in 2020 when humans were writing all the code by hand. As a senior reviewer at the time with many review requests, I often wore the hat of “how does this change impact the system” rather than checking for logical correctness. Other reviewers that were closer to the business requirements were able to validate those types of changes. Different humans looked at the change from differing perspectives.
Types of mistakes I see and how I mitigate them
So what do I do with this realization? Next I’ll highlight the three biggest leverage changes I’ve made to my Claude Code workflow. I’ve provided links to the agent definitions, but it’s probably best if you ask your agents to create your own versions of agents to solve the problems you face.
Assumptions
🙋🏻♂️ I’m guilty of making assumptions about how code works and being wrong. Generally this turns up at some point of the software development lifecycle, but I’ve certainly shipped bad assumptions to production before.
🤖 Agents only have the context you give them. Anything they don’t discover for themselves or are told is going to be an assumption.
The Skeptic
False assumptions are what led me to create my first subagent, The Skeptic. The skeptic is in charge of, well, being skeptical of all claims. It confirms assumptions and pushes back when it cannot find facts to support an assumption.
While trying to eliminate some false positives a few months back, an agent claimed that it could remove a piece of code because it was covered by another code path. The suggested fix would have added a blind spot to the Greenflagged alerting system. As part of the code review process (yes, code reviews are still important), the skeptic agent identified this new gap and prevented it.
Logical errors
🙋🏻♂️ When learning to code, you encounter logic errors and syntax errors. Syntax errors generally mean you typed something wrong, and machines can usually reformat the issues. Logic errors are more subtle, and while they can be based off a poor assumption, they aren’t always caused by that.
🤖 A real issue appeared in Greenflagged. When I approve a package, the AI shouldn’t be able to reject it again later. While planning a fix for that issue, a logic agent spotted an issue that could have let the system re-approve a package I had rejected. The review process caught this before any code existed.
The Logician
Enter The Logician. It is in charge of reviewing the structure of an argument, whether the conclusion actually follows from the premises. It returns a confirmation, names gaps that may need to be verified, and completely broken assumptions that more evidence will not verify.
I use this agent as part of the design and review process. It very usefully pushed back on me when comparing different models to do package evaluation. The agent was able to spot logical flaws in my plan and help come up with a solution that made sense.
The short-term thinking trap
Software only needs to be as good as its use requires. A script to sort through some files a single time doesn’t need to run as efficiently as possible. A prototype doesn’t need to scale. Operating software for the longer-term requires bigger and more long-term thinking. Storing 100 package versions and all their large blobs of metadata in a relational database is fine. Doing this for millions of package versions is going to take a lot of space for something that will rarely be used.
🙋🏻♂️ While trying to iterate as fast as possible, I didn’t pay close enough attention to keeping operational and request data away from larger shapes of data. Agents came up with designs that kept most everything in Postgres, when some of that should have been offloaded to object storage. Classifying the data types and access patterns up front and picking appropriate storage for the scale would have saved a lot of migration work later.
🤖 One small login change would have created millions of URLs that search engines would hold onto for months. The steward caught it, and the fix was one HTML attribute (nofollow).
The Steward
If you’re planning long term, The Steward is an invaluable addition to your agent workflow. It is able to narrow its view and focus on the long-term costs of the suggested changes. I advise it not to block, but to gently steer decisions in ways that will help grow gracefully.
Follow the process
An agent properly diagnosed a failing Redis instance. Redis was persisting things to disk, and Sentinel health checks timed out during that. The agent merged a change to disable persistence and assumed the change would take place automatically. It validated that the ConfigMap updated correctly but failed to restart Redis which caused it to continue to save to disk. The agent skipped a production validation step that would have caught and fixed the issue. As a result, I now make sure agents know how to validate the results of their effort.
These three agents are the highest leverage changes that I’ve made to my workflow so far. Shorten feedback cycles for both humans and agents. If it’s useful for a human, it will probably be useful to an agent. The reverse is not necessarily true.
Where the comparison breaks down
Of course agents are not going to behave exactly like humans. They lose context session to session, and model updates improve and change behavior. Even trying to enforce that they stick to a process can be difficult.
Agents also don’t get frustrated. As I said before, they will iterate on things until they find something that works. A robot will be perfectly happy to deal with normally frustrating user experiences. Greenflagged brings this idea to dependency management. Not only does it block malware spread via supply-chain attacks and credential theft, it also removes vulnerable dependencies as issues are discovered. This results in a broken CI build when dependencies fail to pull, which would be infuriating for a human. Greenflagged makes tools and dependency suggestions available to agents, keeping your production software safe when added to an automated development cycle.
Greenflagged currently serves npm packages and is free for individual use.