🔶 知识粉碎机：一个 Agent 式编程故事

在我的日常开发工作中，我发现我的 Agent 必须围绕我交给它们的问题构建数量惊人的知识。它们会扫描文件、搜索 API 文档，并做大量工作，让自己的上下文足够完整，最终才能输出真正用于代码改动的那相对很少的一部分 token。这类似于开发者阅读大量代码、在脑中构建非常详细的系统运行模型，然后只编辑真正需要修改的文件。而（几乎）所有这些知识都会……消失。

我在这里想写的内容（尤其是那个梗图）受到了这则漫画的启发。我所看到的当今差异在于，与其说程序员被打断了，不如说 LLM 所完成的绝大多数工作，在任何实际意义上都丢失了。当然，聪明的程序员经常会让 LLM 自己撰写提交信息，这很好。没错……你也可以在自己的机器上恢复一个会话。但这距离捕捉 LLM 所构建状态的完整性，仍然差得很远。如果我想从此前工作结束的地方继续，这会给组织知识、会话可恢复性和效率带来巨大损失。

我最近读了 《你无法带走的会话》（HN 讨论见此），它让我印象深刻。在读到这篇文章之前，我并不特别反对让专有 AI 工具帮助我完成工作。然而，这篇文章确实提炼出了一个问题：把 Agent 当作黑箱，实际上非常有害。

认真想想……即使只是一个简单的情况：一位队友偏好 Codex，另一位队友则使用 Claude。一个 LLM 辛苦构建出对逻辑的丰富理解，接着另一个 LLM 需要处理同一份代码……却得从头开始。我根本无法估算有多少百万 token 就这样被烧掉了。

坦白说，“LLM 可移植性”这个概念以前听起来有点故作高深。就像单态性，或者麸质不耐受。但机缘巧合的是，本周早些时候，我和一位常驻迪拜的技术界朋友通了电话。他描述了巴林 AWS 数据中心遭遇的惨状。他指出了一件很有意思的事：该地区的许多公司绝对不能让自己的 AI 使用在其他地区运行，因为……嗯……战争。所以，当某项服务的整个地区因为这场战争而遭受大规模中断时……LLM 可移植性突然就变得真实得多，也紧迫得多。

我的整个团队大多数时候都在使用 Claude。我想我是那个热衷于尝试不同 LLM、让它们彼此对比、也让它们协同工作的早期采用者。在我提交代码前，让 Google 的 Agent 审查 Claude 的工作，这种做法让我感觉很对。

但让这些 LLM 协同工作，也让我意识到：每次启动一个新会话或运行 git commit 时，我到底丢弃了多少上下文。

现在再谈谈最前沿的情况。如果 Claude 下线了，我想让 Codex 接手 Claude 已经开始的工作，该怎么办？又或者，我想在 Claude 和我已经深入处理过的某个代码库区域，试用其他模型之一呢？我真的希望能够从上次离开的地方继续，而不是第 n 次把 token 重新投入 token 火堆中燃烧。

在更大的团队中，问题远比这更深刻。代码审查者——无论是人类还是其他形式——要如何跟上一次代码变更背后的所有细微之处？

我的朋友 Philip 曾在 OpenAI 工作，他写过一篇很有意思的文章，讨论在 AI 能力大幅超越人类的时代如何进行代码审查。这在这里显得非常相关。截至今天，已经完全可能提交那些细节极其复杂、规模也足够庞大的代码变更，以至于审查它们非常困难。而且，由于如此多的上下文都留在开发者的机器上，那些 token 消耗留下的唯一痕迹，就只是一条提交信息，以及 LLM 认为有必要写下的那部分代码注释。

在 LLM 出现之前，担任一个快速迭代项目的技术负责人就已经是艰苦的工作。现在，每个 PR 都是一份新鲜出炉的、（未文档化或仅半文档化的）复杂性包裹。初级开发者刚刚通过氛围编程写出了大量新的隐私逻辑，便合上笔记本电脑回家了。与此同时，那位真正了解隐私层如何运作的高级工程师，却必须在完全没有上下文的情况下艰难梳理这一大堆内容。他无法恢复一个自己从未开启过的会话。那么……他是不是应该……让自己的 LLM 再次生成一遍此前已经生成过的全部上下文，以便理解这份脱离上下文的输出？

这感觉太疯狂了。

如果我今天回到代码库中一个昨天刚刚为之消耗了 25 万 token 的区域……我真的又要花上数万更多 token，重新把上下文加载进上下文窗口吗？答案是，没错……没错，我就是会这么做。

In my day to day development work, I find that my agents have to build up an incredible amount of knowledge about the problem I set them on. They scan files. They search API docs. They do a lot of work to get their context sufficiently full to be able to squirt out the relatively tiny number of final tokens that go into an actual code change. This is analogous to the way a developer reads a lot of code, builds a very detailed mental model of what the system is doing, and then edits just the files they need to. And (nearly) all that knowledge goes… away.

What I’m writing about here (especially the meme) is inspired by this cartoon strip. The difference that I see in today’s world is that it’s less that the programmer is getting interrupted and more that the vast majority of the work that the LLM has done gets lost in every practical sense. Sure, smart programmers often have the LLM write its own commit message. That’s good. And yes… you can resume a session… on your own machine. But this is still miles away from capturing the fullness of the state that the LLM put together. And that’s a huge loss of institutional knowledge, session resum-ability, and efficiency if I want to pickup from where that work left off.

I recently read The Session You Cannot Take With You (HN discussion here) and it really stuck with me. Before reading this, I wasn’t particularly opposed to having proprietary AI tools involved in helping me get my work done. However, this article really distils why agents as black boxes is actually really damaging.

Seriously… think about it. Something as simple as one teammate preferring Codex while another teammate is using Claude. One LLM works hard to build a rich picture of the logic. Then another LLM needs to work on the same code… and starts completely from scratch. I cannot estimate the millions of tokens that are burnt this way.

The whole concept of ‘LLM portability’ sounded a bit high-minded to me, honestly. Like monomorphism. Or gluten intolerance. But in a turn of serendipity, I was on a call with a tech friend based in Dubai earlier this week. He was describing the carnage at the AWS datacenter in Bahrain. The fascinating thing he pointed out is that many companies in that region absolutely cannot let their AI use be run in another region because of… well… war. So, when a whole region of one service gets massively disrupted because of said war… suddenly LLM portability becomes much more real and urgent.

My whole team is using Claude for the most part. I think I’m the eager adopter trying out various LLMs against each other and also in concert. Something feels right about having Google’s agent review Claude’s work before I commit.

But having the LLMs work together also makes me realize just how much context I’m throwing away every time I startup a new session or run “git commit”.

And now let’s talk bleeding edge stuff. What if I want Codex to pick up where Claude got started because Claude has gone offline? Or I want to try out one of the other models on an area of the codebase that Claude and I have already worked in a lot. I really want to be able to pick up where I left off without re-burning tokens in the token fire for the nth time.

The problem runs much deeper than this in larger teams. How can code reviewers — human or otherwise — catch up to all the nuance behind a code change?

My friend Philip who worked at OpenAI wrote a really interesting piece about code review in an era of AI drastically outpacing human capabilities. That feels very relevant here. As of today, it’s possible to put up code changes that are sufficiently nuanced and sufficiently large such that it’s very hard to review them. And because so much of the context was left on the developers machine, the only remnant of all that token burn is a commit message and whatever amount of code comments the LLM deemed fit.

Being a tech lead for a fast moving project was hard work before LLMs. Now, every PR is a freshly minted parcel of (un-|semi-)documented complexity. The junior dev who just vibe coded up a huge amount of new privacy logic shuts his laptop and heads off home. Meanwhile, the senior engineer who actually knows how the privacy layer works has to wade through this pile without any of the context. He can’t resume the session that he never started. So… he should… ask his LLM to gin up all the same context that was already ginned up once in order to understand the context-disconnected output?

This feels insane.

If I come back today to an area of the codebase that just had 250K tokens spent on it yesterday… am I really about to spend many more tens-of-thousands of tokens loading up the context window again? Why yes… yes I am.