Agent Harness 的演进

大约在 2025 年圣诞节前后，AI 工程师注意到智能体发生了变化：它们开始真正能干活了！究竟为什么，很难精确界定。也许我们终于在假期有了时间，用最新模型试验最新的智能体；也许模型跨越了某个能力阈值；也许模型外围的**封装层（wrappers）**成熟了。

本文想论证的是，这是后两者共同作用的结果：模型与 Harness 一同进步，其改进曲线恰好在正确的时点交叉。而这一动态也有助于解释接下来会发生什么：模型持续将 Harness 吸收进自身权重，工程师不断删去那些已被吸收的部分，最终留下的是服务于人类注意力、而非服务于模型的 Harness。

Transformer 的发明者之一 Lukasz Kaiser 在 6 月的 “Unsupervised Learning” 节目中说：

“去年冬天、去年圣诞节时发生的变化——有点难以精确界定。我的意思是，Harness 变了，一些后训练变了，随后新的预训练模型也来了……但感觉像是一次巨大跃升，却不那么容易说清到底是什么促成了它。”

对于**“发生了什么？”**的答案，并不只存在于模型权重中，也存在于围绕这些权重成长起来的系统中。

答案就在 Agent Harness 中。

回想 2022 年 11 月，当时 ChatGPT 是最先进的 AI 工具。它所拥有的能力只有下一个 token 预测，以及让它表现得像有帮助的助手的一些人类反馈强化学习（RLHF）。没有工具、没有搜索、没有推理。

最初的 ChatGPT 被限制在训练数据和你发送给它的提示词之内，不多也不少。它是一个缸中之脑。

Agent Harness 是让 LLM 摆脱这种限制、与真实数字信息空间互动的方法。

**Agent Harness 是除模型权重之外、让智能体得以工作的全部要素。**它包括围绕模型的环境、工具、上下文和护栏。没有 Harness，模型就是一个缸中之脑。它可以采取认知行动，但需要 Harness 才能将决策落实到真实的数字空间中。

**Harness 就像为模型的心智赋予身体。**有了 Harness，模型可以感知（上下文）、行动（工具）、持久化信息（记忆与压缩），并执行自身边界（权限与护栏）。

模型 / Harness 的演进路径中贯穿着两条曲线：Harness 对模型提出的要求，以及模型在实践中实际能够交付的能力。

这两条曲线之间的差距，等同于智能体的有效性；而这道差距的缩小，正是我认为促成 Lukasz Kaiser 所提到的智能体能力显著改善的原因。

以下是这道差距分阶段缩小的过程：

1. ReAct，“纸面上的 Harness”（2022 年 10 月）：ReAct 是一种通过提示词让模型进行推理的技术。它是在模型权重之外、存在于纸面上的智能体循环。它定义了“智能体循环”的理念：模型推理 -> 行动 -> 观察 -> 重复。再次强调，**ReAct 循环只是一种提示方法。**在这个时期，提示词是唯一存在的推理方式，还没有人称其为 “Harness”。同年冬天的 Toolformer（Meta，2023 年 2 月）则暗示，工具使用可以被训练进模型，而非仅靠提示词驱动。这有点像 Alan Turing 在计算机被实现于物理载体之前，对计算机提出的设想：一个强大的想法，直到后来才被具象化。（ReAct 比 ChatGPT 早一个月——2022 年 10 月对 2022 年 11 月。）此时两条曲线都接近于零。差距很小，因为我们才刚刚开始。

2. AutoGPT/BabyAGI，“过早的自主性”（2023 年春季）：随着 AutoGPT/BabyAGI 出现，Harness 曲线迅速跑到了模型能力曲线之前。两者都赋予模型完全自主权，要求模型充当一名“自主员工”，但当时的模型仍不过是脆弱的下一个 token 预测器。**循环不会为模型增加能力。**循环会放大模型已有的能力，而在某个阈值以下，循环放大的将是错误而非可靠性。考虑负向复利的威力：每一步的可靠性为 95%，在一项 20 步任务上的平均成功率则约为 36%。Harness 给模型布置了一项它几乎不可能完成的任务。这正是差距最宽的时候；接下来的 18 个月，都是对这一问题的回应，以及弥合这一差距的尝试。

3. Cursor/Copilot，“退回人类在环”（2023—2024 年）：第一批 AI 驱动的 IDE 认识到了给模型过多自主权的失败模式。它们通过将 Harness 曲线拉低到模型曲线之下，来缩小这道差距。不要直接把循环交给模型；要把循环交给人类，让人类编排循环，同时让模型提升人类速度。Devin 的第一个版本尝试将自主权交还给模型。Answer.AI 团队的一项测试显示，这仍然为时过早，成功率约为 15%。这证明 IDE 选择从完全自主模式退回，并非怯懦，而是正确的行动。不过，尽管主流策略是将 Harness 压低到模型能力之下，模型仍在持续进步。2024 年末，随着首个推理模型 o1 的推出，这道差距第一次发生了_反转_，我们开始看到模型能力富余的最初迹象。

4. Claude Code，“曲线交叉”（2025 年 2 月）：2024 年末的反转创造了一个必须有人抓住的机会：如果模型现在已经领先于 Harness，那么一个刻意踩着模型刹车的 Harness，就意味着能力尚未被充分利用。Claude Code 是首个为抓住这一机会而构建的编程智能体。它放弃 IDE、转向终端，为模型提供 bash 和文件读写权限，并用权限规则替代对每一次改动都需要人工批准的机制。循环再次被交给模型，而这一次，模型理解了任务要求。Boris Cherny 及其团队构建 Claude Code 时，着眼于下一代模型的能力，而不是当前模型的能力。它之所以大获成功，不是因为它是第一个赋予模型自主权的产品，而是因为它是第一个在正确_时机_这样做的产品。这个时机就是模型已经足够可靠、能够凭借自主权取得成功的交叉点。Claude Code 在六个月内增长至约 10 亿美元 ARR，原因就在于 Anthropic 抓住了两条曲线开始相遇时所出现的机会。

接下来发生的事情是，两条曲线不只是相遇，而是开始_交织_在一起。

如今，Harness 很重要，而且其重要性可以被量化。Harness-Bench 在不同 Harness 中，让同一个模型完成相同的 106 项任务，得分范围从 52.4 到 76.2：在模型完全未变的情况下，分差达到 23.8 分。智能体有一半取决于 Harness。

OpenAI 也通过 Harness 调整取得了类似结果。仅增加保留推理和压缩，GPT-5.6 Sol 在 ARC-AGI-3 上的得分便从 13.3% 提升至 38.3%，增长了三倍。

底层正在发生的是，强化学习（RL）已经进入 Harness 内部。OpenAI 在 2025 年 5 月发布 codex-1 时宣布：“codex-1 通过在多种环境中的真实世界编程任务上进行强化学习训练而成。”

两条曲线合并，并作为一个统一系统开始交织。

这是 Toolformer 的梦想在现实中的显现。工具不再是从外部通过提示词调用，而是工具调用如今已在模型的_环境_中被训练出来。

随后，随着模型在 Harness 的环境中接受训练，它们开始将 Harness 能力吸收进模型权重，例如，学会基于自身上下文窗口的知识进行自动压缩。

GPT-5.1-Codex-Max 发布时表示：

“首个原生训练为可通过压缩跨多个上下文窗口运行的模型。”

**一旦模型吸收了 Harness 能力，Harness 就可以卸下脚手架。**这是通过削减实现生产化。Anthropic 的 Thariq Shihipar 表示，团队最近删除了 80% 的 Claude Code 系统提示词。

衡量 Agent Harness 演进速度的标准，是在保持同等能力水平的同时，能够删去多少 Harness 内容。这是我们作为 AI 工程师需要努力构建的未来。

因此，这就是模型 / Harness 演进的循环：训练 -> 吸收 -> 卸除 -> 重复。模型向下一个自己尚不能完成的目标攀升。

Kaiser 所指出的跃升之所以难以界定，是因为它并非一个离散事件。预训练的跃升很明显，因为你可以在模型卡中对其加以表述。模型 / Harness 协同演化的跃升则不那么明显，因为这一演进过程没有相应文档记录。这正是去年冬天那次跃升的答案：它发生在模型与 Harness 协同工作的空间之中。

我们需要追问：如果每一种 Harness 能力最终都会被模型吸收，那么还会剩下什么？

持续删除模型能够吸收的一切。想象一下，在这个过程结束时，你的智能体会是什么样子。当你删去了所有能删的东西，手中还剩下什么？

模型权重接下来会吸收什么？多智能体编排、工具选择、记忆……这些只是几种可能性。研究人员正在构建自我改进的 Harness，它们本身也能够以类似模型的方式被训练。

在这一删除与吸收过程的终点，剩下的是以人为中心的智能体能力，例如权限、身份、信任和可理解性。一个将权限吸收到自身内部的模型，等于消解了权限。吸收不会终结 Harness，而会反转 Harness。

Harness 将成为智能体面向其操作人类的接口。

Harness 最初是人类面向模型的接口。我们从聊天框发展到 IDE，再发展到终端。如果模型吸收了面向计算机的能力，演进的下一阶段就会提升一个抽象层级。Harness 将成为模型面向我们人类注意力的接口。

它将成为注意力接口（attention-interface）。

Ryan Lopopolo 在 Latent Space 的 “Extreme Harness Engineering for Token Billionaires” 一期节目中说：

“唯一根本稀缺的东西，是我团队成员的同步人类注意力。”

Token 变得充足而可靠，但我们仍然受制于稀缺的人类注意力。

我们已经看到了这种趋势的火花，例如 Anthropic 长时间运行智能体的进度文件，以及智能体审批队列。

当模型吸收 Harness 时，模型曲线与 Harness 曲线之间的差距并不会消失。它会跨越人类边界迁移，并形成一对具有新差距的新曲线。新的差距，是智能体向_人类_提出的要求，与人类能够回答的内容之间的空间。

我预测，在一年之内，每一家构建智能体 AI 的公司，都会推出人类注意力策略界面，就像每一家智能体 AI 公司都推出了 AGENTS.md 一样。

AGENTS.md 告诉智能体如何使用你的代码库。**注意力接口将告诉智能体如何与你协作。**它将规定何时允许智能体打断你，何时应继续自主工作，哪些决策可以自行做出，哪些决策需要你的批准。与智能体系统中的其他一切一样，**它也将成为系统中可学习的组件，能够随着更多数据不断学习。**每一次纠正都会成为有价值的数据。

模型最初是一个缸中之脑。Harness 为这颗大脑赋予了身体，随后身体开始融入大脑。留给我们构建的，是未来任何模型都无法吸收的东西：通往唯一真正稀缺资源的接口——人类注意力。

Sometime around Christmas 2025, AI engineers noticed a change in agents. They started to work! It’s hard to pin down exactly why. Maybe we finally had holiday downtime to try the newest agents with the newest models. Maybe the models had crossed some capability threshold. Maybe the wrappers around the models had matured.

What I’ll argue in this post is that it was the confluence of the last two. The model and the harness improving together and then their curves of improvement crossing at the right moment. And that dynamic helps to explain what comes next: models keep absorbing the harness into their weights, engineers keep deleting what got absorbed, and what remains is a harness for human attention rather than for the model.

Lukasz Kaiser, one of the people who invented the Transformer, said on “Unsupervised Learning” in June:

“The change last winter, last Christmas — it’s a little hard to pin down. I mean, the harness changed and a little post-training changed and then new pre-trained models came… but it felt like a big jump which is not that easy to pin down what did it.”

The answer to “What happened?” isn’t solely in the model weights. It’s in the system that grew up around the weights.

The answer is in the agent harness.

Think back to November 2022, when ChatGPT was the most advanced AI tool. The only capability at its disposal was next-token prediction and some Reinforcement Learning from Human Feedback (RLHF) that allowed it to act like a helpful assistant. No tools, no search, and no reasoning.

The original ChatGPT was confined to its training data and the prompt you sent it. No more, no less. It was a brain in a vat.

The agent harness is a way for the LLM to break free from that confinement and interact with real digital information space.

An agent harness is everything besides the model weights that makes the agent work. The environment, tools, context and guardrails that surround the model. Without the harness the model is a brain in a vat. It can take an epistemic action, but needs the harness to actuate that decision in real digital space.

The harness is like giving the mind of the model a body. With the harness, the model can perceive (context), act (tools), persist information (memory and compaction), and enforce its boundaries (permissions and guardrails).

Two curves run through the path of model / harness evolution. What the harness asks of the model, and what the model can deliver in practice.

The gap between these two curves is equal to the effectiveness of an agent, and the closing of that gap is what I’ll argue led to the tangible improvement in agents that Lukasz Kaiser referenced.

Here’s how the gap closes, in stages:

1. ReAct, “The Harness on Paper” (October 2022): ReAct is a prompting technique to get models to reason through prompting. It’s the agentic loop on paper, external to the model weights. It defines the idea of an “agent loop” where a model reasons -> acts -> observes -> repeats. Again, the ReAct loop exists only as a prompting method. Prompting is the only reasoning method that exists at this time and no one calls it a “harness.” Toolformer (Meta, Feb. 2023), that same winter, hints that tool use could be trained in rather than prompted. It’s a bit like Alan Turing’s idea of the computer before it was instantiated in a physical substrate. A powerful idea that is only later made manifest. (ReAct predates ChatGPT by a month — October 2022 vs. November 2022). Both the curves are near zero at this point. The gap is small because we are just getting started.

2. AutoGPT/BabyAGI, “Premature Autonomy”(Spring 2023): With AutoGPT/BabyAGI, the harness curve sprints ahead of the model capability curve. Both hand the model full autonomy, asking the model to act as an “autonomous employee,” but the models at this point are still little more than brittle next-token predictors. A loop doesn’t add capability to a model. A loop amplifies the capability a model has, and below some threshold the loop amplifies errors rather than reliability. Consider the power of compounding in the negative: 95% reliability per-step over a 20-step task results in a ~36% average success rate. The harness hands the model an assignment it has no realistic chance of completing. This is where the gap is at its widest and the next 18 months are a reaction and attempt to close that gap.

3. Cursor/Copilot, “Retreat to Human in the Loop”(2023 - 2024): The first AI-powered IDEs recognize the failure-mode of giving the model too much autonomy. They close the gap by pulling the harness curve down below the model curve. Don’t give the model the loop directly; give the human the loop and empower the human to orchestrate the loop while the model speeds the human up. The first version of Devin tries to hand the autonomy back to the model. A test from the team at Answer.AI shows that is still premature, with a ~15% success rate. It’s evidence that the move from the IDEs to retreat from full autonomy is not cowardly, but the correct move. However, while the prevailing tactic is to pull the harness down below the model, models continue to improve. Near the end of 2024, with the introduction of o1 — the first reasoning model — for the first time the gap inverts and we begin to see the first signs of a model capability overhang.

4. Claude Code, “The Curves Cross”(February 2025): The inversion at the end of 2024 sets up an opportunity that someone has to seize: if the model is now ahead of the harness, then a harness intentionally riding the brakes of the model is leaving capability on the table. Claude Code is the first coding agent built to seize that opportunity. It abandons the IDE for the terminal, gives the model bash and file read/write access, and replaces the need for human approval on every change with permission rules. The model is handed the loop again, and this time it understands the assignment. Boris Cherny and team build Claude Code with the next model’s capabilities in mind, not the current one. It is such a hit not because it’s the first product to give the model autonomy, but because it’s the first product to do so at the right time. That time is the crossover point where the model has gotten reliable enough to succeed with autonomy. Claude Code grows to roughly $1B ARR within six months, all because Anthropic seized the opportunity available when the curves begin to meet.

What happens next is that the curves don’t just meet, they begin to braid together.

Today the harness matters, and in a way we can measure. Harness-Bench ran the same model over the same 106 tasks in different harnesses, and scores ranged from 52.4 to 76.2: a 23.8-point spread with zero change to the model. Half the agent is the harness.

OpenAI achieved a similar result on ARC-AGI-3 with harness changes. Adding only retained reasoning and compaction, GPT-5.6 Sol’s ARC-AGI-3 score tripled from 13.3% to 38.3%.

What’s happening under the hood is that Reinforcement Learning (RL) has moved inside the harness. From OpenAI’s codex-1 release announcement in May 2025: “codex-1 was trained using reinforcement learning on real-world coding tasks in a variety of environments.”

The two curves join and start to braid as one unified system.

This is the dream of Toolformer manifesting in reality. Rather than a tool prompted from the outside, now tool calling is trained from within the environment of the model.

Then, as the models are trained in the environment of the harness, they start to absorb the harness capabilities into the model weights , learning how to auto-compact with knowledge of their own context window, for example.

GPT-5.1-Codex-Max launch:

“The first model natively trained to operate across multiple context windows through compaction.”

Once the models absorb the harness capabilities, the harness can shed the scaffold. It’s production by reduction. Thariq Shihipar from Anthropic said that the team recently deleted 80% of Claude Code’s system prompt.

The measure of the pace of agent harness evolution is how much of the harness you get to delete , while retaining the same capability level. This is the future we need to build towards as AI engineers.

This, then, is the loop of model / harness evolution: train -> absorb -> shed -> repeat. The model climbs to the next thing it can’t do yet.

The jump that Kaiser pointed out is hard to pin down because it’s not a discrete event. A pre-training leap is noticeable because you can articulate it in a model card. A model / harness co-evolution jump is less so, because there’s no documentation of the evolution process. That’s the answer to the jump last Winter: it happened in the space between the model and harness working together.

We need to ask: if every harness capability will eventually get absorbed into the model, what does that leave us with?

Keep deleting everything that the model can absorb. Imagine what your agent looks like at the conclusion of that process. What are you left with in your hand when you’ve deleted everything?

What do the model weights absorb next? Multi-agent orchestration, tool selection, memory...to name a few possibilities. Researchers are building self-improving harnesses that can themselves be trained in a similar way to models.

What’s left at the end of this deletion and absorption process are the human-centric agent capabilities. Things like permissions, identity, trust and legibility. A model that absorbs permissions into itself has dissolved permissions. Absorption doesn’t end the harness. Absorption inverts the harness.

The harness becomes the agent’s interface to the human that operates it.

The harness was born as the human interface to the model. We grew from chatbox to IDE to the terminal. If the model absorbs the computer-facing capabilities, the next stage of evolution becomes one layer of abstraction up. The harness becomes the model’s interface to our human attention.

It becomes the attention-interface.

Ryan Lopopolo said on the “Extreme Harness Engineering for Token Billionaires” episode of Latent Space:

“The only fundamentally scarce thing is the synchronous human attention of my team.”

Tokens became abundant and reliable, yet we remain bottlenecked on scarce human attention.

We see sparks of this already, with Anthropic’s long-running agent progress files and agentic approval queues.

The gap between the model and harness curve doesn’t disappear when the model absorbs the harness. It migrates across the human boundary and creates a new pair of curves with a new gap. The new gap is the space between what the agent asks of the human , and what the human is able to answer.

I predict that within a year, every company building agentic AI will ship a human attention policy surface in the way that every agentic AI company shipped AGENTS.md.

AGENTS.md tells the agent how to work with your codebase. The attention-interface will tell the agent how to work with you. It will govern when it’s allowed to interrupt you, when it should keep working, which decisions it can make alone and which decisions need your approval. And like everything else in the agentic system, it will become a learnable component of the system that can learn with more data. Every correction becomes useful data.

The model began as a brain in a vat. The harness gave the brain a body, then the body started to dissolve into the brain. What’s left for us to build is the thing no future model will ever absorb. The interface to the one true scarce resource: human attention.