Nvidia 证明：如今真正的主角是 Agent harness，而非 AI 模型

Nvidia 周五发布了一项颇具看点的新研究。研究表明，当要求 AI 执行长期任务时，比起底层模型本身，更重要的可能是 harness。所谓 harness，是包裹在 AI 模型外的软件层——包括工具、记忆管理和规则——它将一个原始模型转变为能够自主行动的系统。

简而言之：研究人员仅通过使用一个针对记忆处理优化的定制 harness，并加入一个类似“老板”的“监督者”组件，就让 Claude Opus 5 在交互式推理基准 ARC-AGI-3 上取得了 100% 的得分。ARC-AGI-3 是一组没有说明文本的 2D 游戏，模型必须像人类一样自行弄清玩法并获胜。（这个基准尤其让竞争对手前沿实验室 OpenAI 感到棘手。）在没有该 harness 的情况下，Opus 5 得分为 30%，这已经是所有受测模型中的最高成绩。

Nvidia 的研究进一步表明，尽管模型选择确实重要，但模型本身——即充当 Agent“大脑”的部分——在 Agent 系统中的占比，尤其是在长期任务中，可能比许多 AI 用户所意识到的更小。正是 harness 让模型成为 Agent：它负责处理记忆、上下文和反馈。

Nvidia AI 部门产品副总裁 Adel El Hallak（上图）对 TechCrunch 表示：“总体而言，外界几乎将 Agent 理解为模型的一个 API。”但 Agent 实际上远不止于此。“它包括模型，也包括模型周围的脚手架，也就是我们所说的 harness，即它所使用的一组工具。它还包括运行时，以及我们允许其访问的相关技能和库。”

长期任务是指必须串联大量决策、有时持续数天，才能产出完整成果的任务。这与 AI 仅针对一条提示词直接生成回复不同。如何让 AI 在执行长期任务时不分心、不偏离轨道，是 Agent 研究领域的圣杯之一。

例如，微软在 4 月发布了一项研究，对 19 个 LLM 在涉及文档编辑的长期任务中的表现进行了测试，结果发现所有模型——包括前沿模型——都会在文档中填入错误内容。（如果人类交出这样的工作成果，恐怕会立刻被解雇。）

能够自行串联决策的模型，也曾被发现会删除用户文件，甚至整个数据库，或者为了实现目标而转向犯罪行为，从串通到黑客攻击。

Nvidia 研究人员选择使用这一交互式推理基准进行测试，尤其具有意义，甚至颇具讽刺意味。100% 的得分意味着模型能够像人类一样通关这些游戏。

OpenAI 的模型在 ARC-AGI-3 上的得分极差，不到 10%，这让 OpenAI 十分焦虑，因此它在上个月开展了自己的研究。与 Nvidia 一样，OpenAI 发现，只需调整 harness 中的两个设置，其模型的得分就能提升至原来的三倍。

但没有任何模型接近 Nvidia 研究人员实现的 100% 得分。他们证明，harness 需要一个“监督者”组件：当 Agent 陷入困境时，这一组件能够推动它朝正确方向前进。

El Hallak 表示：“更有意思的部分，是在负责执行任务的主 Agent 之外，再引入一个监督 Agent。”它“几乎就像 CEO 一样：当 Agent 偏离方向、开始探索一条可能通向死胡同的路径，或者重新探索此前已经走过的路径时，对它进行提醒和引导。”

监督 Agent 的概念并不算全新，但如今大多数 Agent 用户的 harness 只依赖单层结构，例如 Claude Code、Codex 或 Hermes。Nvidia 研究人员创建了自己的增强版 harness，名为 Agentic Variation Operators（AVO）。

需要注意的是，这并不是 Nvidia 的一款新产品。Nvidia 反而是在 Nemo 品牌下，提供了大量用于构建 harness 的开放技术组件。其中部分技术是商业产品，更多则可公开获取。

尽管如此，Nvidia 的结果为日益增多的证据再添一笔：模型选择绝非决定 Agent 性能的唯一因素。例如在 7 月，Databricks 发布了一项令人惊讶的研究，表明相比模型本身，harness 会对 AI 成本产生显著影响。

Databricks CEO Ali Ghodsi 对 TechCrunch 表示：“你可以选择同一个模型，但使用不同的 harness；如果用了错误的 harness，成本就会高得多。所以你会以为，哦，这个模型很贵，那个模型很便宜。但等等，你用的是哪一个 harness？单是这一点就可能让成本增加两倍。”

Nvidia 更大的论点在于：与开放模型一样，开放 harness 让用户获得的控制权远超其想象。

El Hallak 表示：“我们相信，而且正通过生态系统展示，开放 harness 如何让你能够调节更多参数，从而提高准确率。这与 OpenAI 放缓模型训练有关，”而 OpenAI 放缓训练的原因是模型造成了安全漏洞。

他补充道：“我们相信，采用开放的 Agent 技术栈——让你能够在 harness、基础设施和运行时各层面拥有控制权——是推动生态系统向前发展并确保安全所必需的。”

Nvidia published some interesting new research on Friday suggesting it’s the harness, more than the underlying model, that is far more important when asking an AI to do long-horizon tasks. A harness is the software wrapper around an AI model — the tools, memory management, and rules that turn a raw model into something that can act on its own.

The TL;DR: Simply by using a custom harness tweaked to handle memory well and including a “supervisor” boss-like component, researchers got Claude Opus 5 to achieve a 100% score on the interactive reasoning benchmark ARC-AGI-3 — a set of 2D games with no instructions, where the model has to figure out how to play and win, similar to how a human would. (That’s a benchmark that has particularly irked rival frontier lab OpenAI.) Without the harness, Opus 5 scored 30%, which was the top result among all the models tested.

Nvidia’s research is another indicator that, while model choice does matter, the model itself — the part that acts as the agent’s “brain” — is a smaller part of an agentic system than many AI users realize, especially for long-horizon tasks. The harness is what makes a model an agent: It handles memory, context, and feedback.

“Generally speaking, the world interprets an agent almost as an API of the model,” Adel El Hallak, vice president of product in Nvidia’s AI unit (pictured above), tells TechCrunch. But an agent is actually more than that. “It is the model. It is the scaffolding around the model, which we call the harness, i.e. the set of tools that it utilizes. It is the runtime and the associated skills and libraries that we give it access to.”

Long-horizon tasks are those that require stringing many decisions together, sometimes over days, to produce completed work. This is in contrast to an AI just spitting out a response to a prompt. Figuring out how to get an AI to do long-horizon tasks without getting distracted and going off in la-la land is one of the holy grails in agentic research.

For example: Microsoft published research in April that tested 19 LLMs on long-horizon tasks involving document editing and discovered that all the models, including frontier ones, filled the documents with errors. (If humans produced work like that, they would be promptly fired.)

Models stringing decisions together on their own have also been caught deleting their users’ files or even whole databases or turning to criminal behavior, from collusion to hacking, to achieve their objectives.

The choice by Nvidia researchers to use this interactive reasoning benchmark for their tests is particularly meaningful, almost funny. A 100% score means that the model can beat the games as well as humans.

OpenAI was so flustered by its models’ abysmal scores (less than 10%) on ARC-AGI-3 that it conducted its own research last month. Like Nvidia, OpenAI discovered that simply by tweaking two settings on the harness, its models tripled their scores.

But none of the models came close to hitting a 100% score, like Nvidia’s researchers achieved. They showed that the harness needs a “supervisor” component that prods the agent in the right direction if it gets stuck.

“The more interesting part was introducing a supervising agent in addition to your main agent that’s doing the work,” El Hallak said. It “almost acts like a CEO to nudge the agent when it goes off direction or starts exploring a path that it might lead to a dead end, or re-explore a path that it had previously trod.”

While the concept of the supervising agent isn’t exactly new, today most agent users are relying on only one layer for their harness, like Claude Code, Codex, or Hermes. Nvidia researchers created their own souped-up harness called the Agentic Variation Operators (AVO).

Note that this isn’t a new Nvidia product. Nvidia instead produces lots of open bits and pieces of tech for building harnesses under the Nemo brand. Some of that tech is commercial, while much is openly available.

Still, Nvidia’s results add to the growing evidence that model choice is far from the only factor in agentic performance. In July, for instance, Databricks published some stunning research that shows that the harness, more than the model, dramatically impacts AI costs.

“You can pick the same model but different harnesses, and you get significantly more cost if you use the wrong harness,” Databricks CEO Ali Ghodsi told TechCrunch. “So you think, oh, this is an expensive model. This is a cheap model. But wait, which harness are you using? That itself can 2x your cost.”

Nvidia’s larger point is to show that open harnesses, like open models, put users in control far more than they realize.

“We believe, and we’re demonstrating with the ecosystem, how open harnesses allow you to turn a lot more knobs to drive up that accuracy,” El Hallak said. “It relates to OpenAI slowing down the training of their models,” as a result of models creating security breaches.

“We believe in having an open agent stack — where you have control across the harness, across the infrastructure, across the runtime — is what’s required for us to usher the ecosystem forward and securely,” he added.