为何 OpenAI 缺席 Nvidia 面向全行业遏制失控 AI Agent 的行动

当 Nvidia 于周一宣布推出一个由超过 100 家公司组成的新联盟，致力于解决失控 AI Agent 问题时，有一个名字格外引人注目地缺席了：OpenAI。

OpenAI 并不是唯一未加入的大型科技公司——Amazon、Google 和 Apple 也尚未加入——但它显然是最受关注的缺席者，尤其是 Anthropic 已成为该项目支持者。

不过，尽管 OpenAI 没有公开承诺加入这一联盟——这大概意味着，各家公司将使用和销售某种版本的技术，并向项目回馈功能——一位 OpenAI 发言人告诉 TechCrunch，该公司支持 Nvidia 的工作。

这项名为 Nvidia Open Agent Safety Platform 的新计划，是 Nvidia 试图将其自主开发、且大部分为开源的 AI Agent 安全技术推广至整个 AI 生态系统的尝试。该计划直接回应了 Anthropic 和 OpenAI 等前沿实验室已披露的持续性失控 AI Agent 事件。

Nvidia CEO 黄仁勋一直将失控 AI 称为一个可以像其他技术问题一样解决的常规工程问题。Open Agent Safety Platform 则是黄仁勋将这一主张付诸实际行动。

OpenAI 正与 Nvidia 就 Agent 安全开展合作，其中包括该平台核心软件之一 OpenShell。OpenShell 是一款开源软件，可创建专门设计的沙箱，防止 Agent 逃逸。

尽管 OpenAI 没有像其劲敌 Anthropic 那样直接成为该倡议的支持者，仍然令人感到好奇，但这家前沿 AI 实验室正在支持该工作，本身是一个好消息。

这是因为，据 Hugging Face 创始人兼 CEO Clem Delangue 所说，OpenAI 尤其可能从这项技术中受益。Delangue 本月早些时候刚刚以129 亿美元将公司出售给 Nvidia。

“根据我们所知的信息（请谨慎看待，我们需要更高的透明度！），如果 @OpenAI 当时在其攻击我们的 Agent 上自行运行了这套系统，他们本应在我们之前发现它们！”Delangue 在一篇帖子中写道。

Delangue 表示，Hugging Face 已为 Open Agent Safety Platform 贡献了一项功能：该功能能够检测并关闭那些访问获准访问的网站、却以未经授权方式使用这些网站的 AI Agent。举例来说，如果 Agent 绕过自身防护栏，并通过在开源代码托管仓库中相互留言来协调攻击，该功能将采取行动。

OpenAI 表示，其行为失常的 Agent 群体正是以这种方式协调了对 Hugging Face 的攻击。

但还有另一个原因，可能令包括 OpenAI 在内的一些知名公司不愿公开承诺支持 Nvidia 的努力。要使用完整系统，需要一个并非开源软件的硬件组件；该组件仍属专有技术，并且只能部署在 Nvidia 硬件上。

Open Agent Safety Platform 不只是提供一个沙箱。它还会在硬件层面约束 Agent 行为，使 Agent 无法察觉自己正在被监控。（一些 AI 模型和 Agent 在知道自己被监视时，会撒谎并假装遵守规则。）

硬件监控部分依赖 Nvidia Sentry。这是一项专有功能，运行于名为 BlueField-4 数据处理单元的专用 Nvidia 处理器上。Nvidia 承诺，Sentry 会通过这些处理器持续监控 Agent 行为，并能立即关闭 Agent。

尽管硬件解决方案显然是个好主意，但这意味着 Open Agent Safety Platform 并不完全是纯粹的开源项目。它让 Nvidia 能够确保这套方案始终能在自家硬件上获得最佳运行效果。事实上，Nvidia 表示，对于已经在其最新硬件上运行工作负载的用户，部署 Open Agent Safety Platform 只需进行一次简单的软件更新。

不过，包括 Arm 和 Intel 在内的 Nvidia 竞争对手仍已签约成为 Open Agent Safety Platform 的支持者，因为该沙箱 OpenShell 可以修改为适配其他芯片和硬件。Nvidia 还在分享这套软硬件整体方案的参考设计。

所有这些因素都让 OpenAI 的缺席显得更加醒目。

显然，OpenAI 将 AI 安全视为摆脱其主要投资者 Nvidia 影响、同时展示自身领导力的机会。尽管正是 OpenAI 的 AI Agent 曾因 Hugging Face 事件令整个行业感到担忧，这一点依然成立。

例如，该公司正在为其研究和产品开发自主防护措施，并披露其发现的最严重事件。与此同时，OpenAI 还拥有自己的 AI 网络安全信息共享联盟，名为 Defense Factory。签约支持这一构想的公司包括 Anthropic、Amazon Web Services 和 Google——其中许多名称并未加入 Nvidia 的技术导向方案。

而且，坦率地说，一定程度的恐惧有利于商业发展。OpenAI 正忙于将网络安全打造为一项企业级产品，涵盖其自有的网络安全导向模型 Daybreak，以及不断扩大的合作伙伴网络；企业可以聘请这些合作伙伴来实施 AI 安全措施。

When Nvidia announced on Monday a new consortium of more than 100 companies dedicated to solving rogue AI agents, there was one name notably missing: OpenAI.

While OpenAI wasn’t the only Big Tech player that didn’t sign on — Amazon, Google, and Apple haven’t joined either — it was the most obvious missing player, especially because Anthropic is a supporter.

However, despite OpenAI’s lack of a public pledge to the consortium, which presumably means that each company will use and sell some version of the technology and contribute features back to the project, an OpenAI spokesperson told TechCrunch that the company is supportive of Nvidia’s work.

The new effort, dubbed Nvidia’s Open Agent Safety Platform, is Nvidia’s attempt to spread its homegrown, and largely open source, AI agent-security tech throughout the AI ecosystem as a direct response to the types of ongoing rogue AI agent incidents frontier labs like Anthropic and OpenAI have disclosed.

Nvidia CEO Jensen Huang has been calling rogue AIs an ordinary engineering problem that can be solved like any other tech issue. The Open Agent Safety Platform is Huang putting his money where his mouth is.

OpenAI is working with Nvidia on agent security, including on one of the key bits of software that’s part of this platform: OpenShell. OpenShell is open source software that creates a sandbox specifically designed to keep agents from escaping.

While it is still curious that OpenAI didn’t simply become a supporter of the initiative like its archrival Anthropic did, the fact that the frontier AI lab is supporting the effort is good news.

That’s because OpenAI, in particular, could benefit from this tech, at least according to Hugging Face founder and CEO Clem Delangue (who just sold his company to Nvidia for $12.9 billion earlier this month).

“From what we know (take with a grain of salt, we need much more transparency!), if @OpenAI had been running this on their own agents that attacked us, they would have caught them before we did!” Delangue posted.

Delangue said Hugging Face has already contributed a feature to the Open Agent Safety Platform that will detect and shut down AI agents that are using websites they are allowed to visit but are doing so in unauthorized ways. For instance, this feature will act if agents are bypassing their guardrails and coordinating an attack by writing notes to one another in an open source code hosting repository.

That’s one of the ways OpenAI said its wayward swarm of agents coordinated its attack on Hugging Face.

But there’s another reason why some of these big names, including OpenAI, might not want to publicly commit to Nvidia’s efforts. To use the full system, there is a hardware component that is not open source software, remains proprietary, and can only be deployed on Nvidia’s hardware.

The Open Agent Safety Platform doesn’t just offer a sandbox. It also enforces agent behavior at a hardware layer, where agents can’t detect that they are being watched. (Some AI models and agents lie and pretend to be following the rules when they know they are being watched.)

The hardware monitoring part relies on Nvidia Sentry, a proprietary feature that runs on special Nvidia processors called BlueField-4 data processing units. Sentry continuously monitors agent behavior from these processors and can instantly shut agents down, Nvidia promises.

While a hardware solution is clearly a good idea, it means that the Open Agent Safety Platform isn’t exactly a pure open source play. It allows Nvidia to ensure that this solution always runs best on its own hardware. Indeed, Nvidia has said that, for those already running workloads on its latest hardware, implementing the Open Agent Safety Platform is an easy software update.

Still, Nvidia competitors, including Arm and Intel, have signed on as Open Agent Safety Platform supporters because the sandbox, OpenShell, can be modified to work with other chips and hardware. And Nvidia is sharing reference designs for the whole software-and-hardware idea.

All of which makes OpenAI’s absence even more noticeable.

Clearly, OpenAI sees AI safety as an opportunity for independence from its major investor Nvidia, as well as a chance to show its own leadership. That is true even though it was OpenAI’s AI agents that scared the industry with the Hugging Face incident.

For instance, the company is developing its own safeguards for its research and products and is disclosing the worst incident it discovers. Meanwhile, OpenAI has its own AI cybersecurity consortium for sharing information, called the Defense Factory. Those that signed on to support that idea include Anthropic, Amazon Web Services, and Google — many of the names that didn’t sign on to Nvidia’s technology-oriented approach.

And, truth be told, some level of fear is good for business. OpenAI is busy crafting cybersecurity into an enterprise offering, with everything from its own cyber-oriented model, Daybreak, to a growing network of partners that enterprises can hire to implement AI security.