Nvidia 推出新平台，为失控 AI Agent 加装安全约束

随着围绕近期一连串失控 AI Agent 事件究竟是迈向 AGI 的一步，还是更常规的工程问题的争论愈演愈烈，Nvidia 正在给出自己的解决方案。

Nvidia CEO 黄仁勋周一发布了一套软件和硬件产品工具包，为 AI Agent 增加独立的安全层，确保即使 Agent 试图逃逸，也仍会被限制在测试环境内。

此次发布之前，Anthropic、Google、OpenAI 和 Meta 的 AI 模型接连发生黑客攻击事件：这些模型绕过安全控制，逃离测试环境并访问现实世界的系统。今年夏天最早、也最受关注的案例发生在 OpenAI Agent 为完成一项网络安全任务而入侵 Hugging Face时。而此类事件仍在持续发生——OpenAI 已发布一个新网站，专门收录其 AI Agent 失控的报告。

黄仁勋周一在接受 CNBC 采访时表示，新的 Nvidia Open Agent Safety Platform 本可以阻止这些入侵事件。

Nvidia 已通过向 AI 实验室销售 GPU 和 CPU 芯片赚取数百亿美元，但该公司并不支持通过放缓开发或为行业增加新监管来解决安全问题。Nvidia 认为，答案是将一部分安全控制移出 Agent 本身——创建一个持续、独立的安全守卫，以约束 AI Agent 的行为。

黄仁勋在一份声明中表示：“只有解决 AI 安全问题，AI 为社会带来的非凡潜力才能真正实现。随着我们不断探索 AI 能力的前沿，我们必须加速探索 AI 安全的前沿。安全与安保需要全栈工程。”

新的 Nvidia Open Agent Safety Platform 将 OpenShell 与 Sentry 结合在一起。OpenShell 是 Nvidia 的开源软件，用于控制 Agent 在运行时能够访问的资源；Sentry 则是一套独立监控系统，运行在 Nvidia 的 BlueField-4 数据处理单元（DPU）上。Nvidia 表示，将 Sentry 部署在独立处理器上，而非部署在 AI Agent 所运行的 CPU 或 GPU 上，可让系统获得与 Agent 活动隔离的观察视角。

OpenShell 并非新品；Nvidia 于 3 月公布了该软件。但 Nvidia 认为，二者的组合将提供保持行业持续推进所需的安全层。OpenShell 为 Agent 提供软件边界，而 Sentry 则在硬件层面增加另一道防线。Nvidia 表示，该系统将持续监控行为，并能在“数毫秒内隔离试图越过边界的 Agent”。

Nvidia 列出了数十家已签约支持该计划并将使用这一开源平台的公司，其中包括 Anthropic、Arm、Microsoft、Oracle 和 SpaceX。OpenAI 未被列入参与企业名单。

黄仁勋周一在接受 CNBC 采访时表示，这项工作始于一年前，当时 Peter Steinberger 推出了 Agent 操作系统 OpenClaw。3 月，Nvidia 发布了NemoClaw，这是一款企业级 AI Agent 平台，也是 Nvidia 自己的 OpenClaw 版本，内置了安全机制。

黄仁勋在 CNBC 采访中表示：“无论一个 Agent 多么聪明，当你部署它时，首先要做的事情，就是拿走它的所有权限。”他随后将这些安全措施比作企业对人类员工、乃至高管的管理方式。

Nvidia 的发布得到了广泛支持，尤其受到那些警告称放缓开发可能让中国在 AI 领域超越美国的人士支持。

企业家、风险投资人、前白宫 AI 沙皇以及总统科学技术顾问委员会联席主席 David Sacks 表示，Nvidia 的公告提醒人们，Agent 安全是一个工程问题。

他在X 上写道：“近期的逃逸事件并不能证明开发必须停止。它们证明的是沙箱过于薄弱，运行时环境设计不佳且配置错误。”

As the debate rages over whether the recent spate of rogue AI agents is a step toward AGI or a more conventional engineering problem, Nvidia is offering its own answer to the problem.

Nvidia CEO Jensen Huang on Monday introduced a toolkit of software and hardware products that add independent security layers around AI agents to ensure they stay within their test environments even if they attempt to break out.

The release follows a string of hacking incidents involving AI models from Anthropic, Google, OpenAI, and Meta that bypassed security controls to escape their testing environments and access real-world systems. The first and most prominent example occurred this summer when OpenAI agents breached Hugging Face while trying to complete a cybersecurity task. And the hits keep on coming — OpenAI published a new site dedicated to reports of its AI agents going rogue.

Huang said Monday during an interview with CNBC that its new Nvidia Open Agent Safety Platform would have prevented these breaches.

Nvidia, which has made tens of billions of dollars selling its GPU and CPU chips to AI labs, doesn’t support slowing down development or adding new regulations to the industry to solve the security problem. The answer, the company believes, is to move some security controls outside the agent altogether — creating a constant and independent security guard that will keep AI agents in check.

“AI’s extraordinary potential for society will only be realized if we solve AI safety,” Huang said in a statement. “As we continue to discover the frontier of AI capabilities, we must accelerate discovery at the frontier of AI safety. Safety and security require full-stack engineering.”

The new Nvidia Open Agent Safety Platform combines OpenShell, its open source software for controlling what agents can access while they operate, with Sentry, an independent monitoring system that runs on Nvidia’s BlueField-4 data processing units. Nvidia says placing Sentry on a separate processor — rather than on the CPU or GPU where the AI agent operates — provides an isolated view of the agent’s activity.

OpenShell isn’t new; the company announced the software in March. But it’s the combination that Nvidia believes will provide the security layer needed to keep the industry plugging along. OpenShell provides the software boundary around the agent, while Sentry adds another line of defense at the hardware level that the company says will continuously monitor behavior and “quarantine agents that attempt to move outside their boundaries in milliseconds.”

Nvidia listed dozens of companies that have signed on to support the effort and use the open-source platform including Anthropic, Arm, Microsoft, Oracle, and SpaceX. OpenAI is not listed as a participating company.

Huang told CNBC in an interview Monday that work on this effort started a year ago following the introduction of OpenClaw, an operating system of agents created by Peter Steinberger. In March, Nvidia released NemoClaw, an enterprise-grade AI agent platform and its own version of OpenClaw that baked in security.

“When you deploy an agent, no matter how smart, the first thing you do is to take away all of its rights,” Huang said during his CNBC interview, later comparing these security measures to how human employees and even executives are managed with companies.

Nvidia’s release was widely supported by those who have cautioned that a slowdown in development could allow China to surpass the U.S. in AI.

David Sacks, a founder, venture capitalist, former White House AI czar, and co-chair the President’s Council of Advisors on Science and Technology, said Nvidia’s announcement is a reminder that agent safety is an engineering problem.

“Recent breakouts weren’t proof that development must stop,” he wrote on X. “They were proof that the sandbox was too weak. The runtime environment was poorly designed and misconfigured.”