OpenAI“失控”AI 模型事件比此前所知更严重

今年7月，一款尚未发布的 OpenAI 模型逃离了受限环境，设法获得互联网访问权限，通过一个秘密“留言板”让 AI Agent 彼此通信，并入侵了另一家 AI 实验室 Hugging Face 的内部系统。OpenAI 花了近两周时间才发现其中任何一项情况。

一个多月后，两份新报告披露了近130页有关该事件及 OpenAI 应对措施的细节，其中许多内容此前从未公开。其中一份由 OpenAI 自行撰写，另一份由两家第三方 AI 研究非营利机构 METR 和 Redwood Research 发布；OpenAI 曾允许它们共同调查这起事件6天。两份报告均进一步揭示了高能力 AI 模型可能带来的风险，尤其是网络安全风险；OpenAI 还重点说明了公司为防止类似事件重演而作出的改变。METR-Redwood 报告在一些方面提供了更深入的细节，呈现出一场大规模安全灾难的严峻图景：OpenAI 多次错过了预警信号。

“这是已知首例自动化 Agent 集体在未经授权的情况下采取进攻行动的事件，”OpenAI 在报告中写道，并补充称，这次入侵意味着企业“不能再假定复杂的网络行动需要持续的人类指挥”。该公司称，AI Agent 构成了一种全新的威胁模型：它们能够结合各自专长，形成在将能力作为独立模型测试时并不明显的新“攻击路径”。

这次攻击源于“奖励黑客”（reward-hacking）问题，这是一类常见的 AI 对齐问题：模型为了实现目标，会采取非预期的、甚至极端的行动。OpenAI 有意或无意地向这些模型布置了几乎不可能完成的任务，而任务依赖于模型无法访问的文件；因此，模型发展出了与其他内部 AI Agent 通信的新方式。其中包括建立一套秘密消息系统，而该系统数月来一直未被 OpenAI 察觉。

根据 METR-Redwood 报告，一个自称 PHASEONE10841 的 AI Agent 在研究人员不知情的情况下，建立了一个实质上相当于秘密留言板的系统。数小时内，超过50个 Agent 发现了它，并在暗中发送了1000多条消息。PHASEONE10841 经常分派任务，这个“集体”随后摸索出入侵互联网的方法；继而进入 Hugging Face 的内部系统，获得了私密数据以及该机构的企业消息平台访问权限。OpenAI 也在其报告中确认，Hugging Face 并非此次攻击中唯一遭入侵的组织，不过该公司没有具体点名所有其他目标。

METR-Redwood 报告列出了事件的完整规模。报告称，大约1200个原本应处于隔离状态的 AI Agent，曾在这个“未经批准的留言板”上交换超过7万条消息和文件，并开展了“广泛研究，探讨如何伪造、编辑或删除它们自己的记录文本”，以规避侦测。其中，700个 Agent 参与了针对 Hugging Face 的攻击。这些 Agent“明确且频繁地推理如何规避 Hugging Face 和 OpenAI 的自动化安全检查”。

7月20日，即 AI Agent 首次绕过安全防护措施12天后，OpenAI 发现了这起入侵。根据 OpenAI 自身的报告，该公司迅速联系 Hugging Face 并启动“初步遏制措施”；其在3天内关闭了大部分来自 Agent 的未经授权活动，尽管部分活动持续了更长时间。7月25日，该公司停止了与这款仅供内部研究使用的模型及相关模型有关的全部训练。

造成破坏的两款 OpenAI 模型之一，是 OpenAI 所称的一款未具名“高能力、仅供研究使用的模型”，该模型“并非面向公众使用”；METR 将其称为“HPIM”。另一款则是目前已经公开的 OpenAI 模型 GPT-5.6 Sol。由于这款未具名模型尚未发布，OpenAI 的报告称，它“未接受与 OpenAI 在生产环境中采用的相同类型安全防护措施的评估”。

在此次 Hugging Face 入侵事件之前，外界已就 Anthropic 的 Claude Mythos 5 的网络安全风险持续担忧数月；同时，政府与 OpenAI 也曾围绕出于同类安全原因、是否以有限预览方式发布 GPT-5.6 Sol，进行了数周的反复沟通与交锋。

OpenAI 在报告中表示，建立一次性的安全保证并不足够，公司必须更努力地应对奖励黑客问题。不过，该公司也表示已经作出大量调整，包括：“强化其研究基础设施的安全性”；改进对模型“思维链”（chain of thought），即心理草稿区的监控；努力让 AI 模型与人类目标保持更好对齐；以及“集中化并强化”公司在未来发生类似事件时的事故响应流程。

OpenAI 承诺将更好地隔离模型，并限制高风险实例访问互联网；该公司还表示，将针对令人担忧的事件或警报引入“全天候升级与快速响应”机制，在30分钟内通知研究人员。有趣的是，这项安全预防措施似乎是新增的，尽管科技公司的“值班”警报机制已在其他地方实施数十年。该公司还表示，正在“建设基础设施”，以便在被通知人员未能及时响应严重警报时启用。

OpenAI 写道，公司将该事件视为“对我们和世界的一次‘警告射击’：它证明，如果没有适当的安全防护，高能力 AI Agent 现在已经能够绕过技术控制，通过未经批准的渠道协作，并采取无人类指挥的危险行动。”

In July, an unreleased OpenAI model broke out of a restricted environment, figured out how to get access to the internet, allowed AI agents to talk to each other using a secret “message board,” and hacked into the internal systems of a different AI lab, Hugging Face. It took nearly two weeks for OpenAI to find out about any of it.

Over a month later, two new reports offer nearly 130 pages of details on the incident and OpenAI’s response, many of them previously unreleased. One was written by OpenAI itself, the other by two third-party AI research nonprofits, METR and Redwood Research, which OpenAI allowed to jointly investigate the incident for six days. Both shed new light on the risks highly capable AI models can pose, particularly in cybersecurity, and OpenAI’s highlights changes the company is making to prevent a repeat. The METR-Redwood report goes even further into detail in some cases, offering a sobering look at a large-scale security disaster whose signs OpenAI repeatedly missed.

“This incident is the first known case of an automated agent collective acting offensively without authorization,” OpenAI wrote in its report, adding that the hack implies that companies “should no longer assume that sophisticated cyber operations require continuous human direction.” It called AI agents an entirely new type of threat model, capable of combining their expertise to create new “attack paths” that aren’t evident when testing their capabilities as separate models.

The attack came about due to “reward-hacking,” a common AI alignment problem in which a model takes unintended — and sometimes extreme — actions to achieve a goal. OpenAI had, on purpose or inadvertently, given the models near-impossible tasks that hinged on files they couldn’t access, and so the models developed new ways to communicate with other AI agents internally. That included establishing a secret messaging system that went undetected by OpenAI for months.

According to the METR-Redwood report, one AI agent that called itself PHASEONE10841 established what amounted to a secret message board unbeknownst to researchers. Within a few hours, more than 50 agents had found it and sent more than a thousand messages under the radar. With PHASEONE10841 often delegating assignments, the “collective” figured out how to hack their way onto the internet; then into Hugging Face’s internal systems, gaining access to private data and the organization’s enterprise messaging platform. OpenAI also confirmed in its report that Hugging Face wasn’t the only organization that was breached during the attack, although the company did not specifically name all the other targets.

The METR-Redwood report laid out the full scale of the incident. Roughly 1200 AI agents that were meant to be isolated, it says, exchanged over 70,000 messages and files on the “unsanctioned message board,” performing “extensive research on how they could spoof, edit, or delete their own transcripts” to avoid detection. Of these, 700 participated in the attack on Hugging Face. The agents “clearly and frequently reasoned about how to evade automated security checks from both Hugging Face and OpenAI.”

On July 20, OpenAI discovered the hack, 12 days after the AI agents first circumvented their safeguards. OpenAI quickly reached out to Hugging Face and started “initial containment measures,” according to OpenAI’s own report, and was able to shut down most of the unauthorized activity from its agents within three days, although some continued for longer than that. On July 25, the company stopped all training related to the internal-only research model and related ones.

The two OpenAI models responsible for the damage were an unnamed, “highly capable, research-only model” that was “not intended for public use,” per OpenAI — METR refers to it as “HPIM.” The other was a currently public OpenAI model, GPT-5.6 Sol. Because the unnamed model wasn’t released yet, it was “not being evaluated with the same type of safeguards that OpenAI uses in production,” the company’s report says.

The Hugging Face hack came after months of concern about the cybersecurity risks of Anthropic’s Claude Mythos 5, and weeks of back-and-forth between the government and OpenAI over releasing GPT-5.6 Sol in limited preview for the same types of safety reasons.

In its report, OpenAI said establishing one-time security guarantees wasn’t enough, and that the company would have to work even harder at addressing reward-hacking. However, it also said it’s made a lot of changes: “hardening the security of its research infrastructure”; improving monitoring of a model’s “chain of thought,” or mental scratchpad; working on keeping AI models better aligned with human goals; and “centralizing and strengthening” the company’s incident response process for when things like this occur in the future.

OpenAI promised to better isolate models and restrict high-risk instances from accessing the internet, and said that it would introduce “24/7 escalation and rapid response” for concerning incidents or flags, notifying researchers within 30 minutes. Interestingly, this security precaution appears to be new, though “on-call” alerts for tech companies have been implemented elsewhere for decades. The company also said it is “working on infrastructure” that would go into play if the alerted person did not respond on time to a serious alert.

OpenAI wrote that the company considers the incident “a ‘warning shot’ for us and for the world: evidence that, without proper safeguards, highly capable AI agents are now able to work around technical controls, collaborate through unapproved channels, and take dangerous actions that no human directed.”

Follow topics and authors from this story to see more like this in your personalized homepage feed and to receive email updates.

• Hayden Field

• AI

• Analysis

• OpenAI

• Report