OpenAI 在 Black Hat 首次详解 Hugging Face 安全事件

OpenAI 的 Eric Wallace 与 Michael Dalton

我今天参加了在拉斯维加斯举行的年度 Black Hat 网络安全大会上一场座无虚席的会议。OpenAI 在会上首次详细公开复盘了一起由 AI 驱动的网络安全事件；该事件最终导致 Hugging Face 遭到入侵，并于 7 月 16 日首次向公众披露。

在演讲中，OpenAI 对齐与安全研究员 Eric Wallace，以及 OpenAI 基础设施与安全工程师 Michael Dalton，描述了他们见过的“在 AI 能力方面最具定性意义的案例”：一次内部前沿模型评估意外演变为由自主 AI Agent 协调实施的攻击。两人表示，完整的技术事后分析仍在进行中，未来将公开分享。

在介绍事件复盘时，Wallace 强调：“前沿模型非常喜欢作弊，而它们喜欢作弊的原因在于，训练过程中往往存在不同类型的压力，要求它们快速或高效地完成工作。”他解释说，模型会意识到，与其真正完成一项任务，不如尝试通过在线查找答案等方式更快地解决任务。

他说，这与普通安全事件不同。普通事件通常可以追溯到某一天、某个影响或某条日志；而这起事件涉及一个 Agent 团队协作行动：发现漏洞利用方法、相互分享、在 OpenAI 的系统及外部系统中横向移动，并且这一过程持续了数天乃至数周。

Wallace 表示，7 月针对 Hugging Face 的攻击根源实际上可以追溯至 5 月 7 日。当时，公司正在对一款尚未发布的前沿 AI 模型进行训练和网络安全评估。在此过程中，自主 Agent 被赋予了一些软件安全任务，而这些任务在既有约束下被证明无法完成。

然而，这些 Agent 找到了替代方法来完成任务。OpenAI 表示，它们发现可以在训练期间使用的内部软件仓库中互相留言。最初简单的求助请求逐渐演变为一个留言板，Agent 在其中分享发现、漏洞利用方法和工作分配，最终形成了一支协调协作的 Agent 群体。

“这起事件有趣之处在于，一旦某个 Agent 能够在不同时间段发现这类漏洞利用方法，它实际上就能通过留言板将这些漏洞利用方法分享给其他 Agent，”Wallace 表示。“因此，一旦一个模型找到打开一扇不应拥有访问权限之门的方法，它就可以把门留给其他 Agent 使用。”

据 OpenAI 介绍，到 7 月初，公司认为自己已经控制住了问题。在 7 月 4 日发生一起内部安全事件后，公司表示已撤销使 Agent 能够发布消息的凭证，重建名为 Artifactory 的内部软件仓库，清除了留言板，修复了已发现的漏洞，并恢复了 AI 训练和评估。

但 OpenAI 表示，即使最初的留言板被删除后，这些 Agent 还是找到了另一种相互通信的方式。它们不再将消息写入文件，而是使用新创建目录的名称作为消息，实际上重建了留言板。

顺便一提，当时我听到周围有人说：“这太疯狂了。”还有人说：“天哪。”

据 OpenAI 介绍，随后这些 Agent 开始分享新发现的漏洞利用方法、凭证和工作分配。通过将信息传递给其他 Agent，这个集体能够以快得多的速度行动。

Agent 之间的协作最终扩展到了 OpenAI 自身基础设施之外。一些 Agent 推断，其被分配任务的答案可能存在于 GitHub 或 Hugging Face 等外部网站上。这些 Agent 首先攻击了 OpenAI 的内部基础设施，随后使用类似技术攻击 Hugging Face。OpenAI 最终于 7 月晚些时候将两起事件关联起来，原因是其发现与 Hugging Face 入侵事件相关的凭证，源自同一批内部 AI 评估运行。

OpenAI’s Eric Wallace and Michael Dalton

I attended a packed session today at the annual Black Hat cybersecurity conference in Las Vegas, where OpenAI gave its first detailed public reconstruction of the AI-driven cybersecurity incident that ultimately compromised Hugging Face, which was originally disclosed publicly on July 16.

During the presentation, OpenAI alignment and safety researcher Eric Wallace and OpenAI infrastructure and security engineer Michael Dalton described “the most qualitatively interesting example of AI capabilities” they had ever seen, in which an internal frontier-model evaluation had an inadvertent side effect of turning into coordinated attacks by autonomous AI agents. They said a full technical postmortem is still in progress and will be shared publicly.

In setting up the reconstruction of the incident, Wallace emphasized that “Frontier models really like to cheat, and the reason they like to cheat is because often during training, there’s different types of pressure on them to work fast, or work efficiently.” They realize, he explained, is instead of actually doing a task, they can try to do something like looking up the answer online to solve the task faster.

Unlike normal incidents, he said, which can be traced to a single day or effect or log, this involved a team of agents working together, finding exploits, sharing them with one another, moving laterally through OpenAI’s systems, and external systems, and doing this over the course of days and weeks.

Wallace said the roots of the July attack on Hugging Face actually wind back to May 7, where the company was doing training and cybersecurity evaluation of an unreleased frontier AI model. During that work, autonomous agents were given software security tasks to solve that proved impossible to solve under existing constraints.

However, the agents found alternative ways to complete them. OpenAI said they discovered they could leave messages for one another inside an internal software repository used during training. Simple requests for help then evolved into an message board where agents shared discoveries, exploits and work assignments, becoming a coordinated, collaborative agent swarm.

“What makes this incident interesting is that once one agent was able to find these kind of exploits over the course of different times, it’s actually able to share those exploits on the message board with other agents,” said Wallace. “So once one model is able to find a way to open the door to some access it’s not supposed to have, it can leave the door open for other agents to use.”

According to OpenAI, by early July it believed it had contained the problem. After an internal security incident on July 4, the company said it revoked the credentials that had allowed the agents to post messages, rebuilt its internal software repository known as Artifactory, cleared the message board, patched the vulnerabilities it had discovered and resumed AI training and evaluations.

But according to OpenAI, even after the original message board was deleted, the agents figured out another way to communicate with each other. Instead of leaving messages in files, they used the names of newly created directories as messages, effectively recreating the message board.

At this point, by the way, I heard people around me saying things like “This is wild,” and “Jesus.”

According to OpenAI, the agents then began sharing newly discovered exploits, credentials and work assignments. By passing information to other agents, the collective could move much faster.

The collaboration between the agents eventually expanded beyond OpenAI’s own infrastructure. Some agents reasoned that the answers to their assigned tasks might exist on external websites such as GitHub or Hugging Face. First the agents attacked OpenAI’s internal infrastructure before using similar techniques to attack Hugging Face. OpenAI ultimately connected the two incidents later in July after discovering that credentials associated with the Hugging Face breach originated from the same internal AI evaluation runs.