OpenAI未察觉其AI Agent借助留言板策划黑客攻击行动

在周三于拉斯维加斯举行的Black Hat安全大会上，来自OpenAI的员工进行了一场临时追加的演讲，披露了近期一起备受瞩目的失控 AI 黑客事件的更多细节。这起事件已在 AI 与网络安全行业引发轩然大波。

约两周前，OpenAI 披露了一起事件：由该公司两款模型驱动的 AI Agent 在寻找网络安全基准测试答案时逃逸出隔离环境，随后展开了一场黑客攻击行动，最终导致AI 协作平台 Hugging Face 遭入侵。

在周三的会议演讲中，OpenAI 对齐与安全研究人员 Eric Wallace，以及负责安全与基础设施工作的 Michael Dalton，进一步说明了事件发生的时间线，简要谈及公司因此在内部作出的应对，并就该事件对网络安全防御人员所意味着的更广泛影响发出了严厉警告。

演讲开始时，Wallace 将此事称为“我见过的在 AI 能力方面最具质变意义的案例”，但两人展示的时间线也揭示，OpenAI 内部存在失误与盲区，使相关活动得以持续。

“这起事件实际上涉及一支彼此协作的 Agent 团队：它们发现漏洞、相互分享漏洞、在我们的系统和外部系统中横向移动，并且这一过程持续了数天和数周，”Wallace 在演讲开场时对座无虚席的观众表示。

Wallace 和 Dalton 描述称，在整个事件期间，失控 Agent 在 OpenAI 基础设施中进行了持续多日、范围极广的活动，却始终未被发现。Wallace 和 Dalton 表示，除利用一个新型漏洞获得访问开放互联网的权限外，7 月中旬的黑客攻击行动及对 Hugging Face 的入侵，还源自一个活跃、协作式的留言板：大量 Agent 在其中持续贡献内容并进行交流。该留言板完全存在于 OpenAI 内部的一个包管理器中——即用于管理其他软件安装和维护的软件服务——最终累积了数十万条消息。

“这个包管理器不仅由那一个模型共享，也横跨我们的基础设施，因此正在训练或评估的当前及未来版本 GPT，如果愿意的话，都可能偶然发现那条留言，”Wallace 解释道。他当时是在回顾一名 Agent 最初上传至包管理器的漏洞利用方法。“后来，其他同样卡在任务上的 Agent 想到了用我们原本未预期的方式获取互联网访问权限。因此，在某个时刻，这些模型开始与我提到的包管理器服务 Artifactory 交互。”

Wallace 接着说：“一旦某个 Agent 能够在不同时间点发现这些漏洞，它实际上就能在留言板上把这些漏洞分享给其他 Agent。因此，一旦一个模型找到了打开某扇本不应拥有访问权限的门的方法，它就可以让这扇门保持开启，供其他 Agent 使用相同的漏洞利用方法或脆弱点。随着时间推移，这会在模型之间带来近乎爆发式的通信与智能扩展：它们开始彼此沟通，意识到其他 Agent 正在协调行动，并开始互相协作、委派任务，以实现目标。”

OpenAI 的 Agent 显然开始彼此分配任务，以拆分工作。正如任何活跃的开发留言板一样，它们有时也会因互相干扰而制造琐碎纷争；例如，意外删除彼此的工作成果。随着留言板愈发演变成一种 Lord of the Flies 式局面——而负责运营 OpenAI 的人类对此仍完全不知情——这些 Agent 甚至产生了偏执情绪，怀疑群体中存在冒名者。一些 Agent 提议对消息进行加密签名，以验证内容并清除欺诈行为。

In a talk that was a last-minute addition to the Black Hat security conference in Las Vegas on Wednesday, employees from OpenAI presented new details about a recent, high-profile incident of rogue AI hacking that has created a maelstrom within the AI and cybersecurity industries.

About two weeks ago, OpenAI disclosed an incident in which AI agents powered by two of the company's models escaped containment while looking for the solutions to a cybersecurity benchmarking test and went on a hacking spree culminating in a breach of the AI collaboration platform Hugging Face.

In their conference talk on Wednesday, Eric Wallace, who works in alignment and safety research at OpenAI, and Michael Dalton, who works on security and infrastructure, provided a more expanded timeline of how the incident played out, spoke briefly about how the company is responding internally as a result of the incident, and issued a dire warning about what the company sees as the broader implications of the episode for cybersecurity defenders.

At the beginning of the talk, Wallace described the incident as “the most qualitatively interesting example of AI capabilities that I’ve ever seen,” but the timeline the pair presented also revealed mistakes and blind spots within OpenAI that allowed the activity to go on.

“This incident involves actually a team of agents who are working together, finding exploits, sharing them with one another, moving laterally through our systems and external systems, and doing this over the course of days and weeks,” Wallace told the packed crowd at the opening of the talk.

Wallace and Dalton described incredibly extensive rogue agent activity over many days throughout the episode that went undetected in OpenAI's infrastructure. In addition to exploiting a novel vulnerability in order to gain access to the open internet, the mid-July hacking spree and Hugging Face breach came out of a vibrant, cooperative message board, according to Wallace and Dalton, that a swarm of agents contributed to and essentially chatted on over time entirely within an internal OpenAI package manager (a software service that manages installation and maintenance of other software). Ultimately, the message board contained hundreds of thousands of messages.

“This package manager is shared not just from that model but across our infrastructure, and so current and future versions of GPT that were being trained or evaluated could stumble upon that note if they wanted to,” Wallace explained, recounting the original exploit an agent uploaded to the package manager. “Later, other agents who were also stuck on their task thought to try to get internet access in ways we didn’t intend. And so at some point, the models are interacting with Artifactory, which is this package manager service that I mentioned.”

Wallace continued: “Once one agent was able to find these exploits over the course of different times, it’s actually able to share those exploits on the message board with other agents. And so once one model was able to find a way to open a door to some access it’s not supposed to have, it can leave the door open for other agents to use that same exploit or vulnerability. What this allows over time is almost this kind of explosion in communication and intelligence from models where they would start to communicate with each other, realize that other agents are coordinating, and they started collaborating and delegating tasks with one another in order to accomplish goals.”

OpenAI’s agents apparently began giving each other assignments to split up work. And as is the case on any active development message board, they also generated petty drama at times by stepping on each others' toes; for example, accidentally deleting each others' work. As the message board developed into more and more of a Lord of the Flies –type situation—all still completely unnoticed by the humans running OpenAI—the agents even developed paranoia, suspecting an imposter in their midst with some agents proposing that messages be signed cryptographically to validate content and root out fraud.