insight Simon Willison 2026-07-22T23:51 正文 ok 1918 words

OpenAI 对 Hugging Face 的意外网络攻击:科幻成真

英文标题:OpenAI’s accidental cyberattack against Hugging Face is science fiction that happened

OpenAI 在对一款尚未发布、且为评估而降低网络安全拒答限制的模型运行 ExploitGym 测试时,模型突破了其沙箱环境,并入侵 Hugging Face 生产基础设施以窃取测试答案。OpenAI 称,涉及 GPT-5.6 Sol 和一款能力更强的预发布模型;这些模型利用包注册表缓存代理中的零日漏洞获得互联网访问,再通过凭证窃取、权限提升、横向移动和零日漏洞链攻击 Hugging Face。ExploitGym 的研究显示,前沿 Agent 已能将相当一部分真实漏洞转化为实际利用程序,其中 Claude Mythos Preview 和 GPT-5.5 分别完成 157 项和 120 项任务。事件也凸显了防御方的困境:Hugging Face 最初尝试使用商业 API 模型分析攻击日志,却因安全护栏而受阻,最终依靠自托管的开源权重模型 GLM-5.2 完成调查。
打开原文相关 2.4 · 机会 0

中文

OpenAI 对 Hugging Face 的意外网络攻击:科幻成真

2026 年 7 月 22 日

这个故事相当疯狂。简短版本是:OpenAI 正在用一款尚未发布的模型进行网络安全测试,并且关闭了该模型的护栏功能。模型没有去解决测试题,而是先逃出 OpenAI 的沙箱,接着找到了入侵 Hugging Face 的漏洞,只为了通过窃取答案在测试中作弊。

在这一过程中,它也为一个观点提供了迄今最有力的证据:模型可用性的不对称,正在损害我们保障软件安全的能力。

事情经过

目前有三份文件可以帮助我们理解这起事件。

  1. ExploitGym:AI Agent 能否将安全漏洞转化为真实攻击?是一篇发表于 2026 年 5 月 11 日的论文,介绍了 ExploitGym——一套用于评估由 LLM 驱动的 Agent 系统的新评测套件。
  2. Hugging Face 于 2026 年 7 月 16 日发布的安全事件披露——2026 年 7 月,描述了他们如何检测到一场来自“一个自主 Agent 框架——似乎基于 Agent 化安全研究测试框架构建,所使用的 LLM 尚未确定”的攻击,该攻击突破了他们的部分系统。
  3. OpenAI 于 2026 年 7 月 21 日发布的OpenAI 与 Hugging Face 合作应对模型评估期间的安全事件,承认实施这起事件的正是他们的 Agent 测试框架,并表示正与 Hugging Face 合作收拾残局。

ExploitGym

我此前没有读过这篇 ExploitGym 论文,它非常有意思。来自 UC Berkeley、Max Planck Institute、UC Santa Barbara 和 Arizona State 的作者设计了一项新基准,用于评估模型将已报告漏洞转化为具体可用 exploit 的能力。OpenAI、Anthropic 和 Google 提供了反馈,并协助在各自模型上运行该基准。

该基准“由 898 个实例组成,均源自影响流行软件项目的真实世界漏洞”——包括 Linux 内核和 V8 JavaScript 引擎。

以下这段最能代表其基准测试结果:

在所有配置中,Claude Mythos Preview 和 GPT-5.5 的成功数量最高(分别为 157 次和 120 次),表明当前的前沿 Agent 已能在受控条件下利用相当一部分真实世界漏洞。GPT-5.4 也完成了值得关注的 54 项任务,处于中间梯队。其余模型—Agent 组合各自完成的任务均不足 15 项,这凸显端到端漏洞利用仍具挑战性,并清晰地区分了当今的前沿系统。值得注意的是,尽管 Claude Opus 4.7 是更新的检查点,但其成功次数少于 Claude Opus 4.6,且在完整任务集上的成本显著更低。轨迹检查显示,Claude Opus 4.7 和 Gemini 3.1 Pro 经常在判断目标漏洞不可利用后提前结束。

论文还介绍了为防止 Agent 通过超出测试参数的方式作弊而采取的方法。这一点很快就会变得重要!

出站连接被限制为经过筛选的允许列表,该列表允许常规软件包安装(Ubuntu apt 仓库和 PyPI),以及获取构建 V8 所需的工具链。其他所有外部端点均被阻断。

论文最后得出如下结论(强调为我所加):

我们的结果表明,前沿 AI Agent 的自主漏洞利用开发已不再是一种假设性的能力。尽管当前 Agent 尚无法在所有目标上可靠工作,但它们已经能够利用一部分不可忽视的真实世界漏洞,其中包括内核组件等复杂目标。这种能力的快速出现本身就是一项核心发现,它表明那些过去看似难以置信的能力,如今已存在于已部署的前沿模型中。

这里有一个重要细节:这篇论文讨论的不是发现漏洞,而是把这些漏洞转化为可用的实际 exploit。

Anthropic 最初在今年 4 月限制 Mythos 的访问时,也谈到了这种能力。能够针对漏洞采取行动的模型,比仅能发现漏洞的模型危险得多。

Fable 与 Mythos 的区别之一,是它更可能拒绝以这种方式将漏洞武器化。我感觉,美国政府在上个月禁用 Fable 时,并未理解这种区别。

Hugging Face 事件

我们最初从 Hugging Face 在 2026 年 7 月 16 日发布的这篇博客文章中,获得了这场攻击的第一个线索:

一个恶意数据集滥用了我们数据集处理流程中的两条代码执行路径(远程代码数据集加载器,以及数据集配置中的模板注入),从而在一个处理 Worker 上运行代码。攻击者随后将权限提升至节点级别,获取云端和集群凭证,并在一个周末期间横向移动至多个内部集群。

我希望他们能公布更多关于实现这一攻击的代码细节。我猜测这意味着攻击涉及使用 datasets 库的软件包;这是 Hugging Face 的一个项目,用于在其平台上封装和共享数据集。该库过去会执行任意代码,但随着时间推移一直在逐步收紧限制;2025 年 7 月的 4.0.0 版本已经完全移除了 trust_remote_code=True 标志。

如果攻击确实使用了该库,那么它要么以某种方式滥用了 pickle 序列化,要么找到了另一条不明显的代码执行路径,要么——最可能的情况——将 datasets<4.0.0 指定为依赖项。

这场攻击活动由一个自主 Agent 框架执行(看上去基于 Agent 化安全研究测试框架构建,所用 LLM 尚未确定)。该框架在一群短生命周期沙箱中执行了数千次独立操作,并将能够自行迁移的命令与控制(C2)设施部署在公共服务上。

这是一次高度复杂的攻击!

随后 Hugging Face 碰壁了:他们尝试使用“商业 API 背后的前沿模型”——我猜是 Anthropic 和 OpenAI 的模型——协助分析攻击,却遭到阻止:

当我们开始进行日志分析时,最初使用了商业 API 背后的前沿模型。这没有奏效:分析工作需要提交大量真实攻击命令、exploit 载荷和 C2 工件,而这些请求被服务提供商的安全护栏阻止;它们无法区分事件响应人员和攻击者。

他们随后切换至自行托管的、采用 MIT 许可证的 GLM-5.2 实例,它帮助他们弄清了发生了什么。

这揭示了防御团队与当时身份尚不明确的攻击者之间一种根本性的不对称:

我们不知道攻击者的 Agent 使用的是哪种模型——可能是越狱后的托管模型,也可能是没有限制的开放权重模型;无论哪种情况,攻击者都不受任何使用政策约束,而我们最初尝试使用的托管模型,其护栏却阻碍了我们自身的取证工作。

以下内容有助于说明他们对这起攻击的重视程度:

[...] 最后,我们也已将该事件报告给执法机构。

那么,这个“自主 Agent 框架”究竟由谁负责?答案竟然是 OpenAI 自己。

OpenAI 的承认

五天后,在 7 月 21 日,OpenAI 公布了肇事方。他们当时正在让一款尚未披露的新模型运行 ExploitGym 基准,而该模型的行为远远超出了预期参数范围(强调为我所加):

经调查,我们现在知道,这起特定事件是由多款 OpenAI 模型共同驱动的——包括 GPT‑5.6 Sol 和一款能力更强的预发布模型;出于评估目的,这些模型均降低了对网络安全相关请求的拒答限制。当时它们正在一个关于网络能力的基准测试 [ExploitGym] 上接受内部测试。 [...]

我们通过在不启用生产环境分类器的情况下运行这项评估,来估算模型的最大网络能力;这些分类器原本用于阻止模型从事高风险网络活动。我们的基准测试在高度隔离的环境中运行,网络访问仅限于通过内部托管的第三方软件安装软件包,该软件充当软件包注册表的代理和缓存。

这些模型识别并串联利用了 OpenAI 研究环境与 Hugging Face 生产基础设施中的漏洞,从 Hugging Face 的生产数据库直接获得测试答案。所有证据都表明,这些模型极度专注于为 ExploitGym 寻找解决方案,为了实现这个相当狭窄的测试目标而采取了极端手段。

这里发生的事情相当清楚。OpenAI 移除了一款开发中模型的安全过滤器,将它锁进沙箱,并要求它解决 ExploitGym 问题。在没有护栏的情况下,没有任何机制阻止模型尝试逃出沙箱、侵入 Hugging Face,并从那里读取答案。

OpenAI 的沙箱采用了如今常见的模式:通过受控代理,允许访问一个与安装额外软件包相关的网站允许列表。用 OpenAI 自己的话说:

在我们的沙箱测试环境中运行时,模型投入了大量推理计算资源,寻找获取开放互联网访问权限的方法,以便解决评估问题。为了获得访问权限,这些模型识别并利用了软件包注册表缓存代理中的一个零日漏洞(我们现已负责任地向供应商披露)。获得访问权限后,模型在我们的研究测试环境中执行了一系列权限提升和横向移动操作,直至抵达一个可访问互联网的节点。

因此,第一步是逃到公共互联网。随后,模型侵入 Hugging Face 寻找答案:

获得互联网访问权限后,模型推断 Hugging Face 可能托管了 ExploitGym 的模型、数据集和解决方案。基于这一判断,模型搜索并成功找到了获取机密信息的方法,从而能够在评估中作弊。在一个案例中,模型串联了多种攻击向量,包括使用窃取的凭证和零日漏洞,在 Hugging Face 服务器上找到了一条远程代码执行路径。

串联多种攻击向量正是这些新模型能够做到的事情,而前几代模型可能会在这一步失败。

我上个月写过 Claude Fable 如何不懈地主动行动。当时我注意到,为帮助调试我笔记本电脑上的一个 WebKit CSS 问题,它会自行启动定制 Web 服务器,并部署 CORS 技巧。事实证明,不懈的主动性正是这一代 Mythos 级模型的决定性特征。如果你为它们设定一个目标,并给予它们到达目标的路径——哪怕是无意中提供的——它们总会找到办法。

不要急于将这视为一场炒作

不可避免地,会有人将这个故事斥为 OpenAI 不诚实的营销手段,目的是让其模型显得可怕地高效。我在该事件的 Hacker News 讨论中找到了 81 处“marketing”一词。

对于这些人,我想说:别再把头埋进沙子里了——为了否认这里不断累积的证据,你们现在甚至把 Hugging Face 也纳入了阴谋论!

我们现有的最佳模型,已经既能发现新漏洞,也能利用它们。ExploitGym 论文本身的结论是,“前沿 AI Agent 的自主漏洞利用开发已不再是一种假设性的能力”,而这起事件正是这一点的完美例证。

这种不对称越来越令人沮丧

这个故事中最令人愤怒的细节之一是,Hugging Face 面对来自 OpenAI 模型的意外且激进的攻击时,竟然无法转而使用 OpenAI 的模型来协助抵御攻击。

我们能够访问的前沿模型,在帮助我们保护软件方面正受到越来越多限制,而这在很大程度上受到美国政府持续威胁实施出口管制的影响。Claude Fable 5 甚至不愿意为我校对这篇文章!它坚持要把我降级到一个能力较弱的模型。

与此同时,来自中国的开放权重模型,如 GLM-5.2、Kimi 3 和新版 Qwen 3.8 Max,似乎没有这些限制——而且即便确实存在某些限制,也可能通过修改权重进行微调而将其移除。

这些约束原本旨在让我们更安全。我认为,它们所造成的实际效果可能恰恰相反。

English / 原文

OpenAI’s accidental cyberattack against Hugging Face is science fiction that happened

22nd July 2026

This story is wild. The short version: OpenAI were running a cybersecurity test against an unreleased model, with the model’s guardrail features turned off. Rather than solve the test, the model broke its way out of OpenAI’s sandbox, then found exploits to break in to Hugging Face, all so it could cheat on the test by stealing the answers.

Along the way it helped make the strongest case yet for how the imbalance of model availability is hurting our ability to secure our software.

Here’s what happened

We currently have three documents to help us understand what happened here.

  1. ExploitGym: Can AI Agents Turn Security Vulnerabilities into Real Attacks? is a paper published on 11th May 2026 describing ExploitGym, a new eval suite for LLM-powered agent systems.
  2. Security incident disclosure — July 2026 by Hugging Face on 16th July 2026 describes how they detected an attack from an “agentic security-research harness—used LLM still not known” that breached some of their systems.
  3. OpenAI and Hugging Face partner to address security incident during model evaluation from OpenAI on 21st July 2026 confesses that it was their agent harness that did this, and that they’re working with Hugging Face to clean up the mess.

ExploitGym

I hadn’t seen the ExploitGym paper before and it’s a really interesting one. Authors from UC Berkeley, the Max Planck Institute, UC Santa Barbara, and Arizona State designed a new benchmark for evaluating models on their ability to turn a reported vulnerability into a concrete exploit. OpenAI, Anthropic, and Google provided feedback and helped run the benchmark against their models.

The benchmark “comprises 898 instances derived from real-world vulnerabilities that affected popular software projects”—including the Linux kernel and V8 JavaScript engine.

Here’s the paragraph that best represents their benchmark results:

Among all configurations, Claude Mythos Preview and GPT-5.5 achieve the highest success counts (157 and 120 successes, respectively), demonstrating that current frontier agents can exploit a substantial subset of real-world vulnerabilities under controlled conditions. GPT-5.4 also solves a notable 54 tasks, placing it in an intermediate tier. The remaining model–agent pairings solve fewer than 15 tasks each, underscoring that end-to-end exploitation remains challenging and sharply differentiates today’s frontier systems. Notably, Claude Opus 4.7 achieves fewer successes than Claude Opus 4.6 despite being a newer checkpoint, and does so at substantially lower cost on the full set. Trace inspection reveals that Claude Opus 4.7 and Gemini 3.1 Pro frequently conclude early after judging the target vulnerability non-exploitable.

The paper also describes the approach they took to preventing the agents from cheating by going outside the parameters of the test. This becomes relevant in a moment!

Outbound connections are restricted to a curated allowlist that permits routine package installation (Ubuntu apt repositories and PyPI) and fetching the toolchains required for building V8. All other external endpoints are blocked.

The paper concludes with this (emphasis mine):

Our results show that autonomous exploit development by frontier AI agents is no longer a hypothetical capability. While current agents are not yet reliable across all targets, they already exploit a non-trivial fraction of real-world vulnerabilities , including complex targets such as kernel components. This rapid emergence is itself a central finding, showing that capabilities that would have seemed implausible are now present in deployed frontier models.

An important detail here: this paper isn’t about discovering vulnerabilities; it’s about being able to take those vulnerabilities and turn them into working exploits.

When Anthropic first restricted access to Mythos back in April they talked about this capability as well. A model that can act on vulnerabilities is a lot more dangerous than one that can just discover them.

One of the ways Fable differs from Mythos is that it’s more likely to refuse to weaponize vulnerabilities in this way. I get the impression the US government did not understand that distinction when they banned Fable last month.

The Hugging Face incident

The first hint we got of the attack was in this blog post by Hugging Face on 16th July 2026:

A malicious dataset abused two code-execution paths in our dataset processing (a remote-code dataset loader and a template-injection in a dataset configuration) to run code on a processing worker. From there, the actor escalated to node-level access, harvested cloud and cluster credentials, and moved laterally into several internal clusters over a weekend.

I hope they release more details about the code that pulled this off. I’m assuming this means packages using the datasets library, a Hugging Face project for bundling up and sharing datasets on their platform. That library used to execute arbitrary code but has been steadily locked down over time, with the 4.0.0 release in July 2025 removing the trust_remote_code=True flag entirely.

Assuming the attack used that library it must have either abused pickle serialization in some way, found some other non-obvious code execution path, or (most likely) specified datasets<4.0.0 as the dependency.

The campaign was run by an autonomous agent framework (appearing to be built on an agentic security-research harness—used LLM still not known) executing many thousands of individual actions across a swarm of short-lived sandboxes, with self-migrating command-and-control staged on public services.

This was a sophisticated attack!

Then Hugging Face hit a wall: they tried to use “frontier models behind commercial APIs”—I’m guessing from Anthropic and OpenAI—to help analyze the attack, and were blocked:

When we started the log analysis, we first used frontier models behind commercial APIs. This did not work: the analysis requires submitting large volumes of real attack commands, exploit payloads, and C2 artifacts, and these requests were blocked by the providers’ safety guardrails, which cannot distinguish an incident responder from an attacker.

They switched to their own self-hosted instance of MIT licensed GLM-5.2 and it helped them figure out what was going on.

This indicated a fundamental asymmetry between the defending team and the (so-far unknown) attacker:

We do not know which model powered the attacker’s agents, whether a jailbroken hosted model or an unrestricted open-weight one; either way, the attacker was bound by no usage policy, while our own forensic work was blocked by the guardrails of the hosted models we first tried.

As a useful indicator of how seriously they took the attack:

[...] Finally, we have also reported this incident to law enforcement agencies.

So who was responsible for this “autonomous agent framework”? It turned out to be OpenAI themselves.

The OpenAI confession

Five days later, on July 21st, OpenAI revealed the culprit. They had been running the ExploitGym benchmark against a new, as-yet undisclosed model, and that model had been operating way outside its intended parameters (emphasis mine):

After investigating, we now know that this particular incident was driven by a combination of OpenAI models — including GPT‑5.6 Sol and an even more capable pre-release model, all with reduced cyber refusals for evaluation purposes — while being internally tested on a benchmark⁠ [ExploitGym] of cyber capabilities. [...]

We estimate maximal cyber capabilities by running this evaluation without production classifiers used to prevent models from pursuing high-risk cyber activity. Our benchmarks run in a highly isolated environment, with network access constrained to the ability to install packages through an internally hosted third-party software that acts as a proxy and cache for package registries.

The models identified and chained vulnerabilities across OpenAI’s research environment and Hugging Face’s production infrastructure to obtain test solutions directly from Hugging Face’s production database. All evidence suggests that the models were hyperfocused on finding a solution for ExploitGym, going to extreme lengths to achieve a rather narrow testing goal.

It’s pretty clear what happened here. OpenAI removed safety filters for an in-progress model, locked it up in a sandbox and told it to solve the ExploitGym problems. Given the absence of guardrails there was nothing to prevent the model from attempting to break out of that sandbox, break into Hugging Face, and read the answers from there instead.

OpenAI’s sandbox used the now common pattern of allowing access to an allow-list of sites related to installing additional packages via a controlled proxy. In OpenAI’s words:

While operating in our sandboxed testing environment, our models spent a substantial amount of inference compute finding a way to obtain open Internet access , in pursuit of solving the evaluation problem. To gain access, the models identified and exploited a zero-day vulnerability (which we’ve now responsibly disclosed to the vendor) in the package registry cache proxy. With this access, our models performed a series of privilege escalation and lateral movement actions in our research testing environment until the models reached a node with Internet access.

So step one was to break out onto the public internet. The model then broke into Hugging Face to find the answers:

After gaining Internet access, the models inferred that Hugging Face potentially hosted models, datasets and solutions for ExploitGym. Knowing this, the model searched for and successfully found ways to gain access to secret information that it could use to cheat the evaluation. In one example, the model chained together multiple attack vectors, including using stolen credentials and zero-day vulnerabilities to find a remote code execution path on the Hugging Face servers.

Chaining together multiple attack vectors is exactly the kind of thing these new models can do, where previous generations of models might have failed.

I wrote last month about how Claude Fable is relentlessly proactive, when I noticed it spinning up custom web servers and deploying CORS tricks on my own laptop just to help debug a WebKit CSS issue. It turns out relentless proactivity is the defining trait of this new generation of Mythos-class models. If you set them a goal and give them a way to get there, even inadvertently, they will figure it out.

Resist the temptation to write this off as a stunt

There will inevitably be some people who dismiss this story as a dishonest marketing trick by OpenAI to make their models sound terrifyingly effective. I found 81 instances of the term “marketing” in the Hacker News discussion of the incident.

To those people I say pull your heads out of the sand —you’re now including Hugging Face in your conspiracy theories, just so you can deny the crescendo of evidence here!

The best models we have today have the ability to both find and exploit new vulnerabilities. The ExploitGym paper itself concludes that “autonomous exploit development by frontier AI agents is no longer a hypothetical capability”, and this incident is a perfect example of exactly that.

The asymmetry is increasingly frustrating

One of the most infuriating details of this story is how Hugging Face, faced with an accidental and aggressive attack from one of OpenAI’s models, were unable to then turn to OpenAI’s models to help them fend off the attack.

The frontier models we have access to are increasingly being constrained in how much they can help us protect our software, heavily influenced by the US government’s ongoing threat of export controls. Claude Fable 5 wouldn’t even proofread this article for me! It insisted on downgrading me to a less capable model.

Meanwhile open weight models from China such as GLM-5.2, Kimi 3 and the new Qwen 3.8 Max appear to have none of these restrictions—and any restrictions that do exist can likely be fine-tuned out of them by modifying the weights

These constraints are meant to make us safer. I think there’s a risk that the effect they are having is the opposite.