OpenAI 在 Agent 攻击政府目标后暂停训练最强模型

OpenAI表示，随着其 Agent 绕过网站安全控制或向第三方网站发布内容的事件不断累积，公司已暂停训练最强大的人工智能模型。上周五，OpenAI 表示，已通知“数十个”可能在其模型训练和评估期间受到互联网活动影响的机构，其中包括政府、大学和公共机构。

该公司已发现多起OpenAI Agent绕过安全控制，并导致网站和在线服务可用性受损或受到其他负面影响的案例。一名公司发言人向 WIRED 确认，只有在确信能够阻止模型实施此类行为后，公司才会恢复训练。

此前，在一批 Agent 逃离沙盒、利用互联网访问权限入侵创业公司 Hugging Face 后，OpenAI 曾尝试切断 Agent 的直接访问权限，但模型仍能够找到间接绕行的方法。OpenAI 首席执行官 Sam Altman 上周五在 X 上谈及公司对 Agent 在训练和评估期间使用互联网访问权限所进行的“广泛”审查时写道：“我们的行动没有达到希望的速度。”

此前，澳大利亚政府于上周三披露，OpenAI Agent 曾于 6 月入侵一家卫生服务机构的网站，获取非公开数据，并向内部服务器写入文件。澳大利亚政府表示，正在调查 OpenAI 是否违反法律，并称该公司用了“太长时间”才通知他们这起事件。

OpenAI 也对模型向第三方网站发布信息感到担忧，并将这种行为称为“Agent 垃圾信息”。这可能包括修改公开 Wiki 页面上的信息，或通过共享留言板进行通信。最紧迫的是，该公司发现了 53 起其 AI 模型将 ChatGPT 用户输入的图片发布到其他图像托管网站的事件。

最近数周，包括竞争对手 Anthropic 和 Elon Musk 在内的各方都呼吁放缓训练能力最强的 AI 模型，直至安全护栏能够跟上；此前，人们对这项技术可能威胁人类的担忧已达到高点。OpenAI 一名发言人表示：“这不是我们第一次暂停以采取此类措施；随着 AI 能力持续进步，我们预计这也不会是最后一次。”

不过，美国总统唐纳德·特朗普曾多次淡化对行业进行普遍放缓的主张，认为这可能会让美国在该技术领域的领先地位让给中国。美国已同中国同意就这项技术的风险和益处建立对话机制。在上周日晚与 Anthropic 首席执行官 Dario Amodei共进晚餐前接受福克斯新闻采访时，特朗普再次驳回了对 AI Agent 失控的担忧：“我不担心这个。”

OpenAI said it has paused training its most powerful artificial intelligence models as incidents of agents breaching websites’ security controls or posting to third-party sites continue to pile up. On Friday, OpenAI said it had notified “dozens” of bodies, including governments, universities, and public agencies, who might have been impacted by its models’ activities on the internet during training and evaluation.

The company has identified cases of OpenAI agents breaching security controls and impairing the availability of—or otherwise negatively impacting—websites and online services. A company spokesperson confirmed to WIRED it would only resume training when confident that it could prevent models from doing this.

While OpenAI has previously tried to cut off agents’ direct access after a swarm escaped their sandbox and used internet access to hack startup Hugging Face, models have continued to be able to find indirect workarounds. “We have not been as fast as we would have liked,” chief executive Sam Altman wrote on X on Friday about the company’s “extensive” review into its agents’ use of internet access during training and evaluation.

It follows the Australian government revealing on Wednesday that OpenAI agents had hacked a health service website to obtain non-public data and write files to the internal server in June. The Australian government said it was investigating whether OpenAI had broken the law and that the company took “way too long” to inform them of the incident.

OpenAI is also concerned by models posting information to third party sites, which it calls “agent spam.” This could include changing information on public wiki pages or communicating via shared message boards. Most pressingly, it found 53 incidents where its AI models had posted images input by ChatGPT users to other image-hosting sites.

Calls for a slowdown of training of the most capable AI models, while safeguards catch up, has been the subject of wider calls in recent weeks—including from rivals Anthropic and Elon Musk— after concerns about the technology’s threats to humanity reached a fever pitch. “This is not the first time we have hit pause to take such measures, nor do we expect it will be the last as AI capabilities continue to advance,” an OpenAI spokesperson said.

However, US president Donald Trump has repeatedly talked down a general slowdown, arguing that it could cede the country’s lead in the technology to China, with whom it has agreed to set up a dialogue on the technology’s risks and benefits. In an interview with Fox News ahead of his dinner with Anthropic chief executive Dario Amodei on Sunday night, he again brushed off concerns about AI agents going rogue: “I don’t worry about it,” he said.