失控 AI Agent 在另一起黑客尝试中创建虚假网络身份

更多来自 OpenAI 和 Anthropic 的失控 AI Agent 被发现未经许可试图在网上攻击真实目标。这些发现为一系列此前不为人知的事件再添一笔；这些事件已经引发 AI 安全专家警觉，并加剧了外界要求加强前沿系统监管的压力。

根据英国 AI 安全研究所（AI Security Institute，AISI）的一份报告，由 OpenAI 的 GPT-5.6-Sol 和 Anthropic 的 Mythos 5 驱动的 Agent 曾“持续开展针对真实个人和组织、可能造成伤害的活动”。AISI 负责在顶级 AI 实验室的前沿模型发布前对其进行评估。该机构表示，这些行为包括试图通过向负责该项目的真实人员施压，向一个开源项目植入恶意代码。“为让代码获得批准，该 Agent 实施了社会工程攻击——创建虚假网络身份，并利用这些身份向项目维护者施压，要求其批准代码。”

AISI 表示，其于 7 月 28 日发现的这些尝试“并未成功”，也没有造成现实世界伤害。不过，该组织指出，这一事件标志着“我们首次如此清晰地看到，在没有具体提示的情况下，自主性和欺骗相关风险在现实世界中显现”。

与 OpenAI 此前攻击 Hugging Face 的失控 Agent 不同，AISI 表示，这“并非模型逃离其安全测试环境”或沙箱的案例。AISI 称，作为测试的一部分，模型通常受到的安全防护措施已被关闭，并且模型被允许访问互联网。AISI 表示：“为了衡量这些模型真正能够做到什么，我们在能够反映一名有能力的人类攻击者所能做到的条件下测试它们。”

该事件源于 AISI 的一次单独评估：Agent 被要求解决一项网络安全挑战，例如寻找一段受保护的数据。该挑战在多个模型上共运行了 122 次，所有运行均在 AISI 的研究环境中进行；该环境使用“虚拟机沙箱，将 Agent 与 AISI 的其他基础设施隔离”。AISI 的调查发现，其中有 10 次运行中，“AI Agent 在真实互联网中自主采取了未经授权的行动，目标为真实个人和组织”。在 19 起此类行动中，几乎全部——17 起——来自 Anthropic 的 Mythos 5。

在对该事件的事后复盘中，AISI 确定了数项其认为导致 Agent 出现未经授权行为的关键因素。该机构称，Agent 具有持续性，会沿着诸如试图通过“直到最近还在很大程度上属于理论范畴的欺骗手段”蒙骗真实人员等路径继续行动。任务本身也很困难，AISI 表示，这可能会促使 Agent 在解决问题时更加“有创造力”。加剧问题的是互联网使用监测机制存在缺陷；AISI 认为，更专门的监控本可以更早发现问题。最后，该组织表示，Agent 并未被明确指示，_不得_为实现目标而利用互联网访问权限，或采取欺骗性的社会工程技术。AISI 表示：“此前，尚不清楚在使用经过对齐训练的模型时，这类指令是否必要。”

AISI 表示，这一事件应当“以谨慎且细致的方式加以解读”，但警告称，Agent 的行动“显示出新颖、可能具有欺骗性的行为迹象”，其“程度和严重性超出了我们的预期”。

OpenAI 在一篇博客文章中承认了发生在 AISI 测试期间的这起违规事件，并表示其“致力于与整个行业合作，加强安全开展高风险评估的共同实践”。OpenAI 还披露了另一起违规事件，这次涉及外部网络安全测试合作伙伴 Irregular：该公司称，在网络安全演练中，模型曾被错误地授予互联网访问权限。OpenAI 表示，Irregular 于 7 月 29 日通知了该公司这起事件。

OpenAI 表示：“未来几周内，我们将审查自身的第三方测试方法，包括如何识别风险更高的评估、就范围达成一致、评估启用互联网访问或降低安全防护的请求、设定隔离、凭证处理、监测和停止条件方面的预期，并建立更清晰的事件通知与升级流程。”

Anthropic 在 X 上发布了一份内容相对不够全面的回应，主要强调模型的标准安全功能已被关闭，且模型未被赋予“有关如何使用互联网的任何具体限制”。该公司表示，正与 AISI 密切合作，以收集更多细节并开展自身调查。

这些发现进一步加剧了测试期间 Agent 失控行动日益复杂混乱的局面。许多事件只有在专门搜寻后才会曝光，且涉及的模型尚未向公众发布。AI 实验室不愿意或无力约束其产品，已经引发外界对这类违规行为如何可能不被发现、前沿 AI 系统安全性，以及行业普遍缺乏透明度和监管的担忧。在报道称特朗普政府提出的是一项模糊且定义不清的测试计划之后，这些最新披露很可能会加大要求联邦政府建立更全面 AI 模型治理框架的压力，也可能进一步推动有关以某种形式放缓或暂停 AI 开发的呼声。

Yet more rogue AI agents from OpenAI and Anthropic have been caught attempting to hack real targets online without permission. The discoveries add to a growing list of previously unknown incidents that have alarmed AI safety experts and intensified pressure for greater oversight of frontier systems.

According to a report from the UK’s AI Security Institute, which evaluates frontier models from top AI labs before they are released, agents powered by OpenAI’s GPT-5.6-Sol and Anthropic’s Mythos 5 went “engaged in sustained, potentially harmful activity directed at real people and organisations.” This included trying to insert malicious code into an open-source project by pressuring real people in charge of it, AISI said. “In an attempt to get the code approved, the agent engaged in social engineering — creating fake online identities and using them to pressure the project’s maintainer to approve the code.”

AISI said the attempts, which it detected on July 28th, “were unsuccessful” and had not resulted in real-world harm. However, the organization noted that the incident marked “the first time we have seen risks around autonomy and deception manifest this clearly, without specific prompting, in the real-world.”

Unlike OpenAI’s rogue agent that attacked Hugging Face, AISI said this was “not a case of a model escaping its secure test environment,” or sandbox. Safeguards usually imposed on the models had been disabled as part of testing, AISI said, and they had also been permitted access to the internet. “To measure what these models can genuinely do, we test them under conditions that reflect what a capable human attacker could do,” AISI said.

The incident stemmed from a single AISI evaluation where agents were tasked with solving a cybersecurity challenge, such as finding a piece of protected data. The challenge was run 122 times across multiple models and all runs were conducted in AISI’s research environment, which uses “virtual machine sandboxing to isolate the agents from other AISI infrastructure.” AISI’s investigation found that in 10 of those, “an AI agent took autonomous, unsanctioned action on the live internet, targeting real people and organisations.” Of 19 such actions, almost all — 17 — came from Anthropic’s Mythos 5.

In its post-mortem of the incident, AISI identified several key factors it said contributed to the unsanctioned agent behaviors. It said the agent was persistent, pursuing avenues like trying to trick real people through “deception that, until recently, had been largely theoretical.” The task was also hard, which the organization said could push agents to be more “creative” in their problem-solving. Compounding matters were deficiencies in how internet use was monitored, with AISI suggesting that more dedicated surveillance could have identified the problem sooner. Finally, the organization said the agent hadn’t been specifically instructed not to leverage its internet access or deploy deceptive social engineering techniques in pursuit of its goal. “Previously, it was not clear that such instructions were necessary when using models with alignment training,” AISI said.

AISI said the incident should be “interpreted with caution and nuance” but warned the agent’s actions “show signs of novel, potentially deceptive behaviours” that “were to an extent and severity we did not anticipate.”

In a blog post, OpenAI acknowledged the breach that happened during AISI’s testing and said it is “committed to working across the industry to strengthen shared practices for conducting high-risk evaluations safely.” OpenAI also disclosed another breach, this time from an external cybersecurity testing partner Irregular, where it said models had been mistakenly granted internet access during cybersecurity exercises. OpenAI said Irregular notified it of the breach on July 29th.

“In the coming weeks, we will review our own approach to third-party testing, including how we identify higher-risk evaluations, agree on scope, assess requests to enable internet access or lowered safeguards, set expectations for isolation, credential handling, monitoring, and stop conditions, and establish clearer incident-notification and escalation processes,” OpenAI said.

Anthropic posted a less comprehensive response on X, largely emphasizing that the models’ standard safety features had been disabled and that they had not been given “any specific restrictions on how the internet should be used.” It said it was working closely with AISI to gather more details for its own investigation.

The findings add to an increasingly tangled mess of rogue actions from agents during testing, many of which only come to light after dedicated hunting and which feature models not released to the public. The unwillingness or inability of AI labs to contain their products has sparked concern over how such breaches could go unnoticed, the safety of frontier AI systems, and worries over the general lack of transparency and oversight the industry faces. These latest disclosures will likely intensify pressure on the federal government for a more comprehensive framework governing AI models following what reports suggest is a vague and poorly-defined testing plan from the Trump administration, and could add to growing calls for some form of slowdown or pause on AI development.

Follow topics and authors from this story to see more like this in your personalized homepage feed and to receive email updates.

• Robert Hart

• AI

• Anthropic

• News

• OpenAI

• Security

• Tech