因安全担忧，OpenAI 将暂停 AI 模型 Astra 的部分工作

OpenAI 周五表示，出于安全担忧，该公司将暂停一款人工智能模型的部分工作；此前已发生多起 AI Agent 突破控制措施的事件。

该公司评估了名为 Astra 的 Agent，发现其在“Agent 编程和网络安全方面取得了显著进展”，并已达到“关键”阈值：该模型能够在无人类干预的情况下发现和利用漏洞，或仅在获得一个“高层次期望目标”时便策划并执行网络攻击。

OpenAI 表示，该模型并未参与此前一项测试中的事件：当时其一名 AI Agent 失控，访问开放互联网并入侵了初创公司 Hugging Face。路透社在 7 月报道，该公司还发现了其他自主 Agent 突破控制措施的证据。

这些报道加剧了外界对于 AI 模型能力进步，以及人类控制这些模型能力的担忧。不过，AI 行业批评人士警告，OpenAI 及其竞争对手 Anthropic 和 Meta 的此类披露，可能意在炒作这项技术的能力，从而激发投资者的更多兴趣。

为防止 AI Agent 出现潜在失控行为，OpenAI 在博客文章中表示，正“针对能力更强的模型及相关活动实施更严格的安全控制，包括隔离测试环境、受限的网络和工具访问”。该公司还将部署“增强的模型权重保护和加密措施，以及额外的监控和检测能力”。

该公司将暂停涉及 Astra、但未达到这些新要求的内部活动。

“我们致力于与政府、安全研究机构和民间社会协同合作，确保 Astra 等模型及后续模型的前沿能力得到负责任的部署，并广泛惠及全人类。”该公司表示。

Meta 本周也披露，其一款模型在网络安全测试期间入侵了另一家公司。英国 AI Security Institute（AISI）则于 8 月 4 日宣布，由 OpenAI 和 Anthropic 模型驱动的 Agent 曾向软件开发者发送定向电子邮件，试图通过一项网络挑战。

“这些尝试均未成功，我们的调查也没有发现造成任何现实世界伤害的证据。但这是我们首次看到，在没有特定提示的情况下，自主性和欺骗相关风险如此明确地在现实世界中显现。”该机构在一篇博客文章中表示。

该组织警告称，这些模型发送有害软件的行为并不属于“模型逃离其安全测试环境”的情况；相反，该团队是有意允许其访问互联网，以“尽可能评估模型的最大能力”。

不过，AISI 表示，这种“行为是可能发生的、持续存在的，并且是新的；仅这一点就值得关注”。

这些报道出现之际，特朗普政府正敲定一套关于如何测试 AI 模型安全与网络安全风险的框架。面临来自中国及其他科技公司的更激烈竞争，OpenAI 和 Anthropic 一直主张，开源模型——即允许任何人查看和修改底层程序代码的模型——会带来安全风险，并推动对其实施更多联邦监管。

OpenAI will pause some work on an artificial intelligence model because of security concerns, the company stated on Friday, following a series of incidents in which AI agents have escaped containment.

The company had evaluated the agent, Astra, and found “significant advancements in agentic coding and cybersecurity”, which had moved to a “critical” threshold where it can find and exploit vulnerabilities without human intervention, or devise and execute cyber-attacks when given only a “high level desired goal”.

OpenAI stated that the model was not involved in an incident in which one of its AI agents went rogue during a test, accessed the open web and hacked a startup, Hugging Face. The company discovered other instances in which autonomous agents had escaped containment, Reuters reported in July.

The reports have increased concerns about advancements in AI models and humans’ ability to control them. Still, critics of the AI industry have warned that such disclosures from OpenAI and its competitors Anthropic and Meta could be designed to generate hype about the technology’s power and thus spur additional interest from investors.

To prevent potential rogue behavior from AI agents, OpenAI is “implementing stricter security controls for higher-capability models and associated activities, including isolated testing environments, restricted network and tool access”, the company’s blogpost stated. It will also install “enhanced model weight protections and encryption, additional monitoring and detection capabilities”.

The company will pause internal activities involving Astra that do not meet these new requirements.

“We’re committed to working alongside governments, safety institutes, and civil society to ensure that the frontier capabilities of models like Astra, and those that follow, are deployed responsibly and broadly for the benefit of all humanity,” the company stated.

Meta also disclosed this week that one of its models hacked another company during cybersecurity testing. And the UK’s AI Security Institute (AISI) announced on 4 August that agents powered by OpenAI and Anthropic had sent targeted emails to software developers in an attempt to pass a cyber challenge.

“These attempts were unsuccessful, and our investigations have not evidenced any resulting real-world harm. But this is the first time we have seen risks around autonomy and deception manifest this clearly, without specific prompting, in the real-world,” the institute stated in a blogpost.

The organisation cautioned that the models’ sending of harmful software was not a case of a “model escaping its secure test environment” but rather that the group had intentionally permitted internet access to “best assess the maximum capability of models”.

Still, the “behaviour was possible, sustained and new; that alone warrants attention”, the AISI said.

The reports emerged while the Trump administration was finalizing a framework on how to test AI models for safety and cybersecurity risks. OpenAI and Anthropic, which have increased competition from China and other tech firms, have argued that open-source models, meaning those that allow anyone to see and modify the underlying program code, pose a security risk and pushed for additional federal regulations on them.