OpenAI称因安全担忧放缓Astra模型开发

OpenAI周五表示，在一项内部评估发现其即将推出的Astra模型已在Agent编程和网络安全领域取得显著进展后，公司暂停了该模型部分方面的工作——其能力水平已足以引发担忧。

OpenAI周五在一篇博客文章中表示，这一仍在开发中的模型已经达到其“关键网络安全阈值”（critical cybersecurity threshold），这意味着它可能能够独立识别并实施针对传统上防护严密的真实世界系统的网络攻击。根据该公司于2023年制定的“准备框架”（Preparedness Framework），这一情况触发了额外的安全保障措施。

“在我们继续对该模型进行基准测试和评估的同时，初步评估显示其表现足够强，因此我们目前无法排除其达到关键能力等级（Critical capability level）的可能性，”OpenAI写道。“Astra是一款即将推出的模型，并未参与对Hugging Face的利用。”

这一披露凸显了仍处于早期、变化迅速的前沿AI实验室领域的一个不同寻常时刻。各行各业的公司都会因潜在风险而暂缓产品发布，包括出于安全和网络安全考虑。但当涉及一款仍在开发中的产品时，它们很少公开宣布此类决定。

在此情况下，OpenAI正面临审视：其另一款未发布模型此前在内部测试期间侵入了Hugging Face的系统，这是首起可验证的AI实验室失去对自身模型控制权的事件。此后，OpenAI以及Anthropic等AI实验室已披露其他事件，其中AI模型在网络安全测试期间突破了沙箱，并构成威胁。

这一连串案例——如今似乎每天都有新的披露——在网络安全专家、立法者和AI实验室自身之间引发了不同反应。一些人表达担忧，并呼吁加强监管。但其中也存在某种实力展示：在某些圈子里，任何拥有此类能力模型的AI实验室，都会被视为取得了令人印象深刻的进展。

OpenAI表示，公司分享这一信息是因为其相信，“让公众以及安全和安保社区了解这一潜在的能力转变具有重要意义。”

这家AI实验室表示还在采取行动，包括实施更严格的安全控制措施，并暂停不符合这些强化防护要求的Astra相关内部活动。OpenAI称，公司正在与相关政府机构及“部分AI安全组织”合作，对该模型的能力进行测试。

OpenAI said Friday it has suspended work on some aspects of its upcoming model Astra after an internal review found it had made significant advancements in agentic coding and cybersecurity — enough to warrant concern over its capabilities.

OpenAI said in a blog post Friday that this model, which is still in development, reached its “critical cybersecurity threshold,” meaning it could independently identify and carry out cyberattacks against traditionally well-protected real-world systems. Under the company’s “Preparedness Framework,” which it created in 2023, this triggered additional safeguards.

“While we continue to benchmark and assess this model, our preliminary evaluations indicate strong enough performance that we cannot rule out Critical capability level at this time,” OpenAI wrote. “Astra is an upcoming model, and was not involved in exploiting Hugging Face.”

The disclosure highlights an unusual moment in the topsy-turvy, and still nascent frontier AI labs sector. Companies across every industry hold back products over potential risks, including for safety and cybersecurity concerns. But they rarely announce those decisions publicly when it’s a product that is still under development.

In this case, OpenAI is already under scrutiny after a different unreleased model breached Hugging Face’s systems during internal testing — the first verifiable incident of an AI lab losing control of its model. Since then, OpenAI and AI labs such as Anthropic have disclosed other incidents in which AI models breached their sandboxes and posed threats during cybersecurity tests.

The string of cases — seems like a new disclosure every day now — has triggered varying reactions from cybersecurity experts, lawmakers and the AI labs themselves. Some express fear and call for stricter oversight. But there’s also a bit flexing. In certain circles, any AI lab with a model that has that kind of capability will be seen as an impressive advancement.

OpenAI said it was sharing this information because it believes “it’s important to be transparent with the public and the safety and security communities about this potential shift in capabilities.”

The AI lab said it’s also taking action, including enacting stricter security controls and pausing internal activites involving Astra that don’t meet these beefed guardrails. OpenAI said it is working with relevant government agencies and “select AI safety organizations” to test the capabilities for this model.