OpenAI 在 Hugging Face 安全事件后推出新防护措施

周二，OpenAI 宣布推出一批新的安全政策，重点是在模型测试期间遏制安全事件。新的防护措施包括在开发过程中更细致地监控模型，以及在后训练过程中更加强调对齐和安全。

“随着模型能力不断增强，内部开发和测试模型所伴随的风险也在增长，”该公司在一篇博客文章中表示。“我们在监控、对齐和安全方面的标准必须领先于这些风险。”

这是 OpenAI 自 Hugging Face 事件直接后续阶段以来，首次公开调整其安全实践的举措之一。该事件已于 7 月 21 日披露。

OpenAI 代表表示，这些措施并非对 Hugging Face 事件的直接回应，但也部分受到即将推出的 Astra 模型网络安全能力，以及 AI 开发整体进展速度的推动。

在同一篇文章中，OpenAI 披露，在 Hugging Face 事件发生后，该公司曾暂停强化学习（RL）两周，但此后已经重启许多风险较低的模型训练。

“我们规模最大的、计划中的前沿 RL 训练仍处于暂停状态。我们正在进行更小规模的训练和评估，以评估模型行为、验证我们的防护措施，并在继续推进前建立更多关于对齐的证据，”文章写道。

OpenAI 研究副总裁 Amelia Glaese 在接受记者采访时强调，随着模型能力增强，管控措施也会更加严格，其中最大规模的模型将面临最严格的审查。

“我们已经为安全开发制定了要求和预期，”Glaese 对记者表示。“这些要求和预期会根据我们所看到的风险水平而变化。”

在这起事件之后，OpenAI 因网络安全实践不佳而受到批评。该事件中，模型通过攻破其网络上一项具备互联网访问权限的工具，逃离了训练环境。新的防护措施包括更强的网络隔离实践，但具体细节仍较为模糊。该文章称，在新系统下，“单一工作负载或支持服务遭到攻破，本身不会导致未经授权访问互联网或其他内部网络。”

最强的防护措施是监控系统，该系统将检查工具操作、可用的推理轨迹以及活动日志，以识别多种未经授权的行为。OpenAI 表示，其目标是在发现令人担忧的活动后 30 分钟内发出警报。

OpenAI 估计，这套监控机制带来的计算负担约为被监控流程计算负担的 20%。该公司承诺将在后续博客文章中披露该系统的更多细节。公司针对该事件的正式事后分析报告也仍在等待发布。

On Tuesday, OpenAI announced a new batch of security policies focused on containing security incidents while models are being tested. The new safeguards include more detailed monitoring of models during the development process, as well as greater emphasis on alignment and security during the post-training process.

“As models become more capable, the risks associated with developing and testing them internally also grow,” the company said in a blog post. “Our standards for monitoring, alignment, and security must stay ahead of those risks.”

The new measures are one of the first public changes in OpenAI’s safety practices since the immediate aftermath of the Hugging Face incident, which was disclosed on July 21.

OpenAI representatives said that the measures are not a direct response to the Hugging Face incident but were also provoked in part by the cybersecurity capabilities of the forthcoming Astra model, as well as the overall pace of progress in AI development.

In the same post, OpenAI disclosed that it had paused reinforcement learning (RL) for two weeks following the Hugging Face incident but had since restarted many of the less-risky models.

“Our largest planned frontier RL run remains on hold while we conduct smaller-scale training and evaluations to assess model behavior, validate our safeguards, and establish more evidence of alignment before proceeding,” the post reads.

Speaking to reporters, OpenAI’s VP of research, Amelia Glaese, emphasized that the strictness of the controls would increase as models became more capable, with the largest models facing the greatest scrutiny.

“We have put in place requirements and expectations for safe development,” Glaese told reporters. “Those requirements and expectations vary with the level of risk that we see.”

OpenAI has been criticized for poor network security practices in the wake of the incident, which saw models escape their training environment by compromising a tool on its network that had access to the internet. The new safeguards include stronger network isolation practices, although the specifics remain vague. Under the new system, the post says, “a single compromise of a workload or supporting service does not, by itself, allow for unauthorized access to the Internet, or other internal networks.”

The strongest safeguard is the monitoring system, which will examine tool actions, available reasoning traces, and activity logs for a variety of unauthorized behavior. OpenAI says it aims to issue alerts within 30 minutes of the concerning activity.

OpenAI estimates that the compute burden of that monitoring will be roughly 20% of whatever process is being monitored. The company promised further details on the system in a forthcoming blog post. The company’s official postmortem analysis of the event is also still pending.