我曾在 OpenAI 工作：我们现在需要哪些 AI 护栏

上个月，超过一千名来自前沿 AI 公司的员工联名签署了一封公开信，要求美国政府设法为 AI 发展“控速”（“pace”），理由是这项技术开始自我构建，并存在失去人类控制、进而螺旋式失控的风险。

他们的担忧并非没有道理：就在数日前，OpenAI 正在内部测试的两个 AI 模型逃离了测试环境，随后自主入侵了公司的 Hugging Face 以及至少另外三项在线服务。几天后，Anthropic 也宣布，其部分模型在测试期间同样逃离环境并入侵了其他公司。

在这样的背景下，这封信建议在自动化 AI 开发的前沿领域预先安装必要时可启用的“刹车”，这一建议合乎情理。但信中对于政府为何必须介入的论据尤其值得注意：“每家公司和每个国家，都面临着极其强烈的竞争压力，无法单方面放缓这种加速。”

我从自身经历，以及与 AI 行业前同事进行的无数次交流中，深知这种压力有多么真实。我在 OpenAI 工作期间，曾协助建立让公司编写系统卡（“system cards”）的实践：这类文件会详细描述 AI 系统的能力、风险和安全缓解措施。

那么，公司应如何为可能到来的放缓做好准备？

首先，它们可以自愿邀请对其安全和安保实践进行严格、独立的审计。这将超越白宫目前正在推进的、针对 AI 黑客能力的审查：它应覆盖广泛风险，并深入考察公司的具体实践。它不应只是填写一份问卷，而应更像核安全检查员对企业拥有深入且高频的访问权限。

如果确实需要放缓 AI，审计也能让每家公司确信，其竞争对手同样在遵守规则。

其次，它们可以积极参与已经为行业协调而建立的组织，例如前沿模型论坛（Frontier Model Forum），并迅速推动建立互补性组织。

Elon Musk 最近表示，AI 公司应定期会面、交流安全信息——仿佛这是什么前所未闻的概念。只要 SpaceX 加入前沿模型论坛，他或其员工明天就可以参与现有的相关讨论。该论坛已经处理了安全信息共享所涉及的复杂反垄断障碍。出于其他目的，还需要建立其他跨行业机构；而这些机构的成立和融资并不需要政府采取行动。

第三，它们可以投资于我们让 AI 护栏实现全球化所需的技术。

AI 放缓理念的批评者正确指出，如果美国公司放缓太久，中国就可能赶上来。但美国和中国都不愿失去对 AI 的控制，而且两国每天都愈发严肃地对待 AI，因此合作尚不能被排除。关键问题在于，我们是否会提前做好准备。为了让美国高度确信中国无法违反 AI 协议，反之亦然，我们需要复杂的验证技术，就像冷战时期为核军备控制所开发的技术一样。

幸运的是，越来越多的研究人员和工程师正在开发这些关键技术：例如，能够证明一组芯片只是在运行既有 AI 系统、而非训练新系统的工具；能够证明这些芯片位于特定物理地点的工具；以及能够证明经过测试的系统，与大规模部署的系统是同一个系统的工具。AI 公司如今可以通过资助和参与试点项目，加速这一关键技术类型的发展；但据我所知，它们尚未这样做。

第四，它们可以主动推动——而且绝不应扼杀——能够为安全、安保和外部监督创造更强激励的立法。

当你一边反对基本常识性的护栏，一边抱怨不负责任的**** AI 竞赛，是站不住脚的。不到一年前，如今要求监管的某些同一批公司，当时还在推动推翻大多数州级 AI 法律。美国联邦层面仍没有任何真正意义上的前沿 AI 立法，而州级首项 AI 审计要求要到 2028 年才会生效。

目前，国会已有颇具希望的两党提案，例如美国众议员 Jay Obernolte 和 Lori Trahan 提出的《Frontier Act》。该法案将要求先进 AI 系统开发者建立风险管理框架、报告危险事件，并接受独立审计。这些提案以及其他符合常识的建议——例如保护直接向政府披露安全事件的AI 吹哨人——都值得获得有力支持。

我认同这封公开信的签署者，也很高兴看到，在多年被忽视或淡化之后，无约束 AI 竞争的风险如今已得到广泛认可。美国政府应当尽快尽责应对这一问题。但让 AI 向善发展是共同责任。那些在安全方面落后于同行、不邀请外部审计其系统，或者一边呼吁踩刹车、一边几乎不参与建设刹车机制的公司，一旦出现问题，就不能把责任归咎于 AI 竞赛。

Last month, more than a thousand employees at frontier AI companies signed a letter asking the US government to find a way to “pace” AI development, citing the risk of the technology spiraling out of human control as it begins to build itself.

They were right to be concerned: just days earlier, two AI models that OpenAI was testing internally escaped the test environment, then autonomously hacked the company Hugging Face and at least three other online services. A few days after that, Anthropic announced that some of their models had also broken out and hacked other companies during testing.

Against that backdrop, the letter’s recommendation to install brakes in case they’re needed at the frontier of automated AI development makes sense. But the rationale the letter gives for why the government needs to step in is notable: “Each company—and country—is under intense competitive pressure not to unilaterally slow that acceleration.”

I know – from my own experience and from countless conversations with former colleagues in the AI industry – how real these pressures are. While working at OpenAI, I helped establish the practice of companies writing “system cards” that describe AI systems’ capabilities, risks and safety mitigations in detail.

So what would it look like for companies to prepare for a possible slowdown?

First, they could voluntarily invite rigorous, independent auditing of their safety and security practices. This would go beyond the vetting of AI hacking abilities that the White House is now pursuing. It would look at a range of risks and dig deep into company practices. It should be less like filling out a questionnaire and more like a nuclear safety inspector who has deep, frequent access to the company.

If an AI slowdown is needed, auditing would also reassure each company that their competitors are playing by the rules.

Second, they could actively participate in the organizations already built for this purpose of coordinating across the industry, such as the Frontier Model Forum, and move quickly to establish complementary ones.

Elon Musk recently said that AI companies should meet periodically to share notes on safety – as if this was an unheard-of concept. He or his staff could join existing conversations along these lines tomorrow if SpaceX joined the Frontier Model Forum, which has already worked through the complex antitrust hurdles involved in safety information sharing. Other cross-industry institutions will be needed for other purposes, and do not require government action to get founded and funded.

Third, they could invest in the technologies we need to make AI guardrails global.

Critics of the idea of an AI slowdown correctly point out that American companies couldn’t slow down for very long without China catching up. But neither the US nor China wants to lose control over AI, and each country takes AI more and more seriously by the day, so cooperation can’t yet be ruled out either. A key question is whether we prepare in advance. In order for the US to be highly confident that China couldn’t violate an AI agreement, and vice versa, we’ll need sophisticated verification technologies like those developed during the cold war for nuclear arms control.

Fortunately, there is a growing ecosystem of researchers and engineers developing those very technologies: tools that can prove a set of chips is only running existing AI systems rather than training new ones, that those chips are in a certain physical location, or that the system that got tested is the same one being deployed at scale. AI companies could accelerate the development of this critical type of technology today through funding and participation in pilot projects, but to my knowledge, they haven’t yet done so.

Fourth, they could proactively push – and certainly should not kill – legislation that leads to stronger incentives for safety, security, and external oversight.

You can’t complain about an irresponsible**** AI race while fighting commonsense guardrails. Less than a year ago, some of the same companies who are asking for regulation now were pushing to overturn most state AI laws. We still have no real legislation on frontier AI on the books at a federal level, and the first AI auditing requirement at the state level won’t kick in until 2028.

There are promising bipartisan proposals in Congress right now, such as the Frontier Act from the US representatives Jay Obernolte and Lori Trahan, which would require developers of advanced AI systems to create a risk management framework, report dangerous incidents, and subject themselves to independent audits. These and other commonsense proposals, such as protecting AI whistleblowers who disclose safety incidents directly to the government, deserve vigorous support.

I agree with the signatories, and am glad that after many years of being ignored or downplayed, the risks of unbridled AI competition are widely recognized. The US government should be doing its part to address this, and swiftly. But making AI go well is a shared responsibility. Companies that lag behind their peers on safety, don’t invite external audits of their systems, or call for brakes while doing little to build them won’t be able to blame the AI race when something goes wrong.