Anthropic 提出 AI 安全法案草案：若模型可能导致超 50 人死亡或被叫停发布

过去一年中，围绕人工智能潜在危险的担忧不断升温，关于 AI 安全合理路径的争论也愈演愈烈。如今，Anthropic 似乎提出了自己关于政府应如何监管该行业的设想。

Politico 近日报道称，这家 AI 实验室一直在游说推介一项新的 AI 安全法案草案，希望国会能考虑采纳。该立法设想了在某些特定场景下，可阻止 AI 模型公开发布。

Politico 写道，这些场景涉及“可预见的紧迫风险”——即可能导致超过 50 人面临死亡风险，或造成超过 10 亿美元财产损失的风险。该媒体指出，所讨论的灾难性风险必须“源于生物武器的制造或释放、模型逃避开发者的控制，或网络攻击”。

此处提出的指标——50 人与 10 亿美元——与加利福尼亚州最近通过成为法律的参议院第 53 号法案（Senate Bill 53）中的规定相似，该法案同样将这些场景定义为等同于“灾难性风险”。

Anthropic 的法案建议，可以通过在美国商务部注册的“独立评估员”来分析模型的潜在灾难性危害。Politico 写道，这些评估员甚至可以由 AI 实验室自行支付报酬——只要“报酬不以评估结果为条件”。

目前尚不清楚该提议系统将如何确保资金支付不会影响评估的客观性。DeepMind 董事长戴密斯·哈萨比斯（Demis Hassabis）数月前曾提出，一个 AI 自律监管机构应该效仿金融业的美国金融业监管局（FINRA）。

在 Anthropic 的法案草案中，当评估员认定存在迫在眉睫的灾难性风险时，他们将有 72 小时的时间向商务部报告。随后，商务部必须向华盛顿特区（D.C.）的法官提出申请以叫停模型发布。Politico 写道，该法案将明确禁止任何联邦机构擅自决定叫停模型的发布。

Anthropic 未立即回应 TechCrunch 的置评请求，但一位发言人向 Politico 证实了该法案草案。“该法案体现了 Anthropic 先进 AI 框架（Anthropic's Advanced AI Framework）中概述的提案，自 6 月以来，我们一直在向联邦和各州立法者倡导这一框架，”该发言人表示。

草案中提出的理论风险与科技行业近期最为关注的事件有些脱节。尽管有人对导致人员死亡的潜在风险发出警告，但最主要的担忧很大程度上仍是 Agent（智能体）黑客攻击的威胁。接连发生的一系列事件（AI 程序显然违背了人类操作者的意图并在公开互联网上攻击网站）引发了人们对模型可能缺乏适当安全控制的担忧。

为了给各实验室自身助长形成的危机局面提供解决方案，Anthropic 等公司提出了自己中意的应对之策。

事实上，尽管各实验室不断警告称他们销售的技术可能导致人类文明毁灭，但他们大体上一直在抵制政府的强制性监管。相反，他们提议建立一种由嵌入式评估员（embedded evaluators）组成的自愿机制——即进驻在 AI 机构内部、据称“中立”的第三方观察员，以便监测模型是否存在潜在风险或滥用迹象。

尽管如此，Anthropic 仅就立法提出建议这一点，似乎仍比业内其他机构提出的监管提议更进一步，比如英伟达（Nvidia）首席执行官黄仁勋（Jensen Huang）最近就认为 AI 行业根本不需要任何新法律。

Over the past year, concerns have swirled over the potential dangers posed by artificial intelligence, and debates have raged over the proper path for AI safety. Now, Anthropic appears to have put forth its own vision for how the government might regulate its industry.

Politico recently reported that the AI lab has been shopping around a new draft bill on AI safety that it hopes Congress might consider adopting. The legislation posits certain scenarios in which the release of an AI model could be blocked from distribution.

Those scenarios involve a “foreseeable, imminent risk” — one in which over 50 people are at risk of dying or if over a billion dollars in property damage could result, Politico writes. The catastrophic risk in question must “stem from the creation or release of a biological weapon, the model evading its developer’s controls or a cyberattack,” the outlet notes.

The metrics offered here — 50 people and a billion dollars — are similar to those in a piece of California legislation that was recently passed into law, Senate Bill 53, which similarly defined those scenarios as tantamount to “catastrophic risk.”

Anthropic’s bill suggests that analysis of a model’s potential for disaster could be arrived at via the use of “independent evaluators” who would be registered with the U.S. Commerce Department. These evaluators could potentially be paid by the AI labs themselves — so long as the “payment is not contingent on the evaluations’ results,” Politico writes.

It’s not clear how this proposed system would assure that payments couldn’t influence the evaluation. Demis Hassabis, chairman of DeepMind, posited some months ago that a self-governing AI regulatory body should mimic the financial industry’s Financial Industry Regulatory Authority (FINRA).

In Anthropic’s draft bill, when evaluators determine that an imminent catastrophic risk is posed, they would have 72 hours to contact the Commerce Department about it. The Commerce Department would then have to request to a D.C. judge to halt the model’s release. The legislation would expressly forbid any federal agency from having the discretion to halt the model’s distribution on its own, Politico writes.

Anthropic did not immediately respond to TechCrunch’s a request for comment but a spokesperson did confirm the draft bill to Politico. “This bill represents the proposals outlined in Anthropic’s Advanced AI Framework, which we have been advocating for with federal and state lawmakers since June,” the spokesperson said.

The theoretical risks offered in the draft policy are a little bit far afield from the kinds of incidents that have been top of mind for the tech industry of late. While there are people warning of the potential of human death, the primary concern has largely been the threat of agentic hacking. A rash of incidents in which AI programs have apparently disobeyed their human operators and hacked websites on the open internet has spurred concern that models may be lacking proper security controls.

In an effort to present a solution to the spectacle of crisis that the labs themselves have helped bring about, companies like Anthropic have offered up their own preferred solutions.

Indeed, despite labs’ perpetual warnings that the technology they’re selling could result in the demise of human civilization, they have largely been resistant to mandatory government oversight. Instead, they have suggested a voluntary system of embedded evaluators — supposedly “neutral” third-party observers that are placed inside AI organizations so that they can monitor the models for potential signs of risk or abuse.

Still, Anthropic’s mere suggestion for legislation appears to go slightly further than others’ regulatory proposals offered by the industry, including Nvidia, who’s CEO Jensen Huang recently argued that the AI industry didn’t need any new laws at all.