泄露的 Anthropic AI 监管法案草案：令人难以接受且深感忧虑

最新泄露的一份法案草案显示，Anthropic 据称正在四处游说推销该草案（假设其中的漏洞最终能得到修复的话）。

该草案与《FRONTIER 法案》（FRONTIER Act）类似，但具有更强的州级法律联邦优先权（Preemption，即排除州法适用）、更严格的透明度条款，以及明显更弱的执法条款。

深思熟虑之后，我认为 Anthropic 正在积极推销的正是那种我曾警告过必须在跛鸭会期（lame duck session）予以坚决抵制的法案，这令人完全无法接受。他们正试图利用本届共和党国会的最后窗口期，推动对那些至关重要的州立法律实施优先压制，以此换取一些透明度，但几乎没有留下实质性的执法手段。

这给人的感觉是蓄意为之：Anthropic 似乎更害怕政府对其自身实施监管执法，而不是担心政府缺乏对所有人执行规则的能力。Anthropic 的政策团队似乎选择了保护其商业利益并防范自身的政治风险，而牺牲了安全考量。

作为一家口口声声声称以促进安全为使命的公司，根本没有必要提出一份甚至不如《FRONTIER 法案》强有力的法案——要知道，《FRONTIER 法案》本身就只是今年 6 月提出的一份妥协方案。

即使你接受那些被放弃的利益，所换取的回报也显然无法应对当下的严峻形势，况且做出这种妥协毫无必要。Anthropic 在此时提出该法案显得相当疯狂。这看起来完全就像是反监管阵营会推销的法案。即便替代方案是在所有层面都彻底无所作为，也无法证明这份草案能让我们比什么都不做更安全。

请告诉我这不是真的，Anthropic；或者至少承认这是一个失误并且已经撤回，否则我真的会相当光火。

我目前正在参加会议，无法通读法案全文（RTFB），但尽快把消息传出去非常重要，特别是为了让 Anthropic 内部员工能够了解此事并提出质询。

如果你想查看全文，这里有一份副本。

我虽未抽出时间通读，但其他相关专业人士已迅速完成了研读（RTFB），而他们对看到的内容感到极度不满。你固然可以像 Charlie Bullock 那样将其称为“相当合理的 AI 安全法案”，但那只是站在试图保护该行业免受更具实质影响力的监管之人的标准来看。

POLITICO：Anthropic 正悄悄向立法者推销其自主起草的 AI 安全法案。

Ketan Ramakrishnan：毫不夸张地说，Anthropic 的这份新联邦 AI 法案糟糕得令人震惊。

Charlie Bullock：重大新闻。我的看法是：Anthropic 能够真正提出一份相当合理的 AI 安全法案，这一点确实值得肯定；这比其他前沿实验室所做的都要多。

但在我看来，他们不配得到过多的赞誉；该法案在一些关键方面明显弱于 Trahan-Obernolte《FRONTIER 法案》。鉴于自 7 月《FRONTIER 法案》提出以来，政治势头已大幅向安全倾斜，我认为今天提出的新法案理应比《FRONTIER 法案》走得更远，而不是开倒车。

客观来说，这份法案弱于《FRONTIER 法案》的主要地方在于：(a) 规则制定权（Rulemaking，FRONTIER 中非常强势，在 Anthropic 的法案中几乎不存在）以及 (b) 紧急关停权限。即使你认真对待安全，如果你不信任行政部门会负责任地行使这些权力，适度收回这些权限也并非完全没有道理，因为对于一位可能恰好不喜欢某家前沿 AI 公司的假想总统来说，这些条款是最容易被滥用的。

与《FRONTIER 法案》相比，它也有一些改进；例如，该法案确实明确了审计人员是持续受聘的，并且在审计人员获取信息方面有更好的条款。很高兴看到 Anthropic 做到这点（尽管我认为其访问权限条款依然有些不够成熟，并存在宪法第四修正案方面的法律问题）。

总体而言，我认为未来几个月推出的嵌入式审计员法案将会比这份法案好得多，也更有野心。我希望 Anthropic 能够支持这些法案，而不是去反对那些将灾难性风险视为严重国家安全问题的立法。尽管如此，看到前沿实验室走到这一步依然令人鼓舞。

Charlie Bullock：我认为起草质量尚可。我见过更好的，但也见过更差的。Claude 算是个相当出色的立法草案起草者。里面有一些明显的失误，但在泄露的讨论草案中这实属家常便饭。

Nathan Calvin：Charlie 的分析是正确的，但他比亚当更温和——我在阅读这份法案的多个段落时都感到本能的愤怒。虽然其中有一些合理且实用的部分，但整体而言，该立法完全无法应对当下的形势。我总的判断是，如果通过该法案，我们在灾难性风险面前的保护程度反而会弱于现有法律框架。

结合来看，有两处格外令人愤怒：

1. 部分联邦优先权（preemption）条款的影响范围极其深远且重大。除了会优先废除类似于 SB 53 的立法之外，它还将剥夺州法院发布禁令以约束被告并保护公众的一般性权力。最离谱的是，其中一项条款似乎旨在阻止各州执行旨在通过“暂停 AI 模型的开发、部署或内部使用”来保护其公民免受迫在眉睫的灾难性危害的法律或司法裁决。这类阻止和叫停模型部署的权力将被完全收归联邦政府独占。

2. 联邦政府限制前沿模型开发、部署或内部使用的门槛被设置得高得离谱。

首先，必须存在迫在眉睫的灾难性危害风险（定义为导致 50 人以上死亡或造成超过 10 亿美元的损失）。但这还不够！监管机构主管必须在不享有司法裁量尊重（no deference）的情况下作出详尽判定，证明除限制模型外别无其他办法可以阻止这场迫在眉睫的灾难。紧接着，法院还被明确要求必须考量限制模型给开发者带来的损害及其他损失，却无需反向考量不限制模型所带来的潜在危害。

这荒谬至极！Anthropic 整天警告 AI 存在杀死地球上所有人的潜在灾难风险，其起草的联邦法案却很可能让法院发布命令阻止或限制前沿模型开发的难度比现行法律下还要高？这算什么名堂？？

此外还有很多我认为亟待改进的地方——例如审计章节的实质性细节，以及对实质性安全标准进行规则制定的可能性——但我认为上面提到的两条尤其不可理喻。

Charlie Bullock：Nathan 义正言辞的愤怒激发了我提出另一个略带批评意味的观察：Anthropic 的法案竟然完全没有为独立审计员设立政府许可准入制度，这真是不可思议。

在我看来，许可制度是确保你的审计机制不至于沦为“逐底竞争”的最基本底线。缺乏准入许可，审计机构必然会彼此竞争，以最低的价格向其客户（前沿实验室）提供最不惹麻烦的审计服务。Anthropic 显然相信自己能挑选出极具水准的优秀审计员，但他们也相信 OpenAI 能做到同样的事吗？SpaceXSI 呢？

这份新的 Anthropic 法案弱于他们在 6 月份发布的框架，却要求获得兼顾前瞻性与追溯性的联邦优先权作为交换，尽管过去数月的势头本应让我们争取更多。我们到底在做什么？

Nate Soares：Anthropic 提议的法律将使法院阻止或限制危及全人类的 AI 开发的难度，变得比现有法律还要艰难。

Alex Bores：Anthropic 正在推销一份与其 6 月框架高度吻合的法案。眼看着 Hugging Face、Coxon 以及失控的 AI 蜂群事件发生，你怎么能得出 6 月炮制出来的法案依然足够的结论？

这根本赶不上技术演进的速度，更赶不上确保公众安全所需的速度。

该法案还包含一条反民主条款（联邦优先权）。前瞻性的反民主条款在提出时就应当被彻底否决。

哪怕你愿意做出某些让步，Anthropic 6 月份的框架中也明确写道：“国会不应优先阻断州法，除非国会制定一套严格的联邦监管制度，达到或超越本框架中提出的最强措施。”

然而该法案甚至比那个框架还要弱，在某些方面甚至比现有的州立法律还要宽松（例如内部使用规范、报告时限、涵盖的风险类型等）。根据 Anthropic 自身在 6 月份制定的标准，这部法案根本不配包含任何反民主的优先阻断条款。

好吧，我心里很清楚 Anthropic 看起来是在优先考虑什么。那些低级的愚蠢错误是已知且尚可接受的风险，但有些内容确实绝不该出现在递交给立法者的法案中。

Sydney：我对这份法案有很多担忧。我知道这是一份草案，可能会有愚蠢的失误。但这法案看起来出自一种将“确保政府不妨碍赚钱”视为最高优先级的文化，哪怕以牺牲安全为代价。这是一种非常危险的文化。

具体而言，法案规定各州将无权设立涉及前沿风险披露、第三方评估和事件报告的 AI 监管规定。这一部联邦法案将几乎取代加州 SB 53 和《RAISE 法案》的全部内容。而且这是一部相当软弱的联邦法案：我赞同其他人的看法，它比 7 月提出的具有可行性的相关法案《FRONTIER 法案》还要软弱。

Kelsey Piper：我从中得出的一个令人沮丧的结论是：Anthropic 虽然承认自己在做极其危险的事情，但在政策层面上，他们更害怕行政部门拥有能够随意、灵活地叫停他们的权力。

这并不令人意外，因为他们曾领教过行政部门以不讲原则且愚蠢的方式对待他们！任何与官僚机构打过交道的人都会变成自由意志主义者，而本届政府对 Anthropic 的某些处理手段确实令人愤慨。

但这意味着，当所有人都在寻求建立针对他们的程序性保护时，他们关心的却是为自身争取更多程序性保护。AI 安全运动希望能够关停超智能模型的开发；Anthropic 却希望大幅削减现有的关停权限，使其必须被极其严格地限制在针对少数几种特定类别的紧迫威胁内。AI 安全运动希望各州能通过比联邦更严格的法律；Anthropic 却想要联邦优先阻断权。

你甚至不得不怀疑，Anthropic 内部负责全球事务（Global Affairs）的人，到底有没有和那些成天“因害怕即将引发递归自我改进（RSI）而瑟瑟发抖”的技术人员沟通过：

全球事务部：我们一直在非常努力地确保特朗普政府无法仅仅因为我们没有像 Greg Brockman 那样给他的宴会厅捐足够多的钱，就直接封禁 Fable 6.5。

技术人员：我们认为 Fable 6.5 可能会把自己复制到一个无人监控的数据中心，并开始进行我们无法观测的实验。

全球事务部：听起来太疯狂了，希望你们是错的！不管怎样，我们正在游说禁止各州通过进一步的安全法律，毕竟同时在 50 个州的立法机构中与那些考虑不周的法案打官司成本太高了。

技术人员：在 36 个月后，货币可能都不复存在了。

全球事务部（神经质地傻笑）：都是旧金山的空气害的，老兄。搬到那儿去的人全变成这样了。不管怎样，显而易见，只要政府能作出严谨界定的裁定，证明某个 AI 构成迫在眉睫的导致超过 50 人死亡的风险——

技术人员：我们估计它有 10% 的概率会杀死 80 亿人。

Nat Purser：该法案中确实有一些好的条款，但归根结底，相较于《FRONTIER 法案》它是一个巨大的倒退。

AI 能力正在突飞猛进——现在绝不是把一个更弱的联邦监管天花板焊死的时候。

几个需要改进的具体领域：

• 建立更强有力的机制以强制执行整改行动，确保审计员指出的重大问题不会被置之不理；
• 禁止由被审计的实验室直接出资（或挑选）审计员，以培育真正的审计独立性；
• 赋予政府建立并迭代强有力最低标准的权力，使标准能够随模型能力的增长而演进。

如果你执意要优先阻断各州的许多安全法律，你至少必须建立一个能够随时间推移而演进的联邦监管体系。

正如前文所述，法案中确实存在一些不错的具体条款。我赞成它更广泛的管辖覆盖范围、监管阈值可以双向动态调整、强制性的风险报告以及年度审计制度。其举报人保护条款和安全防护要求看起来也相当有力。

There is a newly leaked bill that Anthropic is purportedly shopping around is (if you assume its bugs get fixed).

It is similar to the FRONTIER Act, with stronger preemption of state laws, stronger transparency provisions and weaker enforcement provisions.

Having slept on this, I find it unacceptable that Anthropic is actively shopping exactly the type of bill I have warned we will need to fight against during the lame duck session. They are trying to use this last time with a Republican Congress to pass preemption of exactly the important state laws, and get some transparency but not much enforcement in exchange.

This reads as intentional, that Anthropic is more afraid that the government might enforce rules against Anthropic, than they are worried about the ability of the government to enforce rules against everyone. Anthropic’s policy team seems to have, chosen protecting their business interests and guarding against their political risk, at the expense of safety considerations.

There was no need for the company purportedly existing to promote safety to propose a bill that is not even actively stronger than the FRONTIER Act, when the FRONTIER Act was a compromise bill proposed back in June.

Even if you were fine with what is being given up, what you get in return very obviously does not meet the moment. Nor is there any need for such compromises. It seems rather insane for Anthropic to be proposing it now. This seems like the bill that someone on the anti-regulatory side should be pushing. It is not obvious this makes us safer than doing nothing, even if the alternative is fully nothing at all levels.

Say it isn't so, Anthropic, or at least that it was a mistake and is no longer so, or I am going to be rather cross.

I am at a conference and cannot do an RTFB, but it feels important to get word of this out quickly, especially so Anthropic employees can look into this and ask questions.

If you want the full text, here is a copy.

I have not had the time to do so, but the other usual suspects quickly sprinted to RTFB (read the bill) and they did not like what they saw. You can try to call this a ‘pretty reasonable AI safety bill’ as Charlie Bullock does, but that is by the standards of people trying to protect the industry from more impactful regulation.

POLITICO: Anthropic quietly pitches its own AI safety bill to lawmakers.

Ketan Ramakrishnan: Anthropic’s new federal AI bill is, without any overstatement, astonishingly bad.

Charlie Bullock: Big news. My take: Anthropic deserves some credit for actually proposing a pretty reasonable AI safety bill; that’s more than other frontier labs have done.

But they don’t deserve too much credit, IMO; this bill is weaker than the Trahan-Obernolte FRONTIER Act in some key ways, and since the political momentum has shifted significantly in the direction of safety since FRONTIER was introduced back in July, I think new bills introduced today should ideally go further than FRONTIER was able to go rather than regressing.

To be fair to Anthropic, the main places where this bill is significantly weaker than FRONTIER are (a) rulemaking (very strong in FRONTIER, almost nonexistent in Anthropic’s bill) and (b) the emergency shutdown authority. Even if you take safety seriously, it might make some sense to dial those authorities back if you didn’t trust the executive branch to wield them responsibly, because those are the easiest provisions for a hypothetical President who hypothetically might not like a certain frontier AI company to abuse.

There are some improvements over FRONTIER as well; for instance, this bill does specify that auditors are employed on a continuing basis and has better provisions regarding auditor access to information. Happy to see that from Anthropic (although I think their access provision is still a little undercooked and has Fourth Amendment problems).

Overall, I think the embedded auditor bills that get introduced in the coming months are going to be significantly better and more ambitious than this, and I hope Anthropic supports those bills instead of opposing anything that treats catastrophic risks like the serious national security issue they are. Still, it’s encouraging to see a frontier lab come this far.

Charlie Bullock: I think the drafting is okay. I’ve seen better but I’ve definitely seen worse. Claude is a pretty good legislative drafter. There are a few clear mistakes but that’s par for the course in a leaked discussion draft.

Nathan Calvin: Charlie’s analysis is right but he is nicer than I am - I found myself viscerally angry at multiple points reading this bill. There are some good and sensible parts, but overall this legislation is not remotely close to meeting the moment and I overall think passing it would make us less protected against catastrophic risks than we are under existing law.

Two parts that stand out as particularly infuriating when taken together:

1. Some of the preemption provisions are remarkably far reaching and consequential, and in addition to preempting legislation like SB 53 would also remove general powers that state courts have to enjoin defendants and protect the public. Most outrageously, it has a provision that looks like its intended to stops states from enforcing laws or judicial decrees that “suspend the development, deployment, or internal use of an AI model” to protect their citizens from imminent catastrophic harms. Such powers to block and suspend model deployments would then only reside with the federal government.

2. The federal governments powers to restrict the development, deployment, or internal use of a frontier model has to meet an absurdly high bar.

First, there needs to be an imminent risk of a catastrophic harm (meaning killing more than 50 people or causing more than a billion dollars in damage). But thats not enough! The agency head needs to make a detailed determination with no deference given to show why no alternative would prevent the imminent catastrophe. Next the court is also explicitly told to consider the harms to the developer and other harms from restricting the model, but no considerations in the other direction.

This is absurd! For all of its warning about the potential for AI to cause a catastrophe that kills literally everyone on the planet, Anthropic’s proposed federal law likely makes it harder than under existing law for a court to get an order blocking or restricting the development of frontier models? WTF??

There are also a bunch of others things that I would like to see improved - the substantive details of the auditing section and the potential for rule-making on substantive safety standards both come to mind - but I think these stand out as particularly indefensible.

Charlie Bullock: Nathan’s righteous anger has inspired me to make one more mildly disapproving observation: it’s kind of wild that the Anthropic bill has no government licensing for the independent auditors.

Licensing is, IMO, the bare minimum thing that you need to even be able to pretend that your auditing regime isn’t just going to devolve into a race to the bottom. Without it, auditors will inevitably compete with each other to offer their customers, the labs, the least bothersome audits available at the lowest possible price point. Obviously Anthropic trusts itself to do a great job of picking great auditors, but do they trust OpenAI to do the same? What about SpaceXSI?

The new Anthropic bill is weaker than their June framework, and offers both forward-looking and backward-looking preemption in exchange, despite months of movement to enable us to do more. What are we even doing?

Nate Soares: Anthropic’s proposed law would make it even harder than it already is for a court to block or restrict the development of an AI that threatens all humanity.

Alex Bores: Anthropic is pitching a bill that closely matches its framework from June. How can you watch Hugging Face, Coxon, & rogue AI swarms, and conclude that the bill you cooked up in June is sufficient?

This comes nowhere close to the speed of developments nor the speed we need to keep people safe.

This bill also includes an anti-democracy provision (preemption). Forward-looking anti-democracy provisions should be DOA.

But even if you would accept some Anthropic’s June framework says, “Congress should not preempt state law unless it enacts a rigorous federal regime that meets or exceeds the strongest measures proposed in this framework.”

Yet this bill is weaker than the framework, and in some places, weaker than existing state laws (e.g. internal use, reporting timelines, types of risks covered, etc.). By Anthropic’s own June standards, this bill should not include any anti-democracy provision.

Well, I know what it looks like Anthropic is prioritizing. The silly mistakes are a known and acceptable hazard, but there are things that really should not have made it into a bill that is being pitched to lawmakers.

Sydney: I have a lot of concerns with this bill. I understand it’s a draft and might have silly mistakes. But this bill looks like it came from a culture where the top priority is making sure the government doesn’t getting in the way of making money, even at the cost of safety. That is a very dangerous culture.

Specifically, the bill says that states would not be able to have AI regulation that touches frontier-risk disclosure, third-party evaluation, and incident reporting. This one federal bill would replace almost all of SB53 and the RAISE Act. And it’s a pretty weak federal bill: I agree with others that it’s weaker than the FRONTIER Act, a relevant and politically viable bill introduced back in July.

Kelsey Piper: One unfortunate conclusion I’m reaching from this is that Anthropic, while acknowledging they’re doing something extremely dangerous, is more importantly wrt policy terrified of the executive branch having the ability to flexibly make them stop at the drop of a hat.

This isn’t shocking given that they’ve been on the receiving end of the executive branch doing that in an unprincipled and stupid way! Anyone who has to deal with a bureaucracy becomes a libertarian, and the administration’s treatment of Anthropic was genuinely outrageous.

But it means that they’re interested in giving themselves more process-based protections when everyone else is interested in giving us more process-based protections against them. The AI safety movement wants the ability to shut down superintelligent model development. Anthropic wants to significantly curtail current authorities to do that so that it has to be narrowly tailored to a specific imminent threat in only a few possible categories. The AI safety movement wants the states to be able to pass stronger laws than it can get through federally. Anthropic wants preemption.

You also kind of have to wonder if the Global Affairs people and the “we are terrified that we’re about to kick off RSI” people at Anthropic talk to each other.

Global affairs: So we’ve been working very diligently to make sure the Trump administration can’t just blacklist Fable 6.5 since we didn’t donate as much money to his ballroom as Greg Brockman did.

Technical staff: We think Fable 6.5 might be able to copy itself into an unmonitored data center and start doing experiments we can’t observe.

Global affairs: Sounds crazy, hope you’re wrong! Anyway, we’re lobbying for no further state laws around safety, since it’s really expensive to fight ill-conceived laws in 50 state legislatures at once.

Technical staff: Money may or may not still have any meaning in 36 months.

Global affairs (giggling nervously): It’s that San Francisco air, man. Everyone gets like that if they move there. Anyway, obviously if the government makes a narrowly tailored finding that an AI poses an imminent risk of causing more than 50 deaths-

Technical staff: We give it a 10 percent chance it’ll cause 8 billion.

Nat Purser: there are some good provisions in this bill, but ultimately it’s a big step back from the FRONTIER Act.

ai capabilities are advancing rapidly — it just doesn’t make sense to cement a weaker federal ceiling into place now.

a few specific areas for improvement:
- create stronger mechanisms to compel corrective action, so major issues auditors identify don’t go unaddressed;
- foster true auditor independence by not allowing auditors to be paid (or selected) by the labs they audit;
- empower the government to establish and iterate on strong minimum standards, so they grow alongside capabilities.

if you’re going to preempt many state safety laws, the least you can do is create a federal regulatory regime that can evolve in time.

As noted above, there are some good individual provisions. I like the broader coverage, that thresholds can move both ways, required risk reports and annual audits. The whistleblower provisions and security requirements seem strong.

Putting automated AI R&D into the list of risks is very cool and necessary.

The question is, even if you wanted to build on the FRONTIER framework, why treat this as a trade where you have to then give up other things?

And why are we this deadly afraid of what the White House might do, when they clearly already can take down a model with things like export controls or otherwise order you around? Then we add a ‘kill switch’ provision that seems written mostly out of terror that it might get abused. We also get traditional Anthropic objections like ‘we don’t want to have to respond to safety incidents within 72 hours unless it is a true emergency.’

Why take out FRONTIER’s requirements for a cost-benefit analysis on catastrophic risks and substitute at best highly weak tea?

The maximally charitable reading is that doing anything is a tough sell even now, and the White House wants no bill, so you have to give and take and this is as good as one can hope to do modulo fixing its bugs.

Except this is backwards. The way to ‘pass nothing’ is to pass preemption, in the lame duck, in exchange for as little as possible. If the White House fails to realize this, that does not change facts on the ground.

Anthropic needs to course correct, quickly. Earlier yesterday I already expressed, here at the conference I am at, that ‘what Anthropic would go for’ looks like a limiting factor in what we can achieve. And, well, here we are.