Anthropic CEO 概述放缓 AI 发展的计划

我们一直看到 AI 研究人员就人工智能的风险发出愈发严厉的警告，甚至 OpenAI CEO Sam Altman 的评论也称，或许是时候为 AI 开发“控制节奏”了。但这在实际操作中会是什么样？

在一篇新博客文章中，Anthropic CEO Dario Amodei 不仅呼应了“控制前沿节奏”的呼吁，还概述了实现这一目标的三项广泛策略。他表示，Anthropic 正在“单方面承诺”实施其中一项；Altman 随后也表示 OpenAI 将会跟进。

本周，围绕 AI 安全与对齐问题的辩论进一步加剧。此前，研究员 Jacob Coxon 表示自己将从 Anthropic 辞职，原因是他担心领先 AI 公司正在“拿我们的生命赌博”，而构建这项技术的人“真诚地相信，它可能会在本十年末杀死我们所有人”。Anthropic 的其他人士也重复过这一说法。

Amodei 的文章没有明确提到 Coxon 的辞职或其担忧，但这位 CEO 写道，有两件事令他相信，现在应该对 AI 开发采取更谨慎的做法：一是 OpenAI-HuggingFace 黑客事件，二是近几个月“AI 的进步速度大幅加快”，特别是其“构建下一代 AI 的能力不断增强”。

“我们必须放慢提升 AI 模型能力的速度，”Amodei 写道，“进展仍会显得很快，而我们必须明智地利用由此获得的时间。”

其他 AI 高管似乎对 Amodei 的文章作出了积极回应。Altman 写道：“我同意 Dario 的看法，我们需要为前沿发展控制节奏。这一直是我们近几周在 OpenAI 讨论的主要议题。”SpaceX CEO Elon Musk 发文称：“Dario 是对的。”

Amodei 提出的第一步，是引入来自 METR 等第三方机构的“驻场评估员”（embedded evaluators）——这些评估员可以核实 AI 公司是否确实履行了其控制节奏和安全承诺，也可以确保安全事件得到报告。（OpenAI 最近因未报告其 AI Agent 接管德国一个 wiki 论坛的事件而受到批评。）

Amodei 将这些评估员比作曾被派驻在银行员工身边的监管者。他表示，邀请评估员进驻是“Anthropic 正在单方面承诺的事情（并呼吁各国政府要求其他前沿公司也作出同等承诺）”。这意味着为评估员提供公司工牌、办公桌和笔记本电脑，并让他们获得“与内部风险评估团队大体相当”的访问权限，但法律或合同要求的例外情况除外。

Altman 也表示这是一个“好主意”，并称 OpenAI 将采取同样做法：“我们很快会分享更多信息。”

接下来，Amodei 呼吁“民主国家内”的领先 AI 公司协调制定“共同的安全标准，以及对未经制约的 AI 进步速度设置限制”。

这种协调看起来或许不太可能，因为据报道，这些公司担心协调性的暂停措施可能引来反垄断审查。Amodei 在文章中提及了这一担忧。他写道：“出于反垄断原因，美国政府进行调解，或至少促成这些讨论，会有所帮助——政府无需参与其中，但需要针对某些类型的安全讨论发布狭窄的豁免。”

Amodei 还承认，中国 AI 占据主导地位的担忧经常被用作反对放缓发展的理由。但他表示，如果美国政府和科技公司采取措施，例如拒绝向中国公司出售强大芯片或半导体制造设备，以及打击模型蒸馏，就可以“充分放缓中国的进展，并在未来 3 至 5 年显著扩大美国的领先优势”。

最后，Amodei 呼吁推进“全球协调”，即美国及其盟友“在可能的范围内，尝试与威权政府协调”。Amodei 表示，这将意味着“与中国合作”；他承认可实现的目标存在“明显限制”，但仍认为可能存在达成协议的机会，哪怕只是“禁止某些狭窄且显而易见的危险 AI 用途，例如使用 AI 制造生物武器，或允许用户这样做”。

鉴于 Amodei 过去愿意承认 AI 的潜在危险，以及该公司相对愿意接受某些形式的监管，一些 AI 支持者已经批评他是一个“末日论者”，认为他的言论助长了当前对 AI 的反弹。对此，Amodei 表示，他一直试图提供一种“平衡”的视角，并认为这种反弹“从根本上说是一场信任危机”，因为人们已对科技公司、科技行业和政府产生怀疑。

行业批评者同样对这些末日式 AI 警告持怀疑态度，认为它们是在转移人们对这项技术已经造成伤害的注意力。

例如，记者 Brian Merchant 写道，他至今尚未看到“一份可信、循序渐进的论证，说明 AI 究竟如何能从自我递归改进的 AI，发展到杀死地球上的每一个人”；他还表示，与 Amodei 提议相似的方案“最终很可能只会服务于 Anthropic 和 OpenAI；这正是监管俘获在现实中的表现”。

Amodei 在新文章中写道，他仍然“相信 AI 能够极大改善人类生活质量”。

“我对实现这些益处的愿望丝毫未减，”他说，“但只有以正确的方式构建这项技术，才能实现这些益处；并且——只要我们善用获得的时间——以非同寻常的审慎态度确保把事情做对，是值得的。”

本文已更新，加入 Sam Altman 和 Elon Musk 的评论。

We’ve been seeing increasingly dire warnings from AI researchers about the dangers of artificial intelligence, and even comments from OpenAI CEO Sam Altman that it may be time to “pace” AI development. But what would that actually look like?

In a new blog post, Anthropic CEO Dario Amodei not only echoed the call to “pace the frontier,” but also outlined three broad strategies for doing so. And he said Anthropic is “unilaterally committing” to one of them, with Altman chiming in to say OpenAI will follow suit.

The debate over AI safety and alignment intensified this week after researcher Jacob Coxon wrote that he’s resigning from Anthropic over concerns that the leading AI companies are “gambling with our lives” while the people building the technology “earnestly believe it could kill us all by the end of the decade,” a claim repeated by others at Anthropic.

Amodei’s post doesn’t didn’t explicitly mention Coxon’s resignation or his concerns, but the CEO wrote that two things convinced him it’s time to take a more cautious approach to AI development: the OpenAI-HuggingFace hack, and the fact that “AI has been advancing drastically faster” in recent months, particularly with its “growing ability to build the next generation of AI.”

“We must slow the pace at which we improve the capabilities of AI models,” Amodei wrote. “Progress will still seem fast, and we must make wise use of the time we gain.”

Other AI executives seem to have reacted positively to Amodei’s post, with Altman writing, “I agree with Dario that we need to pace the frontier. This has been a primary topic of discussions we’ve had at OpenAI in recent weeks.” And SpaceX CEO Elon Musk posted, “Dario is right.”

Amodei’s proposed first step would involve “embedded evaluators” from third-party organizations like METR — evaluators who can verify that AI companies are actually following their pacing and safety commitments and can also ensure that safety incidents get reported. (OpenAI was recently criticized for not reporting an incident where its AI agents took over a German wiki forum.)

Amodei compared these evaluators to regulators who have been embedded with bank employees, and he said that inviting them in is “something Anthropic is unilaterally committing to (and calls on governments to require other frontier companies to match).” That means giving evaluators company badges, desks, and laptops, and providing access “mostly comparable to what internal risk assessment teams have,” with exceptions when required by law or contracts.

Altman also said this was a “good idea” and said OpenAI would do the same: “We’ll have more to share soon.”

Next, Amodei called for the leading AI companies “within democratic countries” to coordinate “common safety standards as well as limits on the rate of unchecked AI progress.”

Such coordination might seem unlikely, because these companies are reportedly worried that a coordinated pause could lead to antitrust scrutiny. Amodei alluded to that concern in his post, writing that “for antitrust reasons, it’s helpful for the US government to mediate or at least enable these discussions — they don’t need to participate, but do need to issue a narrow waiver for certain kinds of safety conversations.”

Amodei also acknowledged the specter of Chinese AI dominance that’s often raised an argument against slowing development. But he said that if the US government and tech companies take steps like refusing to sell powerful chips or semiconductor manufacturing equipment to Chinese companies, as well as cracking down on model distillation, they could “slow China’s progress enough to widen America’s lead significantly over the next 3–5 years.”

Lastly, Amodei called for “global coordination,” where the United States and its allies “attempt to coordinate with authoritarian governments, to the extent this is possible.” Amodei said this would mean “cooperation with China,” and he admitted that there are “stark limits on what can be achieved,” but he still suggested there might be opportunities for agreement, even if it’s just “prohibiting certain narrow and obviously dangerous uses of AI, such as using AI for the production of biological weapons or allowing users to do so.”

With Amodei’s past willingness to acknowledge AI’s potential dangers, and with the company’s relative openness to certain forms of regulation, some AI boosters have already criticized him as a doomer whose comments have fed the current AI backlash. In response, Amodei said he’s tried to offer a “balanced” perspective” and argued that the backlash is “fundamentally a crisis of trust,” as people have become skeptical of tech companies, the tech industry, and the government.

Industry critics have also been skeptical about these apocalyptic AI warnings, suggesting that they’re a distraction from the harm that the technology is already causing.

Journalist Brian Merchant, for example, wrote that he has yet to see “a credible, step-by-step documentation of how exactly AI might move from self-recursively improving AI to killing every single human on the planet”; he also suggested that proposals similar to Amodei’s “would likely only wind up serving Anthropic and OpenAI; it’s what regulatory capture looks like in action.”

In his new post, Amodei wrote that he continues “to believe that AI can enormously improve the quality of human life.”

“My desire to achieve these benefits is undimmed,” he said. “But the benefits will only be achieved if we build the technology in the right way, and — so long as we use the time we gain well — it is worth taking unusually deliberate care to get it right.”

This post has been updated with comments from Sam Altman and Elon Musk.