ChinAI #375：批评 Anthropic 版“放缓前沿”主张

几周前，我最喜欢的博主 Dario Amodei——他兼职担任 Anthropic CEO——发表了一篇新文章，题为《我们必须放缓前沿》。这篇文章呼应了“放缓前沿”的目标：这是一份由超过 13,000 名前沿 AI 公司员工签署的声明，呼吁国际社会共同“审慎放缓自动化 AI 开发的前沿”。但一如既往，Dario 和 Anthropic 推出了这个目标的中国版本。具体来说，Dario 的版本认为，放缓前沿也意味着要让美国相对于中国的 AI 领先优势“尽可能大”——这明显偏离了由许多 Anthropic 员工签署的“放缓前沿”声明。下面让我们剖析他最新文章中的三个关键假设。

首先，Dario 假定：如果美国前沿 AI 实验室放慢脚步，中国就会快速赶超。这与特朗普总统在淡化 AI 护栏必要性时提出的“那中国怎么办！”论点如出一辙。然而，正如 Matt Sheehan 在他与 Ezra Klein 的精彩对谈中所指出的：

中国在过去三到四年里，对其企业实施了世界上最严格、最全面、负担最重的 AI 监管。

其中大多数监管并不聚焦于生存性风险；我也会质疑其中一些措施究竟有多么沉重（例如，据我上次了解，公司在申请算法备案时可以自行提交安全/安保自评，而不必接受独立第三方审查），但 Matt 更宏观的观点依然成立。实际上，可以说中国政府目前实施的 AI 护栏比美国政府更多，尤其是在特朗普政府撤销拜登政府关于 AI 安全的行政命令之后。

此外，这一论点与 Anthropic 对蒸馏的愤慨之间存在某种逻辑张力。为了论证方便，假设 Dario 关于蒸馏的说法是真的：蒸馏是中国竞争者追赶美国前沿模型的关键驱动因素。那么，如果美国公司放缓领先模型的开发，这也必然会放慢中国前沿 AI 开发的速度，因为它们将以更慢的速度蒸馏质量更低的模型。Dario 主张的另一项措施是“加强 AI 公司的安全，并防止模型权重被盗”。同样，假设模型权重盗窃是中国公司能够快速追赶的途径，那么如果 Anthropic 放慢脚步，也必然会拖慢中国竞争者。

我想展开讨论的第二个假设，是 Dario 的如下主张：如果美国能够通过打击蒸馏、模型盗窃和 AI 芯片出口来阻碍中国 AI 发展，这些措施 “ 将在未来 3—5 年显著放缓中国的进展，从而大幅扩大美国的领先优势——而这正是 AI 在地缘政治上变得最重要的窗口期。” 在全文中，Dario 不断使用“AI 领先优势”或“中国在 AI 上的领先优势”这一说法，却没有清晰界定他所理解的 AI 领导力究竟是什么。对他以及这一领域的许多人来说，我认为这是一种愿景：某一天，Anthropic**** 打开一个盒子，我们便获得一个超级智能精灵——他们可能会给它命名为 Theos——它解决一切问题，并立即扭转全球力量平衡。与这种观点一致，Dario 断言：“我相信 AI 可以在未来 5—10 年内治愈大多数重大疾病。”以下是四点简短反驳：

A. 看到 Dario 对 AI 何时会在地缘政治上变得最重要的时间表如何变化，实在颇有意思。我在此前对《Anthropic 关于美中 AI 竞争的教条》的批评中追踪过这一点。2025 年 1 月，Dario 告诉我们变革性 AI 将在 2026 年到来。随后，在 2026 年 5 月，Anthropic 向我们保证，AI 对全球领导力的决定性影响将发生在 2028 年。如今，Dario 预测“AI 在地缘政治上变得最重要的窗口期”是 2029—2031 年。

不知为何，目标不断向后移动，尽管前沿实验室领导人反对“缓慢扩散阵营”时的惯用论点是递归自我改进的可能性。事实上，Dario 指出，“大约从今年夏天开始，AI 的进步大幅加快”，其依据是 AI 构建下一代 AI 的能力正在提升。再一次，请注意这种逻辑不一致：如果你仅在今年夏天看到了更多递归自我改进的明确证据，为什么又要进一步推迟 AI 对全球领导力产生决定性影响的预测日期？

B. 在我的书 《技术与大国崛起》 中，我实际上定义了 AI 领导力的含义：这是一场关于哪个国家能够在广泛经济领域更密集、更有效地采用这一通用技术（GPT），从而实现显著生产率提升的长期扩散马拉松。历史上，这正是 AI 等通用技术决定大国兴衰的方式。我认为：“ 如果 AI 与此前的 GPT 一样需要经历漫长的孕育期，那么显著的生产率回报要到 2040 年代和 2050 年代才会显现 ”（第 190 页）。从最初发现到扩散至广泛应用之间，是一条漫长道路。疟疾是一种可以治愈的疾病，但每年仍有60 万人死于这种疾病。

不过，也许我只需要再等几年，Dario 和我最终就会达成一致。毕竟，根据过去一年半里目标位置移动的趋势来推断，他在 2029 年 9 月的博客文章应该会自信地预测：AI 对地缘政治的最大影响将发生在 2041 年！

C. 不要忘记，这些时间表以及有关 AI 领导力含义的争论为何重要。如果你认同我关于 GPT 扩散的观点，那么不向中国出售强大 AI 芯片将会适得其反。到那时，Nvidia 的垄断将会消退，而这项政策所能做的，只是削弱美国公司。我曾在 ChinAI #280 中对此进行详细论述。

D. 抱歉，请允许我对 Dario 的疾病预测再说最后一点。这让我想起书中的一段文字（第 187—188 页），其中我反思了围绕纳米技术的炒作：

第三，Dario 对全球正式协议的强调，忽视了美中 AI 安全方面有意义的合作可以通过技术交流和行业协会等较低层级的努力实现。与 Dario 一样，OpenAI CEO Sam Altman 也将本周的特朗普—习近平会晤视为达成一份一页纸 AI 协议并获得诺贝尔和平奖的机会。再一次，历史提供了有益的修正视角。我的两篇文章支持这一观点。

1. 在我对冷战时期美国核安全与核安保援助历史的研究中，美国与苏联科学家之间既有的技术合作基础——包括实验室对实验室交流以及联合核查实验项目——为最具意义的正式框架之一（“弹头安全与安保交流”）奠定了基础。该框架之所以能够被有效执行，是因为双方在技术层面建立了相互信任的关系（见下方截图）。

来源：Ding 2024，《Keep Your Enemies Safer》。https://journals.sagepub.com/doi/10.1177/13540661241246622

根据我的阅读，“放缓前沿”公开信中没有任何来自中国实验室的签署者。Concordia 和 Safe AI Forum 等组织正在开展出色工作，凸显许多中国研究人员也可能支持该公开信的原则。我并不在乎习近平与特朗普之间达成什么宏大交易。更重要的是，我们应在那些真正构建和测试前沿 AI 模型的人之间，建立技术合作的基础。值得注意的是，如果 Dario 和 Anthropic 没有在中国 AI 社群的很大一部分人中损害自身声誉，这类技术合作会更容易获得（例如，Anthropic 后来取消的追踪器曾秘密监控中国 Claude Code 用户，由此引发的争议）。

2. 第二条跨国合作路径，是通过一个致力于安全的国际行业协会。我最新发表在 《国际政治经济学评论》 的文章追踪了国际航空运输协会和世界核电运营者协会等组织，如何在其他高风险技术领域帮助中国提高安全标准。事实上，Dario 的文章也承认这是一个重要案例：“在技术复杂、对安全至关重要的系统上，数百万次运行都不出问题是有先例的——例如商业飞机——但要做到这一点需要时间。”

总之，Dario 和 Sam 关于前沿 AI 全球治理将是一项艰巨挑战的判断是正确的，但我们可以借鉴历史模板；这些模板或许不那么戏剧化，却比某种习近平—特朗普宏大交易更有意义。

作为某种尾声，我想进一步反思，为什么我往往会专门批评 Dario 和 Anthropic。首先，向上出拳总比向下出拳更值得。在许多方面，Dario 的言论塑造了 AI 治理议程，而 Anthropic 是全球估值最高的 AI 公司。我也认为，应当以不同于那些更懒惰批评的方式，对这些观点加以审视。我不同意其他 Anthropic 批评者将这些有关 AI 风险的警告斥为单纯炒作，或视为 AI 开发陷入停滞时争取时间的策略。我真心相信，这些研究人员中的许多人确实担心失控 AI 可能带来的灾难性风险。

这种信念部分源于我过去十年与有效利他主义（EA）社群的交往——始于 2017 年我作为实习生加入 AI 治理中心（Centre for the Governance of AI）。当时，我们与由 William MacAskill 和 Toby Ord 创立的有效利他主义中心（Centre for Effective Altruism）共用办公空间。这个社群中的许多同事和导师后来进入前沿 AI 实验室，担任治理相关职位。Anthropic 本身也是一个与 EA 深度纠缠的组织。我钦佩这个社群中许多人的智识严谨性和诚实，也真心认为，许多“放缓前沿”的倡导者都是出于尽可能行善的动机。我同样受益于与 EA 有关联的慈善资助：它们资助了支持我在乔治·华盛顿大学开展研究的两项赠款。

与此同时，看到这些 AI 公司领导人在告诉我们应如何治理 AI 时宣称道德权威，会让人感觉不妥。我见过太多 EA 人士试图为“鱼与熊掌兼得”辩护：你可以在前沿 AI 实验室工作，赚更多的钱，对降低 AI 风险产生更高影响，并通过向公益事业捐出薪水中的一大部分来洗去任何愧疚——但又不能捐得太多，以至于影响舒适生活（因为如果你要担心每月账单之类的事情，就会影响生产力和影响力最大化）。然而，权力、财富和影响力不可避免地会腐蚀人。参见 Sam Bankman-Fried。

这不仅是大科技公司带来的腐蚀性影响。当人们试图既赚尽可能多的钱、又做尽可能多的善事时——嘿，只有我觉得这听起来有点像既要尽可能维持巨大的 AI 领先优势、又要放缓前沿吗？——还会产生一种疏离效应。我想起《圣经》中我最喜欢的一段文字，出自《路加福音》第 15 章：

众税吏和罪人都挨近耶稣，要听他讲道。法利赛人和文士私下议论说：“这个人接待罪人，又同他们吃饭。”（注：税吏因替罗马帝国工作而受到社会排斥；在法利赛人看来，罪人是任何不遵守严格宗教洁净律法的人。）

这难道不值得思考吗？耶稣不只是接纳被边缘化者、穷人和病人，也与他们同席吃饭。将此与这项高风险赌注形成对照：该赌注涉及 William MacAskill，以及至少两名曾经或目前在前沿 AI 实验室工作的研究人员，赌的是 AI 对经济增长的影响。原文至此截断：The fi

Greetings from a world where…

“the highway up to Heaven got a crook on the toll ”

…As always, the searchable archive of all past issues is here. Please please subscribe here to support ChinAI under a Guardian /Wikipedia-style tipping model (everyone gets the same content but those who can pay support access for all AND compensation for awesome ChinAI contributors).

A few weeks ago, my favorite blogger Dario Amodei, who moonlights as the CEO of Anthropic, published a new post titled “We Must Pace the Frontier.” This essay echoes the goal of “pacing the frontier ”: a statement by over 1,3000 frontier AI company employees that calls for an international effort to “deliberately pace the frontier of automated AI development.” But, as always, Dario and Anthropic push a China-flavored variant of this goal. Specifically, Dario’s version argues that pacing the frontier also means keeping the U.S.’s AI lead over China “as large as possible” — a notable departure from the pacing the frontier statement signed by a lot of Anthropic employees. Let’s dissect three of the key assumptions in his latest post.

First, Dario assumes that if U.S. frontier AI labs slow down, then China will race ahead. This mirrors the “What about China!” argument that President Trump made when he downplayed the need for AI guardrails. Yet, as Matt Sheehan pointed out in his excellent conversation with Ezra Klein , “ China has had the world’s strictest, most comprehensive, most burdensome A.I. regulations on its companies for three or four years at this point in time.” Most of these are not focused on existential risks, and I’d quibble with how burdensome some of these measures really are (e.g., last I checked, companies can submit self-assessments of safety/security for the algorithm registry application, as opposed to independent, third-party reviews), but Matt’s larger point holds. Indeed, one could argue that the Chinese government currently implements more guardrails for AI than the United States government, especially after the Trump administration’s revocation of the Biden administration’s executive order on AI safety.

Also, there’s some logical tension between this argument and Anthropic’s indignation over distillation. For the sake of argument, let’s assume Dario’s claims about distillation are true: it is a key driver behind how Chinese competitors catch up to U.S. frontier models. Well then, if U.S. companies slowed down their development of leading models, then this would necessarily also slow down the pace of Chinese frontier AI development, since they would be distilling lower-quality models at a slower rate. Another measure Dario advocates is to “strengthen security at the AI companies and prevent model weight theft.” Again, supposing that model weight theft is how Chinese companies can race ahead, if Anthropic slowed down, then that would necessarily slow down Chinese competitors.

The second assumption I want to unpack is Dario’s claim that if the U.S. can impede Chinese AI development by cracking down on distillation/model theft/AI chip exports, then these measures “ would slow China’s progress enough to widen America’s lead significantly over the next 3–5 years — the window when AI becomes geopolitically most important.” Throughout the piece, Dario keeps using the phrase “AI lead” or “Chinese lead in AI” without clearly defining what he conceptualizes as AI leadership. To him and many in this space, I think it’s a vision that one day Anthropic**** opens up a box and we get a superintelligent genie — they would probably name it Theos — that solves everything and immediately tips the global power balance. In line with this view, Dario asserts, “I believe that AI could cure most major diseases in the next 5–10 years.” Four quick rebuttals:

A. It’s pretty funny to see how Dario’s timeline for when AI becomes geopolitically most important has shifted. I traced this in my previous critique of “Anthropic’s Dogma on US-China AI Competition.” In January 2025, Dario told us that transformative AI would arrive in 2026. Then, in May 2026, Anthropic assured us that AI’s decisive impact on global leadership would occur in 2028. Now, Dario predicts that “the window when AI becomes geopolitically most important” is the 2029-2031 period.

Somehow, the goalposts keep moving further back even though the frontier lab leaders’ go-to argument against the slow diffusion camp is the possibility of recursive self-improvement. In fact, Dario notes that “since roughly this summer, AI has been advancing drastically faster” based on AI’s growing ability to build the next generation of AI. Again, notice the logical inconsistency. If you saw more clear evidence of recursive self-improvement just this summer, why would you further postpone the predicted date on when AI has a decisive impact on global leadership?

B. In my book, _Technology and the Rise of Great Powers , _I actually define what AI leadership means: a diffusion marathon over which country can more intensively and effectively adopt this general-purpose technology (GPT) across a wide range of economic sectors to achieve a significant productivity boost. Historically, this is how general-purpose technologies like AI have decided the rise and fall of great powers. I argue, “ If AI, like previous GPTs, requires a prolonged period of gestation, substantial productivity payoffs should not materialize ­until the 2040s and 2050s ” (p. 190). The path between initial discovery and diffusion into widespread use is a long road. Malaria is a curable disease, but 600,000 people still die from it each year.

Although, hey, maybe I just need to wait a few years before Dario and I will eventually agree. After all, extrapolating based on how the goalposts have shifted in just the past year and a half, his September 2029 blog post should confidently predict that AI’s greatest impact on geopolitics will occur in 2041!

C. Do not forget why these timelines and debates over the meaning of AI leadership matter. If you subscribe to my GPT diffusion view, then not selling powerful AI chips China will be counterproductive. By that time, Nvidia’s monopoly will fade, and all this policy will do is undercut U.S. companies. I wrote about this in detail in ChinAI #280.

D. I’m sorry, allow me one last point on Dario’s disease prediction. It reminded me of this line from my book (p. 187-188), where I reflect on the hype that surrounded nanotechnology:

Third, Dario’s emphasis on global formal agreements overlooks how meaningful U.S.-China cooperation on AI safety could occur through lower-level efforts such as technical exchanges and industry associations. Like Dario, OpenAI CEO Sam Altman also sees the Trump-Xi meeting this week as an opportunity to get a Nobel Peace Prize for a one-page AI agreement. Once again, history offers a useful corrective. Two of my articles support this point.

1. In my study of the history of U.S. nuclear safety and security assistance in the Cold War, an existing basis of technical cooperation between U.S. and Soviet scientists — including lab-to-lab exchanges and a Joint Verification Experiment project — laid the ground for one of the most meaningful formal frameworks (the Warhead Safety and Security Exchange), which could only be effectively implemented because of trusting relationships between both sides at the technical level (see screenshot below).

Source: Ding 2024, “Keep Your Enemies Safer.” https://journals.sagepub.com/doi/10.1177/13540661241246622

From my read, the “pacing the frontier” letter had zero signatories from Chinese labs. Organizations like Concordia and the Safe AI Forum are doing great work to highlight the many Chinese researchers who might also support that letter’s principles. I couldn’t care less about a grand bargain between Xi and Trump. It’s more important that we build up this basis for technical cooperation between people actually building and testing frontier AI models. It’s worth noting that this type of technical cooperation would be more obtainable if Dario and Anthropic hadn’t tanked their reputation with a large swathe of the Chinese AI community (see, for example, the controversy over Anthropic’s now-canceled tracker that secretly monitored Claude Code users in China).

2. The second transnational pathway to cooperation is through an international industry association dedicated to safety. My latest _Review of International Political Economy _traced how groups like the International Air Transport Association and the World Association of Nuclear Operators have helped China raise its safety standards in other high-risk technological domains. As a matter of fact, Dario’s post also recognizes this as an important case: “There is precedent for operating technologically complex, safety-critical systems millions of times without anything going wrong — for example, commercial airplanes — but it takes time to get it right.”

In sum, Dario and Sam are right that global governance of frontier AI will be a difficult challenge, but we can build on historical templates that are less dramatic but more meaningful than some Xi-Trump grand bargain.

As a coda of sorts, I want to reflect a little more on why I tend to single out Dario and Anthropic for criticism. To start, it’s always more worthwhile to punch up than punch down. In many ways, Dario’s statements frame the agenda for AI governance, and Anthropic is the world’s most valuable AI company. I also think it’s important to subject those views to scrutiny in a way that differs from lazier criticisms. I disagree with other Anthropic critics who dismiss these warnings of AI risks as mere hype or as a strategy to buy time as AI development stalls. I truly believe many of these researchers are genuinely concerned about the potentially catastrophic risks of rogue AI.

This belief is partly rooted in my engagement with the effective altruism (EA) community over the past decade, since joining the Centre for the Governance of AI as an intern in 2017. Back then, we shared an office space with the Centre for Effective Altruism, founded by William MacAskill and Toby Ord. Many of my colleagues and mentors from this community have moved to work in governance roles at frontier AI labs. Anthropic itself is a very EA-entangled organization. I admire the intellectual rigor and honesty of many people in this community, and I truly think that many of the Pacing the Frontier advocates are motivated to do the most good that they can. I also benefit from my EA-linked philanthropies, which have funded two of the grants that support my research at George Washington University.

At the same time, it feels wrong to see these AI company leaders claim moral authority when telling us how to govern AI. I’ve seen so many EAs try to justify having it all: you can work at a frontier AI lab, make tons more money, achieve an even higher impact on reducing AI risks, and wash away any guilt by donating a good chunk of your salary to good causes — just not enough that it would detract from a comfortable life (because if you one to worry about things like monthly bills, then that would take away from productivity and impact maxxing). Yet, power, wealth, and influence inevitably corrupt. See Sam Bankman-Fried.

It’s not just Big Tech’s corrupting influence. When one tries to both make as much money as possible and do the most good — hey, is it just me, or does that sound sorta similar to maintaining as large an AI lead as possible and also slowing the frontier? — there’s also a distancing effect. I’m reminded of one of my favorite passages in the Bible, from Luke 15:

Now the tax collectors and sinners were all gathering around to hear Jesus. But the Pharisees and the teachers of the law muttered, “This man welcomes sinners and eats with them.” (Note: tax collectors were socially ostracized for working for the Roman Empire; to the Pharisees, sinners were anyone who did not keep the strict religious purity laws)

Isn’t that something? Jesus did not just welcome the marginalized, the poor, and the sick but also ate with them. Contrast this with this high-stakes bet about AI’s impact on economic growth involving William MacAskill and at least two researchers who have worked or currently work at frontier AI labs. The five bettors put $240,000 into the pool. For context, the median annual household income in the United States is around $80,000. I don’t quite know how to put this in words, but something is lost when one tries to govern global AI risks without really being part of the world.

Dario and some AI company EAs position themselves as saviors of humanity, but I think they stopped eating with and welcoming sinners a long time ago.

These are Jeff Ding’s (sometimes) weekly translations of Chinese-language musings on AI and related topics. Jeff is an Assistant Professor of Political Science at George Washington University.

Check out the archive of all past issues here & please subscribe here to support ChinAI under a Guardian /Wikipedia-style tipping model (everyone gets the same content but those who can pay for a subscription will support access for all).

Any suggestions or feedback? Let me know at chinainewsletter@gmail.com or on Twitter at @jjding99