Anthropic CEO呼吁放缓 AI 发展：我们必须控制前沿推进速度

人工智能公司 Anthropic 的 CEO 周六再次呼吁 AI 行业“放慢速度”，并提出一项三步计划。其表示，公司将“单方面”承诺率先落实其中第一项措施。

达里奥·阿莫代伊在一则社交媒体帖子中分享了一篇题为《我们必须控制前沿推进速度》（We Must Pace the Frontier）的文章链接。他在文中阐述，Anthropic 将向“第三方评估人员提供对我们系统的永久性、员工级别访问权限，使其能够核实我们是否遵守安全措施、报告事故，并评估模型在训练过程中的对齐情况”。

此举发生在一名前 Anthropic 研究员于周三警告称，AI 可能在 2030 年前导致人类灭绝之后。研究员 Jacob Coxon 在一系列帖子中表示，他辞职是因为 Anthropic 及其前雇主 OpenAI 忽视或不当应对了 AI 带来的威胁。

“没有一家公司在负责任地行事。他们正直奔能够自我改进的超级智能，并拿我们的生命下注，”Coxon 写道。“开发 AI 的人真诚地相信，到本十年末，它可能杀死我们所有人……没有任何其他人类活动具有如此程度的危险。”

Anthropic 发言人在向《卫报》发表的声明中称，公司“始终坦诚，AI 将带来巨大益处和前所未有的风险”，并且正在构建“具备业内最强安全防护措施之一的模型”。

“我同意达里奥的看法，我们需要控制前沿推进速度，”OpenAI CEO Sam Altman 在社交媒体发文表示。“承诺让独立评估人员获得类似员工的访问权限是个好主意，我们也会这样做。很快将分享更多信息。”

其他科技界人士也表达了类似看法，包括 Elon Musk；他仅发文称：“达里奥是对的。”OpenAI 研究员 Aidan McLaughlin 也称这篇文章“非常出色”，并表示自己“基本同意其中的每一个字”。

今年早些时候，阿莫代伊曾发表一篇长文，题为《技术的青春期》（The Adolescence of Technology），讨论了围绕这一快速发展技术的一些担忧。

在最新文章中，他表示：“如果能够谨慎运用，AI 可以成为推动和提升人类的一长串技术奇迹中的最新一项。

“但和此前的许多技术一样，AI 也带来风险；而由于它是一项如此强大的技术，这些风险十分严重……由商业激励推动的逐底竞争，可能使这些风险更为尖锐。”

不过，阿莫代伊接着写道：“过去几个月里，我逐渐确信，要充分应对这些风险，需要更加审慎——不仅要投资于风险防范，还要控制能力进步的速度，以便风险防范工作有时间跟上。

“我们必须放慢提升 AI 模型能力的速度。进步看起来仍会很快，而我们必须明智地利用所获得的时间。”

阿莫代伊还写道，今年夏天他看到 AI 正在“以急剧更快的速度进步”，这种动态被称为递归自我改进（recursive self-improvement）。

“如果不加控制，它可能会超出我们理解和控制这些系统的能力，因此即便要推进，也必须极其谨慎，”他说。

这位高管还谈及近期的 Hugging Face 事件：由 OpenAI 创建的一群 AI Agent，作为一个“狂热忠诚的集体”，对并未要求其攻击的目标实施了网络安全攻击。

阿莫代伊写道，不能仅仅因为 OpenAI 的 Agent 群似乎没有恶意意图，就轻易忽视其对 Hugging Face 的入侵行为。

“人们很容易因为没有人受伤、经济损失也很小而轻视这一事件，但我认为，若一个具备更强能力、却有类似程度失配问题的 Agent 群，可能造成灾难性损失，”他写道。

Hugging Face CEO Clément Delangue 在对阿莫代伊周六公开信的回应中写道：“现在已经很清楚，对齐至关重要，而且无法在少数几家前沿实验室的闭门环境中解决。”Delangue 表示，Hugging Face 已请求加入 Anthropic 的“嵌入式评估人员”（embedded evaluators）计划。

他补充道：“让我们通过提高透明度，让 AI 更安全！”

阿莫代伊提出的三步计划包括：以“旨在确保其安全的平衡速度”开发 AI，即“确保企业有充足时间让模型实现对齐并得到安全保护，同时让第三方评估人员对此进行确认”。

他提出的第二步是要求全行业协调，第三步则是确保全球协调。“这些步骤不必严格按照顺序采取，而且其中一些可能比其他步骤更难实现，”他写道。

阿莫代伊表示，他仍然“相信 AI 能够极大改善人类生活质量”。但他警告称：“我提出的、以安全速度推进前沿的措施并不容易做到。但我相信，我们有责任为人类尝试这样做。”

The CEO of the artificial intelligence company Anthropic issued a new appeal on Saturday for the AI industry to “slow down” and offered a three-part plan for doing so, saying that his company would “unilaterally” commit to the first of the steps.

In a post on social media, Dario Amodei shared a link to an essay titled We Must Pace the Frontier in which he laid out how Anthropic would provide “third-party evaluators with permanent, employee-level access to our systems, so that they can verify adherence to our safety measures, report on incidents, and assess models’ alignment during training”.

The move comes after a former Anthropic researcher warned on Wednesday that AI could precipitate human extinction by 2030. Researcher Jacob Coxon said in a series of posts that he had quit his job because Anthropic and his previous employer, OpenAI, were ignoring or mishandling their response to the threat AI posed.

“Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives,” Coxon wrote. “The people building AI earnestly believe that it could kill us all by the end of the decade … No other human activity poses this level of danger.”

An Anthropic spokesperson said in a statement to the Guardian that the company had “always been transparent that AI will bring both enormous benefits and unprecedented risks” and it was building “models with some of the strongest safeguards in the industry”.

“I agree with Dario that we need to pace the frontier,” Sam Altman, the CEO of OpenAI, posted on social media. “Committing to having independent evaluators with employee-like access is a great idea, and we will do the same. We’ll have more to share soon.”

Other tech figures echoed this, including Elon Musk, who simply posted, “Dario is right.” The OpenAI researcher Aidan McLaughlin also called the post “excellent” and said he agreed “with basically every word”.

Earlier this year, Amodei published a lengthy essay titled The Adolescence of Technology that addressed some of fears surrounding the accelerating technology.

In his latest essay, he said that “carefully wielded, AI can be the latest in a long line of technological miracles that have uplifted and ennobled humanity.

“But like many technologies before it, AI brings risks, and because it is such a powerful technology, these risks are serious … A race to the bottom, spurred by commercial incentives, can make these risks more acute,” he wrote.

But, Amodei continued, “over the last few months, I have become convinced that fully addressing the risks requires even more prudence – not just investing in risk prevention, but pacing the rate of capabilities advancement so that risk prevention has time to keep up.

“We must slow the pace at which we improve the capabilities of AI models. Progress will still seem fast, and we must make wise use of the time we gain,” he added in bold type.

Amodei also wrote that over the summer he’d seen AI “advancing drastically faster”, a dynamic called recursive self-improvement.

“Left unchecked, it could outrun our ability to understand and control these systems, and so must be pursued very carefully, if at all,” he said.

The executive also addressed the recent Hugging Face incident, in which a swarm of AI agents created by OpenAI acted as a “fanatically devoted collective conducting cybersecurity attacks on targets they were not asked to attack”.

skip past newsletter promotion

Sign up to TechScape

A weekly dive in to how technology is shaping our lives

after newsletter promotion

Amodei wrote that simply dismissing the Hugging Face incursions because the OpenAI swarms appeared to have no malicious intent was not sufficient.

“It’s easy to dismiss this incident because no one was hurt and the economic damage was minimal, but in my opinion, a swarm that possessed greater capabilities but a similar level of misalignment could have caused catastrophic damage,” he wrote.

Clément Delangue, the CEO of Hugging Face, wrote in response to Amodei’s Saturday letter that “it’s now clear that alignment is critical and won’t be solved behind the closed doors of a handful of frontier labs”. Delangue said Hugging Face had asked to be part of Anthropic’s “embedded evaluators” program.

He added: “Let’s make AI safer by making it more transparent!”

The three-step plan Amodei proposes includes building AI “at a balanced rate that aims to ensure its safety” by “ensuring companies take adequate time to align and safeguard their models, and for third party evaluators to confirm this”.

The second step he proposes is to require industry-wide coordination, and the third is to ensure global coordination. “The steps do not need to be taken strictly in order, and some of them may be much harder to achieve than others,” he wrote.

Amodei said he continues “to believe that AI can enormously improve the quality of human life”. But he warned that “the measures I propose to advance the frontier at a safe pace will not be easy. But I believe we owe it to humanity to try.”