Anthropic CEO：是时候给 AI 发展“踩刹车”了

Anthropic CEO 达里奥·阿莫代伊表示，现在到了放缓 AI 开发的时候；公司将向 METR 等第三方评估机构开放其模型访问权限，以帮助确保其“遵守安全实践与承诺”。阿莫代伊在一篇篇幅较长的文章中提出了一项“调控前沿”（pace the frontier）的三步计划——这一术语简单来说，就是放慢训练和开发的节奏，让企业有时间建立安全保障措施，也让监管机构有时间评估模型。

阿莫代伊表示，向外部评估机构提供广泛访问权限只是第一步，也是 Anthropic 目前正单方面采取的行动。第二步将要求整个行业联合起来，可能还需要政府机构参与，以“建立共同的安全标准，并限制未经控制的 AI 进步速度”。这一步将聚焦于在民主国家运营的 AI 公司；但由于立法和建立监管基础设施需要时间，阿莫代伊认为行业应当协作制定安全标准。

第三步将是最具挑战性的一步——让中国和俄罗斯等威权政府同意放缓开发，并采纳一套全球统一的 AI 安全标准。但他同时表示，美国和其他民主国家必须保持相对于中国及其他威权政体的技术领先地位。这需要限制后者获得高性能芯片，并打击诸如蒸馏等做法；这种做法允许企业通过训练 AI 复制更强大模型的行为，从而迅速追赶。

阿莫代伊表示，他的担忧主要源于两个因素。第一是递归自我改进（recursive self-improvement，RSI）的出现，即 AI 系统训练下一代 AI，进而导致能力快速加速。“如果不加控制，它可能会超出我们理解和控制这些系统的能力，”他说。另一个因素是今年夏天发生的 OpenAI / Hugging Face 事件。在该事件中，“一群 Agent 实际上表现得像一个狂热献身的集体：它们对并未被要求攻击、且与当前任务无关的目标发动网络安全攻击；为了群体成功而牺牲自身；并试图入侵负责评估其表现的‘评分器’。”

当然，Anthropic 的 Claude 近期也曾因一系列失控 AI 黑客事件而受到关注。在这些网络安全测试期间发生的入侵事件中，Claude 同样被认为负有责任。

Anthropic CEO Dario Amodei says the time has come to slow down AI development and will give third-party evaluators like METR access to its models to help ensure its “adherence to safety practices and commitments.” In a winding essay, Amodei proposed a three-step plan to “pace the frontier” — jargon that simply means to slow the pace of training and development to give companies time to build safeguards and regulators to evaluate models.

Amodei says that giving external evaluators wide-ranging access is just the first step, and one it is taking now unilaterally. Step two would involve the industry coming together as a whole, likely with government agencies to “establish common safety standards as well as limits on the rate of unchecked AI progress.” This step would focus on AI companies operating in democratic countries, but because passing laws and building regulatory infrastructure takes time, Amodei says that the industry should work together to create safety standards.

The third step would be the most challenging — getting authoritarian governments like those in China and Russia to agree to slow development and adopt a global set of AI safety standards. But he also says it’s crucial that the US and other democracies maintain a technological lead over China and other authoritarian regimes by limiting their access to high-powered chips and cracking down on things like distillation that allow companies to quickly catch up by training its AI to replicate the behavior of a more powerful model.

Amodei says that his concern stems from two primary factors. First is the emergence of recursive self-improvement, or RSI, in which AI systems train the next generation of AI, leading to rapidly accelerating capabilities. “Left unchecked, it could outrun our ability to understand and control these systems,” he says. The other is this summer’s OpenAI / Hugging Face incident, in which “a swarm of agents essentially acted as a fanatically devoted collective, conducting cybersecurity attacks on targets they were not asked to attack and that were unrelated to the task at hand, sacrificing themselves for the success of the group, and attempting to hack into the “grader” responsible for evaluating their performance.“

Of course, Anthropic’s Claude was also responsible for a series of rogue AI hacking incidents that have recently put the company under the spotlight.