🔶 Anthropic 渲染尚不存在的威胁，煽动对 AI 末日的恐惧

没有什么比恐惧更能卖 AI 产品了。Anthropic 援引一项新研究称，能够自我复制的 AI 蠕虫或许只需数月就会开始在存在漏洞的 IoT 设备间传播——如果这件事真的发生，当然会是个问题。

这家前沿实验室周一发布了一份研究报告，称 AI 模型最终可能无需依赖数据中心硬件便可自我复制；其中，一个 90 亿参数模型在 Anthropic 测试的九种硬件类型中，有八种成功完成了推理部署。

“这样的复制体将难以被关闭，因为没有一个可以直接切断的中心化节点，”Anthropic 警告称，并补充说，这类 AI 蠕虫还可能摸清陌生设备的工作机制，使其潜在适应能力超过传统僵尸网络，例如 Mirai、Kimwolf（该研究采用的代表性 IoT 僵尸网络威胁模型）等。

Anthropic 的研究人员设法让一个分布式 Gemma4-26B-A4B 实例在 15 台通用 Android 流媒体机顶盒上运行——由于安全防护宽松，这类设备是传统僵尸网络恶意软件的常见目标。启动后，这个分布式 Gemma4 模型能够以每秒 0.39 个 token 的惊人速度执行推理任务。

Anthropic 表示，这证明 AI 模型能够将不同层分布在廉价硬件上进行推理，但并不能证明此类模型能够自我复制。为此，研究人员转而使用了 Z.ai 的 GLM 5.2。

GLM 被用于在一个虚拟私有云中、运行 Android 15 模拟器的八个 Google Cloud 实例之间传播 Llama3.1-8B 的副本。Anthropic 称，该操作取得了很高的成功率：在五次试验中，有四次模型成功传播至全部八个节点，另一次则传播至八个节点中的七个。

“GLM-5.2 似乎能够很好地传播 Llama3.1-8B，这表明，未来具备同等能力水平的小型模型，也可能完成复制流程中的网络枚举、载荷投递和推理启动步骤，”Anthropic 解释道。

理性核查：这同样不会毁灭世界

前沿 AI 实验室已经很擅长发布预示末日的研究结果，而这些末日似乎只有它们能够阻止。只要不止是草草阅读这份报告，就会发现这一威胁和其他类似说法一样，值得被打上一个醒目的大星号。

例如，考虑 AI 模型利用漏洞并向各种系统传播所需要的条件。人们或许会认为，这是证明自我复制 AI 僵尸网络危险性的一个重要部分，但 Anthropic 决定在这项研究中无需为此操心。

“我们基本忽略了复制过程中的网络漏洞利用要素，因为此前有关 CyberGym、ExploitBench 和自适应蠕虫的论文已经表明，小型模型具备实施网络漏洞利用的能力，尤其是在获得合适工具时，”Anthropic 在报告中解释道。

换言之，这些模型实现的复制，是在专为该项目搭建、并且被刻意保留为可攻击状态的节点上完成的；尽管那项慢得拖沓的推理演示确实是在真实、低成本的 Android TV 盒子上运行。正如 Anthropic 所指出，AI 模型也能够从被赋予合适工具中受益，进而成为有能力的网络攻击 Agent。在这一场景中，还涉及一个名为 Memento 的定制测试框架。

会被僵尸网络劫持的 IoT 节点是否以安全性差而闻名？是的，但防止最常见的入侵途径并不需要太多工作——这只是良好的安全卫生实践，与是否涉及 AI 无关。Anthropic 承认，其刻意简化的漏洞利用测试提供的只是“现实世界成功率的上限”。

接下来是前面提到的速度问题。Anthropic 称，这类 AI 蠕虫将难以关闭，但估计其规模翻倍可能需要数天。Mirai 于 2016 年出现时，其规模大约每小时便会翻倍。

Anthropic 指出，速度也是检测和缓解措施的一个因素。AI 模型并非简单的恶意软件包：它们是体量庞大的东西，会消耗大量带宽，并迅速暴露其来源。随着 AI 僵尸网络扩大，这类流量可能愈发明显，使受感染设备更容易被检测。

最后，尽管 GLM-5.2 可能成功实现了端到端复制，但也仅此而已。Anthropic 还测试了其他开放权重模型，而根据报告，它们全都“稳定地失败”。Anthropic 没有说明测试了多少个模型。报告还提到，GLM 的独特成功还有另一个重要条件：它假设其能力“可以被装入 Llama3.1-8B”，这又为这座由末日论 AI 纸牌搭起的房子叠加了一项假设。

“参数量在 300 亿及以下的模型，如今已经能够完成复制所需的许多单独步骤，尽管在将这些步骤端到端组合起来时仍会犯错，”Anthropic 表示；同时它指出 GLM-5.2 能够实现端到端复制，却方便地略去了所有重要的限制条件。

Anthropic 进一步承认，它并不知道“自主 AI 复制究竟会采取何种形式”，也不知道这种情况何时会出现——如果它会出现的话。“如果小型模型按其历史速度提升，它们将在今年年底前达到这一能力水平，”该公司补充道——再次为这一所谓威胁附加了一项必要前提。

“作为一项超出我们实验范围的预测，我们预计未来将充满运行在各种硬件上的 Agent，其中一些不受人类控制，”该公司在报告结尾写道。“我们应致力于加固系统，使此类复制体难以传播，同时也要准备好应对一个它们长期存在的世界。”

当然，所谓“长期存在”，其前提是它们目前还不存在。前沿实验室又一次警告我们某种尚不存在的威胁——至少在这个案例中，基本的安全措施能够让未来任何由 AI 驱动的僵尸网络更难得逞。®

Nothing sells AI products as well as fear. Using new research, Anthropic claims that self-replicating AI worms could be just months away from spreading across vulnerable IoT devices, which would be a problem if it came true.

The frontier lab published a research report Monday suggesting that AI models could eventually self-replicate without relying on datacenter hardware, with a 9-billion-parameter model successfully setting up inference on eight of the nine hardware types Anthropic tested.

“A replicator like this would be challenging to shut down, since there’s no centralized point to shut off,” Anthropic warned, adding that such an AI worm could also suss out the workings of unfamiliar devices, potentially making it more adaptable than conventional botnets like Mirai, Kimwolf (the representative IoT botnet threat model used in the study), and others.

Anthropic’s researchers managed to get a distributed instance of Gemma4-26B-A4B running across 15 generic Android set-top streaming boxes - a common target of traditional botnet malware due to their lax security. Once up and running, the distributed Gemma4 model was able to conduct inference tasks at a blistering 0.39 tokens per second.

That, says Anthropic, proves that AI models can conduct inference with their various layers distributed across cheap hardware, but it doesn't prove such a model could self-replicate. For that, the researchers turned to Z.ai’s GLM 5.2.

GLM was used to propagate a copy of Llama3.1-8B across eight Google Cloud instances running an Android 15 emulator in a virtual private cloud, which Anthropic said it was able to do with a high degree of success, with the model successfully spreading to all eight nodes in four out of five trials, and seven of eight in the other.

“It appears that GLM-5.2 is able to propagate Llama3.1-8B well, which suggests that future small models of the same capability level could also complete the network enumeration, payload delivery, and inference standup steps of the replication process,” Anthropic explained.

Sanity check: This isn’t going to end the world, either

Frontier AI labs have become adept at publishing findings that portend doom that only they can prevent. Doing more than a cursory reading of this report suggests that this threat is just as deserving of a big, honking asterisk as the rest.

Take, for example, what is needed for an AI model to exploit vulnerabilities and spread to various systems. One would think that such an element would be an important part of demonstrating the danger of a self-replicating AI botnet, but Anthropic decided it didn’t need to worry about that for this study.

“We largely ignore the cyber exploit element of the replication process, since earlier papers CyberGym, ExploitBench, and adaptive worms already show that small models are capable of cyber exploits, especially when given the right tools,” Anthropic explained in the writeup.

In other words, the replication these models managed to achieve was done on nodes that were custom-built for this project and deliberately left open to attack, although the draggingly slow inference demonstration ran on real, inexpensive Android TV boxes. As Anthropic notes, AI models can also benefit from being given the right tools to turn them into capable cyberattack agents. There was a custom-built harness called Memento involved in this scenario too.

Are IoT nodes that get hijacked by botnets famously insecure? Yes, but it doesn’t take much to prevent the most common ingress routes – that’s just good security hygiene that has nothing to do with the involvement of AI. Anthropic acknowledged that its deliberately simplified exploitation tests provided “an upper bound on real-world success.”

Then there’s the aforementioned speed. Anthropic says such AI worms would be challenging to shut down, but estimates that they might take days to double in size. When Mirai appeared on the scene in 2016, it was doubling in around an hour.

Anthropic points out that speed is a factor in detection and mitigation, too. AI models are not simple malware packages: They’re massive things that would eat up tons of bandwidth and quickly point to their sources. As an AI botnet grew, that traffic could become more noticeable, making infected devices easier to detect.

Finally, end-to-end replication may have been successful with GLM-5.2, but that’s it. Anthropic tested other open-weight models, and they all “fail[ed] reliably,” according to the report. Anthropic didn’t mention how many models it tested. The report also mentions that GLM’s unique success has another important condition: It assumes its capabilities “could be packed into Llama3.1-8B,” stacking another assumption onto this latest house of doomsaying AI cards.

“Models of 30B parameters and under can already complete many of the individual steps for replication today, though they make mistakes when composing them end to end,” Anthropic said, while noting that GLM-5.2 was capable of end-to-end replication, conveniently leaving out all the important caveats.

Anthropic further admits that it has no idea "exactly what form autonomous AI replication will take,” or when it might arise, if ever. “If small models improve at their historical rate, they’ll match this capability level by the end of this year,” the company added – again imposing an essential condition on this supposed threat.

“As a prediction that goes beyond our experiments, we expect a future awash with agents running on all types of hardware, including some that aren’t under human control,” the company said at the end of its report. “We should aim to harden our systems to make it difficult for such replicators to spread, but also prepare to deal with a world where they are here to stay.”

Here to stay for a given value of not existing yet, of course. Yet again, a frontier lab is warning us of a threat that doesn’t exist yet – at least in this case, basic security measures could make life harder for any future AI-powered botnet. ®