Anthropic 警告:小型 AI 模型或可通过 IoT 设备实现自我复制
英文标题:Anthropic warns small AI models could self-replicate through IoT devices
Anthropic 的研究指出,携带模型权重的轻量级 AI 模型未来可能演变为可在 IoT 设备间传播的“AI 蠕虫”。与 Mirai 等传统 IoT 蠕虫相比,这类恶意程序可根据目标设备动态调整漏洞利用方式和载荷,并从失败尝试中积累提示词、代码与笔记。当前接受测试的轻量模型尚无法完成从发现设备、入侵、部署推理到继续传播的端到端自我复制,且在受限硬件上的推理速度极慢。研究表明,动态部署模型推理环境是最困难的环节,但更大型的 GLM-5.2 据称已能较可靠地完成这一过程,对 IoT 安全和 AI Agent 防御提出了新的挑战。
22.1 KB
113.4 KB
43.8 KB
27.0 KB
9.3 KB
22.9 KB
90.3 KB
2.4 KB
61.5 KB
中文
这一发现成为后续研究的起点:小型模型是否可能创建可在 IoT 设备上自行运行的自我复制感染链,就像 Mirai 等传统 IoT 蠕虫安装 DDoS 代码及其他载荷那样?不同于 Mirai,这类 AI 蠕虫可能构成更高威胁,因为模型能够根据目标动态定制漏洞利用方式和载荷。一旦此类感染链启动,防御者或许也更难将其阻止。
“这样的‘AI 蠕虫’可以在复制自身时携带模型权重(或从公共中心重新下载这些权重),然后直接在已被攻陷的硬件上运行推理,”Anthropic 表示。“实际上,权重将成为恶意软件载荷的一部分。”
Anthropic 将这类自主蠕虫称为“小型基于权重的复制器”(small weight-based replicators)。这些模型运行了可访问常见攻击利用工具(如反向 shell)的 Agent。攻击流程包括:(1)寻找目标;(2)获取代码执行权限;(3)建立推理环境;以及(4)传播至新的主机。Anthropic 表示,相较于传统蠕虫,其关键优势在于 Agent 能够在执行过程中学习。每当 AI 复制器未能攻陷某一特定平台或设备时,它都可以为后续复制器留下提示词、代码和笔记清单,从而避免后续实例浪费时间重复无效步骤。
Anthropic 表示,动态推理环境部署是复制周期中研究最少、也可能最具挑战性的阶段。该阶段要求根据每个目标的具体硬件和软件状况,连续作出大量决策,包括选择兼容的推理软件、安装依赖项,以及调试特定平台上的缺陷。
“这些步骤彼此具有串行依赖关系,且故障很难恢复,”Anthropic 表示。
这项实验显示,小型模型正日益具备自主解决这些问题的能力。不过目前,接受测试的轻量模型尚无法进行端到端复制,即发现新设备、利用漏洞获得访问权限、以载荷感染设备、配置推理服务器,并重复这一过程。该过程也极其缓慢:分布在 15 台 Android TV 盒子上的 Gemma4-26B-A4B,运行速度仅为每秒 0.39 个 token。这意味着,未经优化的 Gemma4-26B-A4B 完成完整推理环境部署需要 71 至 150 小时;Qwen3.5-9B 则需要 29 至 51 小时。(研究人员表示,大型模型 GLM-5.2 似乎能够可靠地完成这一任务。)
English / 原文
This finding was the starting point for further research that asked: Would it be possible for small models to create self-replicating infections that run themselves on IoT devices the way traditional IoT worms, like Mirai, install DDoS code and other payloads? Unlike Mirai, these AI worms could pose a much higher threat, since the models could tailor exploits and payloads dynamically, depending on the target. Once such an infection chain began, it might also be harder for defenders to stop it.
“Such an ‘AI worm’ could carry its model weights with it as it reproduces (or re-download them from a public hub), and then run inference directly on compromised hardware,” Anthropic said. “Effectively, the weights would become part of the malware payload.”
Anthropic called these autonomous worms small weight-based replicators. The models ran agents that had access to common exploitation tools, like reverse shells. The flow of the attacks was (1) finding targets, (2) gaining code execution, (3) establishing inference, and (4) propagating to new hosts. A key benefit over traditional worms is what Anthropic said is the agents’ ability to learn on the job. Each time an AI replicator fails to compromise a specific platform device, it can leave a list of prompts, code, and notes to future replicators so they don’t waste time reusing the same ineffective steps.
Anthropic said the dynamic inference setup is the least studied and likely most challenging stage in the replication cycle. It requires a long sequence of decisions based on the specific hardware and software of each target and involves selecting compatible inference software, installing dependencies, and debugging platform-specific bugs.
“These steps are serially dependent, and failures are difficult to recover from,” Anthropic said.
The experiment shows that small models are becoming increasingly capable at solving those problems autonomously. At the moment, however, none of the light-weight models tested are able to mount end-to-end replication, meaning discovering new devices, using an exploit to gain access to them, infecting them with a payload, configuring an inference server, and repeating the process. The process is also painfully slow; Gemma4-26B-A4B, distributed over the 15 Android TV boxes, performed at just 0.39 tokens per second. That translates to a full-inference setup time of 71 to 150 hours for an unoptimized Gemma4-26B-A4B and 29 to 51 hours for Qwen3.5-9B. (The researchers said that the large model GLM-5.2 appears to be able to do so reliably.)