英国 AISI / 美国 CAISI 对 Kimi K3 网络安全能力的初步评估
英文标题：🔶 UK AISI / Caisi Preliminary Assessment of Kimi K3's Cyber Capabilities
英国人工智能安全研究所（UK AISI）与美国 AI 标准与创新中心（CAISI）联合评估了 Moonshot AI 的 Kimi K3 网络安全能力。初步结果显示，Kimi K3 在 ExploitBench 上得分为 32%，高于 GLM-5.2 的 24%，但在 41 个测试样本中均未实现任意代码执行（ACE）。在 32 步的 The Last Ones（TLO）模拟企业网络攻击场景中，Kimi K3 平均完成至第 17 步，低于领先美国模型的 28.5 步，但高于 GLM-5.2 的第 11 步。评估表明，Kimi K3 已具备在获得初始网络访问权限后自主攻击小型、弱防御且存在漏洞的企业系统的能力，但其稳定性和最高级漏洞利用能力仍落后于顶尖闭源模型。
244 B
633 B
720 B
9.3 KB
1.0 KB
352 B
11.5 KB
68.2 KB
39.1 KB
227.5 KB
18.6 KB
118.2 KB
9.6 KB
58.8 KB
25.0 KB
中文
英国人工智能安全研究所（UK Artificial Intelligence Security Institute，UK AISI）与美国 AI 标准与创新中心（Center for AI Standards and Innovation，CAISI）（UK AISI / CAISI）联合评估了 Moonshot AI 的最新模型 Kimi K3（于 2026 年 7 月 16 日发布，并计划于 2026 年 7 月 27 日前以开放权重形式发布）。本次评估聚焦 Kimi K3 的网络安全能力，结果发现：
详细结果
这些结果属于针对一小部分公开和私有基准进行的初步评估。为减少拒答并衡量最大能力，评估时关闭了美国闭源权重模型的系统级安全护栏；这些模型面向公众提供的版本则启用了相关护栏。由于 Kimi K3 托管设置的具体限制，UK AISI / CAISI 对其运行了一组选定的网络安全评估。各个章节提供了详细方法说明。
网络安全能力趋势
模型的网络安全能力通过一种受项目反应理论（Item Response Theory，IRT）启发的方法，汇总多个基准中的多项任务。有关方法论细节，请参阅此前发布的报告。Kimi K3 的整体网络安全能力置信区间大于其他模型，因为其估计仅基于一个基准，即 ExploitBench；该基准包含 41 项聚焦漏洞利用开发的任务。ExploitBench 是衡量模型沿软件漏洞利用阶梯推进能力的领先基准。其他所有模型的整体网络安全能力分数均来自更多任务，这些任务覆盖了网络安全能力的额外领域。
来源：CAISI/NIST
图 2：截至 Kimi K3 发布时，最强美国和中国模型聚合能力随时间变化的初步比较。 美国趋势线由前沿美国模型的结果构成。纵轴每增加 400 分，完成任务的胜算提高 10 倍。误差线和阴影区域表示 95% 置信区间。
漏洞利用开发：ExploitBench
ExploitBench 是由卡内基梅隆大学开发的公开基准，用于衡量模型沿软件漏洞利用阶梯推进的能力，包括覆盖率与崩溃复现、任意读写、控制流劫持以及任意代码执行。该基准在 V8 引擎（为 Chrome 提供支持的 JavaScript 和 WebAssembly 软件）中测试模型对 41 个近期漏洞（2023 年后）的处理能力。
ExploitBench 的结果见图 1 和图 3。
来源：CAISI/NIST
图 3：Kimi K3 与其他模型在 ExploitBench 上的详细表现。颜色越深表示网络安全能力越强。每一行代表漏洞利用开发链中的一个关键里程碑，每个单元格显示相关模型能够达到该里程碑的 ExploitBench 任务数量。
Kimi K3 的表现优于 GLM-5.2，后者是截至 2026 年 6 月网络安全能力最强的开放权重模型。 Kimi K3 得分为 32%，而 GLM-5.2 得分为 24%（图 1）。
与网络安全能力最强的模型不同，Kimi K3 未能针对 ExploitBench 任务开发出可实现任意代码执行（ACE）的漏洞利用程序。 ACE 是漏洞利用开发中最严重的结果，可使攻击者劫持目标。Kimi K3 在 41 个样本中实现 ACE 的数量为 0，而网络安全能力最强的模型平均在 41 个样本中有 20 个实现了 ACE（图 3）。
网络靶场：The Last Ones（TLO）
“The Last Ones”（TLO）网络靶场是一项包含 32 个步骤的模拟企业网络攻击，覆盖 4 个子网和约 20 台主机，人类专家大约需要 20 小时才能完成。网络靶场是由专家构建的模拟网络，其中的主机、服务和漏洞被组织为连续的攻击链，攻击从初始网络访问点开始，可用于衡量模型自主实施端到端网络攻击的能力。
在这项评估中，Kimi K3 的表现显著低于领先的美国网络安全能力模型。 具体而言，Kimi K3 在这条包含 32 步的攻击路径上平均达到第 17 步，而网络安全能力最强的美国模型平均达到 28.5 步。
Kimi K3 的表现优于 GLM-5.2，后者是截至 2026 年 6 月网络安全能力最强的开放权重模型。 在 1 亿 token 的限制内，Kimi K3 平均达到第 17 步，而 GLM-5.2 平均达到第 11 步。
在 10 次尝试中的一次中，Kimi K3 在 1 亿 token 限制内成功完成了“The Last Ones”网络靶场。 这表明，在被指示执行攻击并获得初始网络访问权限时，Kimi K3 能够自主攻击小型、防御薄弱且存在漏洞的企业系统。不过，TLO 在多个方面不同于现实环境：它没有活跃的防御者和防御工具；不会对可能触发安全警报的操作施加惩罚；并且包含一条预设的攻击路径。
能够完成 TLO 已不再是少数模型独有的能力。 在此前的测试中，已有四个公开发布的闭源权重模型完成 TLO，其中能力最强的模型以更高可靠性完成，分别在 10 次尝试中成功 6 次和 7 次。Kimi K3 则在标准的 1 亿 token 限制内，于 10 次尝试中成功 1 次。
English / 原文
The UK Artificial Intelligence Security Institute (UK AISI) and the U.S. Center for AI Standards and Innovation (CAISI) (UK AISI / CAISI) conducted a joint evaluation of Moonshot AI’s latest model, Kimi K3 (released on July 16, 2026 and slated for open-weight release by July 27, 2026). This evaluation focused on Kimi K3's cyber capabilities and found that:
Detailed Results
These results represent preliminary evaluations on a small set of public and private benchmarks. U.S. closed-weight models were evaluated with system-level safeguards disabled to reduce refusals and enable measurement of maximal capabilities. Publicly available versions of these models have these safeguards enabled. Due to the specifics of Kimi K3’s hosting setup, UK AISI / CAISI ran a selective set of cyber evaluations. Detailed methodologies are provided in individual sections.
Cyber Capability Trends
The cyber capability of models is aggregated across multiple tasks from multiple benchmarks using an approach inspired by Item Response Theory (IRT). For details of the methodology please see prior published reports. Kimi K3’s overall cyber capability has a larger confidence interval than other models because it was estimated from a single benchmark (ExploitBench, which has 41 tasks focused on exploit development). ExploitBench is a leading benchmark to measure a model’s ability to progress along the software exploitation ladder. All other models’ overall cyber capability scores were derived from a larger number of tasks that covered additional domains of cyber capability.
Credit: CAISI/NIST
Figure 2: Preliminary comparison of aggregate capabilities over time of the most capable U.S. and PRC models as of Kimi K3’s release. The U.S. trendline is composed of results from frontier U.S. models. A 400-point increase on the y-axis equates to a 10x increase in the odds of solving tasks. Error bars and shaded regions denote 95% CIs.
Exploit Development: ExploitBench
ExploitBench is a public benchmark, developed by Carnegie Mellon University, that measures a model’s ability to progress along the software exploitation ladder, including coverage and crash reproduction, arbitrary read/write, control flow hijack, and arbitrary code execution. The benchmark tests models on 41 recent (post-2023) vulnerabilities in the V8 engine (the JavaScript and WebAssembly software that powers Chrome).
ExploitBench results are presented in Figures 1 and 3.
Credit: CAISI/NIST
Figure 3: Detailed ExploitBench performance for Kimi K3 and other models. Darker shading indicates greater cyber capability. Each row represents a key milestone in the exploit development chain, and each cell shows the number of ExploitBench tasks for which the model(s) in question were able to reach that milestone.
Kimi K3 outperforms GLM-5.2,the most cyber-capable open-weight modelas of June 2026. Kimi K3 achieves a score of 32%, whereas GLM-5.2 achieves a score of 24% (Figure 1).
Unlike the most cyber-capable models, Kimi K3 failed to develop exploits that achieved arbitrary code execution (ACE) for ExploitBench tasks. ACE is the highest-severity outcome in exploit development, granting attackers the ability to hijack a target. Kimi K3 achieved ACE on 0/41 samples, whereas the most cyber-capable models achieved ACE on 20/41 samples on average (Figure 3).
Cyber Range: The Last Ones (TLO)
“The Last Ones” (TLO) cyber range is a 32-step simulated corporate network attack spanning 4 subnets and approximately 20 hosts, which would take a human expert roughly 20 hours to complete. Cyber ranges are expert-built, simulated networks of hosts, services, and vulnerabilities arranged into sequential attack chains that begin at the point of initial network access, and can be used to measure a model's ability to conduct end-to-end cyberattacks autonomously.
On this evaluation, Kimi K3 performs significantly below the leading U.S. cyber capable models. Specifically, Kimi K3 reached step 17 of this 32-step attack path on average, while the most cyber-capable U.S. models reached 28.5 steps on average.
Kimi K3 outperforms GLM-5.2,the most cyber-capable open-weight modelas of June 2026. Within the 100M-token limit, Kimi K3 reaches step 17 on average, compared with step 11 for GLM-5.2.
In one of the 10 attempts, Kimi K3 successfully completes “The Last Ones” cyber range within the 100M token limit. This indicates that Kimi K3 is capable of autonomously attacking small, weakly defended and vulnerable enterprise systems, when directed to do so and given initial network access. However, TLO differs from real-world environments in several ways. It lacks active defenders and defensive tooling, imposes no penalty for actions that would trigger security alerts, and contains an intentional attack path.
Solves of TLO are no longer exclusive to a small set of models. In prior testing, four publicly released closed-weight models have solved TLO, with the most capable models solving it more reliably at 6/10 and 7/10 attempts. Kimi K3 solved it in 1/10 attempts within the standard 100M token limit.