英国 AISI 携手 EvalEval：推动前沿 AI 基准评测结果的可复现与标准化

EvalEval 联盟非常高兴地宣布，英国人工智能安全研究所（UK AISI）正使用 EvalEval 的基础设施公开发布评测结果，以支持更具可复现性与可验证性的评测科学。

AISI 和 EvalEval 此前已开展过研究合作，该合作始于 NeurIPS 2025 联合研讨会，研究所的反馈帮助塑造了 Every Eval Ever (EEE) 模式（schema）。双方合作的下一阶段正是将这一共享基础设施付诸实践。

为什么可复现的评测报告至关重要

随着 AI 部署的加速，评测正日益成为评估模型和系统性能的重要证据来源。然而，目前的评测结果散落在诸多不同格式、平台和发布渠道中，往往缺乏足以复现这些结果的信息。而重新运行评测本身的成本可能高到令人难以承受。

EvalEval 的使命是通过共享报告模式 Every Eval Ever 以及开放平台 Evaluation Cards 来改善这一生态系统，将评测结果及解读它们所需的信息整合进一个统一的结构中。

这自然建立在 AISI 过往工作的基础之上：通过 OptStop 提升评测效率，通过 HiBayES 增强统计严谨性，以及在交互记录分析（transcript analysis）和能力激发（capability elicitation）等领域推进标准化。AISI 与 EvalEval 正在携手诊断评测报告中的脱节问题，并构建共享基础设施来消除这些差距。

AISI 正在公开分享的内容

交互记录级别的透明度不仅对可复现性至关重要，对深入分析与诊断同样不可或缺。在双方合作的新阶段，AISI 正适时通过 Evaluation Cards 公开其评测方法和发现。本次发布包含了该论文主要实验中五个基准的已验证结果、背景及配置信息：

• HealthBench
• FrontierMath
• Humanity's Last Exam
• SWE-Bench Pro
• Terminal-Bench 2.0

这些结果涵盖了六款前沿模型：Claude Opus 4、Claude Opus 4.5、Claude Opus 4.6、GPT-5、GPT-5.2 和 GPT-5.4。本次发布还包含了来自两项相关网络安全评测——Cyber CTFs 和 The Last Ones 的结果，这两项评测使用了一组不同但部分重合的模型集合。这些数据与 AISI 的论文《推理计算如何重塑前沿大语言模型评测》（How Inference Compute Shapes Frontier LLM Evaluation）相配套，该论文研究了基准性能如何取决于推理阶段计算量（inference-time compute）以及评测协议（evaluation protocol）。

Humanity's Last Exam 上的表现在不同评测协议和推理计算量下发生变化。每条曲线显示了在给定 token 预算内解决的尝试任务的累积份额（采用每个任务最早观察到的成功结果）。当模型在每次尝试后收到来自 oracle 的正确性反馈时，随着 token 使用量的增加，它们能继续解决更多任务。

当评测结果连同配置信息一起公开发布时，研究人员和从业者可以更仔细地审查单项研究，并在更广泛的生态系统中对比各项发现。在其他报告缺乏这些细节的情况下，像 AISI 这样的公开发布提供了经过验证的参考基准，以便在具体上下文中解读各项评测——例如，帮助研究人员理解实验配置的选择如何影响所报告的性能。随着更多评测机构采用 EEE，此类开放对比将能够支持更广泛、更可靠的元研究（meta-research）。

AISI 的 Terminal-Bench 2.0 结果与同一模型在不同评测设置下报告的其他评测结果并列展示。

我们对这一落地应用感到振奋，并期待与 AISI 及其他 AI 评测机构进一步推进评测的标准化与共享。

贡献力量，共担使命

关于 EvalEval 联盟

EvalEval 联盟（The EvalEval Coalition）是一个致力于为评测生态构建具有扎实科学依据的研究以及稳健部署基础设施的研究社区。其目标是提升评测科学水平，解决评测适用性与实用性记录缺乏共识的问题，并拓宽对科学研究和政策分析至关重要的影响范围。

该联盟的旗舰项目包括：Every Eval Ever（用于评测结果的共享模式与代码仓库），以及 Evaluation Cards（将基准元数据、评测运行数据和模型元数据组合为可解读的记录）。通过这些项目，能够更容易地厘清表面看似相近的得分是在何种显著不同的条件下产生的。

关于英国人工智能安全研究所

英国人工智能安全研究所（UK AISI）是英国政府科学、创新与技术部旗下的研究机构。其使命是为政府提供对先进 AI 所带来风险的科学认知。AISI 开展研究并构建基础设施，以理解前沿 AI 的能力与影响，开发并测试风险缓解措施，并为政策制定提供参考依据。

延伸阅读

The

EvalEval Coalition

is thrilled to share that the

UK AI Security Institute (AISI)

is using EvalEval's infrastructure to openly share evaluation results, supporting more reproducible and verifiable evaluation science.

AISI and EvalEval have previously collaborated on research that began at a joint workshop alongside NeurIPS 2025, and feedback from the Institute has helped shape the Every Eval Ever (EEE) schema. This next phase of the collaboration puts that shared infrastructure into practice.

Why reproducible evaluation reporting matters

As AI deployment accelerates, evaluations are becoming increasingly important sources of evidence about model and system performance. Yet results are reported across many formats, platforms, and outlets, often without enough information to reproduce them. Running the evaluations again may itself be prohibitively expensive.

EvalEval's mission is to improve this ecosystem through a shared reporting schema, Every Eval Ever, and an open platform, Evaluation Cards, that brings evaluation results and the information needed to interpret them into a common structure.

This builds naturally on AISI's work to make evaluation more efficient through OptStop, more statistically rigorous through HiBayES, and more standardised in areas including transcript analysis and capability elicitation. Together, AISI and EvalEval are working to diagnose gaps in evaluation reporting and build shared infrastructure to close them.

What AISI is sharing

Transcript-level transparency matters not only for reproducibility, but also for analysis and diagnosis. In this new phase of the collaboration, AISI is making publicly reported evaluation methods and findings available through Evaluation Cards where appropriate. The release includes verified results, context, and configuration information for the five benchmarks in the paper's main experiment:

• HealthBench
• FrontierMath
• Humanity's Last Exam
• SWE-Bench Pro
• Terminal-Bench 2.0

These results cover six frontier models: Claude Opus 4, Claude Opus 4.5, Claude Opus 4.6, GPT-5, GPT-5.2, and GPT-5.4. The release also includes results from two related cyber evaluations—Cyber CTFs and The Last Ones—which use a different, partially overlapping set of models. The data accompany AISI's paper, How Inference Compute Shapes Frontier LLM Evaluation, which studies how benchmark performance depends on inference-time compute and evaluation protocol.

Performance on Humanity's Last Exam changes with evaluation protocol and inference compute. Each curve shows the cumulative share of attempted tasks solved within a given token count, using the earliest observed success per task. When models received correctness feedback from an oracle after each attempt, they continued to solve additional tasks as token use increased.

When results are openly released with setup information, researchers and practitioners can examine individual studies more closely and compare findings across the wider ecosystem. Where other reports lack these details, releases like AISI's provide verified reference points for interpreting evaluations in context—for example, by helping researchers understand how setup choices may influence reported performance. As more evaluators adopt EEE, open comparisons like these can support broader and more reliable meta-research.

AISI's Terminal-Bench 2.0 results alongside other reported evaluations for the same models, under different evaluation setups.

We are excited about this adoption and look forward to further standardising and sharing evaluations with AISI and other AI evaluation organisations.

Contribute to the shared mission

About the EvalEval Coalition

The EvalEval Coalition is a research community developing scientifically grounded research and robust deployment infrastructure for the evaluation ecosystem. Its goal is to improve evaluation science, address the lack of consensus around documenting evaluation applicability and utility, and broaden coverage of the impacts that matter for scientific research and policy analysis.

The coalition's flagship projects include Every Eval Ever, a shared schema and repository for evaluation results, and Evaluation Cards, which combines benchmark metadata, evaluation-run data, and model metadata into interpretable records. Together, they make it easier to understand when apparently similar scores were produced under meaningfully different conditions.

About the UK AI Security Institute

The UK AI Security Institute is a research organisation within the UK government's Department for Science, Innovation and Technology. Its mission is to equip governments with a scientific understanding of the risks posed by advanced AI. AISI conducts research and builds infrastructure to understand advanced AI capabilities and impacts, develop and test mitigations, and inform policy.

Further reading