摘要：基准测试得分能告诉我们大语言模型（LLM）具备哪些能力，却无法说明实现这些能力需要消耗多少能量。我们直接测量了这一权衡：在七项基准测试和六种硬件平台上评估了 86 个开放权重模型检查点，硬件范围从消费级 GPU 到八 GPU 服务器。我们发现，即使能力相近的模型之间，能效也存在显著差异。多数配置偏离得分—能耗帕累托前沿：与得分至少相同的其他配置相比，其能耗中位数高出 2.6 至 4.8 倍，最大差异达到 27 倍。推理可以提升能力，但启用思考模式平均会使能耗增加约 3.7 倍，其收益高度依赖模型家族和输出预算。生成 token 数量也不能准确替代能耗指标，会将 18% 至 24% 的模型配对能耗高低排序弄反。相比之下，FP8 可在得分几乎不变的情况下将能耗降低约四分之一；更换硬件则主要改变能耗的整体量级，而不改变模型之间的相对排序。这些结果表明，仅凭能力无法全面衡量效率。我们主张，在报告基准测试得分时，应同时报告特定服务配置和输出预算下的实测能耗。

Abstract:Benchmark scores tell us what an LLM can do, but not how much energy is required to deliver that capability. We directly measure this trade-off for 86 open-weight model checkpoints, evaluated on seven benchmarks and six hardware platforms ranging from a consumer GPU to an eight-GPU server. We find large differences in energy efficiency even among models with similar capability. Most configurations lie away from the score--energy Pareto frontier: they use a median of 2.6--4.8$\times$ more energy than another configuration that scores at least as well, with differences reaching 27$\times$. Reasoning can improve capability, but enabling thinking increases energy by about 3.7$\times$ on average, and the benefit depends strongly on the model family and output budget. Generated-token count is also an imperfect proxy for energy, misordering 18--24% of model pairs. In contrast, FP8 reduces energy by roughly a quarter with little change in score, while changing hardware largely rescales energy without changing the relative ranking of models. These results show that capability alone gives an incomplete picture of efficiency. We argue that benchmark scores should be reported together with measured energy consumption under a specified serving regime and output budget.