摘要：能力和效率是大型语言模型（LLM）推理的两个关键维度。能力是指正确解决给定问题的能力，而效率是指在有限资源下完成这一任务的能力。当 LLM 使用思维链（Chain-of-Thought，CoT）推理解决具有可控难度的问题时，正确解决的问题数量以及得到正确答案所需的 token 数量，都取决于问题难度和模型规模。然而，这些因素如何共同塑造能力与效率，仍缺乏充分理解。本文使用分层贝叶斯模型，评估 DeepSeek-R1-Distill 模型家族在四类算术和算法推理问题上的能力与效率。在固定模型规模下，正确解决一个实例的概率会随着作为问题难度代理指标的实例规模增大而近似呈指数衰减。该衰减尺度随模型规模次线性增长，表明更大的模型能力更强，但能力增益会随规模扩大而递减。作为难度代理指标的输出长度，会随着实例规模按幂律增长。然而，这一幂律的参数并未随模型规模发生系统性变化，表明更大的模型并未变得更高效。综合来看，这些发现揭示了将朴素缩放作为开发更强 AI 系统策略的潜在局限：能力虽有提升但回报递减，而效率几乎没有改善。

Abstract:Capability and efficiency are two key dimensions of reasoning in large language models (LLMs). Capability refers to the ability to solve a given problem correctly, whereas efficiency refers to the ability to do so with limited resources. When LLMs use Chain-of-Thought (CoT) reasoning to solve problems of controlled hardness, both the number of problems solved correctly and the number of tokens required to reach a correct answer depend on problem hardness and model size. However, how these factors jointly shape capability and efficiency remains poorly understood. Here, we use hierarchical Bayesian models to evaluate the capability and efficiency of LLMs from the DeepSeek-R1-Distill model family across four classes of arithmetic and algorithmic reasoning problems. At a fixed model size, the probability of correctly solving an instance decays approximately exponentially with instance size, our proxy for problem hardness. The decay scale grows sublinearly with model size, indicating that larger models are more capable, but that capability gains diminish with scale. Output length grows as a power law with instance size, which serves as a proxy for difficulty. However, the parameters of this power law do not vary systematically with model size, suggesting that larger models do not become more efficient. Together, these findings reveal potential limitations of naive scaling as a strategy for developing more capable AI systems: capability improves with diminishing returns, while efficiency shows little to no improvement.