摘要：测试时扩展可通过生成并组合多个候选回答，提升大语言模型的推理能力。在基于采样的方法中，推理预算通常以生成候选答案的数量 N 来描述。然而，N 只能说明生成了多少候选答案，而不能说明这些候选答案是如何执行生成的。相同的候选预算既可以通过一次批量生成调用完成，也可以拆分为多次使用较小批大小的串行调用。我们首先使用 Phi-3-mini 和 Qwen2.5-1.5B，在 500 条 GSM8K 提示上研究提高 N 对推理准确率的影响。如预期所示，将 N 从 1 增至 8 后，Phi-3-mini 的准确率提高了 8.4 个百分点，Qwen2.5-1.5B 的准确率提高了 18.4 个百分点。然而，准确率本身无法反映使用更大候选预算所带来的系统成本。因此，我们固定 N = 8，并比较四种生成调度：1x8、2x4、4x2 和 8x1，其中 axb 表示进行 a 次生成调用、每次调用生成 b 个候选答案。在保持候选总数固定的条件下，我们测量了延迟、吞吐量、GPU 小时数和 GPU 设备总能耗。在 A100 GPU 上，8 次串行调用的 GPU 设备总能耗是一次批量生成 8 个候选答案调用的 4.64—4.86 倍，其 P95 延迟则为后者的 5.77—6.12 倍。针对每个模型，在 3 个独立调度的 A100 节点上，以及在短输出的 SciQ/V100 实验中，也观察到了相同模式。这些结果表明，仅凭候选数量不足以描述多候选测试时扩展的系统成本。当候选答案彼此独立且显存容量允许时，采用更少的生成调用和更大的批大小会更高效。因此，评测除应报告候选数量和准确率外，也应报告生成调度方式以及 GPU 层面的系统指标。

Abstract:Test-time scaling can improve large language model reasoning by generating and combining multiple candidate responses. In sampling-based methods, the inference budget is often described by the number of generated candidates, N. However, N tells us how many candidates are generated, not how they are executed. The same candidate budget can be produced in one batched generation call or split across several sequential calls with smaller batch sizes. We first study the effect of increasing N on reasoning accuracy using Phi-3-mini and Qwen2.5-1.5B on 500 GSM8K prompts. As expected, increasing N from 1 to 8 improves accuracy by 8.4 percentage points for Phi-3-mini and 18.4 points for Qwen2.5-1.5B. However, accuracy alone does not show the systems cost of using a larger candidate budget. We therefore fix N = 8 and compare four generation schedules: 1x8, 2x4, 4x2, and 8x1, where axb denotes a generation calls with b candidates per call. We measure latency, throughput, GPU-hours, and gross GPU-device energy while keeping the total candidate count fixed. On A100 GPUs, eight serial calls use 4.64-4.86x as much gross GPU-device energy and have 5.77-6.12x the P95 latency of one batched call with eight candidates. The same pattern appears across three independently scheduled A100 nodes per model and in short-output SciQ/V100 experiments. These results show that candidate count alone is not enough to describe the systems cost of multi-candidate test-time scaling. When candidates are independent and memory allows it, fewer generation calls with larger batch sizes are more efficient. Evaluations should therefore report not only candidate count and accuracy, but also generation schedule and GPU-level systems metrics.