摘要：Mixture-of-Experts（MoE）模型会为每个 token 仅激活少数专家，却需要存储完整的专家池。整专家剪枝能够缩小该专家池，但对于推理模型而言，必须在不削弱推理能力的前提下移除专家。常见评分方法依据路由频率或输出幅度对专家排序，衡量的是孤立贡献，而非删除所造成的损害。决定损害程度的是功能可替代性，即保留下来的计算是否能够复现被移除部分的功能。一个贡献较大的专家可能可由剩余混合专家替代，而一个贡献较小的专家则可能承载幸存专家无法恢复的方向信息。我们提出 RAZOR，这是一种免训练方法，通过共识残差评估可替代性；共识残差是各个专家输出相对于其原始加权混合输出的偏差。在固定层输入的情况下，这些残差能够给出删除单个专家导致的精确输出变化，其中包括幸存专家的重新加权，以及由路由器补位机制提升的替代专家。RAZOR 在校准 token 上聚合这种变化，并仅通过前向推理、无需梯度、子集搜索或恢复训练，即可按逐层预算完成剪枝。在 GLM-4.7-Flash、Qwen3.6-35B-A3B、DeepSeek-V4-Flash-0731 和 Hy3 上，以 25% 和 50% 的专家移除比例进行测试时，RAZOR 在全部 8 组模型—预算设置中，均在所评估剪枝方法里取得 9 项以推理为中心任务的最高宏平均分。与 GLM-4.7-Flash 和 Qwen3.6-35B-A3B 上的 REAP 相比，RAZOR 在该平均分上提升 2.12 至 5.59 分，并在全部 4 项比较中降低 reverse KL。保留的准确率并非全貌：经过剪枝的 Qwen3.6-35B-A3B 在响应多样性、格式和终止行为方面仍会发生变化。

Abstract:Mixture-of-experts (MoE) models activate only a few experts per token yet store the entire expert pool. Whole-expert pruning shrinks that pool, but for reasoning models it must remove experts without eroding reasoning ability. Common scores rank experts by routing frequency or output magnitude, which measures isolated contribution rather than deletion damage. What decides the damage is functional replaceability, whether the surviving computation can reproduce what is removed. A large contribution may be replaceable by the remaining mixture, whereas a small one may carry a direction the survivors cannot recover. We introduce RAZOR, a training-free method that scores replaceability from consensus residuals, the deviations of individual expert outputs from their original weighted mixture. Holding the layer input fixed, these residuals yield the exact output change from deleting one expert, including survivor reweighting and the replacement expert promoted by router refill. RAZOR aggregates this change over calibration tokens and prunes to a layerwise budget using forward passes alone, without gradients, subset search, or recovery training. On GLM-4.7-Flash, Qwen3.6-35B-A3B, DeepSeek-V4-Flash-0731, and Hy3 at 25% and 50% expert removal, RAZOR attains the highest macro average over nine reasoning-centered tasks among the evaluated pruning methods in all eight model-budget settings. Against REAP on GLM-4.7-Flash and Qwen3.6-35B-A3B, it gains 2.12-5.59 points on this average and lowers reverse KL in all four comparisons. Retained accuracy is not the whole picture, as pruned Qwen3.6-35B-A3B still shifts in response diversity, formatting, and termination.