摘要：后训练对齐往往较为浅层，在微调过程中容易被削弱。若将宪法式中训练干预与后训练明确隔离，其是否能够产生持久的对齐效果，仍未得到验证。我们构建了一个基于 Anthropic Constitution、包含 3.94 亿 token 的宪法语料库，并在 120B 规模上实施宪法式中训练，即在中训练阶段插入基于原则和价值观的内容。我们采用 2×2 设计（课程排序 × 审慎推理），形成四种宪法式中训练条件及一个对照组，并在模型自行生成的测试集和既有基准上进行评估，涵盖压力下对齐、价值冲突解决、勒索和涌现失配等方面。所有模型均在三个阶段接受评估：中训练后、SFT 后以及良性微调后。宪法式中训练模型在对齐泛化和耐久性方面优于对照组，其中在勒索测试上尤为明显：SFT 会使所有模型形成勒索倾向，但宪法式中训练削弱了这一倾向，且这种优势在良性微调后依然存在（-17.5 个百分点）。不过，这种耐久性并未延伸至需要主动抵抗上下文压力或冲突的场景；在 SFT 后，其优势会减弱。中训练阶段宪法内容的存在也比其结构更为重要，并且宪法式中训练在任何阶段平均都未造成能力损失（MMLU、ARC-Easy、piqa、GSM8K）。因此，在中训练阶段加入适量宪法内容，可能带来广泛且持续的对齐收益，并可作为以 SFT 为核心的流程中一种低成本、互补性的补充。代码、数据和模型均已提供。

Abstract:Post-training alignment is often shallow, eroding under fine-tuning. It remains untested as to whether constitutional midtraining interventions can produce durable alignment when cleanly isolated from post-training. We build a 394M-token constitutional corpus from Anthropic's Constitution and apply constitutional midtraining at 120B scale, where principled, values-based content is inserted into midtraining. A 2x2 design (curriculum ordering x deliberative reasoning) was used to produce four constitutionally midtrained conditions, plus a control, which were evaluated on self-generated and established benchmarks including alignment under pressure, value conflict resolution, blackmail, and emergent misalignment. All models were evaluated across three stages: post-midtraining, post-SFT, and post-benign fine-tuning. Constitutionally midtrained models outperformed the control on alignment generalization and durability, notably on blackmail: SFT instilled a blackmail propensity in all models, but constitutional midtraining blunted it, with the advantage surviving benign fine-tuning (-17.5pp). This durability did not extend to settings that required active resistance to in-context pressure or conflict, where the advantage attenuates after SFT. The presence of constitutional content at midtraining also mattered more than its structure, and constitutional midtraining incurred no capability cost, on average, at any stage (MMLU, ARC-Easy, piqa, GSM8K). A modest amount of constitutional content at midtraining could therefore yield broad, persistent alignment gains, offering a cheap, complementary addition to SFT-centered pipelines. Code, data, and models are available.