大模型全阶段训练综合框架：从数据与分词、预训练到后训练对齐
英文标题：A Comprehensive Framework for All Stages of Large-Model Training: From Data and Tokenization, through Pretraining, to Post-Training Alignment
本研究提出了一个覆盖大语言模型（LLM）与多模态大模型（MLLM）全生命周期的端到端训练分析框架，打破了以往综述割裂单一阶段的局限。论文系统梳理了2017至2026年间的395篇核心文献，从数据、目标函数、优化策略、系统约束、评测及风险治理六个维度构建了统一的方法图谱。研究指出，Scaling Laws 的有效性高度依赖被长期忽视的数据质量与配比，而偏好对齐与带可验证奖励的推理强化学习正重塑后训练边界，多模态训练亦对纯文本范式提出了结构性挑战。该框架为大模型开发者与研究团队提供了涵盖跨阶段效应图、实操决策树及10项开放课题的系统工程指引。
2.0 KB
483 B
612 B
380 B
24.2 KB
34.7 KB
中文
[提交日期：2026年10月24日]
标题：大模型全阶段训练综合框架：从数据与分词、预训练到后训练对齐
查看由 Xiangjing An 与其他 2 位作者撰写的题为《A Comprehensive Framework for All Stages of Large-Model Training: From Data and Tokenization, through Pretraining, to Post-Training Alignment》的论文 PDF
摘要：大语言模型（LLM）与多模态大语言模型（MLLM）的能力源自整个训练流水线（涵盖数据与分词、预训练，以及包括监督微调、偏好对齐、强化学习和持续优化在内的后训练）的协同作用，而非任何单一阶段的独立产物。现有的综述大多采用单阶段或单一技术的视角，将相关研究视为孤立系统，缺乏统一的阶段划分与可横向对比的基准。 本文提出了统一的阶段划分方案以及涵盖数据、目标函数、优化策略、系统约束、评测、风险与治理的六维分析框架。我们系统梳理了 2017 年至 2026 年间的 395 篇核心文献，并绘制了跨阶段效应的概念图谱：预训练对后训练的前向约束，以及后训练对数据与预训练的反向赋能。 我们提炼出三大核心发现：首先，缩放定律（Scaling Laws）的可比性取决于一个长期被忽视的变量——数据质量与数据混合比例；其次，偏好对齐方法（RLHF、DPO）和基于可验证奖励的推理强化学习正在重新定义后训练的边界，而 Pass@1 与 Pass@K 之间的争论仍未形成定论；第三，多模态训练在分词器（Tokenizer）设计、训练目标和评测维度上对纯文本范式构成了结构性挑战。 本文提供了一幅跨阶段效应图、方法族图谱、实用决策树以及至少 10 个可验证的开放性问题，并为各阶段深入的子方向综述构建了可扩展的骨干框架。截至检索日期，在 395 篇参考文献中，经过同行评审的实证文献（A+B类）占比约为 79%。
提交历史
来自：An Xiangjing [
]
[v1] 2026年10月24日 星期六 05:09:05 UTC (498 KB)
English / 原文
[Submitted on 24 Oct 2026]
Title:A Comprehensive Framework for All Stages of Large-Model Training: From Data and Tokenization, through Pretraining, to Post-Training Alignment
View a PDF of the paper titled A Comprehensive Framework for All Stages of Large-Model Training: From Data and Tokenization, through Pretraining, to Post-Training Alignment, by Xiangjing An and 2 other authors
Abstract:The capabilities of large language models (LLMs) and multimodal LLMs (MLLMs) arise from the synergy of the full training pipeline, including data and tokenization, pretraining, and post-training (supervised fine-tuning, preference alignment, reinforcement learning, and continual optimization), rather than any individual stage. Existing surveys mostly adopt a single-stage or single-technique perspective, treating related research as isolated systems without unified stage division or a horizontally comparable basis.
This paper proposes a unified stage division and a six-dimensional analytical framework covering data, objective function, optimization strategy, system constraints, evaluation, and risk and governance. We systematically review 395 core references from 2017 to 2026, and present a conceptual map of cross-stage effects: forward constraints of pretraining on post-training, and reverse empowerment of post-training on data and pretraining.
We derive three core findings. First, the comparability of scaling laws depends on data quality and mixture, a long-neglected variable. Second, preference alignment methods (RLHF, DPO) and reasoning reinforcement learning with verifiable rewards redefine post-training boundaries, while the Pass@1 versus Pass@K debate remains unresolved. Third, multimodal training poses structural challenges to the pure-text paradigm in tokenizer design, training objectives, and evaluation.
The paper provides a cross-stage effects diagram, a method-family map, a practical decision tree, and at least 10 verifiable open problems, with an extensible skeleton for focused sub-surveys per stage. As of the retrieval date, peer-reviewed evidence (A+B) accounts for approximately 79% of the 395 references.
Submission history
From: An Xiangjing [
]
[v1]
Sat, 24 Oct 2026 05:09:05 UTC (498 KB)