摘要：大语言模型（LLM）通过将链式思维（CoT）用作中间思考阶段的草稿板，已经展现出卓越的推理能力。然而，CoT 技术需要对思维 Token 进行显式监督，因此需要丰富、任务专用的数据。本文提出抽象 Token 课程学习（Abstract Token Curriculum，ATC），这是一种新颖的课程学习框架，能够在无需直接监督或人工设计草稿板的情况下，诱导出有效的连续中间表示。ATC 通过一系列分布逐步提高问题复杂度，训练模型在连续表示空间中发展内部的抽象“思维”。本文从理论和实验两方面证明了 ATC 的益处，以及它相对于既有连续思维训练方法的优势。从理论上看，我们表明：在使用 ATC 和单层 softmax 注意力学习奇偶函数时，注意力会自然聚焦于上下文中的 CoT Token；这些 Token 为预测下一个 Token 提供了“最容易的路径”。在实验上，我们展示了 ATC 在图可达性和算术学习任务上的有效性。

Abstract:Large Language Models (LLMs) have achieved remarkable reasoning capabilities by utilizing chain-of-thought (CoT) as a scratchpad for intermediate stages of thinking. However, CoT techniques require explicit supervision on thinking tokens, which requires rich, task-specific data. In this work, we propose Abstract Token Curriculum (ATC), a novel curriculum learning framework that elicits effective continuous intermediate representations without direct supervision or manual scratchpad design. ATC gradually increases problem complexity through a sequence of distributions, training the model to develop internal abstract thoughts'' in the continuous representation space. This paper provides both theoretical and experimental evidence for the benefits of ATC and its advantages over previous methods for training continuous thoughts. Theoretically, we show that for learning parity functions with single-layer softmax attention using ATC, attention naturally focuses on the CoT tokens in the context that provide the easiest path'' to predicting the next token. Experimentally, we show ATC's effectiveness on graph reachability and arithmetic learning tasks.