🔶 为什么机器学习研究 Agent 不会过拟合？

机器学习的核心在于泛化，而不是记忆。你把一批训练样本交给学习算法，并用它们拟合模型；但目标并不是让模型在训练样本上表现良好——这很容易，你完全可以直接记住答案。目标是在此前从未见过的新样本上也有良好表现。如果一个模型在训练数据上表现出色、在新数据上却表现糟糕，那么它实际上什么也没有学会；你只是在欺骗自己，以为它学会了。这种失败模式有一个名字：过拟合。

任何上过统计学或机器学习入门课的人，都知道标准的防御方法：留出一部分数据，拒绝用它训练模型。在实践中，这些留出数据承担两种角色。验证集会在模型构建过程中被反复查阅——用于比较候选方案、调节超参数，以及决定下一步尝试什么。最终的测试集（或称留出集）则应当只在最后使用一次：由于训练过程从未见过它，其上的强劲表现才可以作为模型在现实世界中将遇到的新样本上的表现的可靠代理。

机器学习的核心在于泛化，而不是记忆

不过，“留出”这一条件至关重要。只有当留出集始终真正未被看到时，它作为可靠代理的保证才成立。如果你查看其上的表现，据此调整训练流程，再次查看并不断迭代、追逐越来越好的数字，那么该数据集便不再是未见数据；它已经成为训练流程的一部分。这样做足够多次，你就会像对训练集过拟合一样对它过拟合，从而失去未见数据的代理。这对任何以此方式重复使用的留出集都成立，包括按设计本就会被重复使用的验证集。

机器学习核心中的一个谜题

现实中的机器学习研究看起来恰恰就是我们刚才描述的迭代改进循环。人们使用少数几个多年不变的基准数据集来衡量性能。研究社区进行着一个庞大、分布式的循环：在基准上评估模型，修订训练流程，再次评估，发表结果，然后让下一个团队再榨取出一点提升。

这正是针对留出集进行爬山搜索的做法；按照教科书的说法，它本应导致大规模过拟合。到现在为止，排行榜理应充斥着那些在基准上看似出色、在其他地方却表现平平的模型。

然而，事实并非如此。针对旧的、被大量重复使用的基准，构建全新测试集的研究发现，改进大多能够迁移：模型在新数据上展现出的增益，与其在旧基准上展现的增益相同。与教科书的预测相反，基准驱动的机器学习带来了快速且在很大程度上真实的进步。为什么？

对此并不缺乏假设，但它们很难通过实证检验，因为实验的“研究对象”是整个科研共同体。你无法重置一个领域、抹去它的记忆，再在受控条件下重跑过去十年。

但我们可以做类似的事情。如今，我们拥有能力较强、基于 LLM 的研究 Agent，它们能够自主运行与人类社区相同的机器学习优化循环。它们同样进行基准爬山搜索——耐人寻味的是，它们似乎同样不会过拟合。不同之处在于，Agent 与研究社区不同，是可以被重置的。你可以清除其记忆，精确控制它看到的信息，然后再次运行实验。在最近的论文《What fits (into few tokens) doesn't overfit: Compression and generalization in ML research agents》中，我们正是这样做的——并在此过程中为这个长期谜题提出了具体解释。

奥卡姆剃刀的精确定义

这一解释始于一个非常古老的思想。奥卡姆剃刀原则认为，在同样能够解释数据的假设中，更简单的那个更可能是正确的。事实证明，这一直觉有精确的数学形式，并且它构成了整个故事的基础。

假设你可以用很少的比特描述你的假设——也就是你的模型、你的策略——所需比特数远少于记住训练数据所需的比特数。如果这个紧凑的假设在训练数据上表现非常好，那么它也必须在新数据上表现良好。

奥卡姆剃刀的形式化表述是：在同样能够解释数据的假设中，更简单的那个——即可以用更少比特描述的那个——更可能泛化到新样本。

其推理依赖于一个计数论证。短描述的数量本就不多，因为短字符串的数量有限。候选假设越少，其中任意一个只是偶然在训练集上欺骗你的可能性就越低——即使你确实使用训练集来引导搜索。

还可以换一种方式理解：如果你的压缩描述小到无法暗中记录训练数据，那么它在训练数据上表现良好，就不可能是因为记住了答案——它根本没有足够空间这样做。它必定是捕捉到了数据结构中的某些真实内容。短描述无法作弊，因为没有容纳作弊的空间。

这里有一个颇具吸引力的假设：*成功的机器学习策略具有很高的可压缩性。*研究者在一个项目期间可能会盯着数千个基准分数，但最终保留下来的策略通常只是由一些熟悉选择构成的短清单——某个架构家族、一个优化器、一个学习率调度方案、一个数据处理配方、一套正则化机制。如果这套最终方案只需少量比特就能传达，那么模型对基准的实际依赖，远小于漫长而曲折的实验记录所暗示的程度。爬山搜索过程很广泛，但从另一端产出的东西却是——或本可以是——极小的。

压缩、智能与知识丰富的听众的力量

设想一下：你要向一名聪明的高中生解释一条具体的机器学习流水线，并提供足够细节让对方实际复现它。这将是一场漫长而费力的对话。你需要解释什么是梯度下降、什么是神经网络、PyTorch、JAX 或 TensorFlow 做什么、什么是学习率，等等。其中几乎没有内容是针对你的问题的；它们都是有关机器学习如何运作的通用背景知识。

现在，设想向一位资深 ML 工程师解释同一条流水线。对话会立刻缩减为几句话。你跳过所有属于共同知识的内容，只传达对这个问题真正特定的信息：架构选择、batch size、优化器、几个超参数。听众对世界已有的知识越多，你需要发送的消息就越短，也就能压缩得越激进。这些“世界知识”不计入奥卡姆剃刀论证中的复杂度，因为即使你没有看过训练集，也本来可以把这些内容全都写下来。

这正是大语言模型进入图景的地方。现代 LLM 携带了海量的世界知识。它们知道 ML 工具如何工作；知道标准优化算法；知道惯用的超参数选择和常见默认值。如果某个细节没有明确指定，它们可以填入合理的取值。这使它们成为极其出色的压缩解码器：向 LLM 交付一条简洁的专家对专家消息，它就能将其展开成完整、可运行的流程。仔细想想，这恰好就是它们如此强大的原因。

实验：让策略穿过信息瓶颈

这启发了一个清晰的实验。让一个 ML 研究 Agent——探索者（explorer）——尝试解决一个新的机器学习问题。给予它对验证集的完整访问权限，让它自由实验和迭代，在数百轮中追逐更好的验证性能。此处，验证集扮演基准的角色：一个可被反复查询的可复用留出集。这正是理论上应当过拟合的爬山循环。

然后测试该解决方案的可压缩性。第二个 Agent，即压缩者（compressor），读取探索者工作的完整记录，并尝试将获胜策略提炼为非常短的提示词——仅仅几个 token。该提示词被交给第三个 Agent，即复现者（reproducer），后者必须只利用该提示词和训练数据从头实现策略。关键在于，复现者无法访问验证集、探索者的代码或其工作记录。这条短提示词是从验证集中学到的任何信息能够到达它的唯一通道。（在我们论文报告的研究中，压缩者和复现者均为 Claude 模型。）

如果复现者——在冷启动状态下、仅凭几个 token——能够匹配探索者的性能，那么指定该策略所需的全部依赖验证集的信息，就都穿过了这条极窄通道。该策略是可压缩的。我们将此称为*输出压缩（output compression）*的证书。

该设置具有一个人类研究共同体不具备的实用性质：复现者可以被一次又一次地重置。压缩者可以尝试许多不同的压缩方式，并观察每一种方式被如何解码，因为每一次尝试都会交给一个对上一次毫无记忆的全新复现者。这有点像电影《记忆碎片》（Memento）：你正在为一个会在阅读前被清除记忆的自己留下简短便条。你逐渐学会写出可供一个知识丰富、但失忆的自己执行的便条；这些便条可以很短，因为接收者会像你一样补全所有未说出的内容。

在研究者的实验中，探索 Agent 的策略被挤过一个狭窄的信息瓶颈。凡是能在压缩后保留下来的内容，都必须反映真实结构，而不是被记忆的数据。

另一端产出了什么

结果显示，这些压缩极其短小。在涵盖表格分类、图像分类、语言建模、扩散建模和奖励建模的 8 个数据集中，32-token 提示词已足以让全新的复现者，在绝大多数问题上匹配探索者经自适应优化得到的模型。其中，一种语言建模策略甚至被压缩至仅 16 个 token，且留出集性能毫无损失。

这些提示词实际上是什么样子？最有揭示性的例子，恰好位于简洁性的边界，也就是压缩几乎失效的地方。在一次语言建模实验中，探索者发现了一套自定义的 GPT 风格训练配方。在 16-token 预算下，以下内容仍足以让全新的复现者匹配未经压缩的探索者：

QKn 12L768 Mu .1 R² b2M 4x

对人类读者而言，这看上去十分晦涩，但对于另一个 ML Agent，它表达了具体含义：QKn 表示“QK normalization”，12L768 表示一个 12 层、768 维的 Transformer，Mu .1 表示学习率为 0.1 的 Muon 优化器，R² 表示 squared-ReLU 激活函数，b2M 表示 200 万 token 的 batch，而 4x 表示四倍宽度的前馈模块。可是，将预算削减到 8 个 token 后，提示词变为：

12L768 Mu .1 R²

此时，复现者不再能匹配探索者。缺失部分指定了真正的训练选择：这些选择是根据数据作出的，而且不同于最显而易见的默认值。这一边界展示了可压缩性的限度，因而十分重要。它表明，复现者的成功并不单纯来自先验知识。一些经过压缩的 token 携带了确实从数据本身学到的信息；当这些 token 消失时，性能也随之消失。

我们还进行了一组从另一方向施加信息瓶颈的实验。我们并非压缩探索者的输出，而是压缩其输入：不再告诉探索者每个模型的数值验证分数，而只返回单个比特——这个模型是否击败了当前最佳结果？即便每次查询的反馈都被缩减到 1 个比特，探索者找到的策略仍与获得完整数值分数时一样好。验证集和最终策略之间的通道在两个方向上都很狭窄，而这个单比特版本甚至附带严格的泛化数学保证。

在 8 个数据集上，经数百次迭代实验产生的策略可以被压缩为短至 16 至 32 个 token 的提示词——短到一个毫无记忆的全新 Agent 仍能复现原始结果。

抓住作弊者

一个好的实证理论应当可被证伪——这个理论也是如此。如果低过拟合确实由可压缩性解释，那么那些真正过拟合的模型，就应当无法通过这条流水线被压缩。

为进行检验，我们通过向 Agent 提供对验证集的直接访问权限，并提示它们不惜一切代价最大化验证性能，故意推动它们走向过拟合。Agent 上钩了：在 102 次实验运行中的 38 次里，验证准确率比真实留出集准确率高出超过 10%。

该理论预测，这些增益不应当穿过压缩瓶颈，因为它们编码的是特定验证样本的偶然特性，而不是可迁移的结构。果然，当这些策略经由短提示词被传递给全新的复现者时，验证集特有的优势消失了。压缩以非常高的准确率区分出了合理策略和过拟合策略。

因此，压缩不仅解释了为什么自主研究 Agent 往往不会过拟合；它还提供了一种在过拟合确实发生时捕捉过拟合的工具：标记那些无法通过任何短描述复现结果的情况。

这告诉了我们什么——以及没有告诉我们什么

有几项注意事项需要说明。整个框架假设

Machine learning, at its core, is about generalization, not memorization. You hand your learning algorithm a pile of training examples and use them to fit a model. But the goal is not to perform well on the training examples — that's easy, you could just memorize the answers. The goal is to perform well on new examples that you have never before seen. If a model does well on the data it was trained on but poorly on fresh data, it hasn’t actually learned anything; you have only fooled yourself into thinking it has. This failure mode has a name: overfitting.

Anyone who has taken an introductory statistics or machine learning class knows the standard defense. You hold out some of your data and refuse to train on it. In practice, this held-out data plays two roles. A validation set is one you consult repeatedly while building the model — to compare candidates, tune hyperparameters, and decide what to try next. A final test set (or holdout) is meant to be touched only once, at the very end: because the training procedure never saw it, strong performance there is a correct proxy for the new examples you will encounter in the wild.

Machine learning, at its core, is about generalization, not memorization

The “holdout” condition is crucial, though. The correct-proxy guarantee holds if the held-out set stays genuinely unseen. If you check your performance on it, tweak your training procedure in response, recheck, and iterate, chasing better and better numbers, that set is no longer unseen; it has become part of your training procedure. Do this enough times, and you can overfit it just as you might have overfit the training set, and you have lost your proxy for unseen data. This is true of any held-out set you reuse this way, including a validation set, which is reused by design.

A puzzle at the heart of machine learning

Real machine learning research looks exactly like the iterative improvement loop we just described. Everyone gauges performance using a handful of benchmark datasets that go unrevised for years. The research community repeats an enormous, distributed loop: evaluate a model on the benchmark, revise the training procedure, re-evaluate, publish, and let the next group eke out a little more improvement.

This is precisely the kind of hill-climbing against a held-out set that, by the textbook account, ought to produce rampant overfitting. By now, the leaderboards should be saturated with models that look great on the benchmark and mediocre everywhere else.

And yet that is not what happens. Studies that build entirely fresh test sets for old, heavily reused benchmarks have found that improvements largely transfer : on the new data, models demonstrate the same gains they did on the old benchmark. Benchmark-driven machine learning, against the textbook's prediction, has produced rapid and largely real progress. Why?

There is no shortage of hypotheses, but they have been hard to test empirically, because the "subject" of the experiment is the entire human research community. You cannot reset a field, wipe its memory, and rerun the last decade under controlled conditions.

But we can do something similar. We now have capable, LLM-based research agents that can autonomously run the same machine-learning optimization loops that human communities run. They engage in the same benchmark hill-climbing — and, intriguingly, they too seem not to overfit. The difference is that an agent, unlike a research community, is something you can reset. You can clear its memory, control exactly what information it sees, and run the experiment again. In a recent paper, "What fits (into few tokens) doesn't overfit: Compression and generalization in ML research agents", we do exactly that — and in the process offer a concrete explanation for the long-standing mystery.

Occam's razor, made precise

The explanation begins with a very old idea. Occam's razor says that among hypotheses that explain the data equally well, the simpler one is more likely to be correct. It turns out this intuition has a precise mathematical form, and it is what underlies the whole story.

Suppose you can describe your hypothesis — your model, your strategy — in a small number of bits, far fewer than it would take to memorize the training data. If that compact hypothesis performs very well on the training data, it must also perform well on new data.

Occam's razor, formalized: among hypotheses that explain the data equally well, the simpler one — describable in fewer bits — is more likely to generalize to new examples.

The reasoning runs through a counting argument. There simply are not very many short descriptions, because there are not very many short strings. The fewer candidate hypotheses there are, the less likely it is that any one of them fooled you on the training set by luck — even though you used the training set to guide your search.

Another way to get the intuition: if your compressed description is too small to secretly record the training data, then when it performs well on the training data, it cannot be because it memorized the answers — it didn't have space to do that. It must be because it captured something true about the data's structure. Short descriptions cannot cheat because there isn't room.

Here is an attractive hypothesis: successful machine learning strategies are highly compressible. A researcher might stare at thousands of benchmark scores over the course of a project, but the strategy that ultimately survives is usually a short list of familiar choices — an architecture family, an optimizer, a learning-rate schedule, a data-handling recipe, a regularization scheme. If that final recipe can be communicated in just a few bits, then the model's true dependence on the benchmark is far smaller than the long, winding transcript of experiments would suggest. The hill-climbing was extensive, but the thing that came out the other end was — or could have been — tiny.

Compression, intelligence, and the power of a knowledgeable listener

Imagine trying to explain a specific machine learning pipeline to a bright high-school student, in enough detail that they could actually reproduce it. It would be a long, laborious conversation. You would have to explain what gradient descent is, what a neural network is, what PyTorch or JAX or TensorFlow does, what a learning rate is, and on and on. Almost none of that is specific to your problem; it is general background about how machine learning works.

Now imagine explaining the same pipeline to an expert ML engineer. The conversation now collapses to a few sentences. You skip everything that counts as common knowledge and communicate only what is genuinely specific to this problem: the architecture choice, the batch size, the optimizer, a couple of hyperparameters. The more your listener already knows about the world, the shorter the message you need to send — and the more aggressively you can compress. None of this "world knowledge" counts against you in the Occam's-razor argument, because you could have written all of that down without having looked at the training set.

This is where large language models enter the picture. Modern LLMs carry an enormous amount of world knowledge. They know how ML tooling works; they know the standard optimization algorithms; they know the conventional hyperparameter choices and the common defaults. If a detail is left unspecified, they can fill in a plausible value. That makes them extraordinarily good compression decoders : hand an LLM a terse, expert-to-expert message, and it can unpack it into a full, working procedure. If you think about it, this is exactly why they are so powerful.

The experiment: Squeezing a strategy through a bottleneck

This suggests a clean experiment. Have an ML research agent — the explorer — try to solve a new machine learning problem. Give it full access to a validation set and let it experiment and iterate freely, chasing better validation performance over hundreds of rounds. Here the validation set plays the role of the benchmark: a reusable holdout the agent queries again and again. This is the hill-climbing loop that ought to overfit.

Then test how compressible the solution is. A second agent, the compressor , reads the entire transcript of the explorer's work and tries to distill the winning strategy into a very short prompt — just a handful of tokens. That prompt is handed to a third agent, the reproducer , which must implement the strategy from scratch using only the prompt and the training data. Critically, the reproducer has no access to the validation set, the explorer's code, or its transcript. The short prompt is the only channel through which anything learned from the validation set can reach it. (In the study we report in our paper, the compressor and reproducer are both Claude models.)

If the reproducer — starting cold, armed only with a few tokens — matches the explorer's performance, then all the validation-dependent information needed to specify the strategy fit through that tiny channel. The strategy was compressible. We call this a certificate of output compression.

The setup has a very useful property that human research communities lack: the reproducer can be reset over and over. The compressor can try many different compressions and see how well each is decoded, because every attempt lands on a fresh reproducer with no memory of the last one. It is a little like the film Memento — you are leaving a terse note for a version of yourself whose memory will be wiped before reading it. You learn to write notes that a knowledgeable but amnesiac copy of you can act on; those notes can be very short because the receiver will fill in anything you leave unsaid exactly as you would have.

In the researchers' experiments, an explorer agent's strategy is squeezed through a narrow information bottleneck. Whatever survives compression must reflect real structure, not memorized data.

What comes out the other end

The compressions turn out to be remarkably small. Across eight datasets — spanning tabular classification, image classification, language modeling, diffusion modeling, and reward modeling — 32-token prompts were enough for a fresh reproducer to match the explorer's adaptively optimized models on the large majority of problems. One language-modeling strategy survived compression down to just 16 tokens with no loss in held-out performance.

What do these prompts actually look like? The most revealing examples are right at the border of conciseness where the compression almost breaks. In one language-modeling experiment, the explorer discovered a custom GPT-style training recipe. Under a 16-token budget, this was still enough for fresh reproducers to match the uncompressed explorer:

QKn 12L768 Mu .1 R² b2M 4x

To a human reader this looks cryptic, but to another ML agent it says something concrete: QKn means “QK normalization”, 12L768 means a 12-layer, 768-dimensional transformer, Mu .1 means the Muon optimizer with learning rate 0.1, R² means squared-ReLU activations, b2M means a two-million-token batch, and 4x means a fourfold feed-forward block. Cut the budget to eight tokens, however, and the prompt becomes

12L768 Mu .1 R²

Now the reproducer no longer matches the explorer. The missing pieces specified real training choices that were made as a function of the data and differ from the most obvious defaults. This boundary shows the limits of compressibility and is important. It shows that the reproducer is not succeeding from prior knowledge alone. A few compressed tokens are carrying genuine information learned from the data itself, and when those tokens disappear, so does the performance.

We also ran a set of experiments that imposed an information bottleneck from the other direction. Instead of compressing the explorer's output , we compressed its input : rather than telling the explorer each model's numerical validation score, we returned only a single bit — did this model beat the running best, or not? Even reduced to one bit of feedback per query, the explorer found strategies as good as those it found with full numerical scores. The channel between the validation set and the final strategy is narrow in both directions, and the one-bit version even comes with a rigorous mathematical guarantee on generalization.

Across eight datasets, strategies that emerged from hundreds of iterative experiments could be compressed into prompts as short as 16 to 32 tokens — small enough for a fresh agent with no memory to reproduce the original results.

Catching cheaters

A good empirical theory should be falsifiable — and this one is. If low overfitting is really explained by compressibility, then models that genuinely overfit should fail to be compressible via this pipeline.

To check, we deliberately pushed agents into overfitting by handing them direct validation-set access and prompting them to maximize validation performance at any cost. The agents took the bait: in 38 of 102 experimental runs, validation accuracy ran more than 10% ahead of true held-out accuracy.

The theory predicts that these gains should not survive the compression bottleneck, because they encode idiosyncrasies of specific validation examples, not transferable structure. Sure enough, when squeezed through a short prompt to a fresh reproducer, the validation-specific advantages vanished. Compression separated the legitimate strategies from the overfitting ones with very high accuracy.

So compression does not merely explain why autonomous research agents tend not to overfit but offers a tool for catching overfitting when it does occur, by flagging the cases where no short description can reproduce the result.

What this tells us — and what it doesn't

A few caveats are in order. The whole framework assumes that the only path from the validation data to the final model runs through the prompt we feed the reproducer. Of course, if a model had memorized the validation data during pretraining, it would have a side channel that bypasses the information bottleneck we are trying to impose. We don't think that is what is happening in our experiments: agents improve gradually through real search rather than starting at their best, and performance degrades at very short token budgets. But fully resolving this question will likely require experimenting with fresh datasets collected after a model's training cutoff, which we haven’t done.

Most importantly, our results are about LLM agents, because that is where the experiment is possible — where you can reset the subject, control its inputs, and count their length. But the picture they paint is strongly suggestive about human research communities too. When a field spends years climbing a fixed benchmark, and the gains keep transferring to fresh data, it may be for the same reason the agents' strategies survive a 32-token prompt: the recipes that actually work are simple. Or in other words, "What fits (into few tokens) doesn't overfit."

Acknowledgments: Steven Wu