你的 Agent 实际需要多少记忆？

在我们的上一篇文章中，我们比较了 ALTK-Evolve 与 ACE，并展示了如何向 Agent 提供其自蒸馏指南——每项任务检索少量指南，或注入整套指南——会同时影响准确率和成本。本文退一步，讨论在此之前的问题：应该给它多少？

为 Agent 配备智能体记忆听起来很简单：从其过去的工作中蒸馏经验教训，将这些经验重新放入上下文，经验越多，性能理应越好。但实际并不总是如此。当我们将评测扩展到 8 个模型——从 300 亿参数的稠密模型到前沿闭源系统——有一个发现尤为突出：

智能体记忆不是一个可以直接打开的功能，而是一剂需要针对模型校准的“剂量”。

TL;DR

• ALTK-Evolve 让 Agent 从自身过去的轨迹中学习：蒸馏出可复用的指南，并在推理时将其注入上下文，无需更新权重，也无需人工标注。

• **正确剂量因模型层级而异：**有能力余量的强模型需要完整指南集；较弱模型在使用紧凑的核心指南加按任务检索时表现最佳；而已饱和的模型则没有显示出可衡量的增益。

• **精选检索可以同时是准确率最高且成本最低的方案：**gpt-oss-120b 的任务完成率提升了 +16.1 个百分点，而 Token 仅增加 +5%；同时，Prompt 缓存让生产环境中使用完整指南集也保持可负担。

---

核心洞见：剂量取决于能力

并非每个模型都能从同等数量的记忆中受益。在覆盖不同能力区间的 8 个模型中，我们观察到三种反复出现的模式：

• 有能力余量的强模型需要完整的指南集——包括每一条指南，以及罕见边界情形的经验。它们有能力吸收并应用全部内容。DeepSeek-V3.2（671B MoE）在获得完整的自挖掘指南集后，任务完成率提升了 +9.5 个百分点。

• **规模较小或能力较弱的模型会被庞大的指南集淹没。**对这些模型而言，精炼且高置信度的核心指南，加上每项任务检索到的少数相关指南，效果最好。gpt-oss-120b（117B MoE）通过这一选择性方案获得了 +16.1pp 的提升——而完整指南集带来的提升更小，且 Token 成本高出约 50%。

• **已饱和的模型没有显示出可衡量的增益。**我们称之为饱和模式——这一标签描述的是我们的观察结果，并非已经得到证实的成因。模型可能已经接近在这些任务上的性能上限，指南可能没有覆盖其剩余失败模式，或者模型没有有效应用这些指导。在我们的实验中，GLM-5（745B MoE）属于这一类。

_决定模型落入哪一种模式的因素，并不只是参数量。_基准测试的能力余量、上下文窗口大小、模型架构、指南质量以及任务分布，似乎都会影响模型所处的位置；区分这些因素仍是正在进行的工作。无论如何，实践结论依然成立：正确的记忆剂量取决于模型，而且可以进行校准。

---

学习发生在模型周围，而不是模型内部

这里的“记忆”并不是重放过去的对话记录，而是一套指南集——由 Agent 自身过往轨迹蒸馏而来的有效策略、应避免的错误和边界情形。其流程很直接：

1. Agent 尝试完成任务并产生轨迹。

2. ALTK-Evolve 从成功和失败的运行记录中提取行为指南。

3. 它将这些指南整合为一套可复用的集合。

4. 在推理时，Agent 会获得完整指南集，或其中与当前任务相关的精选内容。

不会更新任何模型权重。这个学习循环改变的是_可供 Agent 使用的指导信息_，而不是底层模型——这也正是其采用成本低、且能在我们测试的 8 个模型之间迁移的原因。

---

跨能力区间的结果

我们在 AppWorld 上进行了评测——该基准涵盖 9 个模拟应用（日历、消息、支付等）中的 585 个多步骤任务（168 个 test_normal + 417 个 test_challenge）。任务采用两种方式评分：Agent 是否完整完成每项任务（TGC — Task Goal Completion，任务目标完成率），以及某个场景的_每一个_变体是否都通过（SGC — Scenario Goal Completion，场景目标完成率，这是更严格的全有或全无标准）。完整定义见附录。

我们比较的三种配置

因为任何记忆研究中最容易令人困惑的问题，就是_上下文窗口中实际放了什么内容_，所以我们先定义这些配置。

两种记忆配置均使用同一套指南集：仅从 AppWorld 的训练集划分中，通过上述循环挖掘一次。两者之间变化的只是_这一套指南的交付方式_：完整指南集会在每一步注入全部内容，精选检索则提供经过选择的子集；指南的生成方式从未改变，也绝不使用任何测试集数据来构建它。

配置	Agent 上下文中的内容
基线	无记忆——按原始状态提供的 Agent。
完整指南集	每一条挖掘出的指南，在每一个 ReAct 步骤注入。
精选检索	来自同一指南集的固定高置信度核心内容，加上为每项任务检索出的少数相关指南（固定部分 + 可变部分）。

一个模型能够挖掘出的指南数量取决于它自身的能力，因此我们按策略报告配置——“完整指南集”与“精选检索”——而不是按无法在模型间直接比较的原始数量报告。

三种模式，一图概览

从 8 个模型的扫描评测中选取代表性模型，按 test_normal 上的任务完成率（TGC）衡量：

图 1. 三种观察到的模式中的代表性模型。柱状图展示 AppWorld test_normal 上基线与最佳记忆配置的 TGC；为使差异更清晰，x 轴从 40% 开始。仅看 TGC 会低估幅度更大的 SGC 提升——请参见下表中的 SGC 列。

为保持图表可读性，图中仅绘制 TGC；下表补充了通常增幅更大的更严格指标 SGC：

模型	模式	基线 TGC / SGC	最佳记忆 TGC / SGC	最佳配置	Δ TGC	Δ SGC
gpt-oss-120b（117B MoE）	弱 / 选择性方案	39.9 / 21.4	56.0 / 37.5	精选检索	+16.1	+16.1
DeepSeek-V3.2（671B MoE）	有能力余量的强模型	79.8 / 64.3	89.3 / 80.4	完整指南集	+9.5	+16.1
Claude Opus 4.6	有能力余量的强模型	90.5 / 87.5	94.6 / 94.6	完整指南集	+4.1	+7.1
GPT-5.5	强模型（接近上限）	92.3 / 82.1	95.2 / 89.3	完整指南集	+2.9	+7.2
GLM-5（745B MoE）	饱和	87.5 / 80.4	87.5 / 80.4	完整指南集	0.0	0.0

从 SGC 列可以看出，这一更严格的指标通常比 TGC 的变化更大——DeepSeek 的 SGC 提升 +16.1pp，而 TGC 提升为 +9.5pp——因为优秀的指南尤其有助于 Agent 通过场景的_每一个_变体，而不仅是应对平均情况。这种效应在能力范围高端也没有消失：TGC 均已接近上限的 GPT-5.5 和 Opus，其 SGC 仍分别提升 +7.2 和 +7.1pp。只要模型仍有可被针对的失败模式，记忆就会持续带来收益。

---

最便宜的记忆策略也可能是最佳策略

一个实际问题是：注入完整指南集会增加每个 ReAct 步骤的输入量，因为这些指南会在每轮重新发送。以下是我们观察到的结果：

模型	配置	Token/任务（基线）	Token/任务（+ 记忆）	额外开销
DeepSeek-V3.2	完整指南集	148K	263K	+78%
gpt-oss-120b	完整指南集	110K	166K	+51%
gpt-oss-120b	精选检索	110K	116K	+5%

表 1. 每项任务的平均 Token 使用量，跨 Agent 步骤累计计算，并以无记忆基线为参照。

有两项要点：

1. 精选检索使成本接近基线。对于选择性方案在准确率上取胜的弱模型而言，它在成本上同样更优——两全其美（gpt-oss-120b 的 TGC 提升 +16.1pp，而 Token 仅增加 +5%）。此处的更高性能不需要更高的推理成本。

2. **记忆不会让推理循环失控膨胀。**DeepSeek 在有无记忆时运行的 ReAct 步骤数大致相同（平均约 18–19 步），因此额外成本来自输入 Token 膨胀，而不是轨迹变得更长。

生产环境中真正的效率杠杆是 Prompt 缓存：指南集的静态部分在不同步骤间完全相同，因此可以被缓存，从而显著降低实际成本。面向缓存的 Prompt 设计——保持共享的指南集前缀稳定，使其能够持续缓存——值得专门投入工程工作。我们还假设，上下文窗口大小也会发挥作用：拥有更大窗口的模型可能更有效地吸收完整指南集，而上下文较小的模型则更受益于让注入内容保持紧凑的检索方案。我们尚未进行能够单独隔离这一因素的受控实验。

---

记忆应该校准，而不只是累积

教训并不是把 Agent 学到的一切都交给它，而是给它实际能够利用的那部分经验。

• 对于弱模型，这意味着紧凑的核心指南加上少数任务特定经验——巧合的是，这也是成本最低的选项。

• 对于有能力余量的强模型，这意味着保留完整指南集，并通过 Prompt 缓存在生产中维持可负担的成本。

• 对于已饱和的模型，这意味着在更好地理解其剩余失败模式之前，不要额外消耗上下文。

整体来看，这些增益是真实的——自动获得、没有数据泄漏、无需人工标注——但前提是记忆剂量适配模型。

---

下一步

这是一个起点，而非终点：

• **学习型选择器。**我们当前的检索方案按余弦相似度对指南进行排序，而我们已经证明，这种相似度无法完美预测哪些指南会帮助特定任务。利用结果信号训练选择器，是顺理成章的下一步。

• **面向极弱模型的记忆。**当模型能力低于某个最低基线时，自蒸馏缺乏有效信号。面向极弱模型的教师蒸馏记忆，是我们正在探索的另一个问题。

• **超越 AppWorld。**这些结果已在 AppWorld 上得到验证——这是一个严格的多步骤基准，但仍只是一个基准。我们正在推进更广泛的 Agent 基准测试和真实世界部署。

• **隔离上下文窗口因素。**如上所述，我们希望开展受控实验，将上下文窗口大小的影响与原始能力区分开来。

试用 ALTK-Evolve 库——其中包含本文所用的提取、整合和检索流水线——或者阅读完整技术报告，了解完整方法和消融实验。

---

附录：理解指标

AppWorld 任务通过两项指标评分，均以百分比报告（越高越好）：

• **TGC — Task Goal Completion（任务目标完成率）。**Agent 完整且正确完成的单项任务占比。这是核心的_“它是否完成了任务”_指标。

• SGC — Scenario Goal Completion（场景目标完成率）。一项更严格的全有或全无指标。每个_场景_捆绑同一任务的多个变体（即同一请求使用不同的数据、措辞或边界条件）。只有 Agent 在每一个变体上都成功，SGC 才将该场景计为通过。它衡量的是可靠性——一个大多数时候能够解决任务、却在某个变体上失败的 Agent，会在 TGC 中得分，但不会在 SGC 中得分。

How Much Memory Does Your Agent Actually Need?

In our

previous post

, we compared

ALTK-Evolve

with

ACE

and showed that

how

you deliver an agent's self-distilled guidelines — a few retrieved per task vs. the whole set injected — drives both accuracy and cost. This post steps back to the question that comes before it:

how much

should you give it?

Equipping an agent with agentic memory sounds simple: distill lessons from its past work, put them back in context, and more experience should mean better performance. It doesn't always work that way. When we scaled the evaluation to eight models — from a 30B dense model to frontier proprietary systems — one finding stood out:

Agentic memory is not a feature you switch on. It's a dose you calibrate to the model.

TL;DR

• ALTK-Evolve lets an agent learn from its own past trajectories: distilling reusable guidelines and injecting them back at inference time, with no weight updates and no human annotation.

• The right dose differs by model tier: strong models with headroom want the full guideline set, weaker models do best with a compact core plus per-task retrieval, and saturated models show no measurable gain.

• Curated retrieval can be both the most accurate and the cheapest option: gpt-oss-120b gained +16.1pp task completion at only +5% tokens — and prompt caching keeps even the full guideline set affordable in production.

---

The Key Insight: Dosage Depends on Capability

Not every model benefits from the same amount of memory. Across eight models spanning the capability spectrum, we saw three recurring patterns:

• Strong models with headroom want the full guideline set — every guideline, including rare edge-case lessons. They have the capacity to absorb and apply all of it. DeepSeek-V3.2 (671B MoE) climbed +9.5 percentage points in task completion when given its full self-mined guideline set.

• Smaller or weaker models get drowned by a large guideline set. For these, a tight, high-confidence core plus a handful of task-relevant guidelines retrieved per task works best. gpt-oss-120b (117B MoE) gained +16.1pp with this selective approach — while the full guideline set gained less and cost ~50% more tokens.

• Already-saturated models show no measurable gain. We call this the saturated pattern — the label describes what we observed, not a proven cause. The model may already have been near its ceiling on these tasks, the guidelines may not have addressed its remaining failures, or it may not have applied the guidance effectively. GLM-5 (745B MoE) sat here in our runs.

What puts a model into one pattern rather than another isn't simply parameter count. Benchmark headroom, context-window size, architecture, guideline quality, and task distribution all appear to shape where a model lands, and separating those factors is ongoing work. The practical takeaway holds either way: the right dose of memory depends on the model, and we can calibrate it.

---

Learning happens around the model, not inside it

"Memory" here doesn't mean replaying a past transcript. It means a guideline set — strategies that worked, mistakes to avoid, and edge cases — distilled from the agent's own prior trajectories. The loop is straightforward:

1. The agent attempts tasks and produces trajectories.

2. ALTK-Evolve extracts behavioral guidelines from both its successful and unsuccessful runs.

3. It consolidates those guidelines into a reusable set.

4. At inference time, the agent receives either the full guideline set or a task-relevant selection of it.

No model weights are updated. The learning loop changes the guidance available to the agent , not the underlying model — which is exactly why it's cheap to adopt and portable across the eight models we tested.

---

Results Across the Spectrum

We evaluated on AppWorld — 585 multi-step tasks (168 test_normal + 417 test_challenge) across 9 simulated apps (calendars, messaging, payments, and so on). Tasks are scored two ways: whether the agent fully completes each task (TGC — Task Goal Completion) and whether every variant of a scenario passes (SGC — Scenario Goal Completion , a stricter, all-or-nothing bar). Full definitions are in the appendix.

The three configurations we compare

Because the confusing part of any memory study is what's actually in the context window , we define the configurations up front.

Both memory configurations draw from the same guideline set , mined once (via the loop above) from AppWorld's training split only. What changes between them is only how that one set is delivered — the full guideline set injects all of it every step, while curated retrieval delivers a selected subset — never how the guidelines were produced, and no test-split data ever goes into building it.

Configuration	What's in the agent's context
Baseline	No memory — the agent as shipped.
Full guideline set	Every mined guideline, injected on every ReAct step.
Curated retrieval	A fixed, high-confidence core of those same guidelines plus a few task-relevant ones retrieved for each task (a fixed portion + a variable portion).

The number of guidelines a model mines depends on its own capability, so we report configurations by strategy — "full guideline set" vs. "curated retrieval" — rather than by raw counts, which aren't comparable across models.

The three patterns, in one view

Representative models from the eight-model sweep, measured by task completion (TGC) on test_normal:

Figure 1. Representative models in the three observed patterns. Bars show TGC on AppWorldtest_normal for baseline vs. the best-memory configuration; the x-axis begins at 40% to make differences visible. TGC alone understates the larger SGC gains — see the SGC columns in the table below.

The figure plots TGC to keep it readable; the table adds the stricter SGC metric, where the gains are often larger:

Model	Pattern	Baseline TGC / SGC	Best-memory TGC / SGC	Best config	Δ TGC	Δ SGC
gpt-oss-120b (117B MoE)	Weak / selective	39.9 / 21.4	56.0 / 37.5	curated retrieval	+16.1	+16.1
DeepSeek-V3.2 (671B MoE)	Strong w/ headroom	79.8 / 64.3	89.3 / 80.4	full guideline set	+9.5	+16.1
Claude Opus 4.6	Strong w/ headroom	90.5 / 87.5	94.6 / 94.6	full guideline set	+4.1	+7.1
GPT-5.5	Strong (near-ceiling)	92.3 / 82.1	95.2 / 89.3	full guideline set	+2.9	+7.2
GLM-5 (745B MoE)	Saturated	87.5 / 80.4	87.5 / 80.4	full guideline set	0.0	0.0

Reading the SGC column, the stricter metric usually moves more than TGC — DeepSeek's SGC jumps +16.1pp against a +9.5pp TGC gain — because good guidelines especially help an agent clear every variant of a scenario, not just the average case. And the effect doesn't disappear at the top of the range: GPT-5.5 and Opus, both near the ceiling on TGC, still gain +7.2 and +7.1pp SGC respectively. Memory keeps paying off as long as a model has a remaining failure mode to target.

---

The Cheapest Memory Strategy Can Also Be the Best

A practical concern: injecting a full guideline set inflates every ReAct step's input, because the guidelines are re-sent each turn. Here's what we observed:

Model	Config	Tokens/task (baseline)	Tokens/task (+ memory)	Overhead
DeepSeek-V3.2	full guideline set	148K	263K	+78%
gpt-oss-120b	full guideline set	110K	166K	+51%
gpt-oss-120b	curated retrieval	110K	116K	+5%

Table 1. Average token use per task, accumulated across agent steps, measured against the no-memory baseline.

Two takeaways:

1. Curated retrieval keeps cost near baseline. For weaker models, where selection wins on accuracy, it also wins on cost — the best of both worlds (+16.1pp TGC at only +5% tokens for gpt-oss-120b). Better performance here does not require more inference cost.

2. Memory doesn't blow up the reasoning loop. DeepSeek runs about the same number of ReAct steps with memory as without (≈18–19 on average), so the added cost is input-token inflation, not longer trajectories.

The real efficiency lever in production is prompt caching : the static portion of the guideline set is identical across steps and can be cached, cutting effective cost substantially. Cache-aware prompt design — keeping the shared guideline-set prefix stable so it stays cacheable — is worth engineering for. We also hypothesize that context-window size plays a role: models with larger windows may absorb the full guideline set more effectively, while smaller-context models benefit more from retrieval that keeps injected content compact. We have not yet run controlled experiments isolating this factor.

---

Memory Should Be Calibrated, Not Merely Accumulated

The lesson isn't to give an agent everything it has learned. It's to give it the amount of experience it can actually use.

• For weak models , that means a compact core plus a few task-specific lessons — which, conveniently, is also the cheapest option.

• For strong models with headroom , it means preserving the full guideline set, kept affordable in production via prompt caching.

• For saturated models , it means spending no extra context until their remaining failure modes are better understood.

The gains are real across the board — automatic, leakage-free, and requiring no human annotation — but only when the dose fits the model.

---

What's Next

This is a starting point, not the finish line:

• A learned selector. Our current retrieval ranks guidelines by cosine similarity, which we've shown doesn't perfectly predict which guidelines help a given task. A selector trained on outcome signal is the natural next step.

• Memory for very weak models. Below a minimum capability baseline, self-distillation lacks signal. Teacher-distilled memory for very weak models is a separate problem we're exploring.

• Beyond AppWorld. These results are validated on AppWorld — a rigorous multi-step benchmark, but a single one. Broader agent benchmarks and real-world deployments are in progress.

• Isolating context window. As above, we want controlled experiments that separate context-window size from raw capability.

Try theALTK-Evolve library — which includes the extraction, consolidation, and retrieval pipeline used here — or read thefull technical report for the complete method and ablations.

---

Appendix: Understanding the Metrics

AppWorld tasks are graded by two metrics, both reported as percentages (higher is better):

• TGC — Task Goal Completion. The share of individual tasks the agent completes fully and correctly. This is the headline "did it get the job done" number.

• SGC — Scenario Goal Completion. A stricter, all-or-nothing metric. Each scenario bundles several variants of the same task (the same request with different data, phrasing, or edge conditions). SGC counts a scenario as passing only if the agent succeeds on every variant. It measures reliability — an agent that solves a task most of the time but fails on one variant scores on TGC but not on SGC.