Kit For AI：大模型上下文窗口不是仓库，而是书桌——AI Agent 的记忆层新思路

非常好的问题 🙌

让我们豁然开朗的转变是：上下文窗口并非存储空间，而是一张书桌。你不会把整个文件柜都堆在书桌上——你只会取出当下需要的那两页纸。

因此，在一个大型项目中，Agent 永远不会加载整个代码库或全部文档。它会调用一个工具，仅获取排名最靠前的几个相关片段——经过重新排序并严格限制在固定的 token 预算内。一个 500 份文档的项目和一个 5 份文档的项目，发送给模型的 token 量大致相同。资料库会增长，但进入上下文的内容并不会随之膨胀。

保持这种精准切片的三件事：

→ 混合搜索（语义 + 精确术语），确保选中的是最正确的片段，而非仅仅是相近的片段

→ 在任何内容进入提示词前，先施加硬性的 token 预算

→ 当旧事实发生变化时，记忆会将其淘汰，从而避免缓慢腐化

说真的，难点不在于窗口大小，而在于检索质量。一个干净利落的 4k token 切片，在准确性和成本上都优于塞得满满的百万 token 窗口。

目前依然棘手的是那种一次性跨越整个项目的推理，这正是我们接下来要攻坚的方向。
你在构建什么？好奇你是否正面临这种情况 :)

Really good question 🙌

The shift that made this click for us: the context window isn't storage, it's a desk. You don't put the whole filing cabinet on your desk — you grab the two pages you need right now.

So on a big project, the agent never loads the codebase or the docs. It calls a tool, and gets back only the top few relevant chunks — reranked and capped to a fixed token budget. A 500-doc project and a 5-doc one send roughly the same amount to the model. The library grows, what hits the context doesn't.

Three things keep that slice sharp:

→ hybrid search (meaning + exact terms) so the right chunk wins, not just a nearby one

→ a hard token budget before anything reaches the prompt

→ memory that retires old facts when they change, so it doesn't slowly rot

Honestly the hard part isn't window size, it's retrieval quality. A clean 4k-token slice beats a stuffed 1M window on accuracy and cost.

Where it still gets tough is reasoning that spans the whole project at once that's what we're chasing next.
What are you building? Curious if it's that case 🙂