🔶 用“梦境”为你的 AI Agent 创建可访问的记忆

Philip Dick 写过一篇短篇小说，题为《机器人会梦见电子羊吗？》。我稍微改动了这个标题，用于自己的一场演讲：《Agent 会梦见二进制羊吗？》。

Agent 会创建或能够访问不同类型的记忆。其中包括短期记忆或工作记忆（可以理解为在一个会话中存储信息），以及长期记忆（Agent 随时间了解某项任务或某个用户后积累的信息）。两者都很重要，但也都需要不同的管理方法。

我会在本周晚些时候发布的另一篇博客中讨论如何管理短期记忆。

对于长期记忆而言，一个关键组成部分——与人类一样——是能够对会话进行反思，并存储相关信息。对于人类，这通常发生在“梦境”中：我们的记忆系统会在其中进行重组和压缩。Agent 的“梦境”同样是这样一种反思过程，用于创建与会话及所发生信息有关的记忆。

没有记忆的 Agent，其能力只取决于我们提供给它的模型和指令。在许多情况下，Agent 会多次遇到相似的问题，尤其是在编程场景中。如果不了解先前的会话，Agent 每次都会以全新的方式解决问题，这会增加非确定性，也会使解决问题所消耗的 Token 数量达到最大。

记忆能够为 Agent 提供有关特定问题或用户的上下文，并据此影响 Agent 对下一步行动的决策。例如，如果 Agent 多次遇到某个特定问题，并且已经“做梦”后存储了某种特定的解决方法，那么它解决问题所需的时间和 Token 都可以大幅减少。此外，这也有助于降低长期运行中的结果差异。

我能给出的最简单类比，是学习加法的过程。在记忆中，你既会学习方法，也会学习完成工作的熟练套路。我们从小做了那么多加法表，以至于可以不去思考加法过程，直接算出 2+2。但当熟练套路不适用时，你也知道该如何进行加法。在“梦境”中，我们可以利用 LLM 提示词，让 Agent 同时获得这两个方面的学习能力。

在许多情况下，当前的“做梦”或分析场景，重点都放在改进 Agent 的代码上：也就是 Agent 可访问的指令、技能和工具。这对许多公司都极具价值；我常说，如果你没有收集公司所花费的每一个 Token，你就是在烧钱。这话有些极端，但如果无法访问会话数据，就不可能改进 Agentic 工作流。

“梦境”的另一种访问方法是使用 RAG（检索增强生成）。我们可以将有关某个意图、用户或任务的信息存储在数据库中。随后，当 Agent 获得一个意图时，它可以通过 RAG 搜索，查看自己是否此前已经发现过这个问题，并按需获取提示。

最简单的例子是对用户信息进行压缩。例如，某个用户可能总是询问 Python 相关问题。我们可以查询该用户的信息，注入其对 Python 的偏好，并获得更好的搜索结果。我们也可以利用“梦境”理解常见意图，甚至预先填入可能相关的代码，或某个特定函数的技能信息。

我构建了一个使用“梦境”的演示项目：它使用 Google ADK 作为 Agent 框架，使用 Gemini（当然你也可以选择任何模型）进行推理，并将 BigQuery 和 Cloud Spanner 用作记忆系统。Cloud Run 用于部署 Agent，并部署了两个版本：一个可访问记忆，另一个则不能。

我发现，借助 vibe coding，在前端展示演示内容会容易得多。如果你部署了代码，就可以按照 UI 步骤操作，比起在不同面板之间来回点击，更容易看到每一个阶段。

记忆系统的工作方式如下：

1. Agent 框架将有关 Agent 会话的全部遥测数据记录到 BigQuery。我们也将其称为轨迹（trajectory）。

2. 我们使用 AI 函数（LLM）在 BigQuery 中对单个会话或跨会话的信息进行总结，作为离线流程来创建记忆。

3. 记忆以向量形式存储在 Spanner 中。根据 Agent 需要访问记忆的不同方式，我们可能会用多个向量存储同一条记忆。

在代码示例中，Agent 会询问你的编程偏好，以及你通常可能偏好的其他事项。“做梦”过程会提取这些信息，并将其存储为 Spanner 可以访问的 Schema。

这一模式具有持久性。通过修改用于“做梦”的提示词和 SQL，我们可以跨会话查看其他属性或偏好，并将其存储以供检索，甚至可以使用输出结果来更新我们的 Agent 代码。

这是我创建的代码。你今天就可以部署它；如果你希望学习同时控制成本，也可以使用比 Spanner 更简单的服务。我已在部署到 Cloud Run 的容器 UI 中提供了分步说明。

欢迎告诉我你的想法！希望这能在你为自己的 Agent 构建“做梦”和记忆系统时提供帮助。

我如何使用 AI：本项目中的全部代码都使用了 AI，而文章文字部分则是 100% 的人类错误。一般来说，对于这样的编程示例，我会深度参与 AI 的使用，并在从 0 到 1 的项目流程中提供相当多的指导。

分享

Philip Dick wrote a short story titled ‘Do robots dream of electric sheep?’. I slightly changed this title for a talk I gave titled ‘Do agents dream of binary sheep?’.

There are different types of memory that an agent creates or has access to. These include short term or working memories (think of storing things in a session), and long term memories (things the agent learns about a task or a user over time). Both are important, but both also require different management methods.

I talk about managing short term memory in a separate blog I’ll be publishing later this week.

For long term memory, a critical component, like in humans, is the ability to reflect over a session and store the relevant information. In humans, this often happens in ‘dreams’ where our memory systems are reorganizing and compacting. Agent dreams are also this process of reflection to create memories about sessions and information that happened.

Agents without memory are only as good as the model and instructions we’ve given them. In many cases, agents see a similar problem many times, in particular for coding examples. Without understanding the prior sessions, agents will take a fresh approach to solving the problem, increasing non-determinism, and also maxing the amount of tokens used to solve the problem.

Memory helps to provide agents context on specific problems or users that inform how an agent may decide to take a next action. For example, if an agent sees a specific problem multiple times, and has ‘dreamed’ and stored that there is a specific way to solve the problem, this can dramatically reduce the time to solve the problem and tokens. In addition, this helps reduce variability over time.

The simplest adage I can give is for learning something like addition. In memories, you learn both methods and rote ways to accomplish work. We did so many addition tables growing up that you can solve 2+2 without thinking about the process of addition. But you also know how to add when the rote methods do not work. In dreams, we can use LLM prompts to give agents access to both sides of learning.

In many cases, current dreaming or analysis scenarios focus on improving the agent’s code: that is the instructions, skills, and tools the agent has access to. This is highly valuable for many companies; I often say that if you aren’t harvesting every token your company is spending, you’re burning money. This is extreme but there is no way to improve agentic flows if you don’t have access to the session data.

Another access method for dreams uses RAG (retrieval augmented generation). We can store information about an intent, a user, or a task in a database. Then, when an agent is given an intent, it can search using RAG to see if it has discovered the problem before, and get hints on demand. The simplest example is compaction about user information. For example, a user may always ask about Python questions. We can look up the information about the user, inject the Python preference, and get a better search result. We can also use dreams to understand common intents and even pre-seed code that might be relevant, or skill information for a specific function.

I built a demonstration using dreams that uses Google ADK as the agent framework, Gemini (although you could pick any model) for inference, and then BigQuery and Cloud Spanner for the memory system. Cloud Run is used for agent deployments, and there are two: one with memory access, and one without.

I’ve found that with vibe coding it makes it a lot easier to show demos on the frontend. If you deploy the code, you can follow the UI steps and see each stage much easier than clicking between panes.

The memory system works like this:

1. The agent framework logs all the telemetry about the agent sessions to BigQuery. We also call this a trajectory.

2. We use AI functions (LLMs) to summarize information in a session or across sessions in BigQuery as an offline process to create memories.

3. Memories are stored in Spanner as vectors. We may store the same memory with multiple vectors depending on the ways that the agent needs to access it.

In the code example, the agent will ask you about your preferences for coding, and other things that you may typically prefer. The ‘dreaming’ takes this information and stores it in a schema that is accessible for Spanner.

This pattern is a durable one. By modifying the prompt used for dreaming and the SQL, we could look across sessions for other attributes or preferences, and store them for retrieval, or even use the output to update our agent code.

Here’s the code that I created. You can deploy it today, and you can also use a simpler service than Spanner if you are looking to learn but keep your costs down. I’ve included step by step instructions in the UI deployed as a container to Cloud Run.

Let me know your thoughts! Hope this is helpful as you build an agent dreaming and memory system for your own agents.

How I used AI: All for code in this, the writing is 100% human errors. Generally with AI I’m very in the loop for a coding example like this, and provide pretty heavy direction in the flow for a 0 → 1 project.

Share