🔶 快照、写时复制与 Agent 沙盒的经济学

在第 1 部分中，我们为一个 Agent 构建了一台 Linux 机器。但如何启动一千台？或者一百万台？

在构建软件工厂 engrams 的过程中，我们意识到，运行数千个隔离 Agent 的经济成本会很快失控。

我们最终采用了一些相当巧妙的技术来尽量降低成本，接下来这一部分将介绍这些技术。

开始吧！

为什么“微型”VM 会变得昂贵

一开始，作为我们首选 VMM 的 Firecracker 让这件事看起来成本很低。它开销极小、启动迅速，并且仅暴露一组最精简的虚拟设备。

……直到我们在其中运行真正的 Agent！

一旦 Agent 启动编译器、语言服务器、浏览器和构建守护进程，Guest 就不再那么“微型”了。无论 Firecracker 进程刚启动时有多小，一个 4 GiB 的 Guest 都可能将数 GiB 页面拉入宿主机 RAM。

我们在磁盘上遇到了同样的问题。如果多数 Agent 只改动少量文件，那么为每个 Agent 复制一个 40 GiB 的 ext4 镜像将十分荒谬。

最小化 microVM ≠ 最小化工作负载。

北极星指标

对于 engrams，我们有两个彼此存在张力的目标：

1. 我们希望最小化成本
2. 我们希望最大化可同时存在的沙盒数量

先谈成本。

为简化讨论，我们将集群成本视为集群节点数量（n）对应的 CPU、内存和磁盘小时数的线性组合：

cost = t(α * vCPU + β * RAM + γ * Disk) * n

将沙盒放置到宿主节点上同样很简单：每台宿主机的 vCPU、RAM 和磁盘都是有限的，而每个沙盒会请求这三类资源中的一定数量。

我制作了一个交互式可视化来说明这个问题：在集群中调度“工作负载”，看看我们会多快耗尽集群资源并开始产生成本：

节点装箱（node_packing） / 自动扩缩容 ≤ 3 成本时钟运行中

自动扩缩容器1 / 3 个节点活跃

host_01，随着 Agent 工作负载被准入而逐渐填充。立方体的填充高度取决于利用率最高的资源。精确的 vCPU、内存和磁盘使用情况显示在相邻仪表中。host_01general_16 / 固定规格节点为空0%

宿主机容量16 · 64 · 400

内存0 GiB / 64 GiB

已分配 0%

磁盘0 GiB / 400 GiB

已分配 0%

运行中的工作负载0 个活跃

该集群中没有正在运行的 Agent。

集群成本**$0.00**

1 个节点 · $0.80/小时00 时 00 分 · 1 秒 = 1 小时

添加工作负载

__+ 编码 Agent2 vCPU · 8 GiB · 40 GiB __+ 构建 Agent4 vCPU · 16 GiB · 120 GiB

每个活跃节点都会为集群成本增加 $0.80/小时，无论该节点是空闲还是已满。

在集群饱和之前，我们只能运行 9 到 24 个并行 Agent！当然，你可以无限制地自动扩缩容，但成本会极快飙升。

不太理想，但我们可以做得更好。

为什么不在它们完成后直接销毁？

我们第一个显而易见的选择，是将这些沙盒视为临时资源。

我们可以给它们设置最长存活时间，选择一个 harness，或通过提示词将 Agent 设计为执行一次性任务，并在任务完成后销毁沙盒。

这会提供与 GitHub Actions 等 CI Runner 相同的约定：具有硬超时限制的有限任务。

但“完成”究竟意味着什么，往往并不明确：如果 Claude Code 停下来问我一个问题，这个会话会被回收吗？又或者，如果我希望在 11 天内将 Bun 代码库迁移到 Rust，这个任务是否应该在 24 小时后终止？

这种方法对 Stripe 有效，因为他们明确将其 Agent 设计为一次性任务。

而对于 engrams，我们希望完全不依赖特定 harness，并复刻在你自己的计算机上运行喜爱 harness 的体验：你可以在数月后回到一个会话，并从完全相同的状态继续工作。

比仅恢复对话更进一步的是：为什么我们不能恢复 Agent 的完整状态，包括其内存、磁盘和正在运行的进程？

对沙盒状态进行快照

我们认为，Agent 大部分时间都在等待用户。我们希望仅为并行沙盒资源付费，同时最大化可同时存在的沙盒数量。

那么，为什么不直接创建快照并将其拆除，以释放宿主机资源？我们可以捕获一个沙盒，将其拆除以释放宿主机，然后在用户回来时恢复它。

为实现这一点，我们需要捕获 Firecracker microVM 的三个部分：

1. CPU 和设备状态
2. Guest 内存
3. Guest 磁盘

Firecracker 已经可以捕获前两项。它将 vCPU、KVM 和模拟设备状态序列化到一个较小的不透明文件（state.bin）中，并将 Guest 内存写入单独的文件（mem.bin）。而磁盘本来就以文件形式存在，因此我们只需复制它。

这使我们得到如下捕获流程：

捕获 Firecracker 沙盒：Engrams 在一个一致的时间点暂停 microVM。Firecracker 将 CPU 和设备状态捕获至 state.bin，并将 Guest 内存捕获至 mem.bin；同时 Engrams 单独捕获 Guest 磁盘。这些部分被组合并作为快照存储，随后 VM 被销毁，从而归还宿主机容量。暂停 VM CPU +设备state.bin 内存mem.bin 磁盘完整快照组合单个工件记录至 Blob 存储销毁 VM容量释放 通过对沙盒状态快照来回收宿主机资源

要恢复一个沙盒，我们反向执行这一过程：

恢复 Firecracker 沙盒：Engrams 获取已存储的快照，并恢复其 CPU 和设备状态、Guest 内存与 Guest 磁盘。这些部分会被重新组装为一个 Firecracker microVM，让 Agent 从停止的位置继续运行。获取快照Blob 存储解包已保存的机器 CPU +设备state.bin 内存mem.bin 磁盘完整快照恢复 VM重新组装恢复 Agent执行继续 将存储的快照重新变为运行中的沙盒

我们起初采用了这种方法，作为概念验证时效果还算不错，但很快就遇到了瓶颈……组合快照实在太大了！

仅以一个请求 16 GiB RAM 和 128 GiB 磁盘的沙盒为例。在最坏情况下，我们需要在 Blob 存储之间传输约 144 GiB 数据；无论是从延迟还是维持长期沙盒的成本来看，这都不可接受。

内容寻址存储 / 写时复制

我们很快意识到，一个 128 GiB 虚拟磁盘并不一定包含 128 GiB 的唯一数据。

其中大部分来自原始镜像：操作系统、编译器、包缓存，以及沙盒启动前安装的其他内容。由该镜像创建的每个沙盒都从完全相同的字节开始。即使 Agent 已经工作了一段时间，它通常也只会修改磁盘中的一小部分。

我们正在重复所有这些工作，并为其支付存储和带宽成本！我们需要一种更好的方式来存储和传输磁盘状态。

我们最终决定将磁盘拆分为固定大小的 16 MiB 块。

我们使用 SHA-256 对每个块进行哈希，并将哈希值作为其在存储中的地址。产生相同哈希的相同块只会被存储一次。完全为空的（全零）块会被跳过。

然后，我们通过一个小型清单记录这些块的组合方式：

offset        chunk
0 MiB         sha256:a8f1...
16 MiB        sha256:4c2e...
32 MiB        sha256:91bd...
...           ...

这样，一个 128 GiB 磁盘只是一份偏移量和哈希列表。这些块不可变，因此从同一镜像创建的每个沙盒都可以共享它们。

创建沙盒时，我们为它提供一个指向基础镜像块的清单。我们复制的是一个元数据对象，而不是 128 GiB 的磁盘。

我们使用 Linux 的*网络块设备（Network Block Device，NBD）*协议，将内容寻址存储与 Firecracker 连接起来。NBD 允许用户空间进程为 Linux 块设备提供读写服务。

我们编写了一个小型 NBD 服务器，将读写操作转换为块获取和上传操作。我们将它连接到 Firecracker，Guest 仍然看到一块普通的 ext4 磁盘，但实际上是我们的服务器在幕后完成转换：

读取时，NBD 服务器会将字节偏移量转换为清单中的块。如果可能，它从本地 NVMe 提供该块；若未命中，则从 Blob 存储获取。我们将高速 NVMe 用作热块的 LRU 缓存，充分利用其速度；缓存绝不超过宿主机容量，并在需要时回退至较慢的 Blob 存储，而非直接失败。

我们不再在启动前组装整个磁盘——只获取沙盒实际读取的内容。

通过 NBD 读取块：Guest 向 NBD 块设备发起普通 ext4 读取。NBD 服务器通过会话清单解析字节偏移量，检查宿主机 NVMe 缓存，并且仅在缓存未命中时从 Blob 存储获取不可变块。

读取请求 offset 48–64 MiB

GUEST ext4 读取 普通磁盘 I/O

块设备 /dev/nbdN VM 所见的同一磁盘

清单 offset → hash 48 MiB → sha256:B

宿主机 NVME 块缓存 命中 → 立即返回

缓存未命中时 Blob 存储 获取不可变块 B

← chunk B 仅请求的 16 MiB 区域返回给 Guest

虚拟磁盘读取会先通过元数据解析。仅请求的块会被获取，而缓存命中永远不会访问 Blob 存储。

对于写入，我们增加了写时复制。首次写入不可变基础块时，会为该沙盒创建一个私有的脏副本，而其他所有沙盒均不受影响。

这意味着两个会话可以从相同的三个块开始，而会话 1 对块 B 的写入只会让该块发生分叉：

块级写时复制：基础镜像和两个会话最初均引用共享的不可变块 A、B 和 C。会话 01 向 B 写入后，只有其中间清单条目指向新的私有块 B 撇号（B prime）。基础镜像和会话 02 继续引用原始 B。不可变块存储 A B C B′ 基础镜像共享模板 A B C session_01 写入块 B A B′ C session_02 未变 A B C 写入 B → 新哈希不可变块存储 A B C B′ 基础镜像 A B C session_01 A B′ C session_02 A B C 一次写入 → 一个新块 三份清单，四个存储块。只有 session_01 的写入会创建新数据。

快照时，我们对脏块进行哈希并上传，然后发布会话清单的新版本。未触及的偏移量会继续引用共享基础块。即使 Guest 看到的是 128 GiB 磁盘，一个弄脏 20 个块的会话最多也只会增加 320 MiB 新磁盘数据。

这同时改善了成本和延迟：

• 挂起时，我们只存储一次基础磁盘，以及各会话的唯一变更；
• 恢复时，我们加载一份小型清单，并按需获取块，因此启动工作量取决于沙盒的工作集，而不是其对内宣称的磁盘容量。

这就从最坏情况下的 144 GiB 快照中移除了 128 GiB 磁盘。

但我们仍要处理一个 16 GiB 的 mem.bin！幸运的是，内存拥有同样有用的特性：大多数沙盒都从相同的字节开始，只有部分字节会随时间发生分歧。

使用 MAP_PRIVATE 和 userfaultfd 实现内存写时复制

至此，我们已将 128 GiB 磁盘从快照中移除。内存更难处理：它没有类似 NBD 的边界可让清单解析每一次读取。每条指令都可能访问它。

我们首先以 512 KiB 块为单位对内存进行内容寻址。这减少了存储重复，但在每次恢复时重建一个 16 GiB 文件，仍保留了大部分延迟和 RAM 成本。我们需要在 VM 运行期间让这些字节保持共享。

因此，我们为每台宿主机提供一个针对镜像规范内存的稀疏共享内存文件：base shm。我们使用 MAP_PRIVATE 将其映射到每个沙盒中，让干净页面共享物理 page-cache 后备，而写入操作则通过内核写时复制获得私有页面。

我们使用 Linux userfaultfd 按需填充 base shm，以保持其稀疏性。发生缺页异常时，我们会比较会话和镜像清单：

1. 对于规范块，我们填充 base shm，并通过 UFFDIO_CONTINUE 映射它，从而保持跨沙盒共享。
2. 对于已变更块，我们从会话快照中获取它，并使用 UFFDIO_COPY 以私有方式安装它。

GUEST 内存后备UFFDIO_CONTINUE

规范缺页Guest 写入恢复已变更页面

session_01session_02物理后备base shm私有

UFFDIO_CONTINUE两个沙盒均从同一个 base-shm page-cache 条目映射页面 B。

35% ΣPSS / ΣRSS三台 Guest · 33% 是理想共享比例

49 MiB 已实际分配4 GiB 表观 base shm · 首次金丝雀测试

规范页面共享一个物理后备；只有写入和恢复产生的分歧会变为私有。

我们本可以在所有地方都使用 UFFDIO_COPY，但那样每个恢复的页面都会变成私有页面。UFFDIO_CONTINUE 保留了我们构建 base shm 所要实现的共享能力。

我们在每次缺页时获取周边的 512 KiB 块以摊销 I/O，而写时复制仍保持页粒度。捕获时，我们保留未变更块的哈希，并为已变更块创建新的哈希。非活跃沙盒会变成一份清单加唯一块，而不再是另一个 16 GiB 文件。

经过这些努力，无论是活跃会话还是静态存放的快照会话，我们所支付的资源都只是工作负载的基础快照，再加上内存和磁盘的差异部分。

从资源密度到会话密度

这一方案生效后，我们可以重新审视最初的容量模型。它适用于 Agent 工作期间，但 Agent 很大一部分时间都在等待人类、网络调用或下一个任务。

我们赋予一个保留会话两种物理形态：

• 运行中：vCPU、私有内存页，以及活跃磁盘工作集
• 非活跃：一个小型 CPU/设备状态文件、两份清单，以及唯一的内存和磁盘块

18 个保留会话 · 6 个活跃会话3 个节点计费

保持 VM 运行将空闲会话快照化

保留会话*

In Part 1, we built one Linux machine for one agent. But how do we launch a thousand of them? Or a million of them?

As we built out our software factory, engrams, we realized that the economics of running thousands of isolated agents gets out of control pretty quickly.

We landed on some pretty clever techniques to minimize cost which we’ll cover in this next part.

Let’s get into it!

How “micro” VMs become expensive

At first, Firecracker, our VMM of choice, made this look cheap. It has very little overhead, boots quickly, and exposes only a minimal set of virtual devices.

…Until we ran real agents inside it!

Once an agent started a compiler, language server, browser, and build daemon, the guest stopped looking so micro. A 4 GiB guest could pull several GiB of pages into host RAM regardless of how small the Firecracker process was at startup.

We hit the same problem with disk. Copying a 40 GiB ext4 image for every agent would be absurd when most changed only a few files.

A minimal microVM ≠ a minimal workload.

North Star

With engrams we have two competing goals in tension with one another:

1. We want to minimize cost
2. We want to maximize the number of possible concurrent sandboxes

Let’s talk about cost first.

As a simplification, let’s think of fleet cost as a linear combination of CPU, memory, and disk hours for the number of fleet nodes (n):

cost = t(α * vCPU + β * RAM + γ * Disk) * n

Sandbox placement onto host nodes is similarly simple: each host has finite vCPU, RAM, and disk, and each sandbox requests some amount of all three.

I’ve whipped up an interactive visualization to illustrate the problem: schedule “workloads” on the fleet and see how quickly we exhaust the fleet and start to incur costs:

node_packing / autoscale ≤ 3 __cost clock running

autoscaler1 / 3 nodes active

host_01, filling as agent workloads are admittedThe cube's fill height follows the most utilized resource. Exact vCPU, memory, and disk usage appears in the adjacent meters.host_01general_16 / fixed sizeNODE EMPTY0%

host capacity16 · 64 · 400

Memory0 GiB / 64 GiB

0% allocated

Disk0 GiB / 400 GiB

0% allocated

running workloads00 active

No agents running in this fleet.

fleet cost**$0.00**

1 node · $0.80/hour00h 00m · 1 sec = 1 hour

add workload

__+ Coding agent2 vCPU · 8 GiB · 40 GiB __+ Build agent4 vCPU · 16 GiB · 120 GiB

Every active node adds $0.80/hour to the fleet cost, whether it is empty or full.

We get anywhere between 9 and 24 parallel agents running until we saturate the fleet! You can of course autoscale to oblivion, but cost explodes very quickly.

Not great, but we can do better.

Why don’t you just kill them when they’re done?

Our first obvious option was to treat these sandboxes as ephemeral.

We could give them a maximum lifetime, choose a harness or prompt engineer agents to one-shot tasks, and destroy the sandbox when the task completed.

That would give us the same contract as CI runners like GitHub Actions: finite tasks with hard timeouts.

But it’s not always clear what “done” even means: if Claude Code stops to ask me a question would the session be reaped? Or if I want to migrate the Bun codebase to Rust over the course of 11 days should that task die after 24 hours?

This approach worked for Stripe, because they explicitly designed their agents for one-shot tasks.

With engrams we wanted to be completely harness agnostic and mirror the experience of running your favorite harness on your own computer, where you can return to a session and pick up exactly where you left off months later.

Even better than just recovering the conversation, why can’t we recover the entire state of the agent, including its memory, disk, and running processes?

Snapshotting sandbox state

We figured that agents spent most of their time sitting around for the user. We wanted to only pay for parallel sandbox resources while still maximizing the number of possible concurrent sandboxes.

So why not just snapshot and tear it down to free host resources? We could capture a sandbox, tear it down to free the host, then restore it when the user returned.

To make that work, we needed to capture three parts of a Firecracker microVM:

1. CPU and device state
2. Guest memory
3. Guest disk

Firecracker already captures the first two. It serializes the vCPU, KVM, and emulated-device state into a small opaque file (state.bin) and writes guest memory into a separate file (mem.bin). And disk is already represented as a file, so we could just copy it.

That gave us a capture flow that looked like this:

Capturing a Firecracker sandbox Engrams pauses a microVM at one coherent point in time. Firecracker captures CPU and device state into state.bin and guest memory into mem.bin, while Engrams captures the guest disk separately. Those pieces are combined, stored as a snapshot, and the VM is destroyed to return host capacity. Pause VM CPU +devicesstate.bin Memorymem.bin Diskfull snapshot Combinesingle artifact Recordin blob store Destroy VMcapacity freed Reclaim host resources by snapshotting sandbox state

To resume a sandbox, we reverse the process:

Resuming a Firecracker sandbox Engrams fetches the stored snapshot and recovers its CPU and device state, guest memory, and guest disk. Those pieces are reassembled into a Firecracker microVM, which resumes the agent from the point where it stopped. Fetchsnapshotblob store Unpacksaved machine CPU +devicesstate.bin Memorymem.bin Diskfull snapshot Restore VMreassemble Resume agentexecution continues Turn a stored snapshot back into a running sandbox

We started with this approach which worked reasonably well for a proof of concept, but fell flat pretty quickly… The combined snapshots were MASSIVE!

Just take one sandbox that has 16 GiB of requested RAM and 128 GiB of requested disk. In the worst case we’re shuttling ~144 GiB of data in and out of blob storage, which is a non-starter in terms of latency and cost of keeping long-lived sandboxes.

Content-addressable storage / copy-on-write

We quickly realized that a 128 GiB virtual disk doesn’t necessarily contain 128 GiB of unique data.

Most of it comes from the original image: the operating system, compilers, package caches, and whatever else was installed before the sandbox booted. Every sandbox created from that image starts with exactly the same bytes. Even after an agent has been working for a while, it usually changes only a small fraction of the disk.

We were duplicating all this effort and paying for it in storage and bandwidth! We needed a better way to store and transfer disk state.

We settled on splitting the disk into fixed-size 16 MiB chunks.

We hash each chunk with SHA-256 and use the hash as its address in storage. Identical chunks that produce the same hash are stored once. We skip completely empty (zeroed) chunks.

Then we record how the chunks fit together in a small manifest:

offset        chunk
0 MiB         sha256:a8f1...
16 MiB        sha256:4c2e...
32 MiB        sha256:91bd...
...           ...

Now a 128 GiB disk is only a list of offsets and hashes. The chunks are immutable, so every sandbox from the same image can share them.

When we create a sandbox, we give it a manifest pointing at the base image’s chunks. We copy a metadata object, not 128 GiB of disk.

We bridged our content-addressed storage with Firecracker using Linux’s Network Block Device (NBD) protocol. NBD lets userspace processes serve reads and writes for a Linux block device.

We wrote a small NBD server that translates reads and writes into chunk fetches and uploads. We attached it to Firecracker, and the guest still saw an ordinary ext4 disk, but was secretly our server doing the translation behind the scenes:

On a read, our NBD server translates the byte offset into a manifest chunk. It serves the chunk from local NVMe when possible and fetches it from blob storage on a miss. We take advantage of fast NVMe by treating it as an LRU cache for hot chunks, never exceeding host capacity, and falling back to slow blob storage instead of outright failure.

We no longer assemble the whole disk before boot — we fetch only what the sandbox reads.

Reading a chunk through NBD The guest issues a normal ext4 read to an NBD block device. The NBD server resolves the byte offset through the session manifest, checks the host NVMe cache, and fetches the immutable chunk from blob storage only on a cache miss.

READ REQUEST offset 48–64 MiB

GUEST ext4 read ordinary disk I/O

BLOCK DEVICE /dev/nbdN same disk the VM sees

MANIFEST offset → hash 48 MiB → sha256:B

HOST NVME chunk cache hit → return immediately

ON CACHE MISS blob storage fetch immutable chunk B

← chunk B Only the requested 16 MiB region returns to the guest

A virtual-disk read resolves through metadata first. Only the requested chunk is fetched, and a cache hit never reaches blob storage.

For writes, we added copy-on-write. The first write to an immutable base chunk creates a private, dirty copy for that sandbox, leaving every other sandbox untouched.

That means two sessions can begin with the same three chunks, and a write to chunk B in session 1 would diverge only that chunk:

Chunk-level copy-on-write The base image and two sessions initially reference shared immutable chunks A, B, and C. After session 01 writes to B, only its middle manifest entry points to a new private chunk B prime. The base image and session 02 continue to reference the original B. IMMUTABLE CHUNK STORE A B C B′ base image shared template A B C session_01 wrote to chunk B A B′ C session_02 unchanged A B C WRITE B → NEW HASH IMMUTABLE CHUNK STORE A B C B′ base image A B C session_01 A B′ C session_02 A B C ONE WRITE → ONE NEW CHUNK Three manifests, four stored chunks. Only session_01's write creates new data.

When it’s time to snapshot, we hash and upload the dirty chunks, then publish a new version of the session’s manifest. Untouched offsets keep referencing the shared base chunks. A session that dirties twenty chunks adds at most 320 MiB of new disk data, even if the guest sees a 128 GiB disk.

That improved both cost and latency:

• On suspend, we store the base disk once plus each session’s unique changes
• On resume, we load a small manifest and fetch chunks on demand, so startup work is tied to the sandbox’s working set rather than its internally advertised disk size

That takes the 128 GiB disk out of our worst-case 144 GiB snapshot.

But we still have a 16 GiB mem.bin to deal with! Fortunately, memory has the same useful property: most sandboxes begin with the same bytes, and only some of those bytes diverge over time.

Copy-on-write memory with MAP_PRIVATE and userfaultfd

At this point we had taken the 128 GiB disk out of the snapshot. Memory was harder: it had no NBD boundary where a manifest could resolve each read. Every instruction could touch it.

We started by content-addressing memory in 512 KiB chunks. That deduplicated storage, but rebuilding a 16 GiB file on every resume kept most of the latency and RAM cost. We needed the bytes to stay shared while VMs ran.

So we gave each host one sparse shared-memory file for an image’s canonical memory: the base shm. We mapped it into each sandbox with MAP_PRIVATE, letting clean pages share physical page-cache backing while writes got private pages through kernel copy-on-write.

We keep the base shm sparse by filling it on demand with Linux userfaultfd. On a fault, we compare the session and image manifests:

1. For a canonical chunk, we populate the base shm and map it with UFFDIO_CONTINUE, preserving sharing across sandboxes.
2. For a changed chunk, we fetch it from the session snapshot and install it privately with UFFDIO_COPY.

GUEST MEMORY BACKINGUFFDIO_CONTINUE

Canonical faultGuest writeRestore changed page

session_01session_02PHYSICAL BACKINGbase shmprivate

UFFDIO_CONTINUEBoth sandboxes map page B from the same base-shm page-cache entry.

35% ΣPSS / ΣRSSthree guests · 33% is ideal sharing

49 MiB physically allocated4 GiB apparent base shm · first canary

Canonical pages share one physical backing; only writes and restored divergence become private.

We could have used UFFDIO_COPY everywhere, but every restored page would be private. UFFDIO_CONTINUE preserved the sharing we built the base shm for.

We fetch the surrounding 512 KiB chunk on each fault to amortize I/O, while copy-on-write stays page-granular. On capture, we keep the hashes of unchanged chunks and created new ones for changed chunks. An inactive sandbox becomes a manifest plus unique chunks, not another 16 GiB file.

Through all this effort, we’re paying for, both in active sessions, as well as snapshotted sessions at rest, only the base snapshots of workloads, plus memory + disk divergences only.

From resource density to session density

Once this worked, we could revisit our original capacity model. It applied while an agent worked, but agents spent much of their time waiting for people, network calls, or their next task.

We had given one retained session two physical shapes:

• Running: vCPU, private memory pages, and an active disk working set
• Inactive: a small CPU/device-state file, two manifests, and unique memory and disk chunks

18 RETAINED · 6 ACTIVE3 NODES BILLING

Keep VMs runningSnapshot idle

retained18 sessions

running18 microVMs

host cost**$2.39** per hour

node_01$0.80/h

010203040506

node_02$0.80/h

070809101112

node_03$0.80/h

131415161718

DURABLE SNAPSHOTSidle VMs still occupy hosts

············

__active __idle but resident __snapshotted

Total session count stays constant. Snapshotting lets host count follow active concurrency instead.

Now we could capture and evict idle sandboxes, then let the autoscaler remove empty hosts. We could size for active work, not every retained session.

And the economics of the fleet cost improved significantly! Thinking through where we started, we had:

cost ∝ α(requested vCPU) + β(requested RAM) + γ(requested disk)

Now it looks more like:

cost ∝ α(requested vCPU) + β(base shm + dirty RAM) + γ(base disk + dirty disk)