Compressor V2：三层压缩，为 LLM 智能体节省 50% 成本

为什么需要压缩？

代码智能体（Coding agents）已成为构建产品的基石。它们运行时间长、上下文繁重。例如，一个标准的 SWE-bench 编程任务可能消耗 30 到 100 轮 API 调用，每个任务 100 万到 1000 万个 Token 十分常见。这些数字乘以整个智能体集群、开发团队或 SaaS 产品后，账单眨眼间快速攀升。

经济压力是压缩的动机之一，但并非唯一。还有其他因素指向同一方向：

瓶颈	限制什么	为什么压缩有帮助
金额成本	单个任务支出；AI 产品的可持续单位经济学	更少（且正确的）Token → 更小的账单
延迟	多数服务商的首 Token 时间与前缀大小成正比	更小的前缀 → 更快的感知响应
上下文窗口	20 万或 100 万 Token 窗口在长会话中仍会被填满	压缩历史扩展有效视野
吞吐量	服务端批量大小与单次请求 Token 数成反比	更小的请求 → 服务商端更多并行；用户端更少排队延迟

因此，一个稳健的压缩层能够解决所有四个问题，并成为智能体工作流性能的关键组成部分。Compressor V2 正是这样一个压缩层，也是 Edgee AI 网关为满足这些需求而构建的组件。

从 V1 到 V2

Edgee 的 V1 压缩器于今年早些时候发布，使用了一种我们称之为“工具结果修剪”的单一策略。该策略受著名的 RTK 项目启发，旨在清理智能体在长会话中收集的冗长工具输出。它在真实的编程工作负载上大致节省了 10% 的成本——安全、简单，但范围有限。

V2 采取了不同的方法：三个正交的策略分层叠加，每个策略针对上下文的不同层面，并可按 API 密钥独立配置。由于这三个策略互不重叠，它们攻击不同的 Token 膨胀源头：

• 简洁性（Brevity） 攻击最昂贵的 Token 类别——输出 Token。
• 工具界面缩减（TSR） 攻击前缀中重复性最高的部分（MCP 工具目录）。
• 工具结果修剪 攻击对话中累积的长尾冗长工具输出。

通过网关，客户可以组合出适合其工作负载的组合，降低 Token 消耗。本文我们将呈现量化这些收益的实验结果。

首先，回顾一些重要概念。

前缀缓存

Anthropic 的前缀缓存是按内容键控（content-keyed）的，两个请求之间任何字节的不同都会使缓存从该点开始失效。一个重复使用相同系统提示和工具目录的长会话只需支付一次 cache_create 费用，并在后续每次交互中以输入价格的 1/10 读取前缀。

这也是为什么“简洁性”或“工具结果修剪”等策略特别适合编码智能体的原因。它们只触及输出部分，从不涉及前缀。因此，在长智能体会话中分摊缓存的能力保持不变，节省的部分集中在最昂贵的类别上。

统计工具

对于我们报告的每个指标，我们都计算了三件事：

• 配对符号检验。 对每个任务，比较每个任务的平均值：edgee 消耗的更少吗？计算获胜次数。在零假设下获胜次数服从 Binomial(n, 0.5) 分布，p < 0.05 在常规阈值上具有显著性。
• Bootstrap 95% 置信区间。 对任务对重采样 10,000 次，在每次重采样上计算统计量，取第 2.5 和第 97.5 百分位数。
• 任务内变异系数。 计算各（任务，后端）单元的平均值。超过 20% 意味着每个重复实验的噪声过大；低于 20% 意味着结果可信。

统计方法论详解

本节详细介绍我们使用的检验、选择它们的原因以及它们所传达的信息。

数据结构

对于每一个（任务，后端）组合，我们收集了 N 次重复实验的测量值，并取平均值以获得每个后端每个任务的单一均值。结果是一个配对数据集：n 个任务，每个任务有一个 vanilla（原生）均值和一个 edgee 均值。配对至关重要——任务难度可能相差几个数量级（有些 SWE-bench 任务是 10 万 Token 的简单修复，另一些则是 1200 万 Token 的重构），因此非配对比较会被任务间差异主导。

配对符号检验

我们用于效应方向判断的主要工具。

设置。 对 n 个任务中的每一个，询问：edgee 消耗的成本是否低于 vanilla？计算“是”的次数。在零假设下（“两个后端没有实际差异”），edgee 获胜次数服从参数为 n、概率为 0.5 的二项分布。

双侧 p 值。 给定 n 中 k 次获胜：

p = 2 · Pr [ X ≥ max( k , n − k ) | X ~ Binomial( n , 0.5) ]

对于 n = 6 个任务和 k = 6 次获胜：p = 2 · (1/2)⁶ = 0.031。例如，简洁性实验的结果就落在这一区域。

为什么选择这个检验。 我们在以下几种合理备选方案中选择了符号检验：

检验	假设条件	我们不采用的原因
配对 t 检验	配对差值服从正态分布	Token 成本差异存在厚尾——正态性被违反。
Wilcoxon 符号秩检验	配对差值关于中位数对称	我们的差值分布是偏态的。
符号检验	仅要求配对数据独立	符合我们的实验设计。假设最少。

使用符号检验的代价是统计功效。它只利用每一对的方向，不利用幅度。好处是，当它确实检测到显著性时，结果在分布的怪异特征下依然成立。

n 的硬性下限。 能达到的最低双侧 p 值为 2 · (1/2)ⁿ：

n 个任务	最低可实现的双侧 p 值
6	0.031
8	0.008
10	0.002
20	≈ 2 × 10⁻⁶

因此，增加重复次数并不能提高符号检验的功效——功效取决于任务数量，而非单元内的精度。

Bootstrap 置信区间

符号检验告诉我们方向，而 Bootstrap 告诉我们幅度大小。

设置。 对于每个后端得到的 n 个任务均值，针对诸如 edgee 与 vanilla 成本的中位数比值等统计量：

1. 对 n 对任务进行 B 次有放回重采样（我们使用 B = 10,000）。
2. 在每一个重采样集合上计算统计量。
3. 取结果分布的第 2.5 和第 97.5 百分位数。

此即为 95% 置信区间。

为什么使用百分位数法。 偏差修正加速（BCa）方法在 Bootstrap 分布偏斜时可以给出稍紧的区间，但代价是对异常值更敏感。在小样本（n = 6 至 n = 10）情况下，我们刻意容忍偏斜，而不是积极校正。百分位数法是更保守的选择。

为什么 B = 10,000。 Bootstrap 置信区间的端点本身是估计值，蒙特卡洛误差以 1/√B 的速度缩小。在 B = 10,000 时，模拟误差远小于 CI 宽度的 1%——小到可以忽略不计。

解释置信区间。 95% CI 排除了 1.0 倍意味着：在重复抽样下，构建的置信区间中有 95% 包含真实的总体中位数，并且观测到的范围不包含“无效应”。实际应用中，将“置信区间排除 1”视为效应确实存在的证据，是一条可捍卫的捷径。

任务内变异系数

对于每个（任务，后端）单元，计算 N 次重复实验的 σ/μ。再对 2n 个单元取均值。平均 CV 若高于约 20%，意味着每个重复实验的噪声与我们试图测量的效应大小处于同一量级。在我们主要的简洁性实验运行中，平均 CV 远低于 20% 阈值。

总体 vs 均值 vs 中位数

对单个指标单说“减少”一词，其实隐藏了三种合理的聚合方式：

聚合方式	公式	回答的问题
总体	1 − (Σ eᵢ) / (Σ vᵢ)	“在总支出中，edgee 节省了多少比例？”——按 Token 量加权
每任务均值	(1/n) × Σ (1 − eᵢ/vᵢ)	“平均任务有什么感受？”
每任务中位数	median(1 − eᵢ/vᵢ)	“典型任务有什么感受？”——不受异常值影响

我们在每个结果中均考虑了这三种方式。

实验方法论

本节解释指导本项工作的设计选择。

工作负载

SWE-bench Lite。 由流行 Python 仓库中 300 个 GitHub 议题组成的标准公开基准测试，每个议题都配有通过测试的提交。我们以 Agent 模式使用 SWE-bench Lite：Claude 获取议题、完整仓库访问权限，并被要求自主修复 Bug。这是编码智能体评估的标准工作负载。

重复实验与随机化

每个（任务，后端）单元重复 N 次。重复实验的顺序按任务随机化：

                每个任务内部随机排序：
   任务 1:    v_1 v_2 e_1 e_2            ─►   e_2 v_1 e_1 v_2
   任务 2:    v_1 v_2 e_1 e_2            ─►   v_1 e_2 v_2 e_1
   …

如果我们总是先跑“全部 vanilla，再全部 edgee”，第二批数据将承受冷缓存的不利影响。随机排序消除了这种系统性偏差。

每次重复的随机数（nonces）

即使进行了随机排序，同一（任务，后端）单元的两次重复也会产生字节完全相同的请求前缀——Anthropic 的缓存将使第二次重复几乎完全从缓存提供服务，从而严重低估其真实成本。我们在首条用户消息前添加一个随机数：

[trial: 8q4r7n2vmp]

You are working autonomously on a bug fix in the django repository. …

这个不同的单行使前缀的其余部分缓存失效，确保每次重复都是冷启动。

Token 计量

Token 使用量直接读取自 Claude Code 的逐会话 JSONL 日志文件，该文件包含 Anthropic 返回的原始 usage 字段。成本根据这些计数和公布的价目表在本地计算。

V2 栈的结果

V2 的三项策略分别对照符合其设计目标的工作负载进行了独立评估。简洁性（Brevity） 是自主编码任务上的主要结果。TSR 是面向工具繁重的 MCP 工作负载的主要结果，而工具结果修剪在长会话中与其他两项策略叠加，可提供额外的节约。

简洁性，每任务成本降低约 30%

针对编码工作负载的主要结果。

设置

• 工作负载： 6 个有代表性的 SWE-bench Lite 任务，Agent 模式（每次会话一条自主提示）。
• 重复次数： 每个（任务，后端）单元 2 次。
• 后端： vanilla Claude Code 对比 启用简洁性的 edgee。
• 总会话数： 24。
• 随机化： 按任务随机排序。
• 每次重复随机数： 已启用（每次重复冷启动）。

主要数据

指标	总体	每任务均值	每任务中位数	符号检验	显著性
成本（美元）	+51.1%	+34.2%	+27.5%	6/6 有利于 edgee	★（p = 0.031）
总 Token	+58.9%	+35.2%	+27.5%	6/6 有利于 edgee	★（p = 0.031）
输出 Token	+53.9%	近似	近似	6/6 有利于 edgee	★（p = 0.031）

需要注意的三点：

1. 全部六个任务在所有指标上均有利于 edgee。 符号检验获胜次数 6/6 达到了 n=6 时可达的最低 p 值 p = 0.031，在常规 α = 0.05 显著性水平上具有统计意义。
2. 三个指标方向一致。 成本下降是因为总 Token 下降，而总 Token 下降又是因为输出 Token 下降。这三个信号相互增强。
3. 我们引用中位数（约 30%）而不是总体（+51%）作为标题数字，以使用更保守且不受异常值影响的度量。

关于 Token 比率的 Bootstrap 置信区间

edgee/vanilla 总 Token 中位数比率的 95% Bootstrap 置信区间：

token ratio = 0.70× (95% CI: [0.41×, 0.84×])

整个置信区间低于 1.0 倍，这意味着：在重复抽样下，构建的区间不包含“无效应”，因此结果稳健。

为什么简洁性如此有效

这个主要数据背后的机制是结构性的。简洁性指示模型精简——不要开场白，不要评论，只做工作。在 Agent 模式的编程任务中，模型通常会在实际编辑之外生成大量元文本：“我先读一下这个文件，然后检查导入，然后进行修改……” 这些评论正是真实且昂贵的输出 Token。

当简洁性激活时，模型依然进行所有相同的工具调用，生成完全相同的最终补丁——它只是停止了叙述其计划。工作本身不变；围绕工作的修饰性文字大幅缩减。

这对成本计算产生三个启示：

• 输出减半
• 缓存读取（cache_read）大致恒定（前缀未被修改；在长智能体循环中缓存分摊持续生效）。
• 缓存创建（cache_create）大致恒定（无前缀失效；工具保持稳定，系统提示保持稳定）。

结果：这一策略将节省集中在最昂贵的 Token 类别上，同时不带来任何缓存创建相关的波动。这是一种将大的 Token 数量信号与更大的成本信号结合的方案——两者同向移动。

成本落地——按类别分解

观察简洁性机制最清晰的方法是按 Token 类别分解两个后端。合并所有 12 次 vanilla 会话和所有 12 次 edgee 会话：

Token 类型	vanilla 成本	edgee 成本	Δ
全新输入	很小	很小	~0
缓存读取	数量可观	数量可观	很小
缓存创建	数量可观	数量可观	很小
输出	占主导	约为一半	−54%

输出类——Anthropic API 上最昂贵的单一 Token 类型——下降超过一半，而这一下降贡献了总体 51% 成本降低的全部。简洁性不仅减少了 Token；它减少了正确的 Token。

MCP 工作负载下的 TSR — 约 10% 成本节约

这是工具繁重工作负载的主要结果。

当智能体接入 MCP 服务器，如 Lin

Why compression matters ?

Coding agents have become the substrate for building products. They are long-running and context-heavy. For example, a single SWE-bench coding task can take 30–100 API turns, where 1–10 million tokens per task is common. These numbers, multiplied across a fleet of agents, a developer team, or a SaaS product make the bill escalates quickly in the blink of an eye.

The economic pressure is one motivation for compression, but not the only one. Other elements point in the same direction:

Bottleneck	What it limits	Why compression helps
Dollar cost	Per-task spend; sustainable unit economics for AI products	Fewer tokens (of the right kind) → smaller bill
Latency	Time-to-first-token scales with prefix size on most providers	Smaller prefix → faster perceived response
Context window	200k or 1M token windows still fill up on long agent sessions	Compressing history extends the effective horizon
Throughput	Server-side batch sizes are inversely related to per-request token count	Smaller requests → more parallelism for the provider; lower queueing delays for the user

Hence, a robust compression layer addresses all four and is therefore a critical part of an agentic workflow performance. Compressor V2 is such a layer and is a component of the Edgee AI gateway that was built with these requirements in mind.

From V1 to V2

Edgee's V1 compressor, shipped earlier this year, used a single strategy we called tool result trimming. This strategy is inspired by the famous RTK project and consists in cleaning up the verbose tool outputs that agents collect over a long session. It delivered roughly 10% cost savings on real coding workloads — safe, simple, but limited in scope.

V2 takes a different approach: three orthogonal strategies layered together , each one targeting a different layer of context and configurable independently per API key. Since the three strategies don't overlap they attack different sources of token bloat:

• Brevity attacks the most expensive token class, output tokens.
• Tool surface reduction (TSR) attacks the most repetitive part of the prefix (MCP tool catalogs).
• Tool result trimming attacks the long tail of verbose tool outputs that accumulate over the conversation.

Through the Gateway, customers compose the combination that matches their workload and reduce their token consumption. In this post, we address the empirical results that quantify exactly these gains.

To start, let us recall some important notions.

Prefix caching

Anthropic's prefix cache is content-keyed, any byte that differs between two requests invalidates the cache from that point on. A long session that reuses the same system prompt and tool catalog pays the cache_create fee once and reads back the prefix at 1/10th the input price on every subsequent turn.

This is also why strategies like brevity or tool result trimming are so well-suited for coding agents. They touch only the output, never the prefix. Therefore, cache amortization across long agent sessions stays intact, and the savings are concentrated in the most expensive class.

Statistical tools

For every metric we report, three things are computed:

• Paired sign test. For each task, compare per-task means: did edgee use less? Count the wins. The win count follows a binomial(n, 0.5) under the null and p < 0.05 is significant at the conventional threshold.
• Bootstrap 95% Confidence Interval. Resample task pairs 10,000 times, compute the statistic on each resample, take the 2.5th and 97.5th percentiles.
• Within-task coefficient of variation. Average across (task, backend) cells. Above 20% means too much per-replicate noise; below 20% means the result is trustworthy.

Statistical methodology in depth

This section walks through the tests we apply, why we chose them, and what they tell us.

The structure of the data

For every (task, backend) combination, we collect N replicate measurements and average across replicates to get a single per-task mean for each backend. The result is a paired dataset: n tasks, each with one vanilla mean and one edgee mean. Pairing matters — task difficulty varies by orders of magnitude (some SWE-bench tasks are 100k-token toy fixes, others are 12M-token deep refactors), so unpaired comparisons would be dominated by between-task variance.

The paired sign test

Our primary tool for direction-of-effect claims.

Setup. For each of n tasks, ask: did edgee use less cost than vanilla? Count the yeses. Under the null hypothesis ("no real difference between backends"), the count of edgee wins follows a binomial distribution with parameters n and probability 0.5.

Two-sided p-value. Given k wins out of n:

p = 2 · Pr [ X ≥ max( k , n − k ) | X ~ Binomial( n , 0.5) ]

For n = 6 tasks and k = 6 wins: p = 2 · (1/2)⁶ = 0.031. The brevity result for example lands in this area.

Why this test, specifically. We chose the sign test over plausible alternatives:

Test	What it assumes	Why we passed
Paired t-test	Paired differences are normally distributed	Token-cost differences have heavy tails — normality is violated.
Wilcoxon signed-rank	Paired differences are symmetric around the median	Our differences are skewed.
Sign test	Only that pairs are independent	Holds for our design. Lowest assumptions.

The cost of using the sign test is statistical power. It uses only the direction of each pair, not the magnitude. The benefit is that when it does detect significance, the result holds under distributional weirdness.

The hard floor on n. The lowest achievable two-sided p-value is 2 · (1/2)ⁿ

n tasks	Min achievable two-sided p
6	0.031
8	0.008
10	0.002
20	≈ 2 × 10⁻⁶

Therefore, adding replicates does not increase sign-test power — power depends on task count, not on within-cell precision.

The bootstrap confidence interval

Where sign test tells us direction; the bootstrap, on the other hand, tells us magnitude.

Setup. Given n per-task means for each backend, for a statistic like the median ratio of edgee-to-vanilla cost:

1. Resample with replacement n task pairs B times (we use B = 10,000).
2. Compute the statistic on each resampled set.
3. Take the 2.5th and 97.5th percentiles of the resulting distribution.

That's the 95% CI.

Why percentile method. The bias-corrected-and-accelerated (BCa) method gives slightly tighter intervals when the bootstrap distribution is skewed, but at the cost of more sensitivity to outliers. With small n (n = 6 to n = 10), we deliberately tolerate skew rather than aggressively correct for it. Percentile is the conservative choice.

Why B = 10,000. Bootstrap CI endpoints are themselves estimates, with Monte Carlo error shrinking as 1/√B. At B = 10,000 the simulation error is well under 1% of the CI width — small enough not to matter.

Interpreting a CI. A 95% CI excluding 1.0× means: under repeated sampling, 95% of the constructed CIs would contain the true population median, and the observed range does not include "no effect." In practice, treating "the CI excludes 1" as evidence that the effect is real is a defensible shortcut.

Within-task coefficient of variation

For each (task, backend) cell, compute σ/μ across the N replicates. Average across the 2n cells. A mean CV above ~20% means per-replicate noise is comparable to the effect sizes we're trying to measure. In our headline brevity run, mean CV was well under the 20% threshold.

Aggregate vs mean vs median

A single per-metric "reduction" hides three legitimate aggregations:

Aggregation	Formula	Answers
Aggregate	1 − (Σ eᵢ) / (Σ vᵢ)	"Of total spend, what fraction did edgee save?" — volume-weighted
Mean per task	(1/n) × Σ (1 − eᵢ/vᵢ)	"What does the average task experience?"
Median per task	median(1 − eᵢ/vᵢ)	"What does the typical task experience?" — outlier-robust

We considered all three on every result.

Experimental methodology

This section explains the design choices that guided this work.

Workload

SWE-bench Lite. The standard public benchmark of 300 GitHub issues from popular Python repositories, each paired with a passing-test commit. We use SWE-bench Lite in agent mode: claude gets the issue, full repo access, and is asked to fix the bug autonomously. This is the canonical workload for coding-agent evaluation.

Replicates and shuffle

Each (task, backend) cell is replicated N times. Replicate ordering is randomized per task :

                                Shuffle within each task:
   Task 1:    v_1 v_2 e_1 e_2            ─►   e_2 v_1 e_1 v_2
   Task 2:    v_1 v_2 e_1 e_2            ─►   v_1 e_2 v_2 e_1
   …

If we always ran "all vanilla, then all edgee," the second batch would inherit a cold-cache disadvantage. Random ordering removes this systematic bias.

Per-replicate nonces

Even with shuffling, two replicates of the same (task, backend) cell would produce byte-identical request prefixes — and Anthropic's cache would serve replicate 2 almost entirely from cache, dramatically underreporting its true cost. We prepend a random nonce to the first user message:

[trial: 8q4r7n2vmp]

You are working autonomously on a bug fix in the django repository. …

That single differing line invalidates the cache for the rest of the prefix making each replicate starts cold.

Token accounting

Token usage is read directly from Claude Code's per-session JSONL log files, which contain raw usage fields returned by Anthropic. Cost is computed locally from these counts and the published price table.

Results across the V2 stack

V2's three strategies were evaluated independently against workloads matched to their design target. Brevity is the headline result on autonomous coding. TSR is the headline result on tool-heavy MCP workloads and Tool result trimming provides incremental savings that compound with the other two over long sessions.

Brevity, ~30% per-task cost reduction

The headline coding-workload result.

Setup

• Workload: 6 representative SWE-bench Lite tasks, agent mode (single autonomous prompt per session).
• Replicates: 2 per (task, backend) cell.
• Backends: vanilla Claude Code vs edgee with brevity enabled.
• Total sessions: 24.
• Shuffle: randomized per task.
• Per-replicate nonces: active (each replicate starts cold).

Headline numbers

Metric	Aggregate	Mean per task	Median per task	Sign test	Significance
Cost ($)	+51.1%	+34.2%	+27.5%	6/6 favor edgee	★ (p = 0.031)
Total tokens	+58.9%	+35.2%	+27.5%	6/6 favor edgee	★ (p = 0.031)
Output tokens	+53.9%	similar	similar	6/6 favor edgee	★ (p = 0.031)

Three things to notice:

1. All six tasks favor edgee on every metric. Sign-test wins of 6/6 hit the minimum achievable p-value at n=6, p = 0.031, which is statistically significant at the conventional α = 0.05 threshold.
2. The three metrics agree directionally. Cost is down because total tokens are down because output tokens are down. The three signals are mutually reinforcing.
3. We cite the median (~30%) as the headline rather than the aggregate (+51%) to have a more conservative and outlier-robust metric.

Bootstrap CI on the token ratio

The 95% bootstrap confidence interval for the median ratio of edgee/vanilla total tokens:

token ratio = 0.70× (95% CI: [0.41×, 0.84×])

The entire CI is below 1.0×, which means: under repeated sampling, the constructed interval does not include "no effect" and hence making the result robust.

Why brevity works so well

The mechanism behind the headline number is structural. Brevity instructs the model to be terse — no preamble, no commentary, just the work. On agent-mode coding tasks, the model normally produces a lot of meta-text alongside its actual edits: "I'll first read the file, then I'll check the imports, then I'll make the change..." That commentary is real output tokens a high price.

When brevity is active, the model still does all the same tool calls and produces the same final patches — it just stops narrating its plan. The work doesn't change; the prose around the work shrinks dramatically.

This has three implications for the cost math:

• Output drops by half
• Cache_read stays roughly constant (the prefix isn't touched; cache amortization across the long agent loop continues working).
• Cache_create stays roughly constant (no prefix invalidation; tools stay stable, system prompt stays stable).

The result: a strategy that concentrates its savings on the most expensive token class without inducing any cache-create churn. That's the strategy that combines a large token-count signal with an even larger dollar signal — both move in the same direction.

Where the cost lands — per-class decomposition

The clearest way to see brevity's mechanism is to break down both backends by token class. For all 12 vanilla sessions and all 12 edgee sessions combined:

Token type	vanilla cost	edgee cost	Δ
Fresh input	small	small	~0
Cache read	meaningful	meaningful	small
Cache create	meaningful	meaningful	small
Output	dominant	roughly half	−54%

The output class — the single most expensive token type on the Anthropic API — drops by more than half, and that drop accounts for the entire +51% aggregate cost reduction. Brevity doesn't just reduce tokens; it reduces the right tokens.

TSR on MCP workloads — ~10% cost

The headline result for tool-heavy workflows.

When an agent connects to MCP servers like Linear, Notion, or GitHub, the request prefix accumulates dozens of tool definitions — typically 30–40 per server, each one a paragraph of JSON describing the tool's name, parameters, and behavior. Tool surface reduction (TSR) rewrites this prefix on the fly, replacing the full tool catalog with a single virtual mcp__edgee_gateway__search tool. The model calls this tool with an {intent, args} payload; the gateway resolves the intent server-side, dispatches the appropriate real tool, and returns the result. The model never sees the full catalog.

Setup

• Workload: 8 synthetic read-only queries against Linear and Notion. Tasks range from single-server lookups ("list 5 recent Linear issues") to cross-server stress queries ("find Linear projects and Notion docs mentioning 'roadmap'").
• Replicates: 3 per (task, backend) cell.
• Backends: vanilla Claude Code vs edgee with TSR enabled.
• Total sessions: 48.

Headline numbers

Metric	Aggregate	Mean per task	Median per task	Sign test	Significance
Total tokens	+33.0%	+31.0%	+30.6%	8/8 favor edgee	★★ (p = 0.008)
Cost ($)	+11.2%	+10.8%	~+10%	5/8 favor edgee	not significant

Three things to notice:

1. All eight tasks favor edgee on token volume. 8/8 hits the minimum achievable two-sided p-value at n=8, p = 0.008 — substantially below the conventional α = 0.05 threshold. This is the strongest direction-of-effect signal in any of our measurements.
2. Cost reduction is real but more modest than tokens. The median per-task cost reduction lands around 10%, with 5/8 tasks favoring edgee. The sign test on cost does not reach conventional significance at n=8 — but the point estimate is directionally favorable and consistent across our two TSR runs.
3. The 95% bootstrap CI on the token ratio is [0.63×, 0.75×] — a tight interval entirely below 1.0×. There is no plausible value of the true population median that is consistent with "no effect" on the token side.

Why the cost effect is smaller than the token effect

A 33% reduction in tokens translates into only ~10% cost reduction because of the per-class pricing asymmetry. TSR's compression removes a lot of cache_read tokens from the prefix — the cheapest token class. Those savings are real but priced at lower rate of output tokens. The strong token signal therefore doesn't multiply 1-to-1 into dollar savings.

This is a feature of the measurement. We treat the token-volume reduction (33%, p = 0.008) and the cost reduction (~10%, trending) as two separate claims , each with its own statistical support. Customers whose binding constraint is throughput, latency, or context-window care primarily about the token claim; customers focused on the bill care primarily about the cost claim. V2 ships both.

Why TSR is the right strategy for tool-heavy workloads

Coding agents have stable tool catalogs that amortize cheaply across long sessions; brevity attacks their bottleneck (output). Tool-using assistants — chatbots integrated with Linear, Slack, Notion, GitHub, etc. — have unstable tool exposure as they connect more services, and their bottleneck is the prefix bloat from dozens or hundreds of MCP tool definitions.

TSR addresses exactly this regime. The math is structural:

• A 40-tool MCP catalog adds roughly 40–80k tokens to every request's tools[] array.
• Under TSR, that becomes ~1k tokens (a single virtual tool definition).
• For products that connect 4–5 MCP servers, the prefix savings compound: 200k tokens of catalog become ~1k.

The 33% volume reduction we measured at an 8-task synthetic benchmark scales upward in regimes where the catalog is larger or where many independent short queries run against the same backend (defeating cache amortization). For products whose latency, context-window, or throughput is the binding constraint, TSR delivers a strong win.

Tool result trimming, 10% cost savings, compounding over long sessions

The third V2 strategy, refined from V1.

Over long sessions, agents accumulate tool outputs — file listings, stack traces, repeated log frames — that bloat the conversation history without proportionally informing the model's next decision. Tool result trimming cleans these up: it preserves the semantically important slices of each tool output and compresses or summarizes the rest. The model retains enough context to reason; it doesn't see thirty pages of identical error frames.

Setup

• Workload: 6 representative SWE-bench Lite tasks, agent mode.
• Replicates: 2 per (task, backend) cell.
• Total sessions: 24.

Result

Metric	Median per task	Sign test
Cost ($)	+10.4%	4/6 favor edgee

Our agent-mode SWE-bench benchmark of the refined V2 version reproduces this at +10.4% median. The sign test (4/6) does not reach conventional significance at n=6. Tool result trimming is a directionally favorable but modest strategy on its own, with the cleanest gains visible on longer sessions where tool-output accumulation dominates the cache_read budget.

The reason we ship it as part of V2 is that it compounds with brevity and TSR. Because tool result trimming operates on conversation history (not on the prefix or the output), it composes cleanly with the other two strategies without overlap. A coding workflow running brevity + tool result trimming gets brevity's ~30% per-task cost reduction on output plus incremental 5–10% from cleaning up the long tail of accumulated tool results.

The clean composability is the design goal of the V2 stack: three strategies, three different layers, additive rather than competing savings.

The V2 stack, composing the strategies

V2's three strategies were designed to be orthogonal so customers can compose them per-workload. Putting the results together:

Workload shape	Recommended V2 combination	Expected effect
Long autonomous coding sessions (SWE-bench shape)	Brevity + tool result trimming	~30% per-task cost from brevity, +5–10% from tool result trimming compounding as session length grows
Tool-heavy queries against MCP servers	TSR + brevity	~10% cost + 33% token-volume reduction from TSR; brevity adds further output-side savings on top
Mixed coding + tool workflows	All three	Each strategy targets a distinct layer; combinations are clean and additive
Latency- or context-bound workloads (regardless of dollar cost)	TSR (priority), brevity (secondary)	Maximum tokens-through-the-model reduction (33% on TSR's target workload)

The strategies don't double-count because they attack different layers of the request:

• Brevity → output tokens (assistant message content)
• TSR → prefix tokens (tools[] array)
• Tool result trimming → message history (cumulative tool outputs)

Each can be toggled independently on a per-API-key basis from the edgee console.

Conclusion

Compressor V2 layers three orthogonal compression strategies, and each delivers a measurable result on the workload it targets:

• Brevity cuts cost by ~30% per task on autonomous coding workloads, with 6/6 tasks favoring edgee, paired sign test p = 0.031. The mechanism is structural: brevity attacks the most expensive token class (output) without disturbing the prefix or cache amortization.
• TSR delivers ~10% per-task cost reduction on tool-heavy MCP workloads. The token-side signal is 8/8 tasks favoring edgee at p = 0.008.
• Tool result trimming , refined from V1 and based on the RTK project, delivers 5–10% per-task cost reduction as a compounding layer that composes cleanly with brevity and TSR across long sessions.

To conclude, users should enable the combination that matches their workload shape, per API key. If you're running coding agents or tool-heavy assistants at scale and the cost bill is what's keeping you up at night, V2 is in production today and saves a combined 50% on the cost of your sessions.