商业产业 Hacker News AI 2026-07-24T00:32 正文 ok 2826 words

Show HN:Turo——面向 CLI AI Agent 的激进 Token 节省代理

英文标题:🔶 Show HN: Turo – An Aggressive Token-Saving Proxy for CLI AI Agents

Turo 是一款为 Claude Code、Codex、Gemini、Cursor 等 CLI AI Agent 设计的文本压缩代理,目标是在不改动代码、路径和标识符的前提下大幅减少上下文 Token 消耗。它通过删除填充语、替换更省 Token 的同义词和释义词、保留并去重内容词等多阶段处理,将示例 PR 审查指令从 138 Token 压缩至 54 Token,`ultra` 模式可降至 41 Token。项目还提供 `gain`、`discover` 和 `doctor` 命令,分别用于统计节省量、回溯 Claude Code 历史中的潜在节省,以及检查安装与 Agent 集成状态。开发者需注意,其 Token 统计为内置 `cl100k` 风格估算;面向 CJK 优化分词器的 `wenyan` 模式适合 Qwen、DeepSeek、GLM 等模型,不适合 OpenAI 模型。
打开原文相关 3 · 机会 1
media602.1 KB media55.2 KB

中文

turo

多指向,少 Token。

一段真实指令块——138 Token(cl100k):

When you are reviewing a pull request, please make sure that you carefully
examine each of the changed files and verify that the new code does not
introduce any regressions in the existing behavior. It is really important that
you check whether the author has added appropriate tests for the new
functionality, because untested code is very likely to break in subtle ways that
are difficult to debug later. You should also confirm that the documentation has
been updated to reflect the changes, and that the commit messages clearly
explain what was changed and why. If you notice any potential security
vulnerabilities, such as unsanitized user input or hardcoded credentials, you
must flag them immediately and request changes before the pull request can be
merged.

会变为 54 Token——减少 61%(turo,语义保持完整):

Reviewing pull request make examine changed files verify new code introduce
regressions existing behavior important check author added appropriate tests
functionality untested break subtle ways difficult debug later also confirm
documentation updated reflect changes commit messages explain notice potential
security vulnerabilities unsanitized user input hardcoded credentials must flag
merged

或者在 --level ultra 下变为 41 Token——减少 70%(按词元去重):

Review pull request make examine change file verify code introduce regression
exist behavior important check author add test untested break debug later
confirm documentation updated reflect commit message notice security
vulnerability unsanitized user input hardcoded must flag merge

或者使用 --level wenyan 时仅为 69 个字符;该模式还会将每个词替换为一个文言单字(适用于 CJK 分词器模型——参见 wenyan):

閱引請作察變檔驗碼引退存為要查者增試 untested 破診後證文更映交訊覺安隙 unsanitized 戶入 hardcoded 須標併

766 -> 69 个字符(35 个汉字)。该输出的 Token 计数:

tokenizer tokens
Qwen / DeepSeek / GLM(约每个汉字 1 Token) 约 42
OpenAI cl100k(每个汉字 2-3 Token) 67

因此,wenyan 在 CJK 分词器模型上胜出(约 42,相比普通 ultra 的约 71),但在 OpenAI 上落败(67 对 41)——仅应搭配 CJK 模型使用。详见下文。

没有冠词。没有介词。没有副词。没有重复词。只保留承载意义的内容词,并按阅读顺序去重。每一条提示、每一轮对话——节省会不断累积。如果压缩结果没有小于输入,turo 会原样透传。

只需安装一次 turo,任何能够调用二进制文件的编码 Agent——Claude Code、Codex、Gemini、Cursor、Windsurf、Cline、Copilot,以及另外 20 多种 Agent——都会将其上下文通过同一个压缩器处理。代码、路径和标识符会原样通过。

Method Command
Homebrew brew install kdeps/tap/turo
Go go install github.com/kdeps/turo@latest
Shell `curl -fsSL https://raw.githubusercontent.com/kdeps/turo/main/install.sh
Manual 从 releases 下载
cat CLAUDE.md | turo              # 文本 -> 去重后的内容词
echo "fox jumps over dog" | turo  # 管道模式
turo -passes 1                    # 单次处理(默认运行至收敛)
turo -filler=false                # 跳过填充语删除
turo -synonyms=false              # 跳过同义词处理(逐字保留原词)
turo -gloss=false                 # 跳过释义词替换(损失更少)
turo -arrows=false                # 原样保留连接短语(跳过 -> 替换)
turo gain                         # 查看至今估算节省的 Token
turo gain --history               # 按时间倒序查看每次压缩历史
turo gain --json                  # 以 JSON 输出相同汇总,供脚本/仪表盘使用
turo discover                     # 查看 turo 可在 Claude Code 历史中节省的 Token
turo discover --json              # 以 JSON 输出 discover 汇总
turo doctor                       # 健康检查:版本、设置、路径、Agent 接线状态
turo --version                    # 输出版本

-gloss(默认开启)会从每个词自身的词典定义中,用词性相同且最短的词替换它(approach -> come)。定义是散文说明,而不是同义词,因此这是损失最大的一阶段——当你需要与原文更接近的措辞时,可通过 -gloss=false / TURO_GLOSS=off 禁用。

-arrows(默认开启)会将多词因果/顺序连接短语(leads to、results in、gives rise to、which produces)替换为单个 -> Token。只有多词短语符合条件,因此这种替换始终至少节省一个 Token;单 Token 连接词(then、becomes、thus)会保持不变,因为 -> 的成本相同。可通过 -arrows=false / TURO_ARROWS=off 禁用。

A cache miss leads to a slow query which produces a timeout
                     |  arrows(默认开启)
Cache miss -> slow query -> timeout

每次压缩(管道模式、代理和 turo run)都会将估算的压缩前/后 Token 数追加至操作系统配置目录中的 JSONL 日志:macOS 为 ~/Library/Application Support/turo/,Linux 为 ~/.config/turo/;可使用 TURO_HOME 覆盖。turo gain 会汇总数据;turo gain --history 则按时间倒序列出近期压缩记录。

turo gain — 42 reductions
  tokens in     8140
  tokens out    2610
  tokens saved  5530 (67%)

by folder:
  ~/Projects/turo        31 reductions  saved 4100 (69%)
  ~/Projects/api          11 reductions  saved 1430 (61%)

每个事件还会记录其运行所在的工作目录,因此 turo gain 会按项目拆分节省量(最活跃的项目优先),turo gain --history 则会显示每次压缩对应的目录。

向任一命令添加 --json,即可获得机器可读格式的相同数字——包括总计、原始(未缩写)Token 数、整数百分比和按目录拆分的数据——因此你可以将节省量接入仪表盘、CI 检查或 jq:

$ turo gain --json | jq '.tokens_saved, .saved_pct'
5530
67

计数来自内置的 cl100k 风格估算,并非真实分词器——应将其视为趋势,而不是账单。

未捕获的节省量——turo discover

turo gain 汇总已经发生的压缩。turo discover 则展示未发生的那些:它扫描现有 Claude Code 历史,并估算若这些会话当时经过代理,turo 本可节省多少 Token。

turo discover — scanned 403 sessions in ~/.claude/projects
  messages       53011 reducible (all roles)
  tokens in      13524093
  would be out   6246528
  would save     7277565 (53%)

by project:
  ~/Projects/api          29490 msgs  saved 3242664 (49%)
  ~/Projects/web          19943 msgs  saved 3142015 (57%)

these sessions ran without turo — capture the savings next time with:
  turo run claude

它会读取 ~/.claude/projects 下按项目存放的会话日志(如果 Claude Code 将其存储在其他位置,请设置 CLAUDE_CONFIG_DIR),并应用与代理相同的角色门控和压缩标志(-level、-filler、-gloss、-arrows、-proxy-all)——因此该估算反映的是 turo run claude 实际会执行的行为。不会向任何地方发送数据;扫描完全在本地进行,且为只读操作。

健康检查——turo doctor

turo doctor 会运行本地健康检查;若任何项目损坏,则以非零状态退出——适用于 CI、安装脚本,或升级后确认 turo 是否已正确接入。

turo doctor

turo
  · version dev (unreleased build)
  · binary /usr/local/bin/turo
  ✓ default level ultra (all roles)

environment
  · no turo env overrides set (using defaults)

gain log
  ✓ writable: ~/Library/Application Support/turo/gain.jsonl (135 events)

claude history (turo discover source)
  ✓ 414 session logs in ~/.claude/projects

pipeline self-test
  ✓ 22 -> 5 tokens (77% smaller) at level ultra

agents
  ✓ Claude Code — detected, skill installed
  ✓ opencode — detected, skill installed
  · Gemini CLI — detected
  · Cursor — detected
  · Qwen Code — detected
  · 5 of 18 supported agents detected (turo -list-agents shows all)

turo is healthy

检查内容:

Section What it verifies
turo 版本字符串、二进制路径、默认 level 的有效性
environment 列出已设置的所有 TURO_* 环境变量覆盖项
gain log Gain 目录是否可创建、日志文件是否可写、事件数量
claude history 是否在 ~/.claude/projects 下找到会话日志
pipeline self-test 对示例句子运行 reduce(),确认 Token 数减少
agents 检测已安装的编码 Agent,并检查原生 Agent 的 skill 安装情况

传入 -level <name> 可测试指定 level;无效 level 会被报告为问题(✗),而非立即硬退出,因此完整报告仍然可见。

turo doctor               # 健康 -> 退出码 0
turo -level bogus doctor  # level 无效 -> 退出码 1

每次运行共有四个阶段,默认均开启:

text -> [1] delete filler -> [2] swap cheaper synonyms -> [3] swap defining words -> [4] reduce to content words
  1. 删除填充语会移除在逐词停用词表中仍会保留的客套语、模糊表达和引导语(please、I think、of course、let me),同时原样保护代码、路径、URL 和标识符。可通过 -filler=false / TURO_FILLER=off 禁用。
  2. 同义词替换会以 Token 数更少的同义词替换词语(见下文)。可通过 -synonyms=false / TURO_SYNONYMS=off 禁用。
  3. 释义词替换会以词典定义中最短的定义词替换词语——这是损失最大的一阶段。可通过 -gloss=false / TURO_GLOSS=off 禁用。
  4. 压缩会删除剩余停用词,按词性保留内容词,进行去重,并在 ultra 模式下按词元合并词形变化。

箭头(默认开启,在压缩前运行)会将多词连接短语改写为 ->,压缩器会将其原样保留在幸存内容词之间。可通过 -arrows=false / TURO_ARROWS=off 禁用。

整个管线会重复运行,直至输出不再变化(默认 -passes 0;正数 -passes N 可限制次数)。第一轮会保留文档结构(标题、各段正文);后续轮次会将其扁平化并跨结构去重,因此大型结构化文档会在收敛前持续缩小——本 README 从 522 Token(1 轮)降至 376 Token(收敛时,约 29 轮)。设置 -passes 1 可仅执行单次处理。

turo 永远不会输出比输入更大的内容:如果某一阶段未能节省 Token,文本会原样通过。

同义词替换(默认开启,有损)

turo 在压缩前会先运行一轮处理,将每个词替换为 Token 数更少的同义词(utilize -> use,demonstrate -> show)。该表由 WordNet synsets(真实同义词)构建,并经过词频筛选以确保替换结果为常用词;随后会限制为相同词性的词,并验证 Token 成本严格更低。如果结果没有更小,turo 仍会原样透传输入。

当你需要逐字保留词语时,可使用 -synonyms=false 或 TURO_SYNONYMS=off 禁用。收益通常较小——现代分词器已将大多数单词编码为单个 Token。WordNet 的一词多义也会留下部分噪声(leverage -> purchase),因此应将其作为散文的可选功能,而不是用于代码。该表由 tools/gensyn.py(WordNet + wordfreq + cl100k tokenizer)生成,因此 Token 数是针对 cl100k 测得,而非目标模型的分词器。

Level 保留内容 压缩率
lite 形容词、名词、动词,以及剩余副词/介词 约 65%
full 形容词、名词、动词 约 70%
ultra(默认) 仅名词和动词,按词元(基本形式)去重 约 70%+
wenyan ultra,然后将幸存词替换为单个文言(Classical Chinese)字符 仅适用于 CJK 模型
echo "the quick brown fox jumps over the lazy dog" | turo --level lite   # quick brown fox jumps over lazy dog
echo "the quick brown fox jumps over the lazy dog" | turo --level full   # quick brown fox jumps lazy dog
echo "the quick brown fox jumps over the lazy dog" | turo --level ultra  # fox jump dog
echo "the wise king studies the old book" | turo --level wenyan    # 智王學舊書

wenyan(仅适用于 CJK 分词器模型)

wenyan 会先按 ultra 模式压缩,再从一个约 380 条、人工整理的词典中,将每个幸存的英语内容词替换为一个文言单字(water -> 水、king -> 王、verify -> 驗)。每个概念一个字符,不使用空格(文言文原本没有空格)。

两个已测量示例:

input ultra wenyan chars cl100k CJK-model(约 1/字符)
The wise king uses water and fire...(80 ch) Wise king use water fire person see mountain old tree 智王用水火人見山舊樹 10 15 约 10
the PR-review paragraph(766 ch) 283 ch / 41 tok 閱引請作察變檔驗碼引退存為要查者增試 untested 破診後證文更映交訊覺安隙 unsanitized 戶入 hardcoded 須標併 69 67 约 42

它可压缩至最少的字符数(该段落从 766 -> 69)。在 OpenAI 的 cl100k 中,一个 CJK 字符需 2-3 Token,因此 wenyan 在该环境下会更大(67 > 41)。它仅在针对 CJK 优化的分词器(Qwen、DeepSeek、GLM)上胜出:常用字符约为 1 Token,此时 69 个字符约为 42 Token,而普通 ultra 为 71 Token。不要将其用于 OpenAI 模型。

turo 自身的 Token 估算器将 CJK 按 1 rune = 1 Token 计算(与这些模型相符),因此会将 wenyan 视为一次压缩,且永不拒绝它。词典外的词会保持英语(如上文的 untested、unsanitized、hardcoded);代码/路径/URL

English / 原文

turo

Point more. Token less.

A real instruction block — 138 tokens (cl100k):

When you are reviewing a pull request, please make sure that you carefully
examine each of the changed files and verify that the new code does not
introduce any regressions in the existing behavior. It is really important that
you check whether the author has added appropriate tests for the new
functionality, because untested code is very likely to break in subtle ways that
are difficult to debug later. You should also confirm that the documentation has
been updated to reflect the changes, and that the commit messages clearly
explain what was changed and why. If you notice any potential security
vulnerabilities, such as unsanitized user input or hardcoded credentials, you
must flag them immediately and request changes before the pull request can be
merged.

becomes 54 tokens — 61% fewer (turo, meaning intact):

Reviewing pull request make examine changed files verify new code introduce
regressions existing behavior important check author added appropriate tests
functionality untested break subtle ways difficult debug later also confirm
documentation updated reflect changes commit messages explain notice potential
security vulnerabilities unsanitized user input hardcoded credentials must flag
merged

or 41 tokens — 70% fewer at --level ultra (deduped by lemma):

Review pull request make examine change file verify code introduce regression
exist behavior important check author add test untested break debug later
confirm documentation updated reflect commit message notice security
vulnerability unsanitized user input hardcoded must flag merge

Or 69 characters with --level wenyan, which additionally swaps each word for a single Classical Chinese character (for CJK-tokenizer models — see wenyan):

閱引請作察變檔驗碼引退存為要查者增試 untested 破診後證文更映交訊覺安隙 unsanitized 戶入 hardcoded 須標併

766 -> 69 characters (35 Han chars). Token counts for that output:

tokenizer tokens
Qwen / DeepSeek / GLM (~1 per Han char) ~42
OpenAI cl100k (2-3 per Han char) 67

So wenyan wins on CJK-tokenizer models (~42 vs plain ultra's ~71) but loses on OpenAI (67 vs 41) — use it only with CJK models. See below.

No articles. No prepositions. No adverbs. No repeated words. Only the content words that carry meaning, deduplicated, in reading order. Every prompt, every turn — the savings compound. If a reduction is not smaller than the input, turo passes the original through unchanged.

Install turo once and any coding agent that can shell out to a binary — Claude Code, Codex, Gemini, Cursor, Windsurf, Cline, Copilot, and 20+ more — pipes its context through the same reducer. Code, paths, and identifiers pass through untouched.

Method Command
Homebrew brew install kdeps/tap/turo
Go go install github.com/kdeps/turo@latest
Shell `curl -fsSL https://raw.githubusercontent.com/kdeps/turo/main/install.sh
Manual Download from releases
cat CLAUDE.md | turo              # text -> deduped content words
echo "fox jumps over dog" | turo  # pipe mode
turo -passes 1                    # single pass (default runs to convergence)
turo -filler=false                # skip filler deletion
turo -synonyms=false              # skip the synonym pass (keep words verbatim)
turo -gloss=false                 # skip the defining-word swap (less lossy)
turo -arrows=false                # keep connective phrases verbatim (skip the -> swap)
turo gain                         # estimated tokens saved so far
turo gain --history               # per-reduction history, newest first
turo gain --json                  # same totals as JSON for scripts/dashboards
turo discover                     # tokens turo could save on your Claude Code history
turo discover --json              # discover totals as JSON
turo doctor                       # health check: version, settings, paths, agent wiring
turo --version                    # print version

-gloss (on by default) replaces each word with the shortest same-part-of-speech word from its own dictionary definition (approach -> come). Definitions are prose, not synonyms, so it is the lossiest stage — disable it with -gloss=false / TURO_GLOSS=off when you need words closer to the original.

-arrows (on by default) replaces multi-word causal/sequential connectives (leads to, results in, gives rise to, which produces) with a single -> token. Only multi-word phrases qualify, so the swap always saves at least one token; single-token connectives (then, becomes, thus) are left alone because -> costs the same. Disable with -arrows=false / TURO_ARROWS=off.

A cache miss leads to a slow query which produces a timeout
                     |  arrows (on by default)
Cache miss -> slow query -> timeout

Every reduction (pipe mode, the proxy, and turo run) appends an estimated before/after token count to a JSONL log in your OS config dir (~/Library/Application Support/turo/ on macOS, ~/.config/turo/ on Linux; override with TURO_HOME). turo gain totals it up; turo gain --history lists recent reductions newest-first.

turo gain — 42 reductions
  tokens in     8140
  tokens out    2610
  tokens saved  5530 (67%)

by folder:
  ~/Projects/turo        31 reductions  saved 4100 (69%)
  ~/Projects/api          11 reductions  saved 1430 (61%)

Each event also records the working folder it ran in, so turo gain breaks the savings down per project (busiest first) and turo gain --history shows the folder for each reduction.

Add --json to either command for the same numbers in machine form — totals, raw (un-abbreviated) token counts, integer percentages, and the per-folder breakdown — so you can pipe savings into a dashboard, a CI check, or jq:

$ turo gain --json | jq '.tokens_saved, .saved_pct'
5530
67

Counts are estimates from the built-in cl100k-style approximation, not a real tokenizer — treat them as a trend, not a bill.

Missed savings — turo discover

turo gain totals reductions that happened. turo discover shows the ones that didn't: it scans your existing Claude Code history and estimates how many tokens turo would have saved had those sessions run through the proxy.

turo discover — scanned 403 sessions in ~/.claude/projects
  messages       53011 reducible (all roles)
  tokens in      13524093
  would be out   6246528
  would save     7277565 (53%)

by project:
  ~/Projects/api          29490 msgs  saved 3242664 (49%)
  ~/Projects/web          19943 msgs  saved 3142015 (57%)

these sessions ran without turo — capture the savings next time with:
  turo run claude

It reads the per-project session logs under ~/.claude/projects (set CLAUDE_CONFIG_DIR if Claude Code stores them elsewhere), applying the same role gating and compression flags (-level, -filler, -gloss, -arrows, -proxy-all) as the proxy — so the estimate reflects what turo run claude would actually do. Nothing is sent anywhere; the scan is local and read-only.

Health check — turo doctor

turo doctor runs a local health check and exits non-zero if anything is broken — useful in CI, install scripts, or after an upgrade to confirm turo is wired up correctly.

turo doctor

turo
  · version dev (unreleased build)
  · binary /usr/local/bin/turo
  ✓ default level ultra (all roles)

environment
  · no turo env overrides set (using defaults)

gain log
  ✓ writable: ~/Library/Application Support/turo/gain.jsonl (135 events)

claude history (turo discover source)
  ✓ 414 session logs in ~/.claude/projects

pipeline self-test
  ✓ 22 -> 5 tokens (77% smaller) at level ultra

agents
  ✓ Claude Code — detected, skill installed
  ✓ opencode — detected, skill installed
  · Gemini CLI — detected
  · Cursor — detected
  · Qwen Code — detected
  · 5 of 18 supported agents detected (turo -list-agents shows all)

turo is healthy

What it checks:

Section What it verifies
turo Version string, binary path, default level validity
environment Lists any TURO_* env overrides that are set
gain log Gain directory creatable, log file writable, event count
claude history Session logs found under ~/.claude/projects
pipeline self-test Runs reduce() on a sample sentence, confirms token count decreased
agents Detects installed coding agents, checks skill installation for native agents

Pass -level <name> to test a specific level; an invalid level is reported as a problem (✗) rather than a hard exit, so the full report is still visible.

turo doctor               # healthy -> exit 0
turo -level bogus doctor  # invalid level -> exit 1

Every run is four stages, each on by default:

text -> [1] delete filler -> [2] swap cheaper synonyms -> [3] swap defining words -> [4] reduce to content words
  1. Filler deletion removes pleasantries, hedges, and leaders that survive word-level stopword lists (please, I think, of course, let me), while protecting code, paths, URLs, and identifiers verbatim. Disable with -filler=false / TURO_FILLER=off.
  2. Synonym swap replaces words with a fewer-token synonym (see below). Disable with -synonyms=false / TURO_SYNONYMS=off.
  3. Gloss swap replaces words with the shortest defining word from their dictionary definition — the lossiest stage. Disable with -gloss=false / TURO_GLOSS=off.
  4. Reduction drops the remaining stopwords, keeps content words by part of speech, deduplicates, and (ultra) collapses inflections by lemma.

Arrows (on by default, runs before reduction) rewrites multi-word connective phrases to ->, which the reducer keeps verbatim between the surviving content words. Disable with -arrows=false / TURO_ARROWS=off.

The whole pipeline repeats until the output stops changing (-passes 0, the default; a positive -passes N caps the count). The first pass keeps document structure (headings, per-section bodies); later passes flatten that and dedupe across it, so large structured docs keep shrinking before converging — this README goes 522 (1 pass) -> 376 tokens (converged, ~29 passes). Set -passes 1 to keep it single-shot.

turo never emits output larger than the input: if a stage does not save tokens, the text passes through unchanged.

Synonym substitution (on by default, lossy)

turo runs a first pass that replaces each word with a fewer-token synonym before reducing (utilize -> use, demonstrate -> show). The table is built from WordNet synsets (real synonyms) and frequency-filtered so swaps land on common words, then gated to same-part-of-speech words and validated to cost strictly fewer tokens. turo still passes the original through if the result is not smaller.

Disable it with -synonyms=false or TURO_SYNONYMS=off when you need words verbatim. The gain is usually small — modern tokenizers already encode most words as a single token. WordNet polysemy also leaves some noise (leverage -> purchase), so keep it opt-in for prose, not code. The table is generated by tools/gensyn.py (WordNet + wordfreq + the cl100k tokenizer), so token counts are measured for cl100k, not the target model's tokenizer.

Level What it keeps Reduction
lite Adjectives, nouns, verbs, and leftover adverbs/prepositions ~65%
full Adjectives, nouns, verbs ~70%
ultra (default) Nouns and verbs only, deduplicated by lemma (base form) ~70%+
wenyan ultra, then swap surviving words for a single 文言 (Classical Chinese) character CJK models only
echo "the quick brown fox jumps over the lazy dog" | turo --level lite   # quick brown fox jumps over lazy dog
echo "the quick brown fox jumps over the lazy dog" | turo --level full   # quick brown fox jumps lazy dog
echo "the quick brown fox jumps over the lazy dog" | turo --level ultra  # fox jump dog
echo "the wise king studies the old book" | turo --level wenyan    # 智王學舊書

wenyan (CJK-tokenizer models only)

wenyan reduces at ultra, then swaps each surviving English content word for one Classical Chinese character (water -> 水, king -> 王, verify -> 驗) from a ~380-entry hand-curated lexicon. One char per concept, no spaces (Classical Chinese has none).

Two examples, measured:

input ultra wenyan chars cl100k CJK-model (~1/char)
The wise king uses water and fire... (80 ch) Wise king use water fire person see mountain old tree 智王用水火人見山舊樹 10 15 ~10
the PR-review paragraph (766 ch) 283 ch / 41 tok 閱引請作察變檔驗碼引退存為要查者增試 untested 破診後證文更映交訊覺安隙 unsanitized 戶入 hardcoded 須標併 69 67 ~42

It collapses to the fewest characters (766 -> 69 on the paragraph). A CJK character is 2-3 tokens on OpenAI's cl100k, so wenyan is larger there (67 > 41). It only wins on CJK-optimized tokenizers (Qwen, DeepSeek, GLM), where a common character is ~1 token — then those 69 chars are ~42 tokens vs plain ultra's 71. Don't use it with OpenAI models.

turo's own token estimator counts CJK as 1 rune = 1 token (matching those models), so it treats wenyan as a reduction and never rejects it. Words outside the lexicon stay English (untested, unsanitized, hardcoded above); code/paths/URLs are preserved verbatim. Extend wenyanMap for more coverage.

In ultra , inflections of the same word collapse to one token by their dictionary base form: goes, went, going -> go; children -> child; servers -> server. A reduction is only applied when it lands on a real dictionary word, so no mangled non-words are ever emitted.

Set default via TURO_LEVEL env var.

Proxy — reduce every request for any agent

To compress all input for an agent that turo can't reach from the inside (Claude Code, Codex, ...), route the agent's requests through turo.

turo run — launch an agent with everything reduced (turnkey)

turo run claude          # every claude request reduced, base URL wired for you
turo run codex           # OPENAI_BASE_URL wired instead
turo run                 # list supported agents

turo run starts an in-process proxy on a free port, points the agent's base-URL env var at it (ANTHROPIC_BASE_URL for claude, OPENAI_BASE_URL for OpenAI-compatible agents), execs the agent, and stops the proxy when it exits. One command, no exports, no /turo inside the agent. Supported: claude, codex, opencode, qwen, aider, crush, goose, amp.

turo -proxy — the proxy on its own

turo -proxy -upstream https://api.openai.com   # silent by default, listens on 127.0.0.1:8787
turo -proxy -proxy-verbose                      # print activity: token summary + before -> after text
turo -proxy -proxy-all=false                    # reduce only user + tool, not every role
export OPENAI_BASE_URL=http://127.0.0.1:8787/v1

Every /chat/completions (and Anthropic /messages) request has its message content reduced before it reaches the real endpoint; the response streams back untouched. By default every role is reduced (-proxy-all is on); pass -proxy-all=false to reduce only user and tool content and leave system and assistant history verbatim. Auth headers pass through; non-chat paths are forwarded unchanged.

The proxy is silent by default. Pass -proxy-verbose to print its activity: the estimated before -> after token count per request plus each message's text before and after (truncated for the terminal). The flag also applies to turo run <agent>.

kdeps does not need this: in agent mode it already pipes the preamble, input, tool results, and history through turo before every call.

npx turo installs the binary and registers the turo skill + /turo command with every coding agent it finds on your machine — Claude Code, Gemini CLI, opencode, Codex, Cursor, Windsurf, Cline, Copilot, and 20+ more. Install once; every agent gets the same reducer.

npx turo                 # binary + register with detected agents
npx turo --list          # show every supported agent and its status
npx turo --only claude   # register with one agent
npx turo --all           # register with every supported agent
npx turo --no-binary     # register agents only (binary already installed)
npx turo --uninstall     # remove binary + registered skills

Under the hood each agent gets one of:

  • Claude Code / opencode — the skill and /turo command are copied into the agent's config dir
  • Gemini CLI — gemini extensions install
  • everything else — npx skills add kdeps/turo --skill turo -a <profile>

Once turo is on PATH, any agent can also pipe context through it directly:

cat CLAUDE.md | turo               # compact system prompt
cat error.log | turo               # reduce log output

Set TURO_LEVEL=ultra for maximum compression. KDEPS_TURO=off or TURO_DISABLED=1 to disable.

  • Code blocks and inline code — passed through unchanged
  • URLs, file paths, version numbers — verbatim
  • Technical terms (API names, CLI commands, error strings) — exact
  1. Embedded English dictionary (120k words, 14MB) classifies every word
  2. Strips articles, prepositions, conjunctions, pronouns (~70 stop words)
  3. Keeps the content words for the level (nouns, verbs, adjectives)
  4. Deduplicates and emits them in reading order — then keeps the result only if it is actually smaller than the input

caveman is a sibling idea with the opposite dial. caveman deletes filler with regex (articles, pleasantries, hedges) and keeps readable prose. turo runs that same filler pass, then keeps going — POS-classifying, deduplicating, lemmatizing, and swapping words for shorter synonyms and glosses — trading readability for a much smaller token count.

Same input, measured with the cl100k tokenizer:

| output | tokens
---|---|---
input | Please, I think you should really just utilize this approach to demonstrate the functionality of the component. | 19
caveman | You should utilize this approach to demonstrate functionality of component. | 11
turo full | Use come show functionality component | 5
turo ultra | Use come show component | 4
| caveman | turo
---|---|---
method | regex filler removal | dict POS + dedup + lemma + synonyms + gloss
output | readable prose | keyword stream
dictionary / WordNet | no | yes
synonym / gloss swaps | no | yes (on by default)
best when | you still need to read it | you only feed it to an LLM

Use caveman when a human reads the result; use turo when only a model does.

System prompts are 50-200k tokens. Most of those words carry grammar, not meaning. Turo points at what matters and drops the rest.

Point more. Token less.