Show HN：我将 DeepSeek V4 Flash 缩至 57GB，它在我的 Mac 上写出了编译器
英文标题：🔶 Show HN: I shrank DeepSeek V4 Flash to 57GB and it wrote a compiler on my Mac
这是一个面向编程与 coding-agent 工作负载的 DeepSeek-V4-Flash-0731 精简版 MoEspresso 包，模型 safetensors 仅为 56.831 GB，较 2.37 bpw 版本减少 32.63%。该包通过移除约 800 亿参数，将总参数量降至 204B，但每个 token 仍约有 13B 激活参数；作者展示了它在 Mac 上编写小型 C 编译器的录屏。测试显示，该版本的代码建模能力保留较多，与 2.37 bpw 包的选定 token 一致率为 88.55%，但通用文本和 API 延续质量存在明显下降。开发者需要使用 MoEspresso 2.1 或更高版本运行，该格式不兼容 llama.cpp、transformers 和 mlx-lm。
4.5 KB
3.6 KB
425.9 KB
中文
DeepSeek V4 Flash 0731 Coder：紧凑型 MoEspresso 包
这是一个从 DeepSeek-V4-Flash-0731 派生的、面向编程任务的 56.869 GB MoEspresso 包。其模型 safetensors 文件占用 56.831 GB，比 84.355 GB 的 MoEspresso 2.37 bpw 包 小 32.63%。该包移除了约 800 亿参数，保留总计 204B 参数，每个 token 约激活 13B 参数。
你可以在这里观看它编写一个小型 C 编译器的录屏：
该包要求使用 MoEspresso 2.1 或更高版本。MoEspresso 是唯一能够提供该模型服务的引擎；llama.cpp、transformers 和 mlx-lm 均无法加载这些文件。
brew install steadfastgaze/tap/moespresso
moespresso serve /path/to/package
该包面向编程和 coding-agent 工作负载，并非 2.37 bpw 包的更小型通用替代品。以下测量结果同时展示了其编程能力保留情况和通用文本能力退化情况。
质量
NLL 和困惑度越低越好。代码面板包含来自 code.utf8 的 12 个固定 2,048-token 窗口，以及 24,564 个下一个 token 目标。
| package | 模型 safetensors | 代码平均 NLL | 代码困惑度 | 与 2.37 bpw 的选定 token 一致率 |
|---|---|---|---|---|
| 此 Coder 包 | 56.831 GB |
1.01758 |
2.7665 |
88.55% |
| MoEspresso 2.37 bpw | 84.355 GB |
0.88582 |
2.4250 |
参考值 |
研究了一个未经剪枝、全部使用 IQ1 且无 REAP 的研究基线，但未予发布。相对于该基线，剪枝和专门化使 WikiText 平均 NLL 增加 0.25170，同时使代码平均 NLL 降低 0.10250。来自 2.37 bpw 包的大部分通用文本能力损失由极端 IQ1 量化造成；后续的专门化则以进一步降低通用文本质量为代价，换取更好的代码建模能力。
通用文本和官方续写面板量化了由此形成的产品边界。
| package | WikiText 平均 NLL | WikiText 困惑度 | API 续写平均 NLL | API 首 token 一致数 |
|---|---|---|---|---|
| 此 Coder 包 | 2.41630 |
11.2043 |
1.23133 |
48/100 |
| MoEspresso 2.37 bpw | 1.71466 |
5.5548 |
0.39634 |
66/100 |
WikiText 面板使用 WikiText-2 测试集中的前 32 个连续 2,048-token 窗口，并对每个窗口的最后 1,023 个 token 打分，共计 32,736 个目标。API 面板使用来自官方 DeepSeek 0731 模型的 100 个提示词和 2,313 个由提供方选定的续写 token。每个本地包均在存储的续写内容下进行 teacher-forced 推理。这并非自由生成状态下的 API 一致性测试。
渲染后的推理和工具验证面板使用转换为 DeepSeek 提示词及工具调用格式的留出对话。
| 类别 | 位置数 | Coder NLL | 2.37 bpw NLL | Coder 选定 token 准确率 | 2.37 bpw 准确率 |
|---|---|---|---|---|---|
| 所有渲染目标 | 8,188 |
1.33567 |
1.31694 |
75.67% |
75.95% |
| 推理 | 1,806 |
1.46925 |
1.47384 |
66.11% |
66.06% |
| 结构化工具调用 | 1,880 |
0.46838 |
0.39007 |
90.05% |
90.69% |
| 工具结果历史 | 850 |
1.12205 |
1.11188 |
76.59% |
78.59% |
该包还通过了额外检查，例如：
| 检查项 | 范围 | 结果 |
|---|---|---|
| 长上下文事实回忆 | 一个 30,000-token 级提示词中的 16 条固定事实 | 16/16 完全匹配 |
| 编程能力 | 16 个私有测试 | 16/16 测试通过 |
这些面板在包开发期间使用。它们用于描述最终产物，但并非未经后续选择影响的基准测试。
配方
三个哈希路由层保留全部 256 个专家。40 个学习型路由层中的每一层保留 256 个专家中的 176 个，合计保留 7,808 行专家参数。服务端路由器仍为每个 token 选择六个专家。
专家保留策略与精度共同确定。在 129 个路由投影单元中，32 个使用 IQ1_S_R4，64 个使用 IQ2_KS，33 个使用 IQ2_K。更稠密的模型权重使用 q6_K。第 12 层下投影被明确提升精度；简短的闭环测试和完整的自回归推理运行表明，仅凭提升数量无法预测持续思考质量。
该包不包含 drafter。可单独分发的 DSpark sidecar 可使用 MoEspresso 2.1 进行校验和服务：
moespresso verify /path/to/package --drafter /path/to/dspark-sidecar
moespresso serve /path/to/package --drafter /path/to/dspark-sidecar
语料库与专门化
权重量化使用了 Bartowski 的 calibration_datav5 和 exllamav3 的 technical.utf8。
专家保留使用了来自 calibration_datav5 的通用窗口、来自 exllamav3 的 code.utf8 的编程窗口、无关的日常聊天内容，以及 Reasoning and Tool Calling 数据集的 conversations 训练集切分。
WikiText-2 仅用于评估。
验证
下载后，请先验证完整目录，再提供服务：
moespresso verify /path/to/package
致谢
基础模型由 DeepSeek 训练并发布。IQ_K 格式由 Iwan Kawrakow 设计，并通过 mlx-iqk 提供服务。稠密 q6_K 路径使用 mlx-kquant。构建数据包含 Bartowski 和 turboderp-org 发布的材料。MoEspresso 运行于 Apple 的 MLX。
完整的第三方来源记录在 MoEspresso 仓库的 THIRD-PARTY-NOTICES 文件中，以及其依赖项随附的通知文件中。
English / 原文
DeepSeek V4 Flash 0731 Coder, compact MoEspresso package
This is a 56.869 GB coding-specialized MoEspresso package derived from DeepSeek-V4-Flash-0731. Its model safetensors occupy 56.831 GB, 32.63% less than the 84.355 GB MoEspresso 2.37 bpw package. The package removes about 80B parameters, leaving 204B total, with about 13B active per token.
You can see a screen recording of it writing a small C compiler here:
The package requires MoEspresso 2.1 or newer. MoEspresso is the only engine that serves it. llama.cpp, transformers, and mlx-lm will not load these files.
brew install steadfastgaze/tap/moespresso
moespresso serve /path/to/package
This package is intended for coding and coding-agent workloads. It is not a smaller general-purpose replacement for the 2.37 bpw package. The measurements below show both the coding retention and the general-text regression.
Quality
Lower NLL and perplexity are better. The code panel contains 12 fixed 2,048-token windows and 24,564 next-token targets from code.utf8.
| package | model safetensors | code mean NLL | code perplexity | selected-token agreement with 2.37 bpw |
|---|---|---|---|---|
| This Coder package | 56.831 GB |
1.01758 |
2.7665 |
88.55% |
| MoEspresso 2.37 bpw | 84.355 GB |
0.88582 |
2.4250 |
reference |
An unpruned all-IQ1 no-REAP research baseline was researched, although not published, and relative to that baseline, pruning and specialization increased WikiText mean NLL by 0.25170 while reducing code mean NLL by 0.10250. Extreme IQ1 quantization accounts for most of the broad-text loss from the 2.37 bpw package. The subsequent specialization trades additional broad-text quality for better code modeling.
The broad-text and official-continuation panels quantify the resulting product boundary.
| package | WikiText mean NLL | WikiText perplexity | API-continuation mean NLL | API first-token agreement |
|---|---|---|---|---|
| This Coder package | 2.41630 |
11.2043 |
1.23133 |
48/100 |
| MoEspresso 2.37 bpw | 1.71466 |
5.5548 |
0.39634 |
66/100 |
The WikiText panel uses the first 32 contiguous 2,048-token windows from the WikiText-2 test split and scores the last 1,023 tokens of each window, for 32,736 targets. The API panel uses 100 prompts and 2,313 provider-selected continuation tokens from the official DeepSeek 0731 model. Each local package is teacher-forced under the stored continuation. This is not free-running API agreement.
The rendered reasoning and tool validation panel uses held-out conversations converted to DeepSeek's prompt and tool-call format.
| category | positions | Coder NLL | 2.37 bpw NLL | Coder selected-token accuracy | 2.37 bpw accuracy |
|---|---|---|---|---|---|
| All rendered targets | 8,188 |
1.33567 |
1.31694 |
75.67% |
75.95% |
| Reasoning | 1,806 |
1.46925 |
1.47384 |
66.11% |
66.06% |
| Structured tool calls | 1,880 |
0.46838 |
0.39007 |
90.05% |
90.69% |
| Tool-result history | 850 |
1.12205 |
1.11188 |
76.59% |
78.59% |
The package also passes additional checks, e.g.:
| check | scope | result |
|---|---|---|
| Long-context fact recall | 16 fixed facts in a 30,000-token-class prompt | 16/16 exact match |
| Coding capability | 16 private tests | 16/16 tests |
These panels were used during package development. They characterize the final artifact but are not untouched post-selection benchmarks.
Recipe
The three hash-routed layers keep all 256 experts. Each of the 40 learned-router layers keeps 176 of 256 experts, for 7,808 retained expert rows in total. The served router still selects six experts per token.
Expert retention and precision were selected together. Of the 129 routed projection cells, 32 use IQ1_S_R4, 64 use IQ2_KS, and 33 use IQ2_K. The denser model weights use q6_K. The layer-12 down projection is explicitly promoted; short closure screens and full autoregressive reasoning runs showed that promotion count alone did not predict sustained-thinking quality.
The package does not include a drafter. A separately distributed DSpark sidecar can be checked and served with MoEspresso 2.1:
moespresso verify /path/to/package --drafter /path/to/dspark-sidecar
moespresso serve /path/to/package --drafter /path/to/dspark-sidecar
Corpus and specialization
Weight quantization used Bartowski's calibration_datav5 and exllamav3's technical.utf8.
Expert retention used general windows from calibration_datav5, coding windows from exllamav3's code.utf8, unrelated casual chat, and the conversations train split of Reasoning and Tool Calling.
WikiText-2 was used only for evaluation.
Verification
After downloading, verify the complete directory before serving it:
moespresso verify /path/to/package
Credits
The base model was trained and released by DeepSeek. The IQ_K formats were designed by Iwan Kawrakow and are served through mlx-iqk. The dense q6_K path uses mlx-kquant. Construction data includes material published by Bartowski and turboderp-org. MoEspresso runs on Apple's MLX.
The complete third-party provenance is recorded in the MoEspresso repository's THIRD-PARTY-NOTICES and in the notices distributed by its dependencies.