💻🤖 Gemma4-12B v2——编程与智能体特化版 ✨

🐣 小巧身材，强大脑力——面向所有人的本地编程与工具使用智能体

无论您的 GPU 如何，无论您的内存多大。 只要空余 ~4.5 GB 显存或统一内存，您现在就可以运行自己的私有、离线编程智能体。🚀 v2 是一次重大的智能体升级——它会先行阅读、推理、使用工具，完成多步技术任务后再行动。🧠🛠️ 完全本地，完全属于您，无需 API，无需云端。

---

📊 头条——它能作为智能体工作（tau2-bench）

v2 专为编程 + 智能体工作打造——编写代码、运行命令、使用工具、调试、多步技术任务。最清晰的信号来自 tau2-benchtelecom，一个智能体工具使用基准，其诊断 → 修复 → 验证循环真实反映了终端/调试工作：

tau2-bench telecom · 20 项任务 · 本地、相同框架、全部 Q8_0	得分
官方 gemma-4-12B-it（基座）	~15%
🟢 Gemma4-12B v2（本模型）	~55%

→ 在技术智能体任务上，大约是基座模型的 3.5 倍。🎯 想了解完整故事——为什么选 telecom、两类模型如何以不同方式失败、诚实的注意事项以及权衡（包括通用知识）？全部拆解在下方。👇

---

🚀 公告

📌 遇到问题？请先查看我置顶的讨论。 ~99% 的问题来自客户端/采样器配置，而非模型权重——那里有快速修复方法。例如：乱码或重复的0000… 输出几乎总是意味着未设置重复惩罚（设置 rep_pen 1.1，temp 1.0）；泄露的 <|tool_call> / <|channel> 令牌意味着您的前端未解析 Gemma 4 的原生工具格式（使用 llama.cpp 的 --jinja）。如果您的疑问未被覆盖，尽管发起讨论——我会尽快阅读并回复。💬

📦 本次发布无 Q2_K 版本。 我完成了 Q2_K（imatrix）构建，但在实际压力测试中表现不佳，所以我将其撤回——我只会在确信质量足够好时才发布量化版本。 最小的可靠选项是 Q3_K_M；Q4_K_M 是推荐的最佳平衡点。🙏

🔮 v3 已经在路上了。 老实说？连我自己都没想到微调后的提升这么大——所以我会继续推进。v3 仍聚焦编程 + 智能体，目标是更进一步。敬请期待！🎉

🐘 还有一个更大的兄弟即将到来——Qwen3.6-27B。 我还开始使用同样的编程 + 智能体配方微调 Qwen3.6-27B，面向那些拥有充足算力、想要更强原始能力的用户。但我没有忘记这个项目的初衷：27B 对某些 GPU/内存来说可能太重了。所以这不是替代品——我会同时并行推进 v3（这个 12B 系列），它只会变得更强。💪 无论您硬件如何，总有一款适合您的模型。 💚

---

💚 个人寄语——感谢与几句真心话（请读）

首先，非常感谢您分享的所有数据和帮助。 🙏 苦乐参半的是：我们谁也没料到 Fable 5 会被退役——只有我自己的数据集保留了 Fable 5 真实的、自撰的思维链。因此，对于社区贡献的每个数据集，我使用 Opus 4.8（xhigh）从头重建了缺失的推理。这或许与原始 Fable 5 样本有出入，但这是唯一可行的路径——而改进结果确实非常、非常巨大（差点让我从椅子上跳起来 😄）。基准数字就在上面。👆

其次——我尽力回复了每一条社区评论，并公开承担了 v1 的训练问题。真的，谢谢：你们的反馈让我得以改进。💚

因为 v1 冲上热榜第一，也吸引了一些恶意言论/捣乱。我将温和而坚定地说：欢迎真正的批评——但纯侮辱性言论不受欢迎。 这是一个本地模型，任何人都能在极小的 RAM/VRAM 上运行的能力，零 API 费用且完全私密；我甚至开源了完整的 safetensors 主文件以供研究和构建。如果有问题，就实际问题发起讨论——我真心愿意倾听并行动。但仅有无意义侮辱的评论对任何人都无益，我会毫不犹豫地删除。🙏

请记住：我是个人——不是为了营销或后续变现而发布“开放”模型的实验室。我不打广告。我用自己的时间和金钱为您构建这一切：合成数据，手工审查与清洗，拆分和重新分段（本轮我甚至构建了一个动态上下文窗口通道来保持智能体先读后做步骤的完整性），阅读最新论文，然后训练 → 评估 → 训练 → 评估。它耗尽了一个完整的 Claude Max 20× 计划（我为自己的工作保留了单独的 Pro），而仅 v2 就花费了 40 多个小时——即使使用 Opus 4.8，数据也抛出层出不穷的难题，我必须亲自验证。真心感谢。🐾

---

🔬 基准测试详解（tau2-bench）

我对 v2 在 tau2-bench（一个智能体工具使用基准）上进行了评估。我没有运行整套测试——这非常耗时——所以我只聚焦于一个最匹配 v2 目标的领域。

为什么是 tau2-benchtelecom？ 电信故障排除要求智能体使用读取/检查工具进行诊断 → 定位问题 → 应用修复 → 验证——在结构上与真实终端/调试工作（检查状态 → 诊断 → 修复 → 确认）相同的循环。这正是本模型应当擅长的领域，因此它是衡量 v2 的正确标尺（比购物/客服领域更有意义）。

tau2-bench telecom · 20 项任务 · 本地、相同框架、全部 Q8_0	得分
官方 gemma-4-12B-it（基座）	~15%
🟢 Gemma4-12B v2（本模型）	~55%

→ 在技术智能体任务上，大约是基座模型的 3.5 倍。🎯

务实，而非凭空捏造。 独立进行的编程/终端虚构探测（任务故意诱使模型编造文件路径/函数签名/值）发现，v2 在行动前先做好铺垫——就像基座模型一样——它会先 grep/read/ls，不会编造任何东西（0% 虚构，与基座模型持平）。

有趣之处——它们如何失败。 基座模型过早放弃：本次运行中它 10 次转向人工坐席（transfer_to_human），而不是完成修复。v2 持续进行——它留在循环中，像一个大得多的模型那样处理问题，这正是它能解决更多任务的原因。它还不完美：有时仍会略微挣扎（过度尝试、重试）。余下的部分失误实际上是基准测试自带 APN 工具的缺陷（它在应能优雅处理的输入上抛出异常），而非模型问题。需要明确：我不会为了抬高分数而修补基准测试工具或泄露测试问题——我宁愿报告一个诚实的数字并改进模型本身。更多训练将在 v3 中进行。 🔧

关于retail（客服购物）： 在 tau2-bench retail 上，基座模型得分略高于 v2。这完全在意料之中，也是设计使然。 Retail 是纯粹的客服（查询用户、处理订单）——不是本模型的目标场景。v2 专精于编程/终端/技术智能体工作，在这些方面（telecom）它远超基座模型。需要客服机器人？这不是它。需要一个本地编程/智能体模型？它正是。💚

对规模保持诚实。 当今的前沿模型——比如 mimo-v2.5-pro 或 Opus 4.8——在此 telecom 基准上全部达到 90%+。它们也非常庞大。对于一个 12B 模型，我粗略猜测 v3 可能顶多达到 60–70%（强调猜测——我甚至还没开始 v3）。所以我们要头脑清醒：与前沿模型之间仍存在真实差距。但请记住规模——这是一个运行在您自己机器上的 12B 模型，尽可能在此尺寸下缩小差距，正是全部意义所在。💪

以及权衡——没有免费的午餐。 我还运行了通用知识基准（MMLU-Pro），v2 在那里的得分略低于基座模型。这对于聚焦的微调来说完全正常且在意料之中：当您大力推进编程 + 智能体时，您会为此牺牲一小部分广泛知识的广度。需要通才？试试我自己的通用用途 Claude Opus 4.6/4.8 蒸馏版——或原始的 google/gemma-4-12B-it 基座。需要一个本地编程/智能体工作者？那就是 v2 调优的目的。

🔬 方法，诚实地说： 这些是本地、相同框架、相对数字（全部模型以 Q8_0 测试，贪婪解码，自模拟用户，20 项任务）。它们不可直接与已发布的 tau2-bench 排行榜数据比较（不同用户模拟器、完整任务集、全精度）——本地自评估的运行得分系统性偏低。请将其解读为**“v2 与基座模型在完全相同的条件下”**的比较，这才是这里真正有意义的对比。

---

📚 v2 新增内容（训练）

v2 延续自 v1 编程器，并增加了一个重大的智能体推进——这正是 v1 所缺失的部分：

• 🛠️ 智能体/终端——真实的多步工具使用轨迹（读取 → 推理 → 行动 → 验证），采用 Gemma 4 原生工具协议。正是这一部分推动了 tau2-bench telecom 的跃升，并修复了 v1“在第一步后停止”的行为。
• 💻 编程——在 Python 任务上经过验证的思维链（真实 CoT，以通过测试为门槛），加上针对困难案例的 Fable-5-重制集。
• 📚 通用——精心挑选的推理/指令数据切片，以保持广泛能力。

所有推理均为蒸馏思维链（关于 Fable 5 样本如何用 Opus 4.8 重建，请参见上方个人寄语）。

---

📦 选择您的尺寸（GGUF 量化版本）

量化类型	大小	适用印象
🟡 Q3_K_M	5.7 GB	适合 8 GB 显存
🔵 Q4_K_M	6.87 GB	最佳平衡点 👌（推荐）
🟣 Q6_K	9.11 GB	趋近无损
⚪ Q8_0	11.8 GB	基本完整质量

ℹ️ 本次发布无Q2_K——它尚未通过压力测试（见公告）。最小可靠量化 = Q3_K_M。

---

🚀 如何运行

方案 A —— llama.cpp（推荐）🦙

⚠️ 需要较新版本的 llama.cpp（这是 gemma4_unified 架构——旧版本构建无法加载）。

@echo off cd /d C:\llama.cpp llama-server.exe ^ -m C:\models\gemma4-v2-Q4_K_M.gguf ^ --ctx-size 16384 ^ --n-gpu-layers 99 ^ --no-mmap -fa on ^ --jinja ^ --temp 1.0 --top-p 0.95 --top-k 64 ^ --host 0.0.0.0 --port 18080 pause

• 🛠️ 智能体使用： 通过 OpenAI 的 tools 字段传递工具（配合 --jinja 使用）。v2 以 Gemma 4 原生协议发出结构化工具调用，并且在智能体循环（read/grep/edit/run，然后 verify）中表现良好。
• 🖱️ 一键式应用： LM Studio / Jan / Ollama ——导入 GGUF，选择量化版本，即可使用。

🧠 思考模式

v2 在回答前以 Gemma 的原生思考通道进行思考（保持 enable_thinking=true，默认聊天模板会处理）。推荐采样参数：temp 1.0, top_p 0.95, top_k 64；编程时也可使用贪婪解码（temp 0）。

---

⚠️ 须知

• 专精于编程/终端/智能体。 通用知识中的事实/数字仍应二次核对。
• 减少拒答： 以任务为导向的训练，而非安全对齐——生产中请自行添加防护栏。负责任地使用。🙏
• 以英语为中心。

---

📚 基础模型与许可

• 许可：Apache 2.0。 Gemma 4 由 Google 依据 Apache 2.0 发布（不同于旧版 Gemma 1/2/3 条款），因此本次微调也是 Apache 2.0——自由使用、修改和重新分发。🎉
• 基础模型： google/gemma-4-12B-it。
• 个人/业余项目——按原样分享，无担保。投入了时间、心血和大量咖啡。玩得开心！🐾✨

---

⚡ 推测解码（MTP 草稿）——已验证构建

MTP/ 文件夹提供 Gemma 4 多令牌预测草稿（unsloth 对 Google 官方 gemma-4-12B-it-assistant 的 GGUF 转换），用于推测解码。Gemma 4 MTP 已合入 llama.cpp 主线（PR #23398）——无需使用分支——但 gemma4-assistant 加载器目前对构建版本敏感，因此请使用下述确切构建：

• ✅ 已验证可用：llama.cppb9553（提交 9e3b928fd）。 我用 gemma4-v2-Q8_0 + MTP-Q8_0 草稿重现：加载顺畅并加速生成（在简单确定性提示上约 88 → ~180 tok/s；实际编程/思考时预计约 1.2–1.3 倍）。无论何种情况均无损。
• ⚠️ 较新构建（如 b9702 / b9717）当前在加载草稿时崩溃，抛出 invalid vector subscript。这是 gemma4-assistant 加载路径的上游回归，并非这些 GGUF 的问题——相同文件在 b9553 上加载正常。继续使用 b9553 直到上游修复。

在 b9553 上的有效命令行（注意旧标志名称——--model-draft，而非 --spec-draft-model）：

llama-server -m gemma4-v2-Q8_0.gguf ^ --model-draft MTP\gemma-4-12B-it-MTP-Q8_0.gguf ^ --spec-type draft-mtp --spec-draft-n-max 4 ^ -ngl 99 -ngld 99 -fa on --jinja

ℹ️ Gemma4Assistant requires ctx_other to be set (this is normal during memory fitting) 这行无害。草稿是通用 Gemma 4 助手（未针对 v2 重新训练），因此接受率略低于模型专属草稿——仍 100% 无损。在小显存卡上，Q8 主模型 + 长上下文 + 草稿可能吃紧；如果遇到 OOM，请降低到 Q6_K/Q4_K_M 或减小 --ctx-size。

💻🤖 Gemma4-12B v2 — Coding + Agentic Edition ✨

🐣 Tiny footprint, big brain — a local coding & tool-using agent for everyone

No matter your GPU. No matter your RAM. With ~4.5 GB of VRAM or unified memory free, you can run your own private, offline coding agent right now. 🚀 v2 is the big agentic upgrade — it reads, reasons, uses tools , and works through multi-step technical tasks before it acts. 🧠🛠️ All local, all yours, no API, no cloud.

---

📊 The headline — it works as an agent (tau2-bench)

v2 is built for coding + agentic work — writing code, running commands, using tools, debugging, multi-step technical tasks. The clearest signal is tau2-benchtelecom, an agentic tool-use benchmark whose diagnose → fix → verify loop mirrors real terminal/debugging work:

tau2-bench telecom · 20 tasks · local, same harness, all Q8_0	score
official gemma-4-12B-it (base)	~15%
🟢 Gemma4-12B v2 (this model)	~55%

→ Roughly 3.5× higher than the base model on technical-agentic tasks. 🎯 Want the full story — why telecom, how the two models fail differently, the honest caveats, and the trade-offs (including general knowledge)? It's all broken down further below. 👇

---

🚀 Announcements

📌 Hitting a problem? Please check my pinned discussion first. ~99% of issues are a client/sampler config, not the weights — and they have a quick fix there. For example: garbled or repeating0000… output almost always means no repetition penalty (set rep_pen 1.1, temp 1.0); and leaked <|tool_call> / <|channel> tokens mean your front-end isn't parsing Gemma 4's native tool format (use llama.cpp --jinja). If your question isn't covered, don't hesitate to open a discussion — I read them and reply as fast as I can. 💬

📦 No Q2_K this release. I finished a Q2_K (imatrix) build, but it didn't hold up under real stress-testing, so I'm holding it back — I only ship a quant once I'm confident it's genuinely good. Smallest reliable option is Q3_K_M ; Q4_K_M is the recommended sweet spot. 🙏

🔮 v3 is already on the way. Honestly? Even I didn't expect the post-training jump to be this large — so I'm pushing further. v3 keeps the coding + agentic focus and aims higher still. Stay tuned! 🎉

🐘 And a bigger sibling is coming — Qwen3.6-27B. I've also started fine-tuning Qwen3.6-27B with the same coding + agentic recipe, for those of you who do have the headroom and want more raw capability. But I haven't forgotten what this project is about: a 27B may be too heavy for some of your GPUs / RAM. So this is not a replacement — I'm pushing v3 (this 12B line) in parallel, at the same time , and it will only get stronger. 💪 No matter your hardware, you'll have a model that fits. 💚

---

💚 A personal note — thank you, and a few honest words (please read)

First, a huge thank-you for all the data and help you've shared. 🙏 The bittersweet part: none of us saw it coming that Fable 5 would be retired — and only my own dataset holds Fable 5's genuine, self-authored chain-of-thought. So for every dataset the community contributed, I rebuilt the missing reasoning from scratch with Opus 4.8 (xhigh). It may diverge from the original Fable 5 traces, but it was the only workable path — and the improvement turned out really,really huge (it nearly launched me out of my chair 😄). The benchmark numbers are right above. 👆

Second — I've tried to reply to every community comment , and I've openly owned v1's training problems. Truly, thank you: your feedback is what lets me improve. 💚

Because v1 hit #1 trending , it also attracted some bad words / trolling. I'll say this gently but firmly: real criticism is always welcome here — pure insults are not. This is a local model that lets anyone run a capable AI on tiny RAM/VRAM, at zero API cost and fully private ; I even open-sourced the full safetensors master to study and build on. If something's off, open a discussion about the actual problem — I genuinely want to hear it and I'll act on it. But comments that are only insults help no one, and I'll remove them without hesitation. 🙏

Please remember: I'm one person — not a lab shipping an "open" model for marketing or to monetize later. I don't advertise. I build this for you on my own time and my own money : synthesizing data, reviewing and cleaning it by hand, splitting and re-segmenting it (this round I even built a dynamic context-window pass to keep the agent's read-before-act steps intact), reading the latest papers, then training → evaluating → training → evaluating. It burned through an entire Claude Max 20× plan (I keep a separate Pro for my own work), and v2 alone cost 40+ hours — even with Opus 4.8, the data threw constant curveballs I had to verify myself. Thank you, truly. 🐾

---

🔬 The benchmarks, in detail (tau2-bench)

I evaluated v2 on tau2-bench (an agentic tool-use benchmark). I did not run the whole suite — it's very time-consuming — so I focused on the single domain that best matches what v2 is for.

Why tau2-benchtelecom? Telecom troubleshooting makes the agent diagnose with read/inspect tools → pinpoint the issue → apply a fix → verify it — structurally the same loop as real terminal/debugging work (check state → diagnose → fix → confirm). That's exactly what this model is meant to be good at, which makes it the right yardstick for v2 (much more so than a shopping/customer-service domain).

tau2-bench telecom · 20 tasks · local, same harness, all Q8_0	score
official gemma-4-12B-it (base)	~15%
🟢 Gemma4-12B v2 (this model)	~55%

→ Roughly 3.5× higher than the base model on technical-agentic tasks. 🎯

Grounded, not made-up. Independently, a coding/terminal fabrication probe (tasks that deliberately tempt the model to invent file paths / function signatures / values) found v2 grounds before it acts just like the base — it grep/read/ls first, and doesn't make things up (0% fabrication, on par with the base model).

The interesting part — how they fail. The base model gives up early : on this run it bailed to a human agent 10 times (transfer_to_human) instead of finishing the fix. v2 keeps going — it stays in the loop and works the problem the way a much bigger model would, which is exactly why it solves so many more. It's not perfect yet: it still flails a little sometimes (over-trying, retrying). And some of the remaining misses are actually a bug in the benchmark's own APN tool (it throws on inputs it should handle gracefully), not the model. To be clear: I will not patch the benchmark's tools or leak its test questions just to inflate my score — I'd rather report an honest number and improve the model itself. More training is coming in v3. 🔧

Aboutretail (customer-service shopping): on tau2-bench retail , the base model scores a bit higher than v2. This is fully expected and by design. Retail is pure customer-service (look up a user, process an order) — not what this model is for. v2 is specialized for coding / terminal / technical-agentic work, and on those (telecom) it dramatically outperforms the base. Need a customer-service bot? This isn't it. Need a local coding/agentic model? It is. 💚

Let's keep it honest about scale. Today's frontier models — think mimo-v2.5-pro or Opus 4.8 — all land 90%+ on this telecom benchmark. They're also enormous. For a 12B model, my rough guess is that v3 might top out somewhere around 60–70% (emphasis on guess — I haven't even started v3 yet). So let's be clear-eyed: there's still a real gap to the frontier. But keep the scale in mind — this is a 12B model running on your own machine , and narrowing that gap as much as possible at this size is the whole point. 💪

And the trade-off — there's no free lunch. I also ran a general-knowledge benchmark (MMLU-Pro), and v2 lands a little below the base model there. That's completely normal and expected for a focused fine-tune: when you push hard on coding + agentic, you trade a sliver of broad-knowledge breadth for it. Need a generalist? Try my own general-purpose Claude Opus 4.6/4.8 distillation — or the original google/gemma-4-12B-it base. Need a local coding/agentic worker? That's what v2 is tuned for.

🔬 Methodology, honestly: these are local, same-harness, relative numbers (all models tested at Q8_0 , greedy decoding, self-simulated user, 20 tasks). They are not directly comparable to published tau2-bench leaderboard figures (different user-simulator, full task sets, full precision) — local self-eval runs systematically lower than published scores. Read them as "v2 vs the base model under identical conditions" , which is the comparison that actually matters here.

---

📚 What's new in v2 (training)

v2 continues from the v1 coder and adds a big agentic push — the piece v1 was missing:

• 🛠️ Agentic / terminal — real multi-step tool-use trajectories (read → reason → act → verify), in Gemma 4's native tool protocol. This is what drove the tau2-bench telecom jump, and it fixes v1's "stops after the first step" behavior.
• 💻 Coding — verified chain-of-thought over Python tasks (real CoT, gated on passing tests) plus the Fable-5-redo set for the hard cases.
• 📚 General — a curated slice of reasoning/instruction data to keep broad competence.

All reasoning is distilled CoT (see the personal note above on how the Fable 5 traces were rebuilt with Opus 4.8).

---

📦 Pick your size (GGUF quants)

Quant	Size	Vibe
🟡 Q3_K_M	5.7 GB	great for 8 GB VRAM
🔵 Q4_K_M	6.87 GB	the sweet spot 👌 (recommended)
🟣 Q6_K	9.11 GB	near-lossless
⚪ Q8_0	11.8 GB	basically full quality

ℹ️ NoQ2_K this release — it didn't pass stress-testing yet (see Announcements). Smallest reliable quant = Q3_K_M.

---

🚀 How to run it

Option A — llama.cpp (recommended) 🦙

⚠️ Needs a recent llama.cpp (this is the gemma4_unified architecture — older builds won't load it).

@echo off
cd /d C:\llama.cpp
llama-server.exe ^
  -m C:\models\gemma4-v2-Q4_K_M.gguf ^
  --ctx-size 16384 ^
  --n-gpu-layers 99 ^
  --no-mmap -fa on ^
  --jinja ^
  --temp 1.0 --top-p 0.95 --top-k 64 ^
  --host 0.0.0.0 --port 18080
pause

• 🛠️ Agentic use: pass your tools via the OpenAI tools field (works with --jinja). v2 emits structured tool-calls in Gemma 4's native protocol and is happy in agent loops (read/grep/edit/run, then verify).
• 🖱️ One-click apps: LM Studio / Jan / Ollama — import the GGUF, pick a quant, go.

🧠 Thinking mode

v2 thinks in Gemma's native thought channel before answering (keep enable_thinking=true, the default chat template handles it). Recommended sampling: temp 1.0, top_p 0.95, top_k 64; for coding you can also go greedy (temp 0).

---

⚠️ Good to know

• Specialized for coding / terminal / agentic. General-knowledge facts/numbers should still be double-checked.
• Reduced refusals: task-focused training, not safety-aligned — add your own guardrails for production. Use responsibly. 🙏
• English-centric.

---

📚 Base & License

• License: Apache 2.0. Gemma 4 is released by Google under Apache 2.0 (unlike the older Gemma 1/2/3 terms), so this fine-tune is Apache 2.0 too — free to use, modify, and redistribute. 🎉
• Base model: google/gemma-4-12B-it.
• Personal/hobby project — shared as-is, no warranty. Built with time, care, and a lot of coffee. Have fun! 🐾✨

---

⚡ Speculative decoding (MTP draft) — verified build

The MTP/ folder ships the Gemma 4 multi-token-prediction draft (unsloth's GGUF conversion of Google's official gemma-4-12B-it-assistant) for speculative decoding. Gemma 4 MTP is in llama.cpp mainline (PR #23398) — no fork needed — but the gemma4-assistant loader is build-sensitive right now , so please use the exact build below:

• ✅ Verified working: llama.cppb9553 (commit 9e3b928fd). I reproduced it with gemma4-v2-Q8_0 + the MTP-Q8_0 draft: loads cleanly and accelerates generation (~88 → ~180 tok/s on a simple deterministic prompt; expect ~1.2–1.3× on real coding/thinking). Lossless either way.
• ⚠️ Newer builds (e.g. b9702 / b9717) currently crash while loading the draft with invalid vector subscript. This is an upstream regression in the gemma4-assistant loader path, not a problem with these GGUFs — the same files load fine on b9553. Stick with b9553 until it's fixed upstream.

Working command on b9553 (note the older flag names — --model-draft, not --spec-draft-model):

llama-server -m gemma4-v2-Q8_0.gguf ^
  --model-draft MTP\gemma-4-12B-it-MTP-Q8_0.gguf ^
  --spec-type draft-mtp --spec-draft-n-max 4 ^
  -ngl 99 -ngld 99 -fa on --jinja

ℹ️ The Gemma4Assistant requires ctx_other to be set (this is normal during memory fitting) line is harmless. The draft is the generic Gemma 4 assistant (not retrained for v2), so acceptance is a touch lower than a model-specific draft would give — still 100% lossless. On small-VRAM cards, Q8 main + long context + the draft can be tight; drop to Q6_K/Q4_K_M or a smaller --ctx-size if you hit OOM.