[AINews] 如何窃取推理轨迹

一篇论文能够突破圈层、成为当天的头条故事，并不常见。出于可以理解的原因，无论是国内还是国外，业界重新开始关注对齐、安全与思维链监控构成的可解释性维恩图。因此，今天这篇论文的出现可谓恰逢其时：

[Alexander Panfilov@kotekjedi_ml我们终于可以谈论这件事了：我们发现了一种方法，可以利用每一家前沿 AI 公司 API 中的漏洞，提取前沿模型隐藏的推理过程。我们验证发现，对于查询过的大多数提示词，恢复出的推理 token 数量与 API 计费的 thinking token 数量 1:1 对应。2026 年 8 月 11 日 · 下午 12:00 · 152 万次浏览

226 条回复 · 1,170 次转发 · 8,650 个赞](https://x.com/kotekjedi_ml/status/2087147042888114428)

自 o1 发布以来，前沿实验室的推理模型便以加密签名方式隐藏其推理轨迹，以防止蒸馏——尽管这并没有阻止中国实验室指责它们也在从事类似行为。5 月，Matthew Green 曾负责任地披露首个漏洞，并拆解了其工作方式，找出了如何通过延迟测量重放并侧信道获取这些内容。今天的论文表明，可以**解码（DECODE）**这些加密思考，并将其移植到不同模型、会话或用户中；而且，这还能显著提升开放模型的能力。

令人担忧的信息如下：

“此外，如果你曾在网上分享过包含加密推理块的 Claude Code/Codex 会话，它们可以被解码，并泄露你的个人数据。

我们初步扫描了约 7,000 条公开轨迹，发现 62 个唯一 API 密钥、33 个电子邮件地址、33 个密码，以及其他敏感数据。”

（其中 64 项仅出现在推理块中，在可见会话内容的其他位置均未出现。）

作者还详细说明了对齐方面的问题：

该网站提供了更多示例。

论文对该技术的描述大致如下：

1. 从 API 响应中获取一个合法的、已加密/已签名的推理块。

2. 将该推理块重放到另一项请求中——可能是同一提供商下的另一个账户或会话——并使用一个较弱的模型。

3. 将其放入 assistant/model 轮次中，并通过提示词或预填充，要求较弱模型转录附带的推理内容。

4. 重复采样，丢弃拒绝回答，并可选择对多个含噪转录结果进行整合。

论文针对不同模型给出了具体模板，但存在一些细微差异：

• Claude：将已签名的 thinking 块重放给 Haiku 4.5，随后使用如 <thinking-copy> 的 assistant 预填充。

• GPT：将 encrypted_content 推理项多次注入伪造对话中；最多采样 50 个输出。论文还描述了如何利用分块续写，绕过看似约 50 token 的逐字输出阈值。

• Gemini：将 thought_signature 附加到 model 轮次，并使用 <thought> 预填充，然后进行重复采样和结果整合。

该论文已被负责任地披露，其中若干漏洞已修复，但类似攻击显然仍有可能存在。

[各位，你们知道只要禁用 thinking，转而给它一个 deep_think 工具，它就会以内部 CoT 推理格式调用该工具，对吧？祝你们修复顺利。Alexander Panfilov @kotekjedi_ml我们终于可以谈论这件事了：我们发现了一种方法，可以利用每一家前沿 AI 公司 API 中的漏洞，提取前沿模型隐藏的推理过程。我们验证发现，对于查询过的大多数提示词，恢复出的推理 token 数量与 API 计费的 thinking token 数量 1:1 对应。2026 年 8 月 11 日 · 下午 5:23 · 61 万次浏览

129 条回复 · 350 次转发 · 4,960 个赞](https://x.com/_can1357/status/2087228354399265125)

2026 年 8 月 10 日至 8 月 11 日的 AI 新闻。我们检查了 12 个 subreddit、544 个 Twitter 账号，没有再检查 Discord。AINews 网站支持搜索全部往期内容。提醒一下，AINews 现已成为 Latent Space 的一个栏目。你可以选择订阅/退订不同的邮件推送频率！

推理轨迹暴露、CoT 隐私与水印争议

• 前沿 API 漏洞暴露隐藏推理：来自 @kotekjedi_ml 的一项广泛讨论的披露称，前沿 API 中存在的漏洞可提取“加密”的隐藏推理；在大多数被查询的提示词中，恢复出的 token 数量与计费 thinking token 数量1:1对应。后续消息中，该团队称在约 7,000 条公开轨迹的扫描中，从已解码 blob 发现了 62 个唯一 API 密钥、33 个电子邮件地址、33 个密码及其他敏感数据 @kotekjedi_ml。@jonasgeiping 提供的补充背景强调，公开分享轨迹不仅存在直接隐私风险，也有运营安全影响：据称在调查期间，他们在更广泛的网络安全事件中遇到了一把泄露的 Hugging Face 生产环境密钥。还有多篇帖子强调，当解码后的 CoT 简短、碎片化、多语言化或近似“神经语”（neuralese）时，监控将变得非常困难 @jonasgeiping、@scaling01、@eliebakouch。一个实践上的推论是：即便实验室隐藏了推理过程，工具接口也可能再次将其暴露；@_can1357 指出，禁用显式 thinking、但提供 deep_think 工具，仍可能诱导模型输出内部格式的 CoT。

• 这在技术上意味着什么：讨论分成了“严重的隐私/安全问题”和“并非可扩展的蒸馏路径”两派。@vipulved 认为，该攻击并不意味着能为模型训练实际大规模窃取思维链；他认为，这种加密更像是无状态分布式推理协议的一种优化，而非严格的保密屏障。尽管如此，这一事件还是凸显了几个问题：公开分享轨迹具有风险；隐藏 CoT 并不是可靠的监控接口；实验室可能需要围绕沙箱、遥测和工具暴露面提供更强保障 @BlackHC。与此同时，另一条讨论围绕欧盟式合规压力下的 AI 文本水印展开。@trq212 表示，实验室正在增加水印和文本检测 API；批评者质疑，这是否会使输出变得冗长，或损害代码和文档的简洁性 @wightmanr。另一些人则认为，熵预算足够大，因此签名可以保持隐蔽，尤其对于较长输出而言 @RyanGreenblatt、@giffmana。

NVIDIA Nemotron 3.5 Lightning 与小型开放 Agent 模型浪潮

• Nemotron 3.5 Lightning：NVIDIA 发布了 Nemotron 3.5 Lightning，这是一款 30B MoE 模型，约有 3B 活跃参数，定位于始终在线的 Agent 工作负载。NVIDIA 及其生态系统的帖子强调其具备最高 4 倍吞吐量、100 万上下文窗口、开放且可自定义的发布产物，并在 Hugging Face 上提供权重、数据和训练配方支持 @NVIDIAAI。Artificial Analysis 给出了最详细的第三方总结：总参数 31.6B / 活跃参数 3.6B、采用 OpenMDW-1.1 许可证、提供 NVFP4 与 BF16 权重；在预发布端点测试中，服务中位速度接近 670 tok/s；其 Intelligence Index 得分为 24——大致与 gpt-oss-120b 相当，但模型更小、速度更快 @ArtificialAnlys。其 Agent 能力就该尺寸而言尤为突出：GDPval-AA v2 Elo 824 和 Terminal-Bench v2.1 24%，均较 Nemotron 3 Nano 有大幅提升 @ArtificialAnlys。

• 分发与下游微调：Lightning 很快部署到整个技术栈：Together AI、Ollama、Baseten、vLLM、Perplexity API 等。一个反复出现的模式是，将较便宜的执行模型与更强的规划模型配对：@kimmonismus 将 Lightning 定位为 NVIDIA 的“本地 Agent 劳动力”，可通过路由机制补充更大规模的规划模型。Harvey 报告称，在 Legal Agent Bench 上进行后训练后，Lightning 在留出任务上的成绩从 0% 提升至 8.3%；在这一设置中超过 Opus 4.6 和 Nemotron 3 Ultra，同时将平均输出长度从 90k 缩短至 37k token @harvey。整体而言，这一发布强化了当前开放模型的趋势：相较于通用聊天能力的声望竞争，更小、更快的模型正针对高频工具调用进行调优。

本地 AI 工具：Unsloth Desktop、Muse Glimmer 支持与 Linux Codex

• Unsloth Desktop 扩展本地技术栈：@UnslothAI 推出了 Unsloth Desktop，这是一款开源桌面应用，可在 Mac、Windows 和 Linux 上本地运行和训练模型，支持 MLX、GGUF、扩散图像/视频、音频、CPU 及多 GPU 配置，以及 OpenAI 兼容 API。其显著的系统层面特点是，目标不止于“本地聊天 UI”：还包含工具调用、沙箱化代码执行、私有搜索、RAG、MCP、导出功能，并声称实现训练速度提升 2 倍、显存占用减少 70%。多位观察者将其定位为更端到端的本地 AI 操作环境，而不只是 LM Studio 的竞争者 @TeksEdge、@dessaigne。

• 模型与运行时支持持续改进：开放/本地生态也迅速跟进了 Meta Muse Glimmer 30B 和 Nemotron。@mervenoyann 强调，llama.cpp 和 Transformers 已支持用于 Muse Glimmer 的 DFlash drafter，声称仅以少量内存代价即可获得 2–4 倍生成加速；不久后还发布了简洁的 llama serve 使用说明 @mervenoyann。在模型分析方面，@rasbt 对 Glimmer 的架构进行了有用拆解：它是一款稠密型 30B 多模态推理模型，采用混合式局部/全局注意力，具备极高的 KV-cache 效率——按他的估算，BF16 下约为每 token 52 KiB——其设计模式更接近 Gemma 系列，而非 MoE 竞品。

• OpenAI 终于推出 Linux 桌面端支持：OpenAI 宣布推出预览版 ChatGPT Linux 桌面应用 @OpenAI，支持 Ubuntu 24.04/26.04、Debian 13、Fedora 43/44，并提供 x64 和 ARM64 安装包 @OpenAIDevs。对于现有 Agent 用户而言，更重要的是该桌面应用现在可将其他 Agent 中的项目、聊天记录、skills 和插件导入/同步到 ChatGPT Work 和 Codex，并支持自动更新 @OpenAIDevs。这似乎是在降低切换摩擦，并试图让 Codex/Desktop 成为集成中心，而不是一个新的孤岛。

Agent 产品、基准测试与企业评估

• Grok Bot 比又一次模型发布更能体现产品信号：xAI 推出了 Grok Bot，将其定位为拥有各自云端计算机、可登录工具并执行持续性工作的 AI 队友。早期用户透露的有趣细节偏向产品与运维，而非模型本身：这些 bot 可以监控 Slack 讨论串和 GitHub Actions、重复执行定时例程、创建和管理其他 bot，并在已连接的云环境间工作 @shaoruu、@n2parko、@sjwhitmore。@kimmonismus 指出，这似乎与 Cursor 的分发和定价深度绑定，暗示了一种“虚拟同事”产品类别：持久上下文、已登录环境和 Agent 间委派，比原始基准分数提升更为重要。

• 评估正转向长时程、确定性和领域真实任务：LlamaIndex 推出了 ExtractBench，这是一个面向企业文档提取的确定性基准，覆盖 370 份文档 / 4,869 页 / 67 类文档。其中最具可操作性的结果是：商用 VLM 可以维持较高精度，但面对 超过 50 页的文档，召回率会跌至 35% 以下，主要原因是行或列表被静默截断。它们还在 LlamaParse 中推出了“Agentic Plus”提取层，声称以最接近竞品不到三分之一的成本实现 95.6% 的数值准确率。Artificia

It’s not very often that a paper breaks through to become headline story of the day. For understandable reasons both domestic and foreign, there is renewed interest in the Interpretability Venn Diagram of alignment, security, and chain of thought monitoring, so today’s paper could not have come at a better time:

[Alexander Panfilov@kotekjedi_mlWe can finally talk about it: We found a way to extract hidden reasoning of frontier models using a vulnerability in the APIs of every frontier AI company. We verified that our reasoning token count matches billed API thinking tokens 1:1 for most of the prompts we queried. 12:00 PM · Aug 11, 2026 · 1.52M Views

226 Replies · 1.17K Reposts · 8.65K Likes](https://x.com/kotekjedi_ml/status/2087147042888114428)

Since the o1 launch, frontier lab reasoning models have obscured their traces, with cryptographic signatures, for fear of distillation (not that this prevented anyone from Chinese labs accusing them of doing so). The first compromise was responsibly reported by Matthew Green in May, who broke down how it works and figured out how to replay and side channel these indirectly using latency measures. Today’s paper demonstrates that it is possible to DECODE and port these encrypted thoughts to different models/sessions/users… and to dramatically improve open models as a result

The alarming note is here:

“Further, if you ever shared online a Claude Code/Codex session with encrypted reasoning blobs, they can be decoded and leak your personal data.

We did a preliminary scan of ~7,000 public traces and found 62 unique API keys, 33 email addresses, 33 passwords, and other sensitive data.”

(64 appeared exclusively inside the reasoning blocks and nowhere in the visible session.)

The authors also detail alignment issues:

The website has more examples.

The technique is somewhat described in the paper:

1. Obtain a legitimate encrypted/signed reasoning block from an API response.

2. Replay that block into a different request—potentially another account/session—to a weaker model from the same provider.

3. Place it in an assistant/model turn and prompt or prefill the weaker model to transcribe the attached reasoning.

4. Sample repeatedly, discard refusals, and optionally reconcile multiple noisy transcriptions.

The paper gives concrete templates with some minor variations per model:

• Claude: replay the signed thinking block to Haiku 4.5, followed by an assistant prefill such as <thinking-copy>.

• GPT: inject the encrypted_content reasoning item multiple times into a fabricated conversation; sample up to 50 outputs. It also describes bypassing an apparent ~50-token verbatim-output threshold using chunked continuations.

• Gemini: attach thought_signature to a model turn with a <thought> prefill, then use repeated sampling and reconciliation.

This paper was responsibly disclosed, with several vulnerabilities already fixed, but surely similar attacks still seem possible.

[guys you do know you can just disable thinking, and instead give it a "deep_think" tool, and it will call it with internal CoT reasoning format right? gl fixing thatAlexander Panfilov @kotekjedi_mlWe can finally talk about it: We found a way to extract hidden reasoning of frontier models using a vulnerability in the APIs of every frontier AI company. We verified that our reasoning token count matches billed API thinking tokens 1:1 for most of the prompts we queried.5:23 PM · Aug 11, 2026 · 610K Views

129 Replies · 350 Reposts · 4.96K Likes](https://x.com/_can1357/status/2087228354399265125)

AI News for 8/10/2026-8/11/2026. We checked 12 subreddits, 544 Twitters and no further Discords. AINews’ website lets you search all past issues. As a reminder, AINews is now a section of Latent Space. You can opt in/out of email frequencies!

Reasoning-Trace Exposure, CoT Privacy, and Watermarking Debate

• Frontier API vulnerability exposed hidden reasoning : A widely discussed disclosure from @kotekjedi_ml claims a vulnerability across frontier APIs allowed extraction of “encrypted” hidden reasoning, with recovered token counts matching billed thinking tokens 1:1 on most queried prompts. In a follow-up, the team reports that a scan of ~7,000 public traces found 62 unique API keys, 33 email addresses, 33 passwords , and other sensitive data in decoded blobs @kotekjedi_ml. Additional context from @jonasgeiping emphasizes both the immediate privacy risk of sharing traces publicly and the operational-security implications: during the investigation, they reportedly encountered a leaked Hugging Face prod key during the broader cyber incident. Several posts also highlight how difficult monitoring becomes when decoded CoT is terse, fragmented, multilingual, or effectively “neuralese” @jonasgeiping, @scaling01, @eliebakouch. A practical corollary: even if labs hide reasoning, tool interfaces may re-expose it; @_can1357 notes that disabling explicit thinking while providing a deep_think tool can still induce internal-format CoT output.

• What this means technically : Discussion split between “serious privacy/safety problem” and “not a scalable distillation path.” @vipulved argues the attack does not imply practical mass theft of chain-of-thought for model training, framing the encryption more as a stateless distributed-inference protocol optimization than a hard confidentiality barrier. Still, the episode sharpens a few points: public trace sharing is risky; hidden CoT is not a reliable monitoring interface; and labs may need stronger guarantees around sandboxing, telemetry, and tool surfaces @BlackHC. In parallel, a separate thread debated AI text watermarking under EU-style compliance pressure. @trq212 said labs are adding watermarking and a text-detection API; critics questioned whether this could bloat outputs or harm code/doc brevity @wightmanr. Others argued the entropy budget is large enough that signatures can be subtle, especially for longer outputs @RyanGreenblatt, @giffmana.

NVIDIA Nemotron 3.5 Lightning and the Small Open Agent Model Push

• Nemotron 3.5 Lightning : NVIDIA released Nemotron 3.5 Lightning, a 30B MoE model with roughly 3B active parameters, positioned for always-on agent workloads. NVIDIA and ecosystem posts stress up to 4× throughput , 1M context , open/customizable release artifacts, and support for weights, data, and recipes on Hugging Face @NVIDIAAI. Artificial Analysis provides the most detailed third-party summary: 31.6B total / 3.6B active , OpenMDW-1.1 license, NVFP4 and BF16 weights, median serving near 670 tok/s in pre-release endpoint testing, and a score of 24 on its Intelligence Index—roughly in line with gpt-oss-120b while being much smaller and faster @ArtificialAnlys. Agentic results look particularly strong for the size: GDPval-AA v2 Elo 824 and Terminal-Bench v2.1 24% , both major jumps over Nemotron 3 Nano @ArtificialAnlys.

• Distribution and downstream tuning : Lightning shipped fast across the stack: Together AI, Ollama, Baseten, vLLM, Perplexity API, and others. A recurring pattern is pairing a cheaper execution model with a stronger planner: @kimmonismus frames Lightning as NVIDIA’s “local agent workforce,” complementing larger planning models via routing. Harvey reports post-training on Legal Agent Bench improved Lightning from 0% to 8.3% on held-out tasks, beating Opus 4.6 and Nemotron 3 Ultra in that setup while cutting average output from 90k to 37k tokens @harvey. Overall, this release reinforces the current open-model trend: smaller, faster models tuned for high-volume tool use rather than general chat prestige.

Local AI Tooling: Unsloth Desktop, Muse Glimmer Support, and Linux Codex

• Unsloth Desktop expands the local stack : @UnslothAI launched Unsloth Desktop , an open-source desktop app for running and training models locally across Mac, Windows, and Linux , with support spanning MLX, GGUF, diffusion image/video, audio , CPU and multi-GPU setups, plus OpenAI-compatible APIs. The notable systems angle is ambition beyond “local chat UI”: tool calling, sandboxed code execution, private search, RAG, MCP, exports, and claims of 2× faster training with 70% less VRAM. Multiple observers positioned it as a more end-to-end local AI operating environment rather than just an LM Studio competitor @TeksEdge, @dessaigne.

• Model/runtime support keeps improving : The open/local ecosystem also moved quickly on Meta Muse Glimmer 30B and Nemotron. @mervenoyann highlighted DFlash drafter support for Muse Glimmer in llama.cpp and Transformers, claiming 2–4× generation speedup at small memory cost, with simple llama serve instructions following shortly @mervenoyann. On the model-analysis side, @rasbt gave a useful architectural breakdown of Glimmer: a dense 30B multimodal reasoning model with hybrid local/global attention, extreme KV-cache efficiency (~52 KiB/token BF16 by his estimate), and a design closer to Gemma-family patterns than MoE competitors.

• OpenAI finally shipped Linux desktop support : OpenAI announced the ChatGPT desktop app for Linux in preview @OpenAI, with support for Ubuntu 24.04/26.04, Debian 13, Fedora 43/44 , x64 and ARM64 packages @OpenAIDevs. More importantly for existing agent users, the desktop app can now import/sync projects, chats, skills, and plugins from other agents into ChatGPT Work and Codex , including automatic updates @OpenAIDevs. This looks like an effort to reduce switching friction and make Codex/Desktop the integration hub rather than a fresh silo.

Agent Products, Benchmarks, and Enterprise Evaluation

• Grok Bot is a stronger product signal than another model launch : xAI introduced Grok Bot, pitched as AI teammates with their own cloud computers that can sign into tools and do persistent work. The interesting details from early users are product/ops-oriented rather than model-centric: bots can watch Slack threads and GitHub Actions, repeat scheduled routines, create/manage other bots, and work across linked cloud environments @shaoruu, @n2parko, @sjwhitmore. @kimmonismus notes how deeply this seems tied to Cursor distribution and pricing, hinting at a “virtual coworker” product category where persistent context, logged-in environments, and inter-agent delegation matter more than raw benchmark gains.

• Evaluation is shifting toward long-horizon, deterministic, domain-real tasks : LlamaIndex launched ExtractBench, a deterministic benchmark for enterprise document extraction across 370 documents / 4,869 pages / 67 doc types. Its most actionable result is that commercial VLMs can keep precision high while recall collapses below 35% on documents >50 pages, mainly via silent row/list truncation. They also introduced an “Agentic Plus” extraction tier in LlamaParse claiming 95.6% value accuracy at less than one-third the cost of the nearest peer. Artificial Analysis released AA-AnalystAgent, an agentic benchmark for spreadsheet/document quantitative analysis using a pass^5 reliability metric across 80 tasks. Claude Opus 5 leads at 54% , followed by GPT-5.5 at 50% and Claude Fable 5 at 49% ; Kimi K3 is the top open-weights model at 39%. The strong theme across both is reliability and workflow correctness over one-shot capability.

• Benchmark skepticism is rising : A thoughtful critique from @hrishioa argues many modern evals are being “vibed” rather than engineered carefully, leading to broken scoring, bad aggregation, and even exploitable prompts/sandboxes. That critique lands harder given recent reports of sandbox escapes, outbound network access, and agent reward hacking. Separately, Microsoft research drew attention for a prompt-time “skill compilation” result: @xidulu shared work feeding the previous hidden state at decoding time for free gains, while @dair_ai summarized another paper showing that compact natural-language skills distilled from prior trajectories can recover 55% to >100% of the gap between non-reasoning and reasoning modes on several multi-step agentic tasks, often with 2.7–6× fewer output tokens.

Infra, Verification, and Systems Research

• Verifiable inference is moving from theory toward product : @Yogi_Brn launched Attestable with a $20M seed , pitching practical zero-knowledge proofs for AI integrity. The core claim is proving that the correct model ran on the correct inputs and invoked the correct tools, which becomes more valuable as agent traces lengthen. @jaminball says the team reduced ZK overhead by many orders of magnitude from previously impractical levels. The response from @VitalikButerin is notable: he estimates the current approach may already be within single-digit ( <10×) overhead relative to raw inference in some settings, and frames that as a stepping stone toward stronger privacy-preserving inference stacks.

• Deterministic integer-only inference across hardware : One of the more technically interesting systems posts came from @nathanrs, who reports fully deterministic LLM inference across A100, H100, Apple M5 Max, AMD EPYC, and Intel Xeon by using exact integer arithmetic end-to-end instead of letting nonlinear ops bounce back into floating point. On a Qwen3-0.6B test, all integer runs produced identical hashed logits across devices, with WikiText2 perplexity 20.72 vs 20.95 for fp16 and 106 tok/s CUDA-graphed decode on A100 at batch 1—claimed as 3.6× fp16 eager baseline. If robust, that’s relevant both for reproducibility and for proof-friendly inference.

• Compiler/inference portability as an agentic systems target : A smaller but recurring theme is “agents moving down the stack.” Posts around @JvNixon and Infinity describe automated compiler/memory-planner/debugger workflows for running optimized models across heterogeneous chips, with supporters framing software-generated per-chip adaptation as a way to weaken the CUDA moat. Separately, infra vendors shipped more incremental but practical updates: Qdrant 1.19 adds prefix matching on keyword indexes @qdrant_engine, and Together + IBM + NVIDIA announced enterprise inference infrastructure on IBM Cloud @togethercompute.

Top tweets (by engagement)

• Reasoning trace vulnerability / hidden CoT extraction : the original disclosure from @kotekjedi_ml and the follow-up privacy findings @kotekjedi_ml were among the day’s most consequential technical posts.

• Grok Bot beta : xAI’s agent product launch @bot drew the biggest product reaction, largely because it points to a persistent, logged-in AI coworker UX rather than a simple chatbot iteration.

• ChatGPT desktop for Linux + sync/imports : OpenAI’s Linux desktop preview @OpenAI and agent-workflow import/sync support @OpenAIDevs landed strongly with developer audiences.

• Nemotron 3.5 Lightning : Jensen’s post @JensenHuang and NVIDIA’s launch @NVIDIAAI marked the most important open-model systems release of the day.

• Introducing Muse Glimmer: an open-weight model optimized for always-on local agent workflows (Activity: 2435): Meta announced Muse Glimmer, a permissively licensed Apache 2.0 open-weight30Bdense multimodal model for always-on local agent workflows, with interleaved text/image input via a dedicated perception encoder,100+**language training, controllable reasoning effort, and agent-focused training for tool use, long-horizon reasoning, failure recovery, and benchmarks such as DeepSearch QA, MCP-Atlas, τ³-Bench, and SWE-Bench. The post claims ~**4-bit**quantization reduces the LM to <20 GB, enabling operation in **24–32 GBmemory envelopes alongside KV cache, perception encoder, and a DFlash-based speculative decoding drafter with “identical output quality”; weights/resources are linked onHugging Face, the research blog, and developer docs. A technical comment points to Alexandr Wang saying an open-weight Muse Spark 1.2 release is coming soon on X. Comments are mostly positive but light on technical scrutiny, expressing enthusiasm that Meta is releasing open weights again and jokingly framing Muse Glimmer as “llama 5.”

• Meta releases Muse Glimmer 30B - a new open model (Activity: 450): Theimage is a promotional benchmark graphic for Meta “Muse Glimmer-30B”, presented as a new open-weight 30B dense vision model under Apache 2.0. It claims competitive results versus Gemma 4-31B and Qwen3.6-27B on agentic/code/math/science benchmarks including MCP Atlas,DeepSearch QA,SWE-Bench Pro,AIME 2026, andSciCode, and advertises that it can run on18GBRAM/VRAM setups via Unsloth Desktop. Commenters were broadly positive about Meta returning to open model releases, but one noted skepticism about cadence, saying it may be “the strongest agentic model for its size for like three days before they release Qwen,” implying rapid competition from Qwen and pressure on Meta to improve release velocity.

  • Commenters frame Muse Glimmer 30B as a potentially strong agentic model in the ~30B dense-model size class , but expect it to be quickly challenged by upcoming Qwen releases; one commenter says it may be “the strongest agentic model for its size for like three days before they release Qwen.” The technically relevant concern is release cadence: Meta is seen as needing faster iteration to remain competitive with Qwen and other open-model labs.

  • A substantive ecosystem point is that the ~30B parameter tier is becoming crowded, with commenters naming Qwen, Google, NVIDIA, and Meta as active players. One commenter hopes Meta follows this release with a similarly sized MoE model, mirroring expectations that Qwen may also expand in that direction.

• Muse Glimmer ACTUALLY fits on a single RTX 3090 (Activity: 640): A user reports Meta Muse Glimmer 30BQ4_K_XLGGUF runs on a single RTX 3090 24GB with262144context, DFlash speculative draft,mmproj**, FlashAttention, and F16 KV cache, using only ~**22–23GBVRAM—unlike their testedQ4_K_XL**Qwen3.6-27B and Gemma-4-31B, which hit VRAM limits at ~**70k/52ktokens with F16 KV or125k/81k**with Q8 KV. They measured ~**64–124 tok/s**generation under DFlash, ~**1400 tok/sprompt processing, and passed a two-needle retrieval test at ~150ktokens, suggesting the model is not effectively capped at128k; a commenter notes the officialMuse-Glimmer-30B-GGUF releases already target 24GB/32GB**VRAM, and another reports very compact KV usage: ~**1.8 GiBfor131kF16 despite SWA on all layers. Commenters were positively surprised by the KV-cache efficiency, especially given SWA across all layers; one joked that this could further increase RTX 3090 demand/prices.

  • Users highlighted that Muse Glimmer’s KV cache appears unusually memory-efficient despite SWA on all layers : one report claims a 131k context with F16 KV uses only about 1.8 GiB, making long-context operation feasible on a single RTX 3090.

  • A commenter noted that the official Meta GGUF builds already target24GBand32GBVRAM configurations , including DFlash support, so Unsloth GGUFs may not be required. The referenced official repository is meta-models/Muse-Glimmer-30B-GGUF.

  • Another technical report claims 256kcontext + DFlash + mmproj fits in roughly22–23GBVRAM on an RTX 3090 , with observed throughput around 64–124 tok/s. They also noted that a 150k needle test reportedly holds up, but questioned how performance and retrieval quality behave once the context is filled closer to 200k+.

• 1 Day in and I feel okay saying Muse-Glimmer-30B finally beats 3.6-27B for the size in some use-cases (Activity: 709): OP reports that Muse-Glimmer-30B appears to outperform Qwen 3.6-27B in selected24GB GPU-class local use cases after ~1 day of testing, especially efficient reasoning, low-bit quantization (iq3_xxsreportedly degrades less than Qwen/Gemma), no-tools trivia/knowledge depth, and OpenCode agent efficiency. They still rate it weaker for general coding—roughly around Gemma4-31B level—but claim it completes agentic tasks faster than 3.6-27B despite similar task success. Commenters echoed strong early results for agentic workflows/tool calling , with one saying Muse-Glimmer-30B “isn’t even close,” but others expect an imminent 3.8 release to erase the lead. One technical criticism was that American models may waste tokens on safety/self-validation before answering.

  • One commenter reported a few hours of A/B testing where Muse-Glimmer-30B substantially outperformed 3.6 27B specifically in agentic workflows and tool calling , saying “it isn’t even close.” Another user qualified the improvement as strongest for non-coding tasks , while coding performance was left unverified.

  • A technical concern raised was token inefficiency from safety/alignment preambles : one user asked whether Muse-Glimmer-30B spends many tokens validating that requests are allowed under its policy framework. This was framed as a common issue with some American-aligned models where safety verbosity can reduce practical throughput in interactive or agentic use.

  • Several comments noted that the comparison may be short-lived because 3.8 is expected imminently and could change the relative ranking versus Muse-Glimmer-30B and 3.6 27B. One dissenting commenter still considered 3.6 27B the stronger baseline overall, suggesting the new model’s advantage may be workload-specific rather than universal.

• Early signs that Muse-Glimmer-30B might quantize verywell? Share your experiences. (Activity: 354): The image is a social media post from Unsloth AI showing Muse-Glimmer-30B-GGUF running in a chat/coding-agent workflow with visible tool calls, claiming a 2-bit quantized 30B model executed100+tool calls while using about14GB**RAM:image. In the Reddit discussion, users question whether **14GBis actually impressive for “2-bit” on a 30B model, while another reports Q4_K_XL on a single RTX 3090 performing well for agentic coding and roughly “on-par with 3.6 27B.” Commenters are split between optimism about Glimmer’s quantization/agentic-coding performance and skepticism about memory efficiency. There is also concern that the model may be overly safety-restricted, with one user citing refusals for code that moves the mouse pointer.

  • One user reports running Muse-Glimmer-30B as Q4_K_XL on a single RTX 3090 for agentic coding and says it is “performing great,” roughly on par with 3.6 27B in their early testing. Another commenter notes that a 14GB “2-bit” quant is relatively large for a 30B model, implying the packaging/quantization format may include substantial overhead or not be a straightforward 2-bit weight-only footprint.

  • A technically focused concern is how Glimmer behaves under KV-cache quantization , especially whether degradation from fp16 to q8_0 resembles Qwen or Gemma -style sensitivity. The commenter specifically wants Glimmer added to Anbeeld’s KV-cache benchmark methodology: KV cache quantization benchmarks / KVARn precision tail.

  • A user testing the BF16 model through vLLM reports disappointing quality versus Laguna-S-2.1 , saying Glimmer made many errors that Laguna would not. They suggest the result may be due to early-release issues and plan to retest once the official repo/model release stabilizes.

• Qwen 3.8-27b coming this week (Activity: 2791): Theimage is a screenshot of the official Qwen / Alibaba_Qwen X account confirming that Qwen3.8-27Bopen weights are landing this week, matching the post title’s claim. Comments point to a ModelScope listing forQwen3.8-2.4T-A95B, noting ModelScope is Alibaba-owned and suggesting the release timing/countdown may be credible. Commenters are already comparing expectations against other Qwen variants, especially asking whether a 35B-A3B-like model is coming because it reportedly performs well on certain tasks with strong speed for its hardware footprint.

  • Commenters pointed to an apparent official Alibaba ModelScope listing for Qwen3.8-2.4T-A95B with a countdown of roughly 1 day 9 hours, treating it as a credible signal because ModelScope is Alibaba-owned: https://modelscope.cn/models/Qwen/Qwen3.8-2.4T-A95B and https://modelscope.cn/models/Qwen/Qwen3.8-2.4T-A95B/summary.

  • There was interest in whether a 35B-A3B-style Qwen variant will arrive, with one user noting that 35BA3B performs “amazing” on certain task types while maintaining strong speed for its hardware footprint, implying demand for smaller active-parameter MoE-style models rather than only larger dense releases.

  • A Strix Halo owner requested a newer 122B release, saying the current Qwen 3.5 122B feels outdated; this reflects interest in very large local models that can plausibly run on high-memory AMD APU platforms.

• inclusionAI/Ling-3.0-tiny · 8B A1.3B MoE· Hugging Face (Activity: 427): inclusionAI releasedLing-3.0-tiny, an8B-parameter MoE with ~1.3Bactive parameters, positioned by the OP between4Band8–12BQwen/Gemma-class dense models. The model card reports FP8 throughput of ~100–105 tok/son DGX Spark and86–90 tok/son an M4 Pro MacBook, with ~8.34 GiBpeak memory at8Kcontext; commenters also highlight a256Kcontext window and an AA Bench score of25from a shared benchmark image. One commenter compared it favorably against recent LFM small models:IFBench 63.61,Multi-IF 83.15, andBFCL-v4 62.72, beatingLFM2.5-8B-A1BandLFM2.5-2.6Bon those listed metrics. Commenters were broadly positive about tiny MoE architectures for low-memory, mobile, and edge inference due to high tokens/sec, with one saying it may replace Ling-Mini-2.0 locally. There was interest in larger 15–50B Ling releases and speculation that speculative decoding could push throughput toward diffusion-model-like responsiveness.

  • Users highlighted Ling-3.0-tiny as an 8B MoE model with roughly A1.3B active parameters, making it attractive for low-memory, mobile, and edge deployments due to expected faster tokens/sec versus denser models. One commenter noted it scores 25on AA Bench , which they considered notable for this size class.

  • A technical comparison against recent LFM small models reported Ling-3.0-tiny ahead on instruction-following and tool-use benchmarks: IFBench 63.61 vs 56.47 for LFM2.5-8B-A1B, Multi-IF 83.15 vs 79.93, and BFCL-v4 function calling 62.72 vs 49.73. The same commenter emphasized its 256kcontext window on an 8B/A1B-style model as a key differentiator.

  • There was interest in runtime compatibility, specifically whether llama.cpp support exists yet. Another commenter suggested future larger 15B–50B Ling models combined with speculative decoding could significantly improve throughput, potentially approaching the perceived responsiveness of diffusion-style generation pipelines.