[AINews] Hot Chips：OpenAI 的 Jalapeño、Cerebras CS-5、Groq 3 LPX 与 Apple M6

迄今为止，第 37 届 Hot Chips 大会最大的发布，是 OpenAI 在自研芯片上取得的惊人进展——距离其与 Broadcom 相关的公告还不到一年……而且它并非 ASIC，而是一个全面的、据称可超越 Blackwell的替代方案。

[自宣布首款定制推理芯片 Jalapeño 以来，我们一直在测试它及其周边系统。结果显示出重大进展：每瓦获得更多智能，并实现更快响应；单一架构可同时提供更高吞吐量和更低延迟，且没有 2026 年 8 月 25 日下午 5:19 · 226 万次浏览

584 条回复 · 1,080 次转发 · 1.36 万次点赞](https://x.com/OpenAI/status/2092300846675505602)

关键指标如今正转向每瓦性能，而 Jalapeño 做到了：

完整的 Hot Chips 演讲内容尚未发布，但下方汇集了多方解读。如需更完整的拆解，可与 OpenAI 的其他内容一并观看：

2026/8/24–2026/8/25 的 AI 新闻。我们检查了 12 个 subreddit、544 个 Twitter 账号，没有进一步查看 Discord。AINews 网站支持搜索全部往期内容。提醒一下，AINews 现已成为 Latent Space 的一个栏目。你可以选择订阅/退订不同的邮件推送频率！

OpenAI 的 Jalapeño 推理芯片，以及推理技术栈的转变

• Jalapeño 公布的数据是当天最大的技术新闻：OpenAI 发布了其定制推理芯片 Jalapeño 的首批基准测试细节，声称在真实模型工作负载中，其效率与延迟均显著优于 NVIDIA GB200/GB300 系统。在 OpenAI 的测试中，Jalapeño 在峰值吞吐量下实现了 1.5–1.9 倍更高的每瓦工作量和 1.7–3.6 倍更低的端到端延迟；对于高度交互式工作负载，其性能高出 2.1–4.1 倍。该芯片额定功耗为 700W，但据称在测试运行中维持在 550W 或以下。OpenAI 表示，将于年底前开始将其部署至自身基础设施；Gen 2 已处于深度开发阶段，Gen 3 也已启动（OpenAI 公告、部署路线图、Sam Altman）。

• 工程师为何关注：这一说法不只是关于原始性能，更在于一种更均衡的推理架构，可减轻通常存在的吞吐量/延迟权衡。多方技术评论强调，部分对比点尤其值得注意：据称 Jalapeño 在某些配置中没有采用激进的 prefill/decode 解耦或投机解码等技巧，仍取得了良好表现，同时击败了使用这些技巧的系统（gdb、kimmonismus 总结、eliebakouch 分析、You Jiacheng）。SemiAnalysis 将其评价为第一代 ASIC 中异常强劲的产品，并将其直接与 Blackwell 和 Rubin 级别系统比较（SemiAnalysis、dylan522p）。

• 另一层重要故事是模型辅助的系统优化：OpenAI 的帖子还表示，GPT-Astra + Codex 协助编写并优化了底层 kernel，使三个此前未规划支持的开放权重模型在约两个月内就在 Jalapeño 上实现高性能；对于选定的 attention 与 MoE 模块，这些实现据称比已有的人类专家编写代码快 1.5–1.8 倍（kimmonismus、eliebakouch）。这是一个重要信号：编译器/kernel 工作正越来越多地被纳入模型改进闭环，而不再只是应用层编码工作。

• 更广泛的基础设施影响：一些帖子将 Jalapeño 与更大的产业转变联系起来——前沿实验室在推理经济性上可能不再完全处于 NVIDIA 的下游，尽管封装和晶圆代工产能仍是严峻瓶颈（Liam Fedus、teortaxesTex 的反应、LearnOpenCV 关于 TSMC/CoWoS 产能的提醒）。

Agent Harness、记忆系统与评测工程正成为一等公民

• Harness 质量正变得与模型选择同样重要：多篇论文和产品发布聚焦于同一个主题：Agent 性能高度依赖其周边系统。一篇由 Microsoft 主导、关于 AutoSaddler 的新论文将 Harness 视为代码，利用失败轨迹离线修补 prompt、工具配置及控制逻辑；相较基础 Harness，其报告在 GAIA2 上提升 +9.0、在 SWE-Bench Pro 上提升 +9.6、在 Terminal-Bench 2.0 上提升 +10.0（论文摘要）。与此同时，另一篇论文直接量化了 Harness 方差，发现更换 Harness 对得分的影响可能远大于更换模型，模型对的排名也会随脚手架而翻转；其提出的解决方法是采用结构化的 Harness Card 信息披露标准（分析、《There Is No Neutral Harness》）。

• 长周期软件工程仍远未解决：SWE Refactor Bench 衡量全仓库迁移任务，例如 C→Rust、Maven→Gradle 和 POSIX→WebAssembly，覆盖 SQLite、zlib 与 libsodium 等真实项目。在 520 次运行中，仅有 28 次通过全部三个阶段，存活率为 5.4%；在 20 项任务中，有 13 项没有被任何系统解决（EinsiaAI）。这为更局部的代码基准测试中那些亮眼的 bug 修复成绩提供了有益的校正视角。

• 记忆系统正被重新设计为可编程状态，而非压缩后的聊天历史：DAIR 总结的一篇阿里巴巴论文使用仅追加事件日志加上持久化 Python kernel来支撑 Agent 会话，将工具输出和派生状态绑定至带类型变量，而不是持续将其序列化到 prompt 中。报告结果包括：在 LongMemEval_S 上达到 94.8%，在 BEAM_10M 上达到 73.1%（较此前最佳公开记忆系统高 +5.1），并使用 Qwen3.8-Max 在 LOCA_256K 上达到 86.7%（总结）。与此相关的 Knowledge Triage 工作表明，朴素的上下文压缩会破坏精确规则的保留：经过五轮压缩后，一种设置仅保留了 10% 的安全规则，而类型感知的保留策略能够保留 2–4 倍更多内容（总结）。

• 实用评测工程正从临时操作走向产品化工作流：LangChain 及合作伙伴分享了一套具体闭环，可将轨迹和人工反馈转化为任务规范、合成环境及评测，用于持续衡量并后训练 Agent（Vtrivedy10、hwchase17）。LangSmith Engine 也发布更新：通过更好的问题检测/聚类、SaaS 和自托管支持、Slack/Linear 集成，以及按成本分层的分析模式，其关键内部基准性能提升了 超过 2 倍（LangChain）。

本地优先 Agent、端侧推理，以及新的个人计算栈

• Perplexity 的 Portable Computer 是当天最清晰的本地 Agent 产品发布：Perplexity 在 NVIDIA DGX Spark 上推出 Portable Computer，将其定位为完全本地化的 Perplexity Computer 版本，其中 orchestrator LLM、subagent LLM 和 agent harness 全部在本地硬件运行，且不依赖云端（Perplexity 发布、模型细节、NVIDIA、Arav Srinivas）。初始本地技术栈采用后训练的 PPLX 27B，同时提供 Qwen 3.8 27B；未来将支持 Nemotron 3.5 Lightning。

• 更深层的趋势是持久化、始终在线的本地 Agent：Srinivas 明确描绘了这样一种未来：后台进程持续从连接器摄取上下文，在一个永久循环中执行多跳推理，并运行于用户自己的硬件之上（Arav Srinivas）。社区反应则出现分歧：一方对隐私与控制能力感到兴奋，另一方则质疑“本地优先”是否应意味着一台 5,000 美元的 DGX Spark，而非普通消费级设备（theo 的批评、theo 的后续帖）。

• Apple/macOS 本地 AI 工具也在成熟：exo 表示，Apple 在新的 M5 Ultra Mac Studio 和 M6/M5 Pro Mac Mini 页面中展示了它，重点强调利用 Thunderbolt 5 上的低延迟 RDMA 对 Mac 进行集群化，并以接近 API 的速度运行 Kimi K3 和 GLM-5.3 等模型；4× M5 Ultra 扩展可实现约 4.8 TB/s 的聚合内存带宽（exo）。相关帖子指出，Apple 更快的 PCIe 存储和基于 ANE 的视觉管线，正使小型本地集群以及混合 CPU/ANE/GPU 推理变得更加实用（anemll、onirenaud）。

• 本地运行时周边工具持续完善：Ollama v0.33 增加了一键集成，让 Claude Desktop 可以将 Ollama 作为第三方网关，用于访问云端及本地模型（Ollama）；OpenCode v2 被展示运行于 Cloudflare Durable Object 内，这说明小型 Agent 运行时正变得可嵌入边缘环境（fayazara）。

模型、检索与搜索基础设施

• Qwen 3.8 正在整个技术栈中出现：围绕 Qwen3.8 发布的热情同时体现在部署与评测帖子中。Together 为 Qwen3.8-27B 增加了微调和专用推理支持（Together）；Unsloth 则声称借助优化 kernel，可在免费的 2× Tesla T4 Kaggle 实例上完成 27B 模型的全量 QLoRA 微调（danielhanchen）。在应用侧，Qwen3.8-27B 在 Image-to-WebDev Arena 中位列开放模型第 1、总榜第 7，定价为每百万输入/输出 token 0.40 美元 / 3 美元（arena）。

• 搜索与检索基础设施迎来多项实质更新：Hugging Face 发布了 Papers with Code 搜索引擎的详细架构说明：PostgreSQL + pgvector、Qwen 3 Embedding 0.6B、混合检索、通过 Hugging Face Jobs 在 NVIDIA L4 上生成 embedding、将产物存入 buckets，并通过 Inference Endpoints 进行在线服务；同一套栈也为论文页面的“相关论文”功能提供支持（Niels Rogge）。Keenable 结束隐身状态，推出面向 AI 的 Web Search API 和 Web Query Language；该公司由前 Yandex Search 负责人创建，获得 2,600 万美元种子轮融资，明确瞄准 Agent 规模的网页检索需求（styskin）。

• 检索模型设计仍是活跃领域：围绕 late interaction / multivector retrieval 的讨论再度升温，有观点认为检索工作负载中的扩展规律终于开始显现，且模型与数据库的协同设计至少与存储格式同样重要（mixedbread 的观点、Silvio Martinico）。

机器人、物理世界模型与具身数据

• Figure 的“Index”是一项重要的机器人数据公告：Figure 推出 Index，将其描述为全球规模最大、最多样化的机器人数据集。其报告的数据包括：每秒摄取 30 分钟的视频上传内容、已上传 1,600 万个视频、已为数据支付 1,500 万美元、并实现 26.4 万次下载。公司还表示，未来 12 个月将在数据和算力上投入 10 亿美元（Brett Adcock、后续帖）。这一规模十分重要，因为许多机器人实验室看起来仍更多受制于示范与感知数据，而不是架构新颖性。

• 大规模物理/世界建模持续推动上下文边界：Anima Anandkumar 重点介绍了 Accelerated Understanding，这是一家为跨模态和 4D 时空物理仿真构建大型 AI 模型的创业公司。该公司声称，在预训练期间使用 1T 参数，训练期间使用 1T 上下文，并且在推理时无需下采样或 patching 即可使用 超过 5T 的上下文（Anima Anandkumar）。细节仍较少，但该帖子值得关注，因为它表明一些前沿非语言模型工作正在迈向的方向：海量上下文

By far the biggest announcement at the 37th Hot Chips conference was OpenAI’s stunning progress on their own chip, less than a year after the Broadcom announcement… and that it isn’t an ASIC; but a full on Blackwell-beating alternative.

[Since announcing Jalapeño, our first custom inference chip, we’ve been testing it and the system around it. The results show a major advance: more intelligence from every watt and faster responses, delivering both higher throughput and lower latency in one architecture without 5:19 PM · Aug 25, 2026 · 2.26M Views

584 Replies · 1.08K Reposts · 13.6K Likes](https://x.com/OpenAI/status/2092300846675505602)

The key metric now is shifting to performance per watt, and Jalapeno delivers:

The full Hot Chips presentation is not yet out but various takes are below. For a fuller breakdown, watch along with the rest of OpenAI:

AI News for 8/24/2026-8/25/2026. We checked 12 subreddits, 544 Twitters and no further Discords. AINews’ website lets you search all past issues. As a reminder, AINews is now a section of Latent Space. You can opt in/out of email frequencies!

OpenAI’s Jalapeño Inference Chip and the Shift in the Inference Stack

• Jalapeño’s published numbers are the day’s biggest technical story : OpenAI released first benchmark details for its custom inference chip Jalapeño , claiming materially better efficiency and latency than NVIDIA GB200/GB300 systems on real model workloads. In OpenAI’s tests, Jalapeño delivered 1.5–1.9× more work per watt at peak throughput and 1.7–3.6× lower end-to-end latency , with 2.1–4.1× higher performance for highly interactive workloads; the chip is rated at 700W but reportedly stayed at or below 550W on the tested runs. OpenAI says deployment into its own infrastructure begins by year-end , with Gen 2 already deep in development and Gen 3 underway (OpenAI announcement, deployment roadmap, Sam Altman).

• Why engineers care : the claim is not just raw perf, but a more balanced inference architecture that reduces the usual throughput/latency tradeoff. Multiple technical reactions highlighted that some comparison points are especially notable because Jalapeño reportedly performed well even without tricks like aggressive prefill/decode disaggregation or speculative decoding in some setups, while beating systems that did use them (gdb, kimmonismus summary, eliebakouch analysis, You Jiacheng). SemiAnalysis framed it as unusually strong for a first-generation ASIC and compared it directly against Blackwell and Rubin -class systems (SemiAnalysis, dylan522p).

• A second-order story is model-assisted systems optimization : OpenAI’s post also said GPT-Astra + Codex helped write and optimize low-level kernels, bringing three previously unplanned open-weight models to high performance on Jalapeño in about two months; for selected attention and MoE blocks, these implementations reportedly ran 1.5–1.8× faster than existing human-expert-written code (kimmonismus, eliebakouch). That is a meaningful signal that compiler/kernel work is increasingly being folded into the model improvement loop, not just application-layer coding.

• Broader infra implication : several posts tie Jalapeño to a larger industry transition in which frontier labs may no longer be strictly downstream of NVIDIA for inference economics, even if packaging and foundry capacity remain a hard bottleneck (Liam Fedus, teortaxesTex reaction, LearnOpenCV caveat on TSMC/CoWoS capacity).

Agent Harnesses, Memory Systems, and Eval Engineering Becoming First-Class

• Harness quality is increasingly as important as model choice : several papers and launches converged on the same theme: agent performance depends heavily on the surrounding system. A new Microsoft-led paper on AutoSaddler treats the harness as code and patches prompts, tool configs, and control logic offline using failure traces, reporting gains of +9.0 on GAIA2 , +9.6 on SWE-Bench Pro , and +10.0 on Terminal-Bench 2.0 over base harnesses (paper summary). In parallel, another paper quantified harness variance directly, finding that swapping harnesses could move scores far more than swapping models, with model-pair rankings flipping across scaffolds; the proposed fix is a structured Harness Card disclosure standard (analysis, “There Is No Neutral Harness”).

• Long-horizon software engineering remains very unsolved : SWE Refactor Bench measures whole-repository migration tasks like C→Rust , Maven→Gradle , and POSIX→WebAssembly across real projects including SQLite , zlib , and libsodium. Across 520 runs , only 28 survived all three stages, for a 5.4% survival rate, and 13/20 tasks were solved by nobody (EinsiaAI). This is a useful corrective to strong bug-fix numbers on more local coding benchmarks.

• Memory systems are being redesigned as programmable state, not compressed chat history : one Alibaba paper summarized by DAIR backs agent sessions with an append-only event log plus a persistent Python kernel , binding tool outputs and derived state to typed variables instead of continually serializing them into prompts. Reported results include 94.8% on LongMemEval_S , 73.1% on BEAM_10M (+5.1 over the previous best published memory system), and 86.7% on LOCA_256K with Qwen3.8-Max (summary). Related work on Knowledge Triage showed that naive context compaction destroys exact-rule retention; after five rounds of compaction, one setup preserved only 10% of safety rules, while type-aware retention policies preserved 2–4× more (summary).

• Practical eval-engineering is moving from ad hoc to productized workflows : LangChain/partners shared a concrete loop for turning traces and human feedback into task specs , synthetic environments, and evals that can be used to measure and post-train agents over time (Vtrivedy10, hwchase17). LangSmith Engine also shipped > 2× better performance on key internal benchmarks with better issue detection/clustering, SaaS and self-hosted support, Slack/Linear integrations, and cost-tiered analysis modes (LangChain).

Local-First Agents, On-Device Inference, and the New Personal Compute Stack

• Perplexity’s Portable Computer is the clearest local-agent product launch of the day : Perplexity launched Portable Computer on NVIDIA DGX Spark , positioning it as a fully local version of Perplexity Computer where the orchestrator LLM , subagent LLM , and agent harness all run on local hardware with no cloud dependency (Perplexity launch, model details, NVIDIA, Arav Srinivas). The initial local stack uses a post-trained PPLX 27B with Qwen 3.8 27B also available; Nemotron 3.5 Lightning support is coming.

• The deeper trend is persistent, always-on local agents : Srinivas explicitly sketched a future of background processes that continuously ingest context from connectors, perform multi-hop reasoning in a perpetual loop, and run on your own hardware (Arav Srinivas). Community reactions were split between excitement about privacy/control and skepticism that “local-first” should mean a $5k DGX Spark rather than commodity consumer devices (theo critique, theo follow-up).

• Apple/macOS local AI tooling is also maturing : exo said Apple featured it on new M5 Ultra Mac Studio and M6/M5 Pro Mac Mini pages, emphasizing low-latency RDMA over Thunderbolt 5 to cluster Macs and run models like Kimi K3 and GLM-5.3 at API-like speeds, with 4× M5 Ultra scaling to about 4.8 TB/s aggregate memory bandwidth (exo). Related posts pointed to Apple’s faster PCIe storage and ANE-based vision pipelines as making small local clusters and mixed CPU/ANE/GPU inference more practical (anemll, onirenaud).

• Tooling continues to fill in around local runtimes : Ollama v0.33 added one-toggle integration to let Claude Desktop use Ollama as a third-party gateway for cloud and local models (Ollama); OpenCode v2 was shown running inside a Cloudflare Durable Object , illustrating how small agent runtimes are becoming embeddable in edge environments (fayazara).

Models, Retrieval, and Search Infrastructure

• Qwen 3.8 is showing up across the stack : enthusiasm around the Qwen3.8 release was visible in both deployment and evaluation posts, with Together adding fine-tuning and dedicated inference support for Qwen3.8-27B (Together) and Unsloth claiming full QLoRA fine-tuning of the 27B model on free 2× Tesla T4 Kaggle instances using optimized kernels (danielhanchen). On the application side, Qwen3.8-27B reached #1 among open models in the Image-to-WebDev Arena and #7 overall , while priced at $0.40 / $3 per million input/output tokens (arena).

• Search and retrieval infra got multiple substantive updates : Hugging Face published a detailed architecture writeup for the Papers with Code search engine: PostgreSQL + pgvector , Qwen 3 Embedding 0.6B , hybrid retrieval, embeddings generated on an NVIDIA L4 via Hugging Face Jobs, artifacts in buckets, and live serving via Inference Endpoints; the same stack powers “related papers” on paper pages (Niels Rogge). Keenable came out of stealth with a Web Search API and Web Query Language for AI, built by former Yandex Search leaders and backed by a $26M seed , explicitly targeting agent-scale web retrieval (styskin).

• Retrieval model design remains active territory : there was renewed discussion around late interaction / multivector retrieval , with claims that scaling behavior is finally becoming visible in retrieval workloads and that model+DB co-design matters at least as much as storage format (mixedbread perspective, Silvio Martinico).

Robotics, Physical World Models, and Embodied Data

• Figure’s “Index” is a major robotics data announcement : Figure introduced Index , described as the largest and most diverse robot dataset in the world, with reported ingestion at 30 minutes of video uploads per second , 16M video uploads , $15M already paid out for data, and 264k downloads. The company also says it will spend $1B over the next 12 months on data and compute (Brett Adcock, follow-up). That scale matters because many robotics labs still appear more bottlenecked on demonstration and perception data than on architecture novelty.

• Large-scale physics/world modeling continues to push context limits : Anima Anandkumar highlighted Accelerated Understanding , a startup building large AI models for physical simulation across modalities and 4D spacetime, claiming 1T parameters during pretraining , 1T context during training, and > 5T context at inference without subsampling or patching (Anima Anandkumar). The details are sparse, but the post is notable as a statement of where some frontier non-language modeling work is heading: massive-context multimodal simulation rather than only text/video generation.

• Embodied policy generalization remains an active benchmark target : a separate robotics post introduced S1 , a manipulation model that can complete tasks from a single demonstration outside its training distribution (anag004). Google Research also shared AgentHands , an XR system that augments conversational agents with synchronized hand gestures for spatial guidance during physical tasks (Google Research).

Top tweets (by engagement)