基准测试显示，OpenAI 的 Jalapeño 芯片专为大规模高速推理而打造

周二在 Hot Chips 大会上，OpenAI 分享了对 Jalapeño 的更详细介绍，其中包括这套新系统的首批基准测试结果。在 SemiAnalysis 的 InferenceX 基准测试中，Jalapeño 在单用户 token 数量和每千瓦吞吐量两项指标上，均超过了当前可用的最先进推理处理器。

OpenAI 硬件负责人 Richard Ho 在一次媒体电话会议上表示：“归根结底，这些结果显示出相对于当前最先进水平非常、非常显著的性能提升。Jalapeño 能够以单位功耗承载更多 AI 工作，同时也能更快返回响应。它既能高效服务大量客户，也能实现极低延迟。”

值得注意的是，这一比较对象是 Nvidia Blackwell 系统；但到 Jalapeño 实现全面部署时，竞争对手的技术可能已取得显著进展。Ho 预计，Jalapeño 将在 2026 年底“以非常小的规模”部署，并在 2027 年迎来更显著的部署。

Jalapeño 于去年 10 月首次公布，由 OpenAI 与 Broadcom 紧密合作开发，OpenAI 自身的模型也参与了开发过程。该公司计划将 Jalapeño 打造成一个跨代平台，使 AI 产品、模型、芯片和内存能够协同开发。

得益于这种全栈方法，OpenAI 能够针对推理过程中常引发性能摩擦的特定阶段进行优化。尤其是，Jalapeño 旨在尽量减少处理过程中的预填充（prefill）和通信阶段延迟；OpenAI 表示，这些阶段往往会成为瓶颈。

该公司在介绍测试结果的博客文章中表示：“我们设计 Jalapeño 的目标是最大限度减少数据移动和通信延迟。这意味着，包括生成响应时使用的 KV cache 在内的模型状态，可以被显式放置并保留在本地；同时，系统会针对每个推理阶段启用恰当组合的计算、内存和网络资源。”

At the Hot Chips conference on Tuesday, OpenAI shared a more detailed look at Jalapeño, including the first batch of benchmark results for the new system. Tested on SemiAnalysis’ InferenceX benchmark, Jalapeño registered both more tokens per user and more throughput per kilowatt than the currently available state-of-the-art inference processors.

“The bottom line is that the results show a very, very significant performance advance over state of the art,” said Richard Ho, OpenAI’s head of hardware, in a press call. “Jalapeño can serve more AI work per unit of power, while also returning responses more quickly. It’s very efficient to serve a lot of customers, but it can also be very low latency.”

Notably, that comparison is against an Nvidia Blackwell system — but by the time Jalapeño reaches full deployment, the competition may have advanced significantly. Ho estimated that Jalapeño would deploy at the end of 2026 “in very small volumes,” with more significant deployment coming in 2027.

First announced last October, Jalapeño was developed by OpenAI in close collaboration with Broadcom, with OpenAI’s own models assisting in the development process. The company plans to make Jalapeño a multigenerational platform, allowing AI products, models, chips, and memory all developed in concert.

Because of that full-stack approach, OpenAI was able to address specific phases in the inference process that often cause friction during inference processing. In particular, Jalapeño is designed to minimize delays during the prefill and communication phases of processing, which OpenAI says often act as bottlenecks.

“We designed Jalapeño to minimize data movement and communication delays,” the company said in a blog post presenting the results. “This means that model state, including the KV cache used while generating a response, can be explicitly placed and kept local while the system activates the right combination of compute, memory, and networking for each inference phase.”