OpenAI published the first measured performance numbers for Jalapeño, its in-house inference chip built with Broadcom, on August 25, and the figures put it well ahead of current Nvidia hardware on the metric the company says matters most: useful AI work delivered per watt. The results, posted to OpenAI’s engineering blog, come from the public SemiAnalysis InferenceX benchmark across GPT-OSS 120B, DeepSeek R1 670B, and the 1-trillion-parameter Kimi K2.5. OpenAI’s full results page shows Jalapeño delivering 1.5 to 1.9 times more AI work per watt at peak throughput and 1.7 to 3.6 times lower end-to-end latency than the comparison systems.
The comparison normalizes each accelerator by its published chip power rating. Jalapeño is rated at 700 watts, but OpenAI says its measured sustained power stayed at or below 550 watts on the tested workloads. The systems it beat carry far higher ratings: Nvidia’s GB200 at 1,200 watts and GB300 at 1,400 watts. On Kimi K2.5, the largest public model in the test, Jalapeño posted about 1.5 times the peak performance per watt and 3.4 times lower end-to-end latency than the comparison system.
Why a chip built only for inference
Jalapeño is a purpose-built ASIC, not a general-purpose GPU. OpenAI designed it around the specific shape of modern language-model serving, where the prompt-processing phase (prefill) is compute-bound and the response-generating phase (decode) is starved by memory bandwidth. The biggest architectural bet is keeping model state, including the key-value cache used while generating tokens, placed locally so the system does not stall moving data between compute, memory, and networking. As we covered in our NPU vs GPU vs CPU explainer, inference leaves very different bottlenecks than training, and a part that optimizes only for decode can serve many more users inside the same power envelope.
The chip pairs one compute die with six HBM4 stacks, providing 216 GiB of memory and 15.4 TB per second of bandwidth. For highly interactive agent workloads, where delays compound across many sequential steps, OpenAI reported 2.1 to 4.1 times higher performance than the comparison systems. Those are vendor-reported benchmark numbers against named Nvidia racks, and they do not cover training or Nvidia’s future Vera Rubin hardware.
What it means for OpenAI’s stack
Jalapeño will not replace Nvidia inside OpenAI. The company confirmed on August 17 a financing agreement with Nvidia of up to $105 billion for an OpenAI-leased data center campus in Ohio, and it says it will keep deploying Nvidia and other partner accelerators for both training and inference. Jalapeño is the first generation of a multigenerational roadmap: the team moved from initial design to tapeout in nine months, with OpenAI’s own models used to explore implementations and shrink design loops, and generation two is already deep in development.
OpenAI plans to begin deploying Jalapeño inside its own compute infrastructure by the end of 2026. Whether the published efficiency lead holds at full production scale depends on HBM4 supply and advanced-packaging capacity, the same constraints that limit every frontier accelerator. If it does, the chip gives OpenAI direct control over the power and latency of serving its own models, a lever that matters more as agentic products make hundreds of sequential calls per task. Our look at optical links for AI racks covers the interconnect side of the same power problem.
