OpenAI's first custom chip "Jalapeño" reportedly beats Nvidia's Blackwell and Rubin in inference benchmarks
decoder ·
OpenAI presented the first benchmarks for its custom inference chip "Jalapeño" at the Hot Chips conference, claiming it outperforms Nvidia's Blackwell and Rubin in throughput per watt and token latency. The chip handles inference only and is a general-purpose LLM accelerator, not tuned specifically to OpenAI's models. OpenAI reports 1.5x to 1.9x more work per watt and 1.7x to 3.6x lower latency across tested models, with results based on SemiAnalysis's InferenceX benchmark and verified in part on-site. The tested models were GPT-OSS 120B, Deepseek R1 670B, and Kimi K2.5 1T, and the chip achieved these numbers without using optimizations like multi-token prediction or speculative decoding. SemiAnalysis said Jalapeño beats Nvidia's Vera Rubin in output tokens per megawatt, though total cost of ownership per token is roughly even. Caveats remain: Jalapeño is only at engineering sample stage, while Nvidia and AMD have results on larger, untested models. OpenAI developed the chip with Broadcom in about nine months, using its own AI models for design and optimization. SemiAnalysis argues this speed suggests Nvidia's CUDA moat may no longer be a durable advantage, though OpenAI's CFO says the chip complements existing partnerships with Nvidia, AMD, AWS, and others.
decoder