OpenAI’s Jalapeño chip is built for fast inference at scale, benchmarks show
techcrunch ·
OpenAI presented benchmark results for its new Jalapeño inference chip at the Hot Chips conference. On Semianalysis's InferenceX benchmark, the chip outperformed current state-of-the-art inference processors in tokens per user and throughput per kilowatt. OpenAI's hardware chief Richard Ho called the results a significant performance advance, noting improved efficiency and low latency. The comparison was against an Nvidia Blackwell system, though competitors may have advanced by the time Jalapeño fully deploys. Jalapeño is expected to reach small-volume deployment at the end of 2026, with broader deployment in 2027. Developed with Broadcom and assisted by OpenAI's own models, Jalapeño is planned as a multigenerational platform. The design specifically reduces bottlenecks in prefill and communication phases by minimizing data movement and keeping model state, including the KV cache, local.
techcrunch