On Wednesday, OpenAI unveiled its first custom-built inference processor, designed and manufactured in collaboration with Broadcom. Named Jalapeño, the brand new processor was designed particularly for the distinctive wants of OpenAI’s inference programs. OpenAI’s personal AI fashions assisted within the improvement of the chip, the corporate mentioned.
Whereas the chip remains to be being examined, OpenAI says early outcomes present considerably higher performance-per-watt than present state-of-the-art options.
The partnership was officially announced in October, however OpenAI’s chip plans have long been rumored as a approach to cut back the corporate’s dependence on Nvidia’s GPUs. Google and Amazon have each constructed {custom} chips to serve the same function, typically referred to as “AI accelerators” — silicon designed particularly to hurry up machine studying workloads.
OpenAI president Greg Brockman defined the corporate’s strategy to chip improvement on its in-house podcast, shortly after the Broadcom partnership was introduced.
“We’ve got a deep understanding of the workload,” Brockman mentioned within the episode. “We’ve actually been searching for particular workloads which can be underserved, [and asking] how can we construct one thing that may be capable to speed up what’s doable?”
Jalapeño is particularly designed for inference, the method of working pre-built AI fashions in response to consumer instructions. Within the announcement, OpenAI emphasised the chip’s low working price when working real-time coding fashions. It’s probably that extra performance-intensive duties like pre-training will nonetheless depend on Nvidia {hardware}, however even small reductions in inference prices might do quite a bit to enhance the corporate’s backside line.
Optimizing that inference system could show to be an important issue within the economics of AI going ahead — and it’s more likely to happen at each stage of the stack. OpenAI is already constructing agentic merchandise like Codex and the fashions that energy them, in addition to knowledge facilities to run these fashions. Shifting into purpose-built chips lets the corporate go even additional in that course of, as the corporate defined in its announcement.
“OpenAI will not be solely creating frontier fashions or constructing merchandise on prime of them; it’s designing the infrastructure beneath them: chip structure, kernels, reminiscence programs, networking, scheduling, deployment programs, and product expertise,” the corporate wrote. “As a result of OpenAI operates throughout the stack, every layer may be optimized across the identical objective: making its fashions quicker, extra dependable, and extra inexpensive for customers.”
While you buy by way of hyperlinks in our articles, we may earn a small commission. This doesn’t have an effect on our editorial independence.