Since the ChatGPT moment nearly four years ago, the AI race has been viewed by many as a contest of model performance. Say what you will about OpenAI co-founder and CEO Sam Altman, but he saw from the very beginning that compute infrastructure would become an important battlefront, and his company is getting ahead in a key element that can make models do more, faster and more cheaply. His long-term strategy was evident earlier this week when OpenAI unveiled its custom inference chip: Jalapeño.
Today, AI tools are capped by a supply-demand mismatch that limits their performance. By designing a custom chip, OpenAI can bring down the cost of serving its models to consumers. AI companies can design chips to work particularly well with their specific models, which improves performance per watt. Eventually, the optimization phase of the AI revolution will be critical to building a good business around AI. While OpenAI might be a little bit behind rival Anthropic on coding performance, it’s ahead on the infrastructure side, having gobbled up more compute than Anthropic, which is now partnered with SpaceX for compute resources.
It’s clear, after this week’s announcement, that OpenAI also has a head start on the custom chip front.