The Scene
AMD’s latest advanced chip could peel customers away from industry leader Nvidia and shake up the overall AI landscape.
The unveiling of the MI400, the company’s most advanced model, drew some of the biggest companies to San Jose to heap praise on AMD CEO Lisa Su, who has made fast progress in a field dominated by Nvidia: Massive, industrial-scale compute clusters that can train and run the world’s most powerful AI models.
“When you first started telling me about the specs, I was like, there’s no way. That just sounds totally crazy,” said Sam Altman, co-founder and CEO of OpenAI, last week during an AMD conference.
But it isn’t just the chip that excites people like Altman. It’s everything else AMD has done to make it perform better as a cohesive system. Gone are the days when chip companies were judged simply by the number of transistors they could fit on a piece of silicon. Massive increases in compute power today are the product of a wide array of engineering breakthroughs, some that require microscopes to see and others that are better observed from the sky.
It took a while for the market to respond to AMD’s announcements last week, but its stock shot up Monday as analysts processed the implications of the event.
For decades, the tech industry had Moore’s Law, the promise that the number of transistors on a chip would double every couple of years. It was dependable and allowed companies like Apple to plan product road maps years in advance. Today’s compute breakthroughs come more from the ability to connect unfathomably large numbers of chips together — an effort that isn’t governed by any “law.”
This is why AMD is selling its MI400 as a complete system, with all of the data center components pre-built in a single server rack called Helios. Nvidia is also moving in this direction, which means pushing the boundaries of chip architecture and everything that is built around them.
So far, Nvidia and now AMD have shown an ability to innovate in this post-Moore’s Law era. The question, though, is for how long. Companies are packing so much hardware into server racks that they are straining the concrete floors that support them and the energy grids that power them.
Mark Papermaster, AMD’s CTO and a veteran of the chip industry, told Semafor the new challenge does require new brainpower. “We’re changing the whole makeup of the company,” Papermaster said. “If you look at our acquisitions, they’ve been broadening our skill base, largely acquiring new software skills and new system skills to allow us to be more effective and move more quickly.”
In this article:
Know More
For instance, Papermaster said the company now employs over 1,000 rack design engineers working on optimizing the solutions. He said the company’s “chief architects,” which used to specialize on specific domains of research, have had to become generalists in order to create more tightly integrated systems for AI compute.
At the same time, AI researchers are working with chip companies on how to improve the efficiency of how models are run and trained on the actual hardware. That is something that would be impossible for general purpose chips that have to run more than just AI workloads.
Nvidia CEO Jensen Huang pioneered the concept of using GPUs for AI, and that vision gave the company a massive first-mover advantage.
But AMD’s fast pursuit, and the changing nature of the industry, have created a dynamic where Nvidia’s biggest competitive moat is now software. It began building a vast and complex suite of programs, called CUDA, that much of the AI industry now uses to train models.
AMD’s CUDA competitor is called ROCm. And while it may be a slog for individual developers to switch to a new system, the biggest AI companies can justify the expense if it means even small gains in compute efficiency.
Cohere CEO Aidan Gomez, an accomplished AI researcher, said on stage last week that it was “super easy” to move one of the company’s AI models over to the AMD platform, and planned to begin training on AMD’s next generation of chips.
Reed’s view
It was interesting that the pop in AMD’s stock price after its big announcement last week was so delayed. That’s probably partly because of what was implied, but never explicitly stated.
The companies represented on stage at the event are undertaking the largest and most ambitious rollouts of compute power in history. And while they signaled they were working with AMD, they didn’t provide any concrete numbers.
What percentage of their new projects will be AMD versus Nvidia?
I asked Papermaster if the MI400 chips could be part of Stargate, the $500 billion project OpenAI is building with Oracle. He said “absolutely .. the 400 is architected for mass scale capability,” but he didn’t confirm any details.
Given that Oracle is already a big AMD customer, it wouldn’t be surprising if those chips are part of the Stargate plan.
After facing shortages of Nvidia chips in the wake of the ChatGPT explosion, AI companies want options. And that is a huge opportunity for AMD.
Room for Disagreement
SemiAnalysis founder Dylan Patel has been a critic of ADM’s software, which he says lags behind Nvidia’s. More recently, Patel has said AMD is catching up and has a renewed sense of urgency, but still has a way to go before it’s on par with Huang’s company.
“Acceptance is the final stage of grief,” he wrote. “AMD has caught a second wind, but Nvidia is still at an all-out sprint, and AMD must match them and even outpace them to catch up.”
Notable
- SemiAnalysis published the most in-depth slicing and dicing of the AMD news that we’ve seen to date.