The News
Google says it is making rapid strides in driving down the energy consumption of AI search queries and, a top engineer told Semafor, is confident those improvements will continue to the point where AI’s carbon footprint “is essentially a non-consideration.”
In a research paper published today, Google reported that a typical query on its Gemini AI “assistant” — which summarizes web search results and can help draft emails, among other functions — uses 0.24 watt-hours of electricity, the equivalent of watching about 9 seconds of television. That’s not only significantly less than figures published by ChatGPT and other AI researchers, it’s about 33 times less energy per query than Gemini used just a year ago. And, as Google keeps investing in low-carbon power sources for its data centers, each query emits about 44 times less carbon than it did a year ago.
“I don’t think we’re anywhere close to where we need to be,” said Amin Vahdat, a vice president of engineering who leads Google’s largest team of machine learning programmers. “But we’ve gotten it to a place where it’s comparable to other sources of information.” And there’s no fundamental reason, Vahdat said, the number won’t fall further, potentially by orders of magnitude.
In this article:
Tim’s view
Widespread public concerns about the enormous energy and carbon footprint of AI, and scary-sounding projections by power companies about how much new electricity infrastructure will be needed to serve it, are often based on dubious data or overly simplistic extrapolations. Most tech companies have been reluctant to disclose much proprietary information at the most granular, energy-per-query level, which can lead to gross overestimations when multiplied across billions of queries.
The main problem, Vahdat said, is that most AI energy forecasts are based on an assumption that AI chips operate continuously at their absolute maximum power consumption. That will lead to a misleadingly high estimate, since in reality they almost always operate at just a fraction of that level. At the same time, many projections leave out other points in the AI data center system in which power is consumed, including backup servers that draw some power while idling and things like lights and air conditioners, which can lead to a misleadingly low estimate.
In performing a more specific analysis of Google’s actual consumption, Vahdat said his goal was both to create a more transparent and credible methodology that other AI engineers could replicate, and to set a concrete metric for his own engineers to work against as they take more steps to drive it down. “We’re making a big shift in our goals, from performance per dollar to performance per watt,” he said.
Jon Koomey, an independent researcher who studies energy efficiency in tech and reviewed Vahdat’s paper, said the paper shows how quickly the assumptions underpinning power demand forecasts can change, as tech companies start to confront energy availability as a tangible constraint on their AI aspirations and are forced to innovate accordingly.
And that has implications across the energy industry: Many hardware manufacturers and utilities are nervous about making expensive infrastructure investments to serve a future power demand boom that may turn out to be far smaller than what today’s most extreme scenarios foresee.
Room for Disagreement
When it comes to their energy use, AI developers are in an arms race between efficiency gains and increasing demand, Koomey said. For now, demand — that is, the skyrocketing number of Gemini queries performed every day — is winning. And the more energy-efficient AI models become, the more incentive there will be to use them freely and thereby raise their total energy footprint even more, a quandary known as the Jevons paradox that is already visible across Big Tech. Google reported last month that its total carbon footprint grew 48% since 2019, moving in exactly the opposite direction of its goal to cut them in half by 2030, thanks to the proliferation of Gemini-based services across many of its products.
Notable
- In Texas, the biggest constraint on new AI data centers isn’t power — it’s water. Data centers are on track to consume 7% of the drought-striken state’s water by 2030.