View / Pacing the frontier is more about business models than AI models

Dario Amodei and other frontier AI lab leaders may care deeply about AI safety, but calls to “pace the frontier” also serve their financial interests.

As Anthropic and OpenAI prepare to go public and try to convince investors they can one day be massively profitable, spending billions on a race to grow the biggest and most capable models may not be the best sales pitch.

Even if large language models progress very little, they already do enough that they’ll be incredibly lucrative for these companies and for the economy.

Right now, the equation is unbalanced: It’s taking exponentially more money to create increased capability that will only marginally improve the utility of the AI product for most customers.

Solving a Millennium Prize Problem is great for bragging rights, and potentially for science, but it doesn’t necessarily translate into better reliability in completing mundane business tasks. For now, the reliability of LLMs comes not from the models themselves — bigger doesn’t and may never mean better — but from all the software built around them.

The next big breakthrough may not come from larger models, but from making AI models that can actually learn after they’re grown, and run more efficiently on everyday computers. That might look something like what AI pioneer Richard Sutton is trying to build at Oak Lab: a capable AI model that could run on 20 watts of power (like the human brain) and continuously update its weights (also like the human brain).

AI safety experts might be more worried about those theoretical, future models than the big ones that exist today because smaller, continually learning models would be more likely to escape the lab and evade attempts to shut them down. It’s a bridge that humanity will have to cross, but likely not anytime soon.

More efficient AI could lower the frontier labs’ costs. But that kind of AI could also be a threat to the business models of Anthropic and OpenAI if customers could get comparable capabilities without tapping those powerful data centers they’ve invested in. (I could be running that kind of AI on my own computer and not worrying about using up my $200 a month plan.)

“You have to wonder about the large language models. They might be at risk when this eventually happens,” Sutton said on a recent podcast. “I’m sure they’ll get a good run. They’ve already had a good run.”