Technology

➚ MOVE FAST: Stepping up. As competition heats up with Microsoft and Google, Amazon Web Services is promoting its sales and marketing head, Matt Garman, to CEO. Meanwhile, Anthropic hired Instagram’s co-founder and former CTO, Mike Krieger, as its Chief Product Officer.

➘ BREAK THINGS: Stepping down. It was only a matter of time before OpenAI chief scientist Ilya Sutskever left the startup he co-founded after he led a failed effort to kick CEO Sam Altman out. Another executive allied with Sutskever quit too. Will their departure hurt OpenAI’s ability to develop the next generation of AI models? (More on that below.)

The biggest standalone new product announced at I/O was Astra, a mobile app in which users can hold up their phone and ask Gemini to analyze the video it captures in real time. Astra is a demonstration of Google’s multimodal approach to training AI models. Instead of using only text, Google went down a more difficult path of training models on text, audio and video, all at once. That was a gamble that seems to have paid off. On Monday, OpenAI announced it had been traveling down a similar path, but Google seems to be ahead.

There were a lot of impressive capabilities in this promo video Google shared (Astra isn’t out yet, so it’s all we have to go off of). But the one that really mattered came at the end, when a Google DeepMind employee asks Astra where she left her glasses. Astra remembered what it saw earlier in the video and reminded her.

This feat is made possible by what the AI industry calls the “context window.” Essentially, it’s the amount of information an AI model can process in a single prompt. Context allows the model to “remember” where the glasses are.

Video requires larger context windows than text. Gemini has achieved, by far, the industry’s largest context window of 2 million tokens, which corresponds to about 2 hours of video. And Google CEO Sundar Pichai says the company will soon reach a virtually infinite context window.

Finding glasses in a demo video is where we are now, but this is where we’re headed: You want to learn to sail, so you ask your AI model to watch a few hours of sailing instructional videos on YouTube. You then get on a boat and ask the model which ropes to pull on and when. (Anybody who knows anything about sailing is probably horrified by this idea.)

Or you’re going to a cricket game for the first time in your life and you have no idea what’s going on (this may or may not be inspired by real events). Your model watches some cricket videos and now you can ask it to explain what’s happening in the game as you watch it.

Some experts in AI believe multimodality is the path toward AI gaining a “world model.” The idea is that something special happens when AI can see the world, instead of just reading about it.

Billionaire tech critic bids for TikTok

THE SCOOP

A billionaire tech critic says he is mounting a bid to buy TikTok, the massive Chinese-owned social media app whose parent company faces US government pressure to sell.

Frank McCourt, the executive chairman of a family real estate giant, McCourt Global, and founder of tech and innovation initiative Project Liberty, told Semafor in an interview Tuesday that he plans to buy and rebuild TikTok as “a new and better version of the internet where individuals are respected and they own and control their identity and their data.”

“TikTok presents the best and worst of the internet. It connects 170 million people and allows them to be creative and build things and enjoy things and do things,” he said. “On the other hand, they don’t get to really share in the value that’s created, and their data is scraped and stolen and shipped to China.”

McCourt said he’s retained the investment bank Guggenheim Securities to advise him on what he calls the “people’s bid,” and said he’ll seek money — TikTok could cost $100 billion — from foundations, endowments, and pension funds, as well as broad based public support.

“We want all the capital to be values-aligned [around] a new and better version of the internet, where individuals are respected and they own and control their identity and their data,” McCourt said.

If he is able to assemble an investor group, he will still likely face intense competition from giant American companies eager to swallow TikTok’s huge advertising business and user base.

Donald Trump’s 2020 attempt to force a sale drew interest from Microsoft and a group that included Oracle and Walmart. (Microsoft’s CEO later called the process “the strangest thing I’ve worked on.“) This time around, former Treasury Secretary Steven Mnuchin, fresh off leading the rescue of a New York bank, has also expressed interest in TikTok.

The amount that the US government should spend on nondefense innovations in AI, though it could take years to dole out funding if such a plan ever came to fruition, according to a plan released by the bipartisan Senate AI working group. That money includes backing production of high-end AI chips.

It’s been an interesting week for OpenAI. On Monday, it announced a new product feature that was remarkably similar to Google’s Astra, which was to be unveiled the next day. And then on Tuesday, it announced that its chief scientist, Ilya Sutskever, was officially leaving after last year’s board upheaval. It’s widely noted that Sutskever played an important role in AI safety at the company, leading the “superalignment” team, and the team’s co-head, Jan Leike, also left.

The question that hasn’t been addressed in most of the news coverage is how Sutskever’s departure will affect OpenAI’s ability to stay ahead in its race with Google and Anthropic.

And if anything was apparent at Google I/O Tuesday, it was that the race is neck and neck. Google, caught off guard by the ChatGPT moment, took a gamble on “multimodality” over a year ago and it seems to have paid off.

OpenAI’s product launch the day before shows how intensely the company is focused on the race with Google. And it’s logical to ask whether upheaval atop its research ranks caused it to lose a step as Google moved quickly to develop multimodality.

I don’t think we’re going back to text-based LLMs from here. The limiting factor now for multimodal models is compute power, something that will be solved in time.

What this shows is that the path toward the most capable frontier AI models is not a straight line of larger data centers and ever more powerful GPUs. We’ll see forks in the road as new breakthroughs offer new methods. Those forks are opportunities for competitors to catch up, or fall behind.

Google’s new product called “Gems” may offer a glimpse into a future in which software is redefined as something personal, rather than a product created by a developer and distributed to the masses.

Gems, which allows users to customize AI models for their specific needs, is the company’s answer to OpenAI’s GPTs, which offer similar functionality and can be shared publicly for others to try. Creating a Gem, or a GPT for that matter, requires no coding or technical knowledge.

When OpenAI announced GPTs and its GPT Store at its first developer conference last year, I was excited to see what people would come up with. But so far, the response to GPTs has been underwhelming. The GPT Store, which could have become the “App Store” of the AI era, hasn’t generated much excitement.

I asked Sissie Hsiao, Google’s Vice President and General Manager for Gemini experiences and Google Assistant, why she thought GPT hadn’t caught on and why Gems would be better.

Hsiao wouldn’t comment on GPTs specifically, but she made an interesting point: In the era of customized software, who wants to use somebody else’s creation?

“They are just so hyper personalized that, even if I gave you my awesome bot, you’d be like, ‘I don’t know how to use it,’” she said. “It isn’t clear that sharing is the thing, versus the ability to construct exactly what you want, quickly.”

Hsiao left open the possibility that people may create apps that are so rich and feature-packed that they are shared like traditional software programs, but she said the technology isn’t quite there to do that with simple natural language prompts.

Hsiao said she created a Gem for a video game she’s currently into, League of Legends: Wild Rift. “I made one that was a Wild Rift super expert,” she said. “It not only had the tone, it became a video game aficionado and spoke to me in that language. It also gave me specific tips on what to do in different situations in the game.”

There are great possible use cases for Gems today. But it’s what it might evolve into that’s really interesting: A sophisticated piece of software created just for you.

• Biden hikes tariffs on $18 billion in Chinese goods.
• Blinken makes surprise trip to Kyiv.
• China gains ground in Iraq’s oil industry.