In today's special edition, we look at the role Microsoft CTO Kevin Scott has been playing at the co͏‌  ͏‌  ͏‌  ͏‌  ͏‌  ͏‌ 
 
rotating globe
May 23, 2023
semafor

Technology

newsletter audience icon
Sign up for our free newsletters
 
Reed Albergotti
Reed Albergotti

Ever since Microsoft announced Bing Chat, I’ve been fascinated by CTO Kevin Scott’s job. Taking OpenAI’s work and turning it into a consumer product meant investing years and hundreds of millions of dollars into infrastructure.

By doing so — and doing it largely under the radar — Microsoft was able to make a massive leap, putting it neck and neck with Google as the leader in AI and giving the company the opportunity to grow its cloud computing services.

I asked Pinterest’s head of engineering, Jeremy King, a longtime friend of Scott’s, whether he thought Microsoft would have made its $1 billion investment in OpenAI and gone all-in on large language models if it weren’t for Scott. “I don’t think so,” he said.

Today is a big day for Microsoft — its first Build developer conference in the ChatGPT era. Bing was always going to be a tiny part of Microsoft’s AI ambitions.

The real impact will be when every organization using Microsoft products and services can build customized AI tools with almost no up front investment. In today's special edition, I spoke with Scott about what’s coming and where he sees the fast-growing AI technology taking us.

Reed Albergotti

The man who put Microsoft in the AI lead

THE NEWS

Microsoft Chief Technology Officer Kevin Scott, who spearheaded the company’s $1 billion investment in OpenAI in 2019, has a message for you: The way you use your computer is about to drastically change.

At Microsoft’s annual Build conference on Tuesday, Scott unveiled new tools that will allow software developers to quickly and easily build custom “copilots,” or artificial intelligence assistants, for virtually any task.

In the background, these copilots will be joined together by a latticework of plugins that give them the ability to do things like book airline tickets, order meals, and schedule meetings.

That means searching for keywords, clicking on links, hitting the “back” button, opening folders, and using documents such as spreadsheets will gradually become less common as large language models like the one that powers ChatGPT carry more of the workload.

While having a conversation with ChatGPT has already fascinated users, Scott believes the biggest wave of transformation will arrive when even the most mundane interactions with computers become conversational.

“We actually believe that the world should be full of these copilots,” Scott said in an interview with Semafor. “There will be copilots for medicine, drug discovery, and all sorts of things where you have a rich interaction between an AI system and a knowledge worker trying to get something done.”

But while Scott has some ideas for what copilots might be good for, the most interesting part is that he actually has no idea what people will do with them. “Platform builders are pretty miserable at answering that question in general,” he said.

Microsoft

STEP BACK

Scott is an anomaly in the tech industry. He grew up poor in a small Virginia town not known for producing C-suite technology executives, and completed most of his PhD in computer science from the University of Virginia.

In his spare time, he constructs leather bags, backpacks — a lot of them, from the looks of his Instagram page — and guitar picks and tools. His Bay Area workshop, which includes laser cutters, drill presses, and industrial sewing machines, is legendary in Silicon Valley.

On paper, he seemed like an unlikely candidate to make Microsoft the leader in artificial intelligence. The field is full of PhDs from MIT, Stanford, and Carnegie Mellon.

“The nice thing about Kevin is he's not bothered with any of that,” said Mike Volpi, a venture capitalist and a friend of Scott’s. “He doesn't seem to need the reinforcement. He's sort of independent of what I would characterize as the mainstream, accepted way of doing things, which lets him do stuff like what he did at Microsoft.”

Pinterest’s head of engineering, Jeremy King, who has been having breakfast with Scott for years, said his friend could see what would happen with AI far ahead of anyone else.

“He's always just a great guy to bounce ideas off of like, ‘hey, we're going this way, we're thinking about combining these two things together.’ Most of the time, he's already thought about it or knew people who thought about it,” King said.

Scott was an engineer at Google before joining LinkedIn. By the time LinkedIn was acquired by Microsoft in 2017, Scott was senior VP of engineering and operations. After the deal, Microsoft CEO Satya Nadella named Scott CTO of the parent company.

At the time of the acquisition, hype around AI had turned to disillusionment. While the technology was everywhere, used to automate processes in just about every industry, consumer applications like virtual assistants and chatbots had failed miserably. Even autonomous driving, once considered right around the corner, had turned into a far-off goal.

But soon, OpenAI was about to make a big bet on “transformer models,” then a new kind of artificial intelligence technique that was eventually used to power the company’s ChatGPT and DALL-E products.

In the past, AI models always plateaued in capability after they reached a certain size. In theory, these transformer models would break the mold and keep “learning” with more and more data.

But that was, by definition, a hypothesis. Testing it was a big technical challenge that involved spending huge sums of money. Under Scott’s direction, Microsoft built a supercomputer with 285,000 central processing unit cores and 10,000 GPUs.

Training OpenAI’s models was ultimately successful, but the journey was not a linear one, Scott said. At times, it would look as if progress was slowing down or stopping completely.

“You think those thoughts because there are a whole bunch of people saying those things all along,” he said. “Whenever you're making a heavy bet, you'll have a full spectrum of people. Some that just completely don't believe that this is a reasonable thing to do or that it's technically flawed, or a dumb allocation of capital.”

In training large language models, it’s difficult to predict exactly when improvements will occur. As the models get bigger, new abilities emerge and then suddenly disappear as the models get even bigger. Then they can reappear later on. Researchers aren’t quite sure why.

Scott said he drew on his experience to steel his resolve. “When you don't have the experience, you've never seen one of these cycles all the way before, it can be really, really disconcerting,” he said. “All of that fear and anxiety tends to make people super cautious, which is exactly the opposite of what you have to do when you're trying to make something very big happen.”

REED’S VIEW

It’s rare when a decades-old public company makes a single decision that changes its fortunes so drastically. Microsoft’s investment in OpenAI and its big bet on a new technology is one of those events.

Scott’s role in that process has flown under the radar, and that probably has more to do with the media than technology.

Nadella was either smart, lucky, or both that Scott was thinking so deeply about artificial intelligence at just the right time.

The best-selling book he co-wrote, Reprogramming the American Dream, laid out his vision for how AI would change labor in the U.S. It was an influential read, but probably would have sold more copies had it not been published as the COVID-19 pandemic took hold in April 2020.

The book also shows his wide network. The foreword was written by Hillbilly Elegy author J.D. Vance who, at the time, had a different reputation in Silicon Valley circles before his current senate rhetoric.

Scott’s perspective in the book — he saw firsthand how changes in labor can upend communities — is that AI will have a net positive impact on jobs. He makes a compelling argument.

His views on AI are some of the more level-headed and logical that you’ll find in this AI-hyped environment in which much of the media coverage has been silly, hysterical, and cynical.

“You absolutely need regulation,” he said of AI. But he noted there’s already regulation that will apply immediately to new AI products, like a copilot for healthcare would have to think carefully about healthcare privacy and drug regulations.

Volpi said he talked with Scott soon after The New York Times wrote an article about Bing Chat going off the rails. Scott was unphased. “He was like ‘ok, bad. We can fix it in a week,’” Volpi said.

These days, Scott is thinking less about how to make AI models get exponentially bigger and more powerful, and more about how to rein in the technology, building in safeguards that prevent hackers from injecting malware into plugins and copilots, and creating layers of “meta prompts” that prevent people from forcing Bing Chat to go off the rails.

ROOM FOR DISAGREEMENT

I was talking recently with a tech executive who balked at the idea that natural language would replace the traditional computer interfaces, largely because people won’t trust these copilots to do their bidding, like booking flights and buying things.

THE VIEW FROM ZURICH

Researchers at ETH Zurich built a programming language that standardizes a way of interacting with large language models. Called LMQL, the language both reduces the cost of queries and makes them more precise.

The takeaway: Development of internet-wide standards are proliferating quickly around the world. It’s still unclear what kind of copilots will become the norm.

NOTABLE

  • OpenAI CEO Sam Altman said it was Kevin Scott who sold him on doing the Microsoft deal, calling it “the most successful partnership at this scale that I know of in tech” in this Stratechery interview.

Sign up for Semafor Technology: The fight for the industry’s future. Read it now.