Microsoft’s Kevin Scott on why he doesn’t hire robotic humans

Reed Albergotti: So, tell me about Microsoft Build this year. What are you going to talk about?

Kevin Scott: So there’s going to be a little bit of me pushing on folks at large, the same way that I do on our engineers inside of Microsoft, which is: You should just let your imaginations run wild. There’s a whole bunch of anti-patterns for even thinking about how to set your ambition right now. One of them is, “Hey, I tried something,” and it marginally works right now. So if that’s true, it’s really going to work real soon.

If you take that sample and decide it’s not ready yet, and then you wait a long time, the entire ecosystem will have skated right past you because you were being too conservative. Same thing with “too expensive.” If you are doing something and say, “Hey, this is super useful, but it’s too expensive right now,” it won’t be very shortly.

And especially when your custom chips start rolling out for the massive inference, right?

Yes. One of the good things about having a platform like this where you’ve got lots of competition, you’ve got just this gravitational pull of the most interesting work, sucking all of these people towards it and a bunch of really powerful competitive dynamics. It’s chips, it’s networks, it’s memory systems. It is power optimization, everything from how are you managing power in terms of picojoules of energy on the chip carrier, all the way up to how you’re managing the transmission losses on power inside of your data center and what your cooling architectures look like.

What’s your favorite optimization? You said there’s all these people working on these little problems in the stack, what’s your favorite story?

One of the really cool ones that a team solved is a big issue that we’ve had for two years now. Since inference demand really took off with GPT-4, we have been power-constrained. It’s not even about how much silicon can you order to go stand up to run these inference calculations on; it’s how fast can you build data centers and get electric power grids to connect or to let you connect to their grids with these gigantic data centers. We’ve done a whole bunch of AI work. Funny enough, we’ve used AI itself to do a bunch of stranded power harvesting inside of our data centers. We need to be running every data center at peak utilization so that we can provision it with as much compute as humanly possible, so that you don’t have a single stranded watt of power anywhere. Doing that is extraordinarily complicated because you’re having to move workloads around. It’s like this giant bin-packing problem, and it’s predicting when things are going to get hot and when things are cooling, and what things you can run together. AI itself is really good at dealing with inhuman levels of complexity, things that are hard to write a classical algorithm to say, “OK, well, I understand the contours of this and all of the factors, and I’m just going to have a closed-form solution to this problem.”

It’s classic Moravec’s Paradox, isn’t it?

Yes. It’s super good in general, when you don’t even understand what all of the contributing factors are to getting to optimal. You can describe “optimal.” “Optimal” in this case is no observable stranded power. Then you just build a system that uses AI to go solve all of the dimensions of that particular optimization problem.

And it worked?

And it worked. Hundreds and hundreds and hundreds of megawatts almost instantaneously made free to serve inference on.

The thing that I’m so stoked about is we are witnessing the birth of an agentic web right now, where you are writing this new sort of software, agents, that you can delegate increasingly complicated tasks to. Eventually, they’re just going to be arbitrarily complicated tasks that you are delegating to these agents to go do on your behalf. Conspicuously missing right now is memory. People approximate it with stuff like retrieval augmented generation. They approximate it with long context windows. We’re seeing our first breakthroughs right now in real agentic memory — where you’re going to start seeing, over the next 12 months, agents that are able to remember all of the stuff that they’ve done for you. 

One of the things that we assume in interactions with human beings is, we have memory. We literally think of it as a disorder when memory isn’t working because so much of our interaction is based on memory.

And the other part of this agentic web that’s emerging really fast and we’re super excited about is the ability of the models to take action on your behalf. And the way that gets sorted out is with things like [model context protocol] and just building all of this engineering buttressing there. It feels to me kind of like the dawn of the web. I feel like I felt back in the ’90s when the internet was blooming. You had all of these protocols, and they were very un-opinionated about what it is you did with them. You didn’t need to go get some gatekeeper to give you permission to go. 

Are you surprised how fast MCP took off? These servers are just everywhere.

Oh, absolutely. If you think about it, it’s like this blindingly obvious need. Agents are fundamentally limited in their utility right now because until you have something MCP, and until it is absolutely ubiquitous, agents have to be hard-wired to go do stuff. It’s too much work, even for a gigantic company to go do. I mean, imagine what the f*cking web would be if one person — the implementer of the web browsers, for instance — had to imagine what the rest of the web ought to look like; that it was their point of view that dictated what the web was going to be. How unbelievably uninteresting would that have been?

Tell me about the memory breakthrough. What’s happening? Is that something you have talked about?

We’ll announce a whole bunch of things, and one of them is this thing called “structured RAG.” It gives you a framework to hang memories into so that you don’t have to go scan, token by token, every single thing that you have ever done inside of an agent or everything that an agent could potentially reason over. It’s a little bit more powerful than a semantic index, which is how retrieval augmented generation looks because it uses the AI systems, the large language models themselves, to impose an ontology or a structure onto the memory space. It’s unbelievably effective at getting you to higher-precision recall over a very large corpora of information. Eventually, we’ll have superhuman memory, where the agents will be able to remember things with a breadth of recall and a higher precision than we can.

I know you’re an optimist on AI. What are the optimists not optimistic enough about?

The thing that I think we’re not optimistic enough about is there are a bunch of these public good things that are very zero-sum games right now, if you look at them from a political point of view. Health care or climate change or education, you look at all of these things and these feel like expensive problems. We’re having to make trade-offs about, you have less of this so you can have more of something over there, and you’re just shuffling things around, and there really isn’t a way to solve the problem where everybody is going to be happy.

I mean, with health care — the fundamental problem is you don’t have enough people getting a consistently high quality of health care. And if anything, you can argue that it’s getting worse because you have, relatively speaking, fewer doctors entering the profession. You’ve got an aging population that’s putting more pressure on the health care system. It’s just going to get worse and worse over time, unless you have some kind of intervention coming in, like there’s just no amount of money. You look at AI, and I would argue right now, we definitely have a capability overhang there. We should be way more optimistic about the benefits that we could have for health care as a public good just by getting things deployed faster. If I look at the AI-for-science work that we’re doing at Microsoft Research —and so it’s both AI and the quantum program that we’re running — the very first things that are going to be beneficiaries of commercial quantum computing and foundation models for science are going to be things like sustainability and climate change. New materials, new carbon capture catalysts, just a whole range of things that hopefully will help climate change be less of a zero-sum. It’s a very zero-sum game right now.

What should the government do to make that actually happen faster?

Every time I talk to policymakers, I try to get them to think about the zero-sum versus non-zero-sum framing. They know exactly what their most contentious problems are. [But] the thing that you need to be doing is figuring out how it is you can leverage existing technology to transform this contentious thing — [and] if you could redefine the rules in a way that would get the capital markets to do more things that have payoffs for these zero-sum, public good problems.

What would it be if you had to steelman the pessimistic argument? What are we not seeing about the challenges today?

I look at my daughter, for instance. I was telling this story this morning.

She’s in the 10th grade. She goes to this school where, at the beginning and the end of the year, they do these immersive programs that are all around social impact and social entrepreneurship.

The project that she just did was with the lady who runs pediatric endocrinology at Stanford who runs the gender clinic there. The particular project is, if you’re a trans kid and you’re on hormone blockers, you have to do something about your bone density. You have to up your calcium intake, and you have to do exercise to help with bone density. Otherwise, you’re going to have osteoporosis way, way, way sooner than you otherwise might have. You’re going to be susceptible to more bone breakage and a bunch of calcium deficiency-related things. There’s this group of three girls, none of whom ever had a computer science class and none of whom have ever programmed anything, none of whom have even had a particular interest in computer science or programming. They decided that they were going to build a mobile app that gamified your calcium intake and exercise. And they had a dad at the school. I wasn’t even aware of this, by the way. My own daughter knows exactly what I do. It didn’t even occur to her to say, “Hey, Dad. Here’s what I’m doing.” Nothing. I knew nothing about this until I walked into the presentation. I was like, “Wow. That’s cool.” But so, there was a dad at the school who had taken an hour and a half to walk them through one of these SUI agent tools. And then they just went to town, and they built an app.

I was just saying this to my team. In 2008, when all of us were writing our first mobile apps, we would have been super pleased with ourselves to have written what these three girls did — 16-year-olds with no programming background whatsoever. It was a really slick, polished app. … It didn’t occur to them that what they were doing was hard.

There are kids graduating college now who majored in CS because they thought maybe they would work on part of one of these mobile apps one day at a tech company. What should they do now?

Well, my prediction is, I don’t think that there’s any kind of upper bound on how much software the world needs. Just because you’ve got a whole bunch of AI tools that let more software be created, [that] actually may increase the demand for computer scientists. This has been true. I’ve been programming for a little bit more than 40 years now. I’m 53, and I wrote my first program when I was 12. Every time we have done something in the past four decades that has increased programmer productivity — and there have been just waves of it and orders of magnitude more leverage that tools have given programmers — we have needed more programmers as a consequence.

Is that true within Microsoft right now, though?

I think it’s sort of herky-jerky. I think over time, absolutely, Microsoft’s total market is going to increase because we are able to do more things than more people want, and that will create the demand for more programmers.

Because right now, you’re not accelerating hiring programmers.

But I think that’s just this weird, transient thing. We’ve certainly had that in moments in history. If you look at the trend line, the trend line is up. And if you rolled back to the 19th century, and you looked at what the labor markets look like, you look at people whose livelihood was agriculture and people whose livelihood was accounting.

Whereas with accountants, they are a larger percentage of the population now. And you could argue that the accountants have even more leverage. I would argue, because of computers, accountants have even more leverage in the 21st century than they did in the 19th.

If I want to get hired at Microsoft out of college today, who are the interns that you’re trying to get right now out of college? What are they good at?

Funny enough, I’ve been having this conversation with our head of school where my daughter goes to.

The educational system that we have is largely structured for the Industrial Revolution. Like, we need people to be literate. We need them to be baseline numerate. We need them to be able to stay on schedule; they have a routine; they can work in groups of people. They understand the teacher is like the proxy for the boss. And all of our structure, even for merit inside of our education system, is built on this Industrial Revolution idea, this idea that my daughter is in the middle of taking a bunch of standardized tests right now. I’ve been telling people for a gazillion years that very soon, the computers will be able to perform better on these standardized tests than any human alive. If you think that the point of education is doing well on standardized tests, you have really misunderstood what the future of work is going to be when a machine can do this, mechanical, stupid. And it’s mechanical and stupid since I took the standardized test a million years ago. So, the thing that every employer is going to look for is, you’re going to want people who are really very good with their social skills. They really understand how to be in dialogue with their fellow human beings, how they can understand what the needs of their fellow human beings are. Where they’re really good at choosing which problem to go solve. Where they are really sensitive to not just short-term opportunity, but long-term needs.

We need people who are really, really thinking in that unusual and human way about how to go do the next creative thing. Some of the conversations I have with my people right now are just so goddamn boring. So there’s more to being a human being than the last app that you wrote or the little programming trick that you did. And all of that stuff is beautiful. I would just really hope that we’re going to start hiring more people and encouraging more people to think about being complete and understanding the full breadth of human history, and understanding how social systems work, and how large groups of humans behave and want to interact with one another. And how to think about what you’re doing like a craftsperson. I’ve come in every day with a very Japanese mentality. I try to get a little bit better at this thing, every day, and I’m doing it because there’s dignity in being better at that thing for myself, independent of everybody else. But also because what I’m doing is in service of other people.

So I think a lot of times, we do a whole bunch of crap now where we just forget. I always try to tell my kids, this is, you have to figure out how to make yourself useful to your fellow human beings. We, none of us are independent. We’re all, in just brutally profound ways, dependent on one another. Given that, that you go through your life and you’re consuming the work of other people, and you are deriving joy from your interactions with other people — you have to figure out how you’re going to reciprocate. What are you doing that other people are going to find valuable and interesting and wonderful and joyful? And I think you just need more of that. When a piece of technology is able to go do more of the mechanical things that are — honestly, they may have been ways that people have defined themselves. But we’re also really super good at redefining ourselves.

It’s kind of like you’re saying we’re moving into this generalist era from when it was very specialized before. It was like you almost wanted to be like this — the Zuckerberg, not to pick on him, but that archetype of the single-minded computer programmer.

Yes. And look, I think expertise is still going to be super good. My daughter wants to be a doctor. It’s like, ‘look, I’m going to try to figure out how human biology works. I’m going to apply that knowledge to help people be healthier.’ I think that’s the right way to go. Think about expertise. Being single-minded and going down a rat hole chasing something super-duper narrow, I don’t know how. And I’m biased, by the way. I’m almost pathologically broad in my curiosity. I’m curious about everything. I used to get yelled at all the time, but it’s like, ‘oh, you need to focus more on one thing.’ I don’t want to focus on one thing. I want all of it.

But do you have some way of finding the right people?

You kind of have to look for signs of curiosity. If you’re thinking about software engineers, you look at folks who make contributions to open-source software. Do they have interests? One of the things I really do look for in software engineers is: Do you have something that you are passionate about that you pursue with intensity that isn’t software engineering? And it’s really interesting. I know software engineers, and they’re invariably among the most successful people in the industry, who are blacksmiths on the side. Or they are some kind of outrageous, competitive athlete on something. You want people who are curious and who have real intensity when they decide that they’re going to follow a lot of curiosity. They’re just really going to go do it. I’ve had people who won international Math Olympiad, ACM coding contests, all of these things where you think this is a good indicator of your intellectual capacity, but who just haven’t been as successful as you might have expected they would be if that were the real merit indicator. I grew up in rural central Virginia. I didn’t know any better than to go to Lynchburg College in Lynchburg, Virginia — this liberal arts school no one’s ever heard of. I didn’t do particularly well on my standardized tests. I dropped out of my PhD program at the University of Virginia to join Google. I think my career has been not bad. If you’re looking at where I went to school and my standardized test results, I would have looked very uninteresting.

My entire life, I’ve wanted to figure out how to get into a position where I didn’t need a job. Like, my greatest comfort in the world — and it has been since I quit my first real job and went to grad school the first time — is to live my life where I can go do the thing that is intellectually most interesting to me. I think you have more capacity to do that right now in the world that we live in today than you ever have been before. It’s a mindset thing.

It is.

It really is. I think this is part of why education really, really, really matters. It’s why I get so frustrated thinking about the amount of time we teach kids to try to get their stupid SAT score from 1500 to 1580. It’s like, the f*ck, who cares? Like, nobody’s ever going to care about that. Five seconds after you get admitted to your university, your university is not going to care. Your future employers aren’t going to care. You’re not going to care. Nobody cares. Unless you have some evidence, which we don’t have, that that is highly correlated, those slight differences [don’t matter]. The difference between 1500 and 1000 on your SATs, there is going to be something there. It may not be the principal thing, by the way. Maybe that you got 1000 on your SATs because you’re in some kind of structural poverty, and you could still be plenty smart and get 1000. But you’ve got some weird circumstances going on that have suboptimized your potential. But these fine differences that we’re optimizing for right now, just unbelievable. It’s a real struggle. It kills me with my 16-year-old. She’s super ambitious, just her mom and dad. The game, as it is presented to her, has her doing all sorts of crap that just kills me that that’s where she is spending her energy: optimizing for an admissions decision at a school that I would honestly prefer she didn’t even go to.

It’s really bizarre. I know in my case, my kids are only nine, but he’s taking a standardized test right now as we speak.

I know. My kid took her AP calc test this week. Two tests a week. She did another round of SAT practice tests on the weekend. Then she took the AP calc exam on Tuesday, yesterday. I know the calc stuff is crazy. I love calculus, but the way that we teach calculus to kids is just nuts.

So you basically have to rethink how you teach mathematics at this point.

One hundred percent. And look, there are some people, and I’ve got probably 100 of them working at Microsoft, who are AI inference kernel people. They are looking at the core numerics of how training and inference work in our AI systems, and there you need to know your calculus. It’s pretty important. The percentage of the population who need to be able to do that relative to the percentage of the population that we shove through AP calculus, it’s ridiculous. The things we need to be teaching kids is, how can you be a good, functioning, well-socialized member of society? Make yourself civically smart. Teach people accountability to your fellow human beings. Then teach people how to solve problems. Real computer science — not vocational programming — will be valuable forever because computer science is about the science of solving problems computationally. It is not about programming languages or mobile apps or any of this crap that some of these programs have become about. It is about real computational thinking. And that is super useful stuff.

And if you think about it in the context of what the big issue is now, with the race with China and all this stuff, they have more PhDs. They have more of everything, basically.

I look at the quality of the PhDs who are coming out of the top computer science programs in China versus the top CS PhD graduates in the United States — the Chinese are better right now. It’s partially because they’re just more rigorous. They really are focusing them on these very enduring, fundamental parts of computer science. A lot of PhDs graduating [in the US] right now are just chasing weird trends rather than really understanding the fundamentals.

I guess the US’ edge is we have a lot of creative people who can get funding.

I think one of the really awesome things about the United States, and we should never forget it, is we’re full of a bunch of rebellious, contrary people, like, ‘don’t f*cking tell me how to go do this thing.’ You may plunge your head straight into a brick wall and fail 10 times, but we have a bunch of people who are willing to do that failing, and eventually that leads to success. A lot of failing is necessary to figure out where the successful thing leads. We don’t have shame in the failure. We have a lot of institutional ability to figure out how to learn from failure and recover from failure. That’s an enormous strength. I don’t see that anywhere else in the world other than here.