AI Copilots and the Future of Knowledge Work

This conversation is part of our AI Revolution series, which features some of the most impactful builders in the field of AI discussing and debating where we are, where we’re going, and the big open questions in AI. Find more content from our AI Revolution series on www.a16z.com/AIRevolution.

Microsoft CTO Kevin Scott, in conversation with CFI’s Bob Swan, explains how AI copilots are keeping developers longer in a flow state and why AI copilots more broadly could be the start of an industrial revolution for knowledge work.

• [00:59] Microsoft’s approach to AI
• [5:55] The next Industrial Revolution
• [10:27] Developer productivity & flow state
• [15:26] Reprogramming the American Dream
• [19:48] Advice to builders

Microsoft’s approach to AI

Bob: Microsoft has played such a huge role in determining the future of AI. Could you take a few minutes and talk about how you see the vision and the future of AI and what is the role of a large tech leader, like Microsoft, in this wave of AI?

Kevin: We really do see AI not as a product and not as a research project, but as a real platform for lots of people to build many interesting new things on top of.

Over the past couple of years and maybe the past 12 months, in particular, with the launch of ChatGPT and GPT-4, you can really see the potential of the platform in the same light as the personal computer or the smartphone. There’s a bundle of technologies that are going to make a whole bunch of new things possible that lots of people are going to be building things on top of, most of which we probably don’t even really understand right now. That’s the exciting thing to me about building a technology. I’ve always been a tool builder, and the idea of building a tool and then having a bunch of people use it in ways that you didn’t anticipate is super interesting.

Bob: In many ways, you’re not doing this on your own. You’ve partnered with leaders in the industry, whether it’s GitHub or OpenAI, as you mentioned, and most recently, Meta Llama 2. Can you talk a little bit more about the role you see partners playing in your vision and how you expect the ecosystem to expand over time? A question that I’m sure you’ve gotten a million times is: where do you get all the GPUs to support growing and expanding relationships?

Kevin: The GPU question is…it’s probably not comical, but I have to laugh at the frequency of that being the number 1 question right now. It’s not even a weekly or monthly thing, the hours in the day that go by where someone is not asking for GPUs are almost nonexistent.

Going back to this whole point of, if you’re building a platform, the platform has to get built with partners and it has to get used by partners. Our partners are extraordinarily important to us. If you think about AI, by far and away the most important partner we have is OpenAI. That partnership has been going for 4 years now and started as a, “Hey, we are a hyperscale cloud. We’re going to build a bunch of AI supercomputing infrastructure in collaboration with you that is then going to train some of the most advanced AI models and systems in the world. Then we’re going to do some work together to figure out how to take those models, those platform building blocks, and get them deployed into products that Microsoft offers, like GitHub Copilot, as well as deploy these things into environments like Azure and Azure OpenAI API, where people can just build their own software on top of it.”

These other partners are super important as well. GitHub, obviously, is one of the most interesting places to deploy AI right now. It is the first illustration of this copilot pattern that we are trying to build out, which is: how can you take a bit of knowledge work or cognitive work that someone is doing and use AI to help them be dramatically more productive at doing that cognitive work? If you look at developers, we just don’t have enough of them to write all of the software that the world needs. A lot of the work that developers do is just toil, so having a tool that can help make the job of software development more delightful and to help software developers be more productive…

You mentioned Meta and Llama 2. I think Llama 2, just like Llama, is going to be an important building block that people are going to want to use to build AI software. We want AI software running on Azure, so we’re doing everything that we can to make that an easy thing for people to do.

The next Industrial Revolution

Bob: The concept of Copilot and how you create these AI tools to assist people with the complex cognitive task: how do you think AI can and will change how we all work over time? What’s your advice for how we get the most out of tools like Copilot?

Kevin: If you think about the Industrial Revolution, for the first time, you had engineered devices, machines that were able to assist people with labor. Most of the labor in the 19th and 18th century and through most of the 20th century was physical. The invention of these new technologies all of a sudden allowed the transformation of the physical labor that almost everyone in the world was doing.

I think a similar thing is going to happen now with cognitive work, or knowledge work, which is becoming a bigger and bigger part of what the workforce is spending their time and energy doing. You and I are both knowledge workers. We haven’t really had a massive breakthrough in productivity for knowledge workers. The PC was an accelerator and the invention of the internet was an accelerator. I don’t think the smartphone helped much at all in terms of productivity and knowledge work. But I think this might be the biggest thing that’s ever happened for this particular type of work.

I think the way that it will get used in the early days, and you can sort of see this right now, is two-fold. One is you have these big categories of knowledge work where there is no way to do more of the particular type of work because productivity is not getting any better—but you’ve got some kind of deficit in society because you actually could use more of it.

On the other hand, you have tons of drudgery in a lot of the work that we all do. If you had some kind of productivity mechanism that could come in and do your least favorite part of your job or the most repetitive and most redundant work, I think we all would be delighted to have those sorts of tools in our lives. I think lots of companies are going to get created very soon to really help tackle some of those problems of, “Hey, where’s the crap work that people would be desperate and happy to have automated?” It would make their work experience better and would let them get more done.

The interesting thing is with some of these big platform shifts that we’ve had in the past, the most valuable things that get done on the platforms are not the things that got deployed in the first year or 2 of the platform change. If you think about the smartphone, the place where you spend most of your time on your smartphone is not the SMS app, it’s not the web browser, it’s not the mail client. It’s in the new things that got created on top of the platform in the years following the availability of the platform.

It’s the hard stuff that you really have to think about. What are the difficult things that have now become possible that were impossible before? That’s the thing that people ought to be thinking about. What are the hard problems I can be working on? There are a lot of easy things that you can do right now that are going to be useful, but I don’t think those are the most valuable things.

Developer productivity & flow state

Bob: One of the use cases that has been pretty powerful has really been code assistant within GitHub Copilot. Going inside the 4 walls of Microsoft, can you talk a little bit about the tools that you’re rolling out to your own engineers, how is it changing how engineers get their work done at Microsoft, and any thoughts or ideas on how you track and measure that kind of productivity in the engineering community?

Kevin: I’m not sure whether any of the AI systems are creating the need for new measures for developer productivity, but I think it’s really, really important. If you are running a software development organization and you don’t have good metrics and visibility into the productivity of developers, you need to get on it right now. It will be very hard to decide which out of a bewildering variety of AI developer tools are coming your way that you want to adopt and what order you want to adopt them in if you don’t have those measurements in place.

The thing that I will say is developer productivity is not lines of code produced. It is how you’re measuring a developer’s ability to very quickly deliver things to users and then measure whether or not those users are benefiting from the things that the developer is producing. It’s instrumenting that entire feedback loop and making sure that you are able to identify points of friction throughout the entire product lifecycle development.

The thing that we’re doing inside of the company right now with AI tools, first and foremost, is getting everybody to use GitHub Copilot, which is a really big productivity win. The thing that it does, in our observation with developers, more than anything else is it helps keep them in flow state longer than they otherwise would. Rather than hitting a blocker when you’re writing a chunk of code or you’re trying to accomplish something and being like, “I don’t know how to go get the next thing done. I’ve got to go consult documentation or ask another engineer who might be busy with something,” being able to get yourself unblocked in the moment before you’re out of flow state is extraordinarily valuable. That’s a thing for folks who are thinking about the utility of these generative AI tools. You’re building for things other than software development.

Last fall, before GPT-4 was announced, I decided, “Could I use these tools to help me write a science fiction book?” Which is something I’ve wanted to do since I was a teenager. It was terrible at writing all of the book. It just wasn’t good at giving you chapters of well-formed prose with character development and all of the things that you would want in a good book. But it was really terrific at keeping me in flow state. So, if I measure my productivity of how many words I could write in a day versus 2X with the tool, it’s easy.

The really interesting thing that everyone should be thinking about, in general, is this notion of flow state. What are the conditions in which people are their absolute most productive or the most delighted at their job? When you’re in flow state, you know it. You’re just killing it. How can you use tools to preserve that as long as humanly possible?

The other place, too, where we’re using a bunch of AI tools is actually in the deployment of AI. A lot of the testing that we’re doing right now, a lot of the responsible AI work that we do, are all using the AI tools themselves to help do that. That’s another interesting thing that just wasn’t obvious. It hadn’t been obvious to me that it was going to be as important as it is when we started doing deployments at scale about 9 months ago.

Reprogramming the American Dream

Bob: Can you talk a little bit about how you, as a leader, are effectively rolling out the tools maybe to motivate and inspire rather than stoke the fears that our employees have about what this technology means for our roles? Then, on a related basis, how do you handle the resistance to change organizationally for disruptive tools like this in your daily job?

Kevin: I think those are 2 very different questions and I’ll dig into them separately. I actually wrote a book about this, honestly, at the worst possible time. The book is called Reprogramming the American Dream. It went to the bookstores and hit shelves in March of 2020, the week that the shutdowns for the COVID pandemic started. The whole point of the book was that AI creates more opportunities for people by far than it creates the potential for harm. It’s not to say that the potential for harm is 0, or that you can ignore it, or that you can make that the umpteen thing you’re thinking about with an AI product. But it does mean that we have to be really insistent and determined around these optimistic scenarios.

Part of the book was rooted in my experience. I grew up in rural Central Virginia in a place where the economy was powered by tobacco farming, furniture manufacturing, and textiles. By the time I was graduating from high school, all 3 of these industries had just collapsed.

When folks in these communities have access to very powerful tools, they tend to do remarkable things that create economic opportunities for themselves, for their families, and their communities. They solve problems that you or I are not going to solve just because we don’t see the whole problem landscape of the world. We don’t have their point of view. These tools of AI are becoming more accessible now than they have been by a dramatically wide margin. You can do interesting things with these tools right now to solve problems like create businesses or be an entrepreneur in small-town Virginia without having a PhD in computer science or expertise in classical AI. You just need to be curious and entrepreneurial.

Your second question was how do you get people resistant to change to embrace some of what’s happening right now with AI?

What we have seen at Microsoft over the past 9 months—and I think you all are seeing this, too, in the entrepreneurial community—is as soon as you have large groups of people enthusiastically building things on top of this technology, it’s very easy to see all of your peers adapting to the change, getting excited about it, and building interesting things, and then just say to yourself, “I’ve got to go do this as well.” I think part of the problem in any organization is figuring out how to get yourself to that point of critical mass of adoption where then it just gets easier. Then you have the opposite problem, which is the thing you started the call with, which is people screaming at you that they don’t have enough GPUs.

Advice to builders

Bob: Let me close with one last, more general question. What do you see as the biggest open questions in AI at the moment?

Kevin: We have proof points of a 20-watt organ that sits between our ears that is artificial general intelligence, which is many, many, many orders of magnitude more efficient than the digital artificial neural networks that we’re building right now. From a technology perspective, there’s probably a whole handful of breakthroughs left to be discovered in figuring out how to close that efficiency gap.

The unlock that, you have to get to where you can do something on your laptop versus having to go buy millions of dollars’ worth of computing infrastructure to do something really enormous. It does that thing for the product development that I was talking about earlier which is, it helps you turn the crank faster and makes the iterations less expensive. Just getting big efficiency breakthroughs would be great for us, obviously, because we’re spending huge amounts of capital on infrastructure to run these models. But I think it would be great for the whole world and the state of pace of innovation that we’re seeing in AI products.

The thing I will say to all of the entrepreneurs in attendance is go find hard stuff to work on. Don’t chase the trivial things. Don’t be the AI moral equivalent of the fart apps that were flooding the AppStore. Just because a thing has become possible doesn’t mean that it’s useful. Really, really focusing on the fundamentals of product building is important. AI is a model, not a product.

Your understanding as an entrepreneur of: who is your user? What is their problem? What can you do to help them? Then determining whether or not this AI stuff is actually a useful piece of infrastructure to go solve that user problem—that is something that is unchanged. AI is just like a new and interesting piece of infrastructure that has come into existence and lets you solve a new category of problems or solve an old category of problems in better ways.

Bob: Kevin, this is great. Thank you very much for taking the time to join us today. I greatly appreciate you sharing the vision you have there at Microsoft and some of the tools and human aspects of bringing this technology to life. It’s great to see. I look forward to next time.

Kevin: Thank you so much for inviting me.