Nvidia’s Xinzhou Wu on EVs, vehicle autonomy, AI, and China

Today, I’m talking with Xinzhou Wu, who is the head of automotive at Nvidia.

Nvidia is obviously in the news constantly because of the AI boom — it’s one of the most valuable companies in the world, because the AI industry can’t get enough of the company’s GPUs.

But Nvidia is also a key supplier to the auto industry. It’s had chips in cars for years now, and Xinzhou has been instrumental in building a complete autonomous driving system that automakers can just use. It’s already in newer Mercedes EVs, for example, as you’ll hear him mention several times.

So I really wanted to get his perspective on how the auto industry is handling the big transition to self-driving EVs. That’s the goal every carmaker and supplier will tell you is coming, but which maybe seems farther away in 2026 than ever. The EV adoption cycle in the United States is fully off track, self-driving seems to forever be stuck trying to solve the final 20 percent of situations, and cars themselves just keep getting more expensive even as consumers are feeling the squeeze of inflation and rising energy prices across the board.

You’ll hear Xinzhou say there’s actually been startling progress in reinventing the fundamental nature of the car itself — something the industry calls the “software-defined vehicle,” controlled by just a handful of powerful computers instead of dozens or even hundreds of independent electronic control units, or ECUs. If you’re a Decoder listener, you have heard so many carmakers talk about the need to get away from ECUs; Xinzhou says that moment is basically here.

We talked a lot about the Chinese car industry and how it’s been able to essentially get a head start because it began building on EV architectures and platforms, instead of having to manage a transition away from gas cars and all those ECUs. Xinzhou used to work at a Chinese original equipment manufacturer (OEM), so he has quite a bit of insight there.

We also talked about working at Nvidia itself. It’s a unique company with a unique leader in Jensen Huang, and Xinzhou said his three years there so far have been a rapid learning experience. He didn’t shy away from the reality of needing to compete for resources and capacity against the company’s booming AI business. His description of what wins those arguments, especially when his customers are as slow and cost-averse as automakers, was fascinating.

Of course, we had to discuss AI and how Nvidia’s approach to autonomy brings together what Xinzhou calls the “classical” stack and the ability for reasoning models to operate the car. There’s a lot here, including the idea that you’ll have an AI model literally talking to itself to figure out how to drive your car, which I find both incredibly interesting and incredibly funny.

And, of course, you can’t talk about electric cars or vehicle autonomy in the US without talking about Elon Musk and Tesla. So I asked Xinzhou pretty directly if Tesla full self-driving can actually do what Elon claims it will be able to do without using lidar. You tell me if you think his answer holds up.

Okay: Xinzhou Wu, head of automotive at Nvidia. Here we go.

This interview has been lightly edited for length and clarity.

Xinzhou Wu, you are the head of automotive at Nvidia. Welcome to Decoder.

I’m really excited to talk to you. It feels like the very nature of what a car is is up for grabs. It feels like the automotive industry is in a period of massive realignment, almost as though there was a sense of where the car was going to end up as a product for several years, and that is because of EV transition difficulties, because of US-China trade war difficulties.

All of that seems messier than ever before. A lot of car makers are retrenching, and it feels like your position in Nvidia gives you a pretty wide view of what’s going on in the car industry, because you supply so many of the major automakers in virtually every country.

So let’s just start there. What’s your view of where the car industry is on this long, winding road to both autonomy and electrification?

That’s an excellent question. I’ve been working in the automotive sector for probably 15 years, starting from my career in Qualcomm. I was heading the Qualcomm automotive team for a while. And obviously, we have heard the phrase “software-defined vehicle.” Right now with AI technology, it’s getting to the next phase, what we call an “AI-defined vehicle” essentially.

With these massive technological innovations, the auto industry has changed pretty rapidly over the last decade. As you know, I also worked as part of a Chinese OEM for five years, heading their autonomous driving team.

Now I’m at Nvidia. So what I have seen over my 15 years of career is the opportunity to witness this massive change. The car went from, let’s say, mostly mechanical, plus electrical machines, to some things that we can upgrade the capability through over-the-air (OTA) software pretty rapidly. That’s what we call the “software-defined vehicle” era. Now, with the technology advancing towards generative AI, we are using AI to rewrite most of the software in the car. That’s what we call the “AI-defined vehicle.”

That has also, on one hand, accelerated the development pace of the vehicle capability. And on the other hand, it’s also changed the way we define “vehicle” as well. AI is impacting the whole industry at every level. It is really exciting to see how the world will evolve from here with these new technological innovations.

Let me pull apart some terms there. I hear them a lot from car makers who love to come on the show and tell me what’s going to happen to cars. But I think some of these terms are a little bit fuzzy on the edges.

So you said “software-defined vehicle.” That’s a pretty fuzzy term. I think the idea there is we’re going to get rid of all of the ECUs in a car that currently control lots and lots of different systems. And we will centralize all of those components into maybe one or two big compute centers in a car. Tesla is very famous for having done this. Rivian has made a huge bet on that. Wassym Bensaid from Rivian was just on the show talking about that.

Other legacy car makers have tried to do this. We had GM on the show. They said, “Look, we don’t need to do that. We’re fine. We’ll do it our way.” Ford tried to do this in big ways. They had to set up a skunkworks and build an entirely new kind of way of making a car that they’re very proud of. There’ll be a truck coming out from that effort sometime soon, we’re told.

I don’t think the industry got there. That’s basically what I’m saying. The startup car makers got to the point where they could claim to have a software-defined vehicle where there were one or two big computers in the car controlling every system. The legacy automakers for the most part have not succeeded yet.

I’ll put an asterisk on that. Maybe Ford will succeed with this new truck, but we don’t know yet. Do you think the industry broadly is going to get to software-defined vehicles or do you think the legacy automakers are going to stay where they are?

100%. I had the opportunity to witness what happened in China from 2018 to 2023. The whole industry went through this massive change just in five years. Over there, not only the new auto OEMs, but also the legacy ones have to adapt. Everybody is adapting to a single central compute kind of electrical architecture because that’s how you compete.

In the rest of the world as well — we have our partners as well through Drive and Drive autonomous vehicle (AV) collaboration, for example, with our partner Mercedes. Their current generation is essential computer-based architecture. It’s going to be in all their vehicles. For the other basic OEMs, we are working with all of them and trying to help them to convert or upgrade the architecture to a one or two computers route, because there will be infotainment, there will be basic driving or advanced driver assistance systems (ADAS), ECU. But I think the world is actually moving pretty rapidly in that direction.

Some of them obviously will be slower. Some of them will be faster. That’s the nature of this business. But I have no doubt that the world is basically evolving in that direction.

I’m actually curious about your history. You worked at XPeng, which is a Chinese car maker. It feels to me, sitting where I sit in the United States and being a car fan for a long time, that Chinese automakers had a fairly unique advantage in that they were not big global automakers. They were not operating at a massive scale. Electrification came. Tesla obviously built a bunch of capability in China to make cars. We all know how the Chinese manufacturing ecosystem works and they got to reset. They got to design a bunch of cars as EVs, clean sheet, basically the way the startup car makers in the United States got to, and build globally competitive cars from a totally new foundation without having to worry about a bunch of the stuff that legacy American car makers would have to worry about. And then, the Chinese government obviously subsidized all that at huge rates.

You worked there. Was that your experience? Is that basically how it went that they got to start fresh?

I think that’s just one side of it. They definitely have less of a legacy, less of a burden to worry about and that is an advantage. But what I also see is not only the new OEMs, but even the global players there have to adapt to the Chinese pace. At least from what I learned over there, everybody is going at that pace. Again, you want to be able to compete.

But as you said, the wave… Software-defined vehicles have been there for a long time and Tesla is the one that’s really taking it to full production. I’m not sure if they’re the first one, but definitely to the largest extent. I have no doubt the OEMs in the rest of the world will follow as well.

I think every OEM right now will have to do this because this is how you compete, that is what you need to do to survive. Autonomy will become almost like a necessity for all the OEMs to have in their vehicles. We all believe in that future. And the only way to get there is to get to… First of all, there is the architecture I described, that enables software upgrades without having many, many discreet ECUs. Actually, I haven’t heard people arguing against that recently. Maybe you heard something different, but I think that’s a necessary step for everybody. At this stage, it’s almost like a table stake for the next generation’s architecture. Obviously we are talking to a lot of OEMs, but this is a consensus that the industry is moving towards.

I’m curious about the pathway there, because I agree with you that many, many people have said that is the end state and that enables everything that’s going to come next. It just feels like the path there has been much bumpier than the industry expected. Part of that is, I don’t know, that the Trump administration doesn’t like EVs. So EV sales and the tax credits here went away and maybe EV sales spiked as all that demand got pulled forward and maybe everybody wants a gas car now. And maybe all of this is harder when you don’t have a giant battery that can power all of these systems in perpetuity and you actually need to start the engine to get power to all these systems instead of having a 12-volt battery.

Or maybe it’s that the Chinese automakers are so competitive and so subsidized that the cost to do it for the legacy automakers is hard to overcome, because they do have the legacy infrastructure and dealer networks in the United States to care for and we’re just going to hold off on it. There’s something about the path to this agreed-upon future state of the car that seems harder than I thought it would be or that anyone on the show over the past five years has said it would be. I’m curious, from your perspective. You’re the supplier, you’re trying to sell the vision, you’re trying to put the chips in all the cars. From your perspective, what has made that path harder?

Well, you have said quite a few things.

The auto industry is very heavy. It involves massive supply chains and lots of companies, lots of employees. And to make a change in the architecture and whenever you push out a car, you have to support it for 10-15 years. Nvidia, as a supplier, is also making a similar commitment to our customers for whatever technology we supply including chipsets, other platforms, and our AV technology. We will have the commitment to support the same generation for 10-15 years, even for the current generation of chips. If you think about it from a Silicon Valley provider kind of perspective, it’s almost insane. But that’s the nature of the auto business. The nature of the business is that it will slow things down a bit. And that’s one thing.

The other thing is, because the technology is changing so fast and so differently from, let’s say, the automotive as we knew before and to the software-defined vehicle, to the AI-defined vehicle, you have to go through a different talent pool to be able to set up the company in a proper way and adapt to this new wave of technology innovations. That’s why Nvidia can come in and help. Because we believe the technology is getting to — we are mainly talking about the autonomous vehicle here, obviously. The technology is getting to a level of maturity. We are going to take this technology to mass production and a supplier can come in.

That’s why we not only provide the AV technology, but we also are providing the whole platform starting from a chip, as well as to operating systems, an open source model, and what we call Halos, the safety operating system that helps the OEM to be able to adapt to this new world faster.

The nature of the business is that not everybody can run at the same speed. So for sure, because of the heaviness of this industry, it will take some time for everybody to get to the finish line. But again, my job in Nvidia is to try to help everybody to get to everything that moves that will be autonomous, to get to this vision as soon as possible.

Let me ask about your part at Nvidia now, because I think this brings us to the Decoder questions. I think everyone listening to the show is probably very familiar with the run Nvidia has been on with AI. It’s one of the most valuable companies in the world. Every GPU that Nvidia can make is accounted for. How many people work at Nvidia Automotive?

We have actually quite a sizable team, in the order of thousands on the automotive team. Because we are working on the whole platform, there’s hardware, software, model, and the infrastructure. It’s a pretty sizable team. Nvidia also has a lot of things we can leverage from the other teams as well. For example, I’m pretty sure you heard about Cosmos and Nemotron. These are our basic open-source foundation models. We are leveraging heavily from work from their side as well.

How is your team organized? You mentioned you’ve got hardware, software, and models. Is that the basic structure of the team or is it organized differently?

Yes. Well, on the engineering side, obviously we have product, we have strategy, we have something behind the scenes. Sometimes we call them unsung heroes. The map team, for example, which is still very critical for L3, L4, the high-level autonomy paths. And the data infrastructure.

The literal navigation maps, that’s what you’re talking about.

Well, there’s HD map as well. So roughly, I divide my team that way. Yes.

And then is that all global? Is that mostly in the United States? Where’s that located?

Mostly in the United States, but we do have a presence in China and Europe as well. Obviously, we are building a global product, a global platform, so we need a support team everywhere.

You mentioned that you rely on some of the foundation models Nvidia has developed more broadly. How is your team structured inside of Nvidia? Does it fit into the AI strategy? Is it set apart? Are you more siloed? How does that work?

Oh, that’s a great question. At Nvidia, we have, let’s say, a centralized hardware team, which is responsible for the hardware roadmap on our GPU, the CPU, all the chipset strategy, and the productization. We have a centralized software team as well. The automotive team is a separate organization, which is very much more automotive-focused, with the mission of building the automotive platform to leverage the work from our hardware team and the software team and adapt to automotive. We have the model team as well.

Nvidia also has a culture of virtual teams. For example, our open-source model for Nemotron and Cosmos, they all sit across from our research team, the software team, and the hardware team. But they are virtual teams that work on these open-source foundation models. We can leverage that work and then in the automotive organization build up a model to help the AV industry have a powerful open-source model to work on.

As I said, basically every GPU Nvidia can manufacture is accounted for in some way. It’s the nature of the AI industry right now. They’re going to go into some neocloud somewhere. Do you have to fight for resources, and attention against that business, which is growing at the speed and the scale it’s growing at?

Yes, believe it or not. [Laughs] Even Nvidia has a limited supply of GPU for compute. We have an internal priority, and I’m working with my colleagues basically almost on a weekly basis to decide how to set aside the different compute, sometimes for training, sometimes for testing, or resources for a different thread of work in the company. And sometimes we need Jensen [Huang, Nvidia CEO] to help, but yeah.

How does that work? What does that debate look like? Is it an ROI debate? “If we put this much money in, we’ll get this much money out from our customers”? Is it a market size debate? What are the parameters of the conversation?

It’s all of the above, as you can imagine. Revenue is important obviously, but also Nvidia, as you know, is a very strategic company. We value what Jensen sometimes calls the zero trillion dollar business. We are looking for new opportunities which can create a trillion-dollar business all the time. So, there need to be strategic priorities we set inside the companies in the new direction we go. You probably also know that we are not a market share company. It’s a balance between basically what brings the money right now and what can create the future, what can create opportunity for the company in the future.

Nvidia is a very uniquely run company. As you’ve mentioned, Jensen’s deeply involved in everything. I’ve seen an interview with him where he said he doesn’t have one-on-one meetings. He just meets with everyone all at once and everyone just hashes it out. What’s that like?

I’ve been at Nvidia for three years. It’s very unique, honestly. And it’s obviously not everybody all at once. It’s different groups. We all have a technical strategy product, a different part of the business reviews with Jensen. It’s super exciting for me, actually, an experience to learn from his strategic thinking and how he thinks about a product, how he thinks about a strategy. He’s also uniquely technically deep. It’s quite an inspiring experience as well to just see how much he’s keep up to date on the technical side. It’s, I would say, a once-in-a-lifetime experience and opportunity for me to be able to learn from Jensen.

When you describe the opportunity for autonomy, particularly in the future, that seems like the big bet. “We’re going to bring to bear Nvidia’s compute excellence, and the power of AI to cars, and have them drive themselves.” What does that revenue model look like? Does it look like you’re just selling chips and software to automakers? Does it look like consumers pay a subscription, and some of that flows back to you? Where does the trillion dollars come from?

That’s also an excellent question. Right now we firmly believe that everything that moves will be autonomous. Every mile driven by the car in the future will be autonomous. If you look at it, among all the cars, we drive 13 trillion miles per year. Right now the percentage of autonomous miles among all the mileage driven is probably negligible. I think it’s 0.006%, or something like that. So this is the opportunity in front of us. Nvidia’s view is that we’ll help the ecosystem together as soon as possible by providing all the foundation technology pieces again, starting from chips to operating systems, and then to what we call Halos. The Halos operating system is really important because it not only provides the SDK and the APIs for folks to develop models on our hardware, it also provides the safety guardrails for developers to put a model on it.

We also define what we call Hyperion as a hardware platform. That’s a production-ready platform, which includes the computer resource, the ECUs, and also the sensor suite. We think it’s necessary to achieve a different level of autonomy. On top of that, we provide Alpamayo, an open-source model, which we trained. It’s open-source not only in the model architecture, but also in the parameters and the data that you can use to fine-tune the model on our platform. On top of that, we also provide all the infrastructure needed. For example, simulation is really important for developing AV right now. We usually say that the AV problem is becoming a three-computer problem. There’s the training computer, there’s the simulation computer, and then there’s the inference computer in the car. All these technology pieces we want to provide to the ecosystem in a platform we call Nvidia Drive, so that folks can develop that technology on top of our platform.

We hope that we can get a percentage of the revenue that the ecosystem can get from every mile that is driven autonomously in the future. This is where the trillion-dollar opportunity can come from.

So, revenue per mile. That sounds like the core metrics that you’re chasing. Where does revenue per mile come from for a user? When I drive a car, do I pay a subscription? Or are you thinking it’s robotaxis everywhere, and they’re being monetized per ride? Where does revenue per mile come from, and how does that number go up?

That’s right. Well, I think the world will embrace both models. One is robotaxis. As you see, there are quite a few successful ones in China, in the US, and in the world. We’ll see more hopefully going down this path. We’ll have a taxi-like fleet where you can enjoy taking you from place A to place B without a driver in the car.

I think the passenger fleet will also continue to exist for a long time because there are many people who still prefer a private space during travel. It’s like many people still prefer to own their house as compared to renting an apartment. There’s an economy behind this as well. We think both models will thrive. That’s why we are working with both the robotaxi companies and the other OEMs, as well as the AV software developer companies to help them by supplying different technology pieces from Nvidia.

One of the interesting dynamics through at least the electrification portion, over the past five years, has been legacy automakers realizing that they had become insurance companies and financing companies, and their suppliers were making the cars. They had lost control of car design in a big way. The tier-one suppliers to the big automakers were in many ways in charge of big subsystems of the cars. When they wanted to do an over-the-air update, they had to go talk to 15 different suppliers to get that done. I’ve heard this complaint dozens and dozens of times on the show. And they all kind of realized, “We need to take back the engineering of the car. We need to be much more firmly in control of the platform of the car.”

It sounds like in autonomy, for a variety of reasons, Nvidia sees an opportunity to become the main supplier to a wide variety of car makers. That’s obviously the intention with them thinking, “We need to take control of the car.” Tesla might use Nvidia chips, but they are very proud of the fact that they wrote every line of that code, and that is their platform, and they’ve made their technology bets. Rivian, I think Wassym is very proud of the fact that he is in charge of that platform company, and he’s going to build that platform. RJ [Scaringe, Rivian CEO] is certainly very proud of the fact that Rivian is that kind of company.

What’s the dynamic there? Because it doesn’t seem like every car maker can stand up the technology bet and forward invest on the hope that the revenue will pay off. They will need a supplier like Nvidia to show up with a ready-made platform and business model. Is that tilting more in your favor now? Have we gotten out of those woods, or is it still up in the air?

I think the beauty of the Nvidia business model on the automotive side is that our platform is completely open. We provide multiple layers of services, and depend on what OEM or robotics companies need. They can select what they want to work with us up to which layer for, as you mentioned, Tesla. Some OEMs are so capable. They even want to build their own inference to paint the car. Even for that, we’re okay. We’ll still continue working with them. Actually, we are working with Tesla and many OEMs who are building using their own inference chip by collaborating with them in the cloud. We even try to help optimize their models. With different OEMs, we have different collaborations, because we still have the simulation computer and the training computing in the infrastructure we are working with them.

For some of the OEMs, they would like to have a more turnkey solution. We are very happy to work with them as well. In that case, we are going to go all the way. We are working like a tier one, or tier 1.5, just to go hand by hand. This is our driver AV kind of partners, for example, Mercedes. We work very closely with them to define the products they want, and then also adapt our driver AV stack to work seamlessly in their vehicle. The engineers from both sides work pretty closely to make it adapt well with the Mercedes design DNA and the customer experience they would like to offer.

This is really important for us. We are not picking winners per se. We try to help OEMs based on their capability at different levels. The openness is really important for our engagement model with OEMs.

One of the reasons I’m so curious about this is that you mentioned training models. You mentioned in other interviews that you’re doing synthetic data to train autonomy in different ways. I’m very curious about that. It just strikes me, looking at the industry. Waymo has this gigantic lead in autonomous miles driven, and they’re very proud of it, and that’s helped make their cars as successful as they are in the markets they’re in. Tesla has a huge number as well because they’re training on the actual cars that are being driven. Not every automaker can figure out how to get to a billion autonomous miles driven.

They’re going to have to rely on some third party to get them to at least the status quo, if not beyond. It feels like Nvidia is sitting there ready to be that third party. Is that a lot of the sales pitch to the automakers? “You can just buy our technology in whatever open capacity that you want, and we will quickly get you to a competitive state”?

I would say this is the one compelling point for OEMs to engage with Nvidia in the Hyperion ecosystem, in the Drive ecosystem. Because one of the key defining things for Hyperion is the compute architecture, and also the sensor architecture is the data sharing. For anybody who engages and becomes an Nvidia Drive partner, we share data through our existing program, through which we collect millions of hours of data. Through the different car programs, we are also accumulating that data from different OEMs. And then we can build a model, first of all, which is trained with all this data. We make sure at least the data collected in our different car programs is shared with the OEM. That’s number one. Number two is in the new era, we strongly believe compute is data as well. So, as you mentioned, there’s a lot of synthetic data.

There’s neuro reconstruct data, which we call NuRec. This is a very important piece of technology and simulation where we collect the data from the field, but we can use neuro reconstruction sometimes to fudge the data to change the background, or change the car trajectory. We can generate a lot of variants of the same data. All this data needs a computer to generate these tens of millions of data points. We can share with everybody who’s engaged in our ecosystem. In this way, collectively from all the players that engaged with the Drive ecosystem, we can catch up on the data gap, which is very important.

So, this is synthetic data, right? You’re going to collect a bunch of real-world driving examples. You’ll put it into a simulator. The simulator will then blur the data. I think the example that I’ve heard you give is there was a pedestrian that came out, and we can just delay the pedestrian, and make that person come out later, and the car will have to react to it as though it’s real.

And you’re going to run lots of training against lots of different variations of the same data. That’s fascinating to me. I understand why all the car makers would buy into that. Why would they buy into the data sharing? Is it just a recognition that collectively they stand a better chance of catching up? Is it because they just don’t want to pay the money? Is it cheaper? Why would they participate with their competitors in that kind of data-sharing arrangement?

Both are absolutely true. Actually, the cost savings are enormous. Data collection running a fleet of huge size is big capital spending for anybody who wants to do that. It’s repetitive as well. If you can find, for example, what we are providing on the Drive platform, or the Drive ecosystem, it can save a lot of effort and money from our customers.

I’m curious about that because the idea is that you’re going to train stuff, and then you’re going to have a model in the car, and we’ll have an AI-defined car. The classical approach to self-driving was that we’re going to throw more and more data at the problem, and eventually the car will know how to do everything, it will have mapped all the roads on top of everything. I have a Cadillac EV, and the way Super Cruise works is, it works on roads that are mapped. Eventually the bet is they’ll map more and more roads, and more and more things, and the car will become more capable.

It feels like Nvidia’s approach is for the car to be smart enough to do anything, with or without the maps. And that requires a different approach to data collection, a different approach to compute, and then obviously a bigger bet on AI. Is that split real? Have you just made that jump? Is that the future of the platform, or are you in the middle?

The approach we take right now for what we call the L2+, which essentially is mapless. As you said correctly, the model will definitely need more data, and to cover more corner cases. And the model is getting bigger as we speak as well, for this generation, next generation. We are going to use a much bigger model with more parameters. Foundation models will also play a big role here. To be able to make this model very capable, more data is very, very critical. But on the other hand, the trend of using a foundation model, which is already trained with internet data, can help as well. That’s why I emphasized quite a few times the connection with the foundation model effort inside Nvidia.

With the reasoning model, and the foundation model, we can leverage from the frontier model perspective, and leverage the internet to scale data to be able to help the vehicle to generalize better, even without the vehicle-specific data. This is the main direction we are betting on, towards a higher level of autonomy, especially Level 4. This is one of the main work threads we are focusing on right now.

Back to OEM, I think being able to leverage what we have built upon through our collaborations with the existing engagement and our massive capability for data generation using synthetic datasets and neuro reconstruction — as well as being able to leverage the foundation model capability which is trained from more general data, but which will help the model to reason better, to generalize better — these are the things we can offer to our customers.

I feel like I have to ask about safety now. I’m sure it’s more complicated than this, but you’re talking about a foundation model reasoning through self-driving. And all I have in my head is ChatGPT apologizing to me because it got it wrong while the car crashes, or one of those horrible long latency loops where the model goes off in the wrong direction and realizes it. And then you can look at the chain of thought and it’s like, oh, it got it totally wrong. It feels bad in the way that Anthropic believes that Claude feels bad. None of that seems compatible with the very real-time nature of driving a car. How do you bridge that gap? Latency, the need to have one of those big models in the background, the sort of reasoning tangents that the models can go on. How is that compatible with driving a car?

Safety is so important to us and it’s so important for the AV industry. Let me answer your question by approaching different layers of our offering. To address safety is obviously not new for the auto industry and we have developed a very sophisticated development protocol and also a validation protocol to be able to prove the software is safe. That’s called ISO 26262. We actually develop our hardware and operating system (OS) level software and the application level software to the highest standard of safety, which is very important and critical to being able to deploy anything to drive the car. That’s number one.

Number two is that we take a slight different approach than some of the players in this space. We actually have a redundant stack even for our L2++ or ADAS function. Other than that is the end-to-end model, which is basically pixel-in, trajectory-out. We also have a classical stack which is more developed based on this safety standard as we know it. It’s a component basically. It’s a stack with many components and each component can be verified using this known standard. That’s what I refer to as a classical stack. And when you have two stacks running in parallel, the classical stack acts like what we sometimes call Big Brother, but essentially it’s a safety guardrail. It tries to verify all the trajectories from the end-to-end model and use the known safety standard to verify it’s safe at every frame.

That’s a very important concept we have. And not only a concept, but the implementation we have in our stack. We will take this, which will be so critical for higher level autonomy, L4. This is also the foundation of our L4 stack where we have full redundancy, not only as a sensor set, but as a software architecture set. This second point is to answer your safety question.

Number three, when we develop the model, we are also trying to make the model reduce the hallucination as much as we can. The way to do that is through massive validation. We are building massive simulation test data for every model we release. Right now we are looking at running five million tests in our program every day.

Roughly every day we have a 10 iteration of the model, the end-to-end model. We are doing really massive validation to make sure in all these scenarios – you can think of that as every tested test scenario – the model is generating the right trajectory. That’s also super critical for us. So this is what we do to make sure our product is safe.

Let me ask you a really dumb question I’m really curious about. You’ve talked a lot about the model and how it will operate the car. And yes, the classical stack is the safety guardrail. Is the model reasoning in language like every other model? Is it sitting there in the background saying, “I see a stop sign. What do I do? I’d better stop. I’m going to go hit the brakes,” the way that any sort of general model reasons in language in the background?

The short answer is yes. In our next-generation model, which we are going to deploy in the next generation of vehicles… Because the current generation has a more or less limited computer, the next generation is SOAR-based. We will have the model trained with language embedded. Being able to reason through language is very important. You can also chat with the model. You can ask the model about what it’s doing and then you can also ask a model to speed up or slow down and make a lane change, for example.

As it’s literally driving, it’s saying to itself, “I see a car over there. I need to change lanes to get ready for the exit that’s coming in a couple miles.” And it’s doing that in language to operate the car?

I think it’s combination of things. Language is already embedding the model, but the vision signal is also super important, as you know.

I want to say it’s multi-model, but language is part of it. As you know, the model is black box. We don’t exactly know what it is exactly doing, but you can ask about it and then the model will answer what it’s trying to do.

I just have this vision of a chatbot model freaking out as it careens on the highway at 55 miles an hour.

Actually, GTC Taiwan released a video that showed that the model is talking constantly. It can be quite annoying if you really try to hear everything the model is trying to reason about.

What’s the latency on that? Obviously you’re deploying the systems, it must be working, but is there an attempt to reduce the latency of that? I feel like language is inherently slow compared to what you need to do to drive. I’m not thinking in language when I drive my car.

100%. That’s why I said it’s multi-model. But to reduce the end-to-end latency is super important. Actually, that’s one of the key advantages of deploying, driving the car with a model. Because if you think about it, the old stack or the classical stack which has multiple components, it usually takes multiple hundred milliseconds. But with a model, because it’s just inference time, it’s separate from input, which is pixel and trajectory. You can reduce the latency, depending on the computer capability you have, obviously. But even in the current generation, we control it to be within 100 milliseconds, which is pretty fast.

Regarding the language reasoning, if you think about it, well, that’s the human brain, right? If you think about it, I would say the information rate is already abstracted. The information rate is not super high. And we are using the internet data to train this kind of language-based reasoning capability. I think the latency is well under control, let me put it that way. And again, you are not driving the car with language only. That’s a key thing, as I said. Usually the reasoning part is, I believe, slower. Again, we don’t know exactly what the model is doing, but the pixel part, that’s what drives the basic instantaneous reaction of the vehicle.

Yeah. If you ask Anthropic, they will tell you that Claude has feelings and emotions and he can get scared.

Do you think about that? Do you think your models have emotions when they’re driving the car?

We’ll use the guardrail to make sure it doesn’t get too moody.

[Laughs] I’m just curious. I mean, like you said, we don’t know how the models are working. I literally have a vision of the model being like, “Oh my God, I’m going so fast.” But maybe the classical system will cut that down.

Is all this running locally in the car?

No, no, no, no. All these validate offline. But the second part with the safety guardrail, when we run two stacks in parallel, that’s definitely in the car. And in the car at every frame, the software in our ADAS ECU, we are comparing the trajectory from both the classical stack and the end-to-end model to make sure the model is outputting a safe basic trajectory.

So do the cars require fast connectivity to be autonomous with your approach?

Not necessarily, but we do require some connectivity to get navigation information and some map information. Most of these are navigation maps. So to help not only the model side and also the classical stack, we do use some of the navigation mapping information to help us understand the world better.

I’m only asking because I covered the launch of 5G networks in great detail and all of the telecom companies promised me that 5G would enable autonomous cars. And it seems like your approach is the one that will lean the most heavily on low latency networks in that way.

This is not wrong, but on the other hand, the car has to drive autonomously in a complete blind spot as well. Real-time low latency, I would say content dependency, has that dependency in the cloud. At least for the ADAS kind of application — L2+ is what we call it — which is meant to work everywhere, building that dependency is not a good idea.

Yeah. When you get to Level 4, Level 5, that’s when you have connectivity dependency.

That’s right. Yes. Yeah.

What happens when you lose the connectivity at Level 4 autonomy? When you’re at Level 5 and you don’t have a steering wheel anymore and you lose connectivity, what happens?

In Level 4, you can think of connectivity as a kind of sensor. The basic driving capability cannot have huge dependency on that. And one of the core concepts of developing Level 4 technology is you have sensor redundancy. That’s not only for GPS, but also for camera, radar, everything you see. For every single point of failure, the car has to be able to drive safely. It’s like if you suddenly lost a GPS, but the car has a local perception, it needs to be able to get to a safe point and pull over. That’s a minimum requirement an L4 system needs to have. This is the basic L4 principle to be able to develop such a system.

I’m very curious about where all of the sensor stacks live in the car, how much compute is in the car, how much RAM we need to put in cars at a time of increasing RAM prices. This all seems like a lot of extra cost to layer into cars which are increasingly getting more expensive and which consumers, at least in the United States, feel like they’re rebelling against in lots of ways.

I can look at our own website traffic; everybody wants to buy a Slate truck for $25,000 and it doesn’t even have a radio. That’s just a battery on wheels. That’s that whole car. It doesn’t even have a paint job. We’re getting rid of paint jobs on the cars now to keep the cost down. You’re talking about a lot of compute in the car, a lot of connectivity, maybe a bunch of RAM to load the models on.

How does that play out? Does that push you more into that robotaxi model or do you think people are just going to buy expensive self-driving cars?

Definitely building autonomous cars needs a lot of hardware, but the other trend is that the hardware cost is dropping pretty rapidly as the technology becomes more mature. For example, radar. Even in my career, I have seen radar prices probably drop by at least four or five times over 15 years, because of the volume getting much bigger and bigger than the cost. I have witnessed the drop of camera sensor prices as well. There are more competitors and the competition brings lower prices when the volume becomes bigger. The scale effect is definitely there right now in ADAS and all the components become much more mature and to some level of commodity.

As you know, the computer’s growing at such a rapid pace. We talked about Moore’s Law in the semiconductor industry some time ago, but in the autonomous driving segment, the computer need has been growing at a really astonishing pace. Roughly we are talking about 10 times every two years. It’s insane. And right now, with the success of AI and obviously Nvidia, we will be able to provide this kind of massive compute to cars at an affordable price.

In the cloud or in the car?

I asked you about fighting for training capacity earlier. Do you have to fight for fabrication capacity too?

Because those costs are going up for everybody.

I’m curious, it’s Nvidia’s demand that’s driving up the cost for everybody. So how do you go get fab capacity when the other divisions at Nvidia are willing to pay whatever rates anyone demands?

Well, it’s the same answer I’ve given you. I don’t know if there’s anything more I can say because we are such a strategic company and our automotive business is doing well as well, but not at the pace of our data center business, obviously. But we are strong believers — Jensen himself, as well — in the AV future. We keep investing in this technology and in this future, not only by allocating internal compute, but with fab capacity as well. That’s definitely one of the things we are looking into.

Actually, most likely even the chip price will need to go up, because of this intense demand for every chip everybody can grab onto. The positive side is that the technology is really getting [good]. I talked about the chip side and also I talked a little bit about the sensor side. I talked about Hyperion, which is a product-ready, compute-plus-sensor kind of platform. So we are really trying to balance between the cost and what we can do. We are looking at what we call the sufficient necessary sensor set to achieve a high level of autonomy.

In Hyperion 10, for example, we really offer two versions. One is a base, which is mostly cameras: 10 cameras, three radars, no lidar. It’s a very cost-effective way to build a L2++ ADAS kind of vehicle. And on the other hand, for the high end, is what we call the Hyperion High: we provide the sensor set required, which has, I think, 14 cameras, three lidars, and seven radars to have enough sensor redundancy to be able to drive L4.

You need ECU redundancy as well. You need our next generation – well, actually, to be more precise, the current generation SOAR-based computer platform. Just imagine you have a car that can really drive by itself. We believe with this sensor set and this computer architecture, we can get to that level of autonomy which can justify the cost.

The minimum sensor set for autonomy feels hotly debated. It’s been hotly debated for a long time. I think Elon Musk saying that he thought lidar was a local maximum ages ago was the beginning of this debate. This debate has not quelled in any way, shape, or form. Do you think Level 4 requires lidar?

The short answer is yes. We believe that lidar is the important sensor to provide the safety and the redundancy required for Level 4 autonomy. But on the other hand, it’s difficult to say it’s 100% necessary. We believe this is a very feasible path based on Hyperion 10’s high-sensor configuration to get to both really high-level urban and highway Level 4 capability. On the other hand, theoretically, people can prove out with massive mileage that lidar may not be necessary. But it will come with the ODD limitation, essentially.

Sorry, what’s the ODD limitation?

Operational design domain (ODD) is basically an applicable domain. You can deploy the technology. We have done quite a bit of analysis on this. Based on our current understanding and the framework we use to do this analysis, we believe that to deploy this L4 technology in all the ODDs that our customer can benefit from, it’s much better to have lidar as compared to not having it.

When you look at where Tesla is with full self-driving and their vehicles and their absolute commitment to being a vision-based system, do you think that they are currently ahead of you? Do you think they’re at parity? Do you think they’re behind you?

There are two levels of answer to this question. For the basic L2++ technology, Elon is probably ahead of everybody. He had a division a long time ago and he has stuck to the vision for a long time to develop and test the technology among a massive fleet. Nobody would argue that Elon is ahead of everybody in the ADAS market and everybody is playing a catch-up game, essentially. And we are very happy, actually, that Elon is so successful. Obviously, Elon is a big customer for us as well, for both SpaceX and Tesla on the GPU computer side. We are supporting him and his team to make sure they’re successful.

For Level 4, I think it’s more open. There are established players who are proven, like Waymo, who are already taking customers to really experience the L4 using the methodologies they use. Tesla is probably still trying to find the path there. We don’t try to pick winners, but we are trying to help everybody to be able to develop that technology. Our mission is really to try to make the AV ecosystem get to this vision of every mile, everything that moves will be autonomous. This kind of vision becomes a reality.

Have you had conversations with Tesla executives about using lidar? It seems oddly religious for no reason, especially if the costs are coming down as you say. At some point, if the better technology solution is right there, it feels like everyone should just use it. Have you had those conversations?

Well, actually no, not myself. My team definitely has. I’m looking forward to having that conversation with them. Much of this is just basic science and reasoning. It’s good to hear their views as well.

I want to wrap up by talking about something that maybe is the least in your control. Models are going to keep getting better, Nvidia is going to keep making chips, maybe customers are going to keep demanding self-driving. That all feels like something you have a handle on.

But the auto market, the cutting edge of the auto market is happening in China. I think we can just agree on this. US consumers open TikTok and see car influencers talking about BYD vehicles and they complain in the comments that they can’t get those cars. I watched a video of a Buick that is in China. It’s a Buick EV that you can’t get in the United States and US customers are furious that Buick is making better cars in China than they’re making here.

There are a lot of trade barriers between the United States and China. Nvidia sits in the middle of that fight in all kinds of ways, whether it’s tariffs on imports of car components, or literal blocks on what chips can be sold and where the revenue from those chips go. As you try to push the car market forward, how does the US-China trade chaos play into it? Is that something you think about? Is it something that’s slowing the industry down? Is it something that you can push through?

Well, I certainly believe the policymakers have their reasoning and rationale to make the policy as we see right now. As Nvidia, we are an open ecosystem player. We still have a lot of customers in China. We try to help… For example, we are still supplying in-car inference chips because they’re still below the threshold of what GPU is allowed to sell in the China market. We are also working with all the Chinese OEMs. Actually, not all of them, obviously, but quite a few of them, to help on the infrastructure side by supplying simulation tools. We are working with them on open-source models, Cosmos and Alpamayo. On one hand, we can help them to make their models better. On the other hand, we can also learn from the competition in the China market.

We are also working very closely with the rest of the world’s OEMs, and we try to supply all Nvidia platforms and at different layers to different OEMs to help them to be successful as well. Again, we don’t pick winners and we try to work with everybody. The mission is super clear and we try to make this vision become reality as much as possible.

When you talk about sharing data between OEMs to train the models better and to make them more capable, are there any regulatory roadblocks or competitive roadblocks between sharing data from Chinese OEMs and American and European OEMs?

Oh yes, of course. Not only China but other regions have restrictions as well. For example, Europe has certain regulations regarding data. We are conformed to all the local regulations to make sure we are compliant with different regions.

Does that mean that regional variants of the models have different capabilities or they’re better at different things? Because if the input data is different, it seems like maybe the output will be different as well.

Absolutely. Well, first of all, for the production model, we try not to fork it as much as we can, but there will be basic regional differences. The model will behave differently in different regions based on the input. Some of the things are what we call country-coded. So obviously the rules are quite different in different regions like in Europe as compared to the US. Some adaptation is required and some parameters are different as well. Yeah, it’s quite an interesting journey trying to scale the technology into different parts of the world.

Do you think that — based on the different regulatory approaches, the different data approaches, different input data, the different configuration of the OEMs and what they’re willing to invest in, the different subsidies from the governments — China will get to Level 4 as a mainstream self-driving experience first? Because if I had to look at it, I would bet that Level 4 self-driving will happen in China way before it happens in the United States as a mainstream experience.

I actually don’t think that’s true. As you know, Waymo is already getting everybody to an L4 experience, at least in certain ODDs in San Francisco, and they’re scaling pretty fast. China is obviously a much more dynamic competing market and there are quite a few players there. But my experience is that all of them have the maturity of Waymo, at least in San Francisco. We are trying to help everybody in the ecosystem.

From an OEM perspective, it’s a different competition landscape, but even on the OEM side, I think different regions have different kind of… Well, one side is that Chinese streets are also much more challenging as compared to US streets. And the Level 4, I would sometimes call it a zero-pne game. Either you have it or you don’t have it. As of today, I think the only one who really has proven that L4 can be safely deployable to every customer without a driver in a city-sized region without any limitation is still in the US or in China.

Yeah, that’s Waymo. I think Waymo is going to be very flattered to hear them described as a mainstream experience. I will accept that for some subset of people in San Francisco, Waymo is a mainstream experience. I think for the vast majority of Americans it is not yet. And that is the big turn, right?

When can a Waymo work in the snow? When are they going to deploy them in Chicago?

As somebody who was in Chicago for a long time, I’m very curious how that goes in Chicago and New York City. The question I have is whether the mainstream experience feels like you just buy a car and Level 2 ADAS is kind of a commodity in cars now. Level 4 will be a mainstream commodity in cars. You push the button and it starts driving itself. How far away do you think we are from that?

That’s exactly my mission, trying to help the industry to get there. I would say if I need to give a time, I wouldn’t say five years, but less than five years.

That is a bold prediction. I think we’re going to leave it there, because we’re at time. You’ve been really great, Xinzhou. I’m excited to talk to you again. We’ll have you back before five years to check in on that prediction. But what should we be looking for next from Nvidia?

There are quite a few things we are planning. First of all, by the end of this year, we are rolling out our technology on the ADAS side in all Mercedes vehicles and some other partners as well, all over the United States. Starting in the next few years, we’ll try to roll out this technology to the rest of the world. Meanwhile, we are also working closely with our partners, for example, Uber. We announced that at GTC, we are trying to roll out our L4 service in the next few years. It’s super exciting.

On top of that, we are, again, an ecosystem player. We are working with almost all OEMs. Right now, I would say 80% of the mass-production OEMs are in Nvidia’s Hyperion ecosystem for L4. We are building this future with everybody. Hopefully you’ll see more exciting announcements from us somewhere down the road.

Well, like I said, we’ll have to have you back soon. Thank you so much for being on Decoder.

Thanks for having me, Nilay. It’s very nice chatting with you.

_{Questions or comments? Hit us up at [email protected]. We really do read every email!}