Cars without drivers? How AGI can “get rid of” humans from autonomous driving

[Editor's Note] The emergence of ChatGPT and GPT-4 has made people truly feel the powerful ability of "big models to change the world." However, most discussions about big models are abstract and non-specific, perhaps too optimistic or too worrying.

As Andrej Karpathy, co-founder of OpenAI and former director of AI and vision at Tesla, wrote in a recent blog post titled “Self-driving as a case study for AGI”: “Unfortunately, much of the discussion has been rather abstract, causing people to go around in circles on the topic and not reach a consensus.”

Karpathy used autonomous driving as an example to talk about his personal views on the future form of AGI, and believed that "the development of autonomous driving capabilities is a good early case study for studying AGI."

The core ideas are as follows:

Copilot and GPT-4 are “secondary” programming automation;

AGI will be limited by demand outstripping supply due to developer self-limitation, regulatory constraints, and simple resource shortages (e.g., the need to build more GPU data centers);

In the wider world of work, many jobs will change, some will disappear, but many new job opportunities will emerge, which is more about restructuring of jobs than outright elimination;

AGI is more like technologies like autonomous driving, where progress is gradual, society is both an observer and a participant, and expansion is limited by a variety of factors, including regulation and educated labor resources, information, materials, and energy;

The world will not collapse, but adapt, change and reconstruct. Take autonomous driving as an example. Traffic automation will make it safer, cities will become cleaner and smoother, and parking lots and roadside parked cars will gradually disappear, making more space for people.

Academic Headlines has made a simple translation without changing the main idea of ​​the original article. The content is as follows:

There has been a lot of discussion recently around artificial general intelligence (AGI), its timeline, and possible forms, as large language models (LLMs) have been developed. Some of these discussions are hopeful and optimistic, but there is also a lot of anxiety and pessimism about the future. Unfortunately, many of these discussions are quite abstract, causing people to go around in circles on this topic and unable to reach a consensus. Therefore, I have been looking for concrete analogies and historical precedents to explore this topic in a more practical way. In particular, when asked about my personal views on the future form of AGI, I tend to use autonomous driving as an example. In this post, I will explain this.

First, let’s look at a common definition of AGI:

AGI: An autonomous system that can surpass human capabilities in most economically valuable tasks. Note that there are two specific requirements in this definition. First, it is a fully autonomous system, that is, it is able to operate independently with little or no human supervision; second, it is able to operate autonomously in most economically valuable tasks. For specifics, I personally like to refer to the U.S. Bureau of Labor Statistics' Occupational Index. A system that has both of these characteristics is AGI.

In this post, I want to argue that our recent developments in self-driving capabilities are a great early case study for the societal dynamics of increasing automation, and by extension, what AGI in general will look like.

I think this because the field of autonomous driving has a few characteristics that can be roughly summarized as “it’s a big thing”: autonomous driving is very intuitive and visible to society (imagine cars on the street without drivers!), it’s a large part of the economy in terms of scale, it currently employs a lot of human work (think Uber/Lyft drivers), and driving is a hard problem to automate, but we did it (ahead of many other areas of the economy), and society has noticed and is responding to it. Of course, there are other industries that are also experiencing huge automation changes, but I’m not personally as familiar with them, or they fall short on some of the above characteristics.

Partial Automation

In the field of artificial intelligence, autonomous driving is considered a "sufficiently complex" problem. It does not appear in a vacuum; it is a gradual process of automating the task of driving, with many "instrumental AI" results in the middle. In terms of automotive autonomy, many cars are now equipped with "level 2" driver assistance systems - an AI that collaborates with humans to complete the task of driving from point A to point B. This system is not fully autonomous, but can handle many low-level driving details.

Sometimes, it can even automate the entire operation (e.g., parking the car for you). The human acts primarily as a supervisor of this activity, but in principle can take over the driving task at any time, or issue high-level instructions (e.g., requesting a lane change). In some cases (e.g., lane following and quick decision-making), the AI ​​outperforms humans, but may still underperform in rare scenarios. This is similar to many tool AIs we are beginning to deploy in other industries, especially as the power of large language models continues to increase. For example, as a programmer, when I use GitHub Copilot to automate a piece of code, or write a larger function with GPT-4, I hand over the low-level details to the automated system, but at the same time, I can "intervene" when needed. In other words, Copilot and GPT-4 are "secondary" programming automation. There are many such second-level automations throughout industry, and they are not all based on LLMs - from TurboTax to robots in Amazon warehouses to many other "tool AIs" in translation, writing, art, law, marketing, and more.

Fully automated

In the evolution of automation, some systems achieve enough reliability to become like Waymo today. They gradually move into the realm of full autonomy. Today in San Francisco, you can open an app and call a Waymo instead of an Uber. A self-driving car will pick you up and take you, the paying passenger, to your destination. It's really amazing. You don't have to know how to drive or pay attention, you can just lie down and take a nap while the system safely takes you from point A to point B. Like many people I've spoken to, I personally prefer riding in Waymo to Uber and have almost exclusively switched to Waymo for in-city transportation. You get a more stable, repeatable experience, the ride is smooth, you can play music, and chat with friends without having to spend mental energy wondering what the driver is thinking while listening to you.

Fully automated hybrid economy

Yet, even with the availability of self-driving technology, many people still choose to hail an Uber instead of a Waymo. Why? First, many people simply don’t know they can hail a Waymo. Even if they do, they may not fully trust the automated system and would prefer to be driven by a human. Also, some people may simply enjoy talking to a human driver, joking around, and interacting with others. Beyond personal preference, Waymo is clearly in short supply, judging by the increasing wait times on the app. There aren’t enough cars to meet the demand. Part of this may be because Waymo is carefully managing and monitoring risk and public opinion. Another possible reason is that Waymo has a quota from regulators on how many cars they are allowed to deploy on the streets. Also, Waymo can’t completely replace Uber overnight. They need to build infrastructure, produce vehicles, and scale operations. I think a similar situation will occur with various automations in other economic sectors—some people or companies will use them right away, but many people 1) don’t know anything about them, 2) don’t trust them even if they know, and 3) would still prefer to hire and work with humans even if they do. Beyond that, demand outstrips supply, and AGI will be limited in the same ways for exactly the same reasons — self-imposed limitations by developers, regulatory constraints, and plain old resource shortages (like the need to build more GPU data centers).

Fully automated globalization

As I mentioned earlier with resource constraints, scaling this technology around the world is still very expensive, labor-intensive, and slow. Today, Waymo is only operating in San Francisco and Phoenix, but the approach is inherently general and scalable, so it may soon be able to expand to Los Angeles, Austin, and other places. Product deployment may also be limited by other environmental factors, such as driving in heavy snow. In some rare cases, it may even require rescue by human operators. Expanding capabilities is not "free." For example, Waymo must expend resources to enter a new city. They must determine the location, map the streets, and adjust the perception and planning/control systems to adapt to some unique situation or local rules and regulations. In our work analogy, many jobs may only be fully automated in certain environments or conditions, and expanding coverage requires work and effort. In both cases, the approach itself is general and scalable, and the frontier will expand, but only gradually over time.

Social reaction

Another aspect that I find interesting is that just a few years ago, there was a lot of commentary and FUD (fear, uncertainty and hesitation) about “will it work” and “will it not work” and people were arguing whether it was even possible. Now, autonomous driving is really here, it’s no longer a research prototype, it’s a product – I can trade money for a fully autonomous transportation service. The industry has achieved full autonomy within the current range of operations. Yet, by and large, almost no one cares about this. Most people I talk to (even those in the tech industry!) are not even aware of this change. When you take a Waymo on the streets of San Francisco, you will see many people seeing it as something exotic. They are surprised, then stare, and then go on with their lives. When full automation is introduced in other industries, perhaps the world will not be taken by storm. Most people may not even realize it at first. When they realize it, they may glance and shrug, and the reaction ranges from denial to acceptance. Some people are so upset about this that they put cones in front of Waymo in protest. Of course, the impact of this is far from being fully felt, but when it happens, I expect it will be a widespread foreshadowing.

Economic impact

Let’s talk about the impact of automation on jobs. Obviously, Waymo replaces drivers, but it also creates a lot of less visible jobs that didn’t exist before—human annotators who help collect data for neural network training, customer service people who connect remotely to support when a vehicle has a problem, people who build and maintain fleets, maps, and so on. To assemble these highly intelligent, high-tech cars, a whole new industry of sensors and related infrastructure is being created in the first place. Similarly, in the broader field of work, many jobs will change, some will disappear, but many new ones will emerge. This is more of a restructuring of jobs than a direct deletion, even if that deletion is the most significant part. It’s hard to say that the overall number of jobs won’t decrease at some point and over time, but it will happen much slower than people who naively look at this situation would imagine.

Competitive Landscape

The last aspect I want to consider is the competitive landscape. A few years ago, there were a plethora of autonomous vehicle companies. However, due to the sheer difficulty of the field (I consider it “just possible” to achieve automation with the current level of AI and computing technology), today, the ecosystem has been significantly consolidated. Waymo has achieved the first functionally complete demonstration of the autonomous future. That said, there are still companies catching up, including Cruise, Zoox, and of course, my personal favorite, Tesla. Given my actual experience and involvement in this field, I will briefly explain it here. I believe that the ultimate goal of the autonomous driving industry is to achieve full autonomy on a global scale. Waymo's strategy is to achieve autonomy first and then expand globally, while Tesla's strategy is to first expand globally and then gradually achieve autonomy. Today, I am a big user of both companies' products, and personally, I am more in favor of the overall technology. However, one of these two companies still has a lot of software work to do, and the other has a lot of hardware work to do. I have my own judgment on which one will develop faster. That being said, as many other economic sectors may experience, after a period of rapid growth and expansion (think autonomous driving circa 2015), if the analogy holds true, only a few companies will emerge victorious from the fierce competition. In the midst of it all, there will be a lot of actively used tool AI (like today’s Level 2 ADAS features) and even some open platforms (like Comma).

General artificial intelligence

That’s the rough outline of what I think AGI looks like. Now just mentally copy and paste it across the economy, with all sorts of interactions and second-order effects that are hard to predict, all happening at different speeds. While this model may not be perfect, I believe it’s a useful thinking tool. In a way, it’s not like a self-recursively improving superintelligence that escapes our control and goes into cyberspace to create deadly pathogens or nanobots that turn the entire galaxy into gray goo. Instead, it’s more like a technology like autonomous driving, where progress is gradual, society is both an observer and a participant, and expansion is limited by a variety of factors, including regulation and educated labor resources, information, materials, and energy. The world won’t collapse because of this, but will adapt, change, and reconstruct. Taking autonomous driving as an example, the automation of transportation will make it safer, cities will become cleaner and smoother, and parking lots and cars parked on the side of the road will gradually disappear, making more space for people. I personally look forward to similar changes brought about by AGI in various fields.

Original link:

https://karpathy.github.io/2024/01/21/selfdriving-agi/