Is autonomous driving equivalent to a frog driving a car? How far do smart cars have to go?

Self-driving cars are already a hot topic that everyone is familiar with, and many high-tech companies are optimistic about its future. So how far are we from self-driving cars? What technical challenges does self-driving cars face? Here are all the things you don't know or want to know about self-driving cars!

(Left) Duan Yulong

Beijing Radio and Television Station host

(Center) Li Huiyun

Researcher at Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences

(Right) Li Shengbo

Tenured Professor, School of Automotive Engineering, Tsinghua University

**Duan Yulong: **No rationality, no truth. Hello, dear audience friends, welcome to the rational party. I am the host Duan Yulong. In this episode, I will talk to you about autonomous driving.

Why don’t we have driverless cars yet?

**Duan Yulong: **The concept of autonomous driving is very popular right now, but why, after so many years, autonomous driving cars have not come to us, and are only on paper or on some closed test roads?

**Li Huiyun: **I think it is developing quite fast. As a complex system engineering technology covering all aspects, including automobile transportation, roads, and communications, it is developing quite rapidly.

In fact, you may have heard some news during this epidemic that our unmanned vehicles have been involved in the distribution of materials, including epidemic prevention materials, and the transportation of some personnel. What you may not know is that there are also some unmanned mining trucks in operation. So in fact, in some specific areas, it has gradually entered our lives.

**Duan Yulong: **But for ordinary people like us, when will we be able to drive a self-driving car to and from get off work or pick up our children? We don’t even need to use the word “drive”, the car will be able to help me complete these tasks?

**Li Shengbo: **Actually, this is a very complicated issue. Generally, when the media talks about driverless driving, they are talking about replacing drivers and humans, which is a very ideal goal. But at present, such pure driverless technology still has many bottlenecks and problems that are difficult to solve. From the perspective of intelligence, the human organism started from a single cell. In about 4 or 5 billion years, it has evolved from single-celled lower organisms, multi-celled lower organisms, fish, reptiles, and mammals, and finally evolved into humans. In this process, our intelligence is also constantly evolving. Humans are the most complex and intelligent organisms on earth that we know of. Our brains have about 80 billion neurons, and only this organism can really drive. At present, the machine neural network used in our autonomous driving technology is about 10 million to 100 million in terms of the scale of neurons. For a living organism, this is probably the level of a frog .

**Duan Yulong: **Do you mean that the current driverless technology is equivalent to a frog driving a car?

**Li Shengbo: **Yes. Our scientists divide autonomous driving or smart cars into five levels, **from L1 to L5. The higher the level, the higher the intelligence. **Unmanned driving actually refers to L5 autonomous driving, which is relatively difficult to achieve.

L5 autonomous driving concept map

Level L1 is a driver assistance system that can automatically help the driver steer or step on the accelerator and brake. Level L2 is an enhanced version of Level L1, which can simultaneously steer, step on the accelerator and brake, but its working conditions are generally limited to relatively simple road scenes such as highways, where there are no pedestrians or bicycles, but only motor vehicles. Level L3 and Level L4 are enhanced versions based on Level L2, and they enter road scenes with more traffic participants and higher complexity.

The difference between L3-L4 and L1-L2 is that the responsible parties for safety accidents are different. According to current regulations, if an accident occurs at L1-L2, the driver is responsible because it is driver-assisted autonomous driving; if an accident occurs at L3-L4, or more precisely, at L4, the system should be responsible. This becomes very complicated. If the system is not as safe as a human, then if it is insured, the premium will be higher than that for a human. In addition, it is also more complicated to identify the responsible party for accidents in such a system. Therefore, the self-driving cars that have entered commercial operation are basically still at L1-L2. There is a leap-forward classification between L2 and L3.

**Duan Yulong: **We have achieved L1 and L2 levels of autonomous driving for many years, but why can't we cross the gap to L3? In addition to the responsibility determination you just mentioned, have you encountered any challenges in technology? How should this challenge be solved?

**Li Huiyun: ** Professor Li just mentioned that in terms of algorithmic computing power, autonomous driving is currently very different from human driving. From the perspective of autonomous driving replacing human hands, feet, eyes, and brains, except for the satisfactory performance in the braking of hands and feet, the performance in the major technical aspects from perception to decision-making, planning, and positioning is not satisfactory.

Schematic diagram of vehicle-mounted sensors

Let me give you two examples. In the area of ​​perception, we may have heard of the debate between LiDAR and vision schools, and of course there is talk of multi-sensor fusion. But the accidents that have happened so far are all caused by this stage. However, if we add a lot of sensors and have a lot of test cases, it is not certain whether we can do this well. Because it involves the subsequent planning, decision-making and the development of artificial intelligence, so this is a closely linked issue.

Then in the positioning part, we are working with the Hong Kong Polytechnic University. Hong Kong is a typical urban canyon. Whether it is a tunnel or a high-rise building, the multipath reflection it produces generally causes the GNSS (Global Positioning System) to reach an error of tens of meters, and there is no way to navigate. Of course, using some technical means, such as selecting specific satellites and adding some prior knowledge when calibrating, the best error is still more than ten meters. So now although high-precision maps can reach the centimeter level, if the positioning still has an error of more than ten meters, it is obviously not acceptable. This can also answer why L3-L4 levels have not reached commercial level.

**Duan Yulong:** How should this problem be solved next? To what extent can the recent advances in artificial intelligence, big data, and cloud computing contribute to the improvement of driverless car technology?

**Li Shengbo: ** For autonomous driving to function, it must be like a human. First, it must have eyes to observe the surrounding environment; second, it must use the brain to make judgments and decisions; and finally, it must output commands to the hands and feet to complete tasks such as steering, accelerator and brake. The entire system of an autonomous vehicle is completed by algorithms, but the algorithm must rely on two points. First, it needs computing power or computers to carry it, and second, it needs data to train it. So autonomous driving is similar to artificial intelligence, and algorithms, data and computing power are its three basic pillars . At present, our computing power cannot meet the requirements of high-level algorithms. The reason is simple. Even if a server is installed on the car, the neural network it can carry is only the scale of hundreds of millions of neurons. The human brain has only about eight to ten watts of power, but it can run 80 billion neurons, and the performance of each neuron is much better than that of a machine. So according to this analogy, our computing power is still not enough.

Schematic diagram of a deep neural network (DNN). It works similarly to the human brain, passing input data through layers of interconnected neurons for processing.

Next is the algorithm. The current mainstream algorithm is deep learning. I think deep neural networks may become a more mainstream autonomous driving algorithm in the future. At present, many people question its poor interpretability and security. However, humans are also intelligent entities composed of neural networks. Is the human brain necessarily interpretable? It is not always able to make accurate judgments. Therefore, as long as the security of the algorithm is good enough, we can still trust its effectiveness . As for whether the network itself is interpretable and whether it has sufficient security, it needs to be judged based on the results.

**Li Huiyun: **I also agree that neural networks are developing rapidly. They can shine in many fields such as autonomous driving perception and human-machine co-driving voice recognition. But I think there is still a part that current deep learning lacks: it does not have the human ability of reasoning, induction, deduction, analogy, etc., and these abilities are the big difference between frogs and humans. I think if we want to transition from frogs to humans, we still need to make great breakthroughs in the two fields of mathematics and artificial intelligence .

But I also agree that we should look at the efficacy. Now our general solution to look at the efficacy is to clearly divide the product's design operation domain (ODD). For example, in the case of low speed, the speed and road model are specified, and then there is a selected route, which will be jointly confirmed by passengers, operators and information and communication providers. In such an area, if we can complete the functions of sensing, decision-making, control, execution, obstacle avoidance, emergency stop, etc. according to our solution, we can make the product work well in the area.

Thought Dinner

Completed: 30% /// / / / /// / ********

Why data is the new oil?

**Duan Yulong: **Just now you mentioned that data is one of the three basic pillars of autonomous driving. The collection, use and analysis of data are all very important. What do you think about data?

**Li Shengbo: **Now data has become a basic means of production like oil in the new era, and its importance is self-evident, especially in the field of autonomous driving. We generate a lot of data every day when we drive, but where this data is stored and how to use it is a very core issue.

Human drivers are a means of improving intelligence similar to the reinforcement learning process. The reinforcement learning principle used for autonomous driving refers to the continuous accumulation of virtual or real data, and then using the data to train perception algorithms, calculation algorithms and control algorithms to improve the performance of the algorithms and gradually transition autonomous driving from low-level to high-level.

Professor Li Shengbo

A typical solution was proposed by Chinese scientists. Since the computing power on the vehicle is relatively small, or the scale of the algorithms that can be run on the vehicle is limited, it is possible to move part of the computing power to the roadside or even to the cloud server for calculation. We call it cloud-controlled driving , or intelligent driving with vehicle-road collaboration , which is a brand-new concept. Through this solution, we may be able to solve the problem of insufficient computing power and insufficient accumulation of single-vehicle data.

In the foreseeable future, if most of the vehicle data is accumulated in the cloud "brain", this "brain" can self-learn, self-update, and self-evolve. For example, if there are 2 million connected cars in Beijing, it can simultaneously learn the data collected by the drivers of 2 million cars for self-learning and updating, and then realize its own intelligent evolution, which may accelerate the evolution of autonomous driving algorithms . In short, data is the basic productivity, which can promote the evolution of computing power.

**Duan Yulong: **Recently, the source of data for self-driving cars has become a focus of attention. For example, a company emphasized that it would never use LiDAR, but only visual radar. However, some manufacturers believe that LiDAR is more efficient. What do you think?

**Li Huiyun: **This needs to be viewed in stages and goals. The original intention of a certain company was to do assisted driving, so to achieve commercialization at a low cost, the use of visual solutions is undoubtedly the way to achieve its goal. Now most of the new car-making forces want to do higher-level autonomous driving, so lidar is indispensable in most scenarios. It is irreplaceable by vision in target detection and even positioning and mapping because it has deep information. If you want to reach the L3-L4 level, the goals of the two parties will eventually converge and achieve multi-sensor fusion.

**Duan Yulong: **After all, the number of cars on the road that can be equipped with autonomous driving related systems is limited. Is it sufficient to rely solely on these cars to obtain data to support the upgrade of the entire autonomous driving system?

**Li Huiyun: **So in the future, vehicle-road collaboration will become inevitable. Because single-vehicle intelligence has its limitations: if each vehicle relies on the cumulative number and type of sensors and computing resources, the cost will be very high; there are also some physical limitations, such as the "ghost peeking" phenomenon caused by vehicles in the lane ahead or right-turn road obstruction. Through the method of vehicle-road collaboration, it is obvious that the problems caused by the limitations of single-vehicle intelligence can be solved at a low cost.

Schematic diagram of vehicle-road collaboration

From another perspective, the intelligence of a single vehicle cannot complete data collection and solve the congestion of the entire city very well. Because if speed or driving comfort is taken into consideration, the optimization of a single vehicle will inevitably cause competition among multiple vehicles , which brings about a game. If you want to achieve an overall effect, you must consider urban traffic, and then vehicle-road collaboration is naturally introduced. From this perspective, with the assistance of sensors, computing power and mobilization capabilities beside the road, the overall traffic in the era of autonomous driving will be optimized, including the number of accidents, average travel speed and comfort level.

**Li Shengbo: **Now autonomous driving has two different directions: single-vehicle intelligence and vehicle-road-cloud collaboration. Single-vehicle intelligence is equivalent to everyone pursuing their own intelligence, such as faster, more agile, more energy-efficient, and more efficient. When driving, single-vehicle intelligence may frequently overtake and change lanes, and get from point A to point B as quickly as possible. But this may affect other people, so it has some global limitations. And vehicle-road-cloud collaboration is equivalent to having a God's perspective to tell everyone how to achieve better performance and solve some global problems such as urban congestion and improved traffic efficiency.

Although there is no unified standard for how much data is needed for autonomous driving, some research institutions predict that autonomous driving that can reach human-level performance will require more than 10 billion kilometers of accumulated data. This is equivalent to a driver driving 24 hours a day for 7,000 years . Therefore, it is difficult to achieve this scale by simply accumulating road data.

There are currently two solutions to this problem. The first is to rely on multiple cars to accumulate data, train strategies and share them for joint use by participants. The second idea is that in addition to using real car data, simulation software can also be used to simulate the driving process and accumulate virtual data. Many universities and companies are focusing on developing autonomous driving software and then letting virtual drivers drive. In this way, driving the car for one hour with virtual software is equivalent to driving it for a day, a week or even a month.

Thought Dinner

Completed: 60% ///****// /**********//// **

The state has its own laws, and the car has its own rules

**Duan Yulong: **Since we cannot rely on the accumulation of individual data, we need everyone to collaborate and cooperate with each other, so there needs to be rules. In terms of autonomous driving, how are the domestic and international regulations progressing?

**Li Huiyun: **This is a complicated matter. Traditional fuel vehicles and new energy vehicles are a kind of means of transport. Their testing and standard systems are mainly concerned with power systems, endurance, comfort, fuel economy emissions, energy consumption, noise, etc. **In addition to being a means of transport, today's smart connected vehicles are also a large mobile terminal data space and computing platform. **In this process, we introduced software, including software-defined vehicles, functional safety and OTA upgrades of software, communication and information security, and other types of software and hardware. In this case, the testing, access standards and regulatory systems should be changed accordingly.

Researcher Li Huiyun

At present, efforts are being made at home and abroad. In July, ISO (International Organization for Standardization) released an international standard called ISO 22737 for low-speed autonomous vehicles. The domestic Automobile Standardization Committee is also organizing the formulation of standards. I heard that the terminology definition has been completed. It should be released in the next two years. The content will include safety specifications for functions, components, communications, etc. Because only with the support of standards can all parties make efforts to invest and limit responsibilities and rights, so we are all looking forward to the implementation of the domestic standard system.

**Li Shengbo: **my country also has some plans and layouts in this regard. Our standardization organization hopes to build a universal data platform. On the one hand, it hopes to store the collected data on a public platform, and on the other hand, it hopes to ensure the uniformity of data format during the analysis process. So this standard is very critical.

**Duan Yulong: **Data related to national economy and people's livelihood must be owned by the whole people.

**Li Shengbo: **The concept of public ownership is good, and this is how it should be.

Thought Dinner

Completed: 70% /// /// / ****///********

Can we trust autonomous driving?

**Duan Yulong: **Although there are many difficulties restricting the commercial use of self-driving cars, we all know that the development speed of human technology and the arrival of certain nodes will exceed our imagination. Assuming that self-driving cars come to us in a few years, will they really be as popular as we imagine? In 2018, a self-driving car in testing hit and killed a pedestrian. What do you think of the safety issues that arise in the specific application of self-driving cars?

**Li Huiyun: **I think we can refer to several breakthroughs in our industrial development. For example, for airplanes and subways that can basically achieve autonomous driving, we have a conventional solution, which is to divide a limited operating area that is confirmed by all parties. As long as there is no unauthorized change, once something goes wrong, the responsibilities of all parties can be determined more clearly. This idea may not be correct, but it does need to be jointly formulated by the legal community, the psychological community, and government departments.

**Li Shengbo: **The question you mentioned about who is responsible for an accident is an important problem that autonomous driving faces today. We have discussed the classification of smart cars before. The positioning of L1-L2 is a driver assistance system. This does not mean that it only assists the driver, it can also complete certain autonomous driving functions. However, if anything happens, the driver is still responsible. This is a solution in our project.

Autopilot can automatically assist the vehicle in steering, accelerating and braking within the lane, but the driver is still required to actively monitor the vehicle. The vehicle has not yet achieved fully autonomous driving.

**Duan Yulong: **A certain company’s cars have had many accidents in the past while in automatic assisted driving mode, so according to the knowledge you shared, is the car driver responsible for such accidents?

Li Shengbo : Yes, because it is only a driver assistance system. There are some requirements in its functional design. For example, after the automatic driving system is turned on, your hands cannot leave the steering wheel; in addition, when using this system, your eyes must be looking forward and your brain must be thinking, so that you can take over the vehicle in case of an emergency.

But for L3-L4, the difficulty is relatively high. L3-L4 requires that if the system is not activated when an accident occurs, the responsibility belongs to the person; if the system has been activated, the responsibility belongs to the system. Where exactly is this difficulty? I still calculate it according to economics. Human driving may have three to five accidents per million kilometers, which is the average level of human drivers, and different countries have higher and lower levels. The number of accidents determines the amount of premiums. If the number of accidents is high, the premium will also be high. If you want to reduce the premium, you need to improve safety. Therefore, the core issue of true high-level autonomous driving is that if both humans and autonomous driving systems can be insured, the safety of the system must be higher than that of humans, so that the premium can be low. At present, most scholars, researchers and engineers are still mainly solving the more critical problem of technology. But in the long run, it will become a philosophical, sociological, and legal and regulatory issue.

**Duan Yulong: **Self-driving cars have already had some alarming problems. For example, a company's smart car had a tragic traffic accident. The cause of the accident was that the vehicle mistook the image of blue sky and white clouds sprayed on the body of the truck in front as the real blue sky and white clouds, and drove straight forward, resulting in an accident. The driver died on the spot. This makes people very worried and afraid. Do you think people can overcome this fear?

**Li Huiyun: **I think so. I have great faith in intelligent, computerized, and shared public transportation tools such as airplanes and subways, and I believe I can quickly adapt to the convenience brought to us by intelligent networked cars.

**Li Shengbo: **My personal judgment is that whether autonomous driving can be truly accepted by the public depends on its safety or the number of accidents. If its number of accidents can be reduced to a level similar to that of humans, we should be able to accept this system.

**Duan Yulong: **One day in the future, taxis, subways and even airplanes will be driverless. Then what will happen to the practitioners in the transportation industry, such as drivers? How do you think the problems of these people should be solved?

**Li Huiyun: ****Social development can provide more career space and ways. **Many of the careers we see now may not be imagined by people in the past, such as live streaming. So as society progresses, some careers will disappear, but it is bound to bring more and more personalized careers. I am still very optimistic.

**Li Shengbo: **For the development of every industry, change and transformation cannot be achieved in an instant. It is a long and gradual replacement process, which may take 30 to 40 years or even longer, and may become a problem of two or three generations. In this process, the jobs that were originally replaced will have some other outlets. As Mr. Li just said, some new jobs will also appear, and people will flock to new fields.

**Duan Yulong: **Do you think we can see fully autonomous driving in our lifetime?

**Li Shengbo: **I am an optimist and I think I should be lucky enough to see it in my lifetime.

Thought Dinner

Completed: 90% /// /// // /****/

What other issues are hindering the advancement of autonomous driving?

**Duan Yulong: **The last question is, in your opinion, what other issues will restrict the large-scale commercial use of self-driving cars in the short term?

**Li Huiyun: **Let me give you an example: Everyone thinks that vehicle-road collaboration is a very good development direction, which can make up for the various deficiencies of the intelligence of autonomous driving vehicles. However, the cost problem brought about by vehicle-road collaboration has been put in front of us . When a friend of mine was working on a smart road project, the shareholders asked a very practical question: How to recover the cost of the extra part when the project cost has increased from 100 million yuan per kilometer to 120 million or even 150 million yuan now? Is it borne by each passing vehicle or by the passing autonomous driving vehicles? For the passing autonomous driving vehicles, how to charge them? This question stumped everyone. So in terms of business model, cost and government organizational model, we still need to work together to solve this problem.

**Li Shengbo: **In fact, there are some cost comparisons between different technical solutions. For example, for the single-vehicle intelligence and the collaborative intelligence of the vehicle, road and cloud, which one has lower cost and which one has better autonomous driving effect, these issues all have economic considerations. In addition to the economy, there are also some peripheral issues that pose challenges. Because after the car becomes intelligent, we will introduce network technology, which will increase the interface between the car and the outside world. The previous car was a closed system, and it was not easy for the outside world to command it or input a little extra information. But with the Internet of Vehicles and various external sensors, more information can be transmitted. From this perspective, if self-driving cars are really going to be on the road, there are still some peripheral issues to deal with, which will restrict whether they can be truly popularized and put on the road.

**Duan Yulong: **Today, we have outlined the beautiful vision of driverless cars in the future. When this vision comes, I believe that the lifestyle of the whole society and the living conditions of every individual will be greatly and revolutionary improved. We also believe that this day will come to us sooner or later.

There is no truth without reason, and this rational party will come to an end here. Thanks again to the two teachers for coming, and thank you to all the audience friends for watching!

Original title: "Is autonomous driving equivalent to a frog driving a car? How much effort is needed to make a frog evolve into a human | Rational Party Season 3"

Source: Gezhi Lundao Forum