Nature News: AI "disrupts" exoskeletons to help build a strong body

Contributing author: Su Hao (Professor at North Carolina State University and University of North Carolina at Chapel Hill)

Although exoskeleton robots can help people walk, they are usually limited to laboratory environments and require half an hour to an hour to adjust the robot's software algorithm to suit each user.

Recently, Professor Su Hao's team from North Carolina State University and the University of North Carolina at Chapel Hill published a groundbreaking research paper titled "Experiment-free Exoskeleton Assistance via Learning in Simulation" in the top international academic journal Nature.

The paper demonstrates a new method for robots to learn control strategies through reinforcement learning in a computer simulation environment. Through this "learning-in-simulation", the study shows that this method can make robots intelligent, especially the generalization ability that can adapt to various people. Not only can they autonomously adapt to various actions such as walking, running, and climbing stairs, but they can also help save a lot of human energy during walking.

This research marks a major breakthrough in exoskeleton technology, making robots more intelligent and practical, and is expected to greatly improve the quality of life of the elderly, people with mobility impairments and people with disabilities.

Innovative artificial intelligence "learning in simulation" framework to achieve exoskeleton intelligence and versatility

An exoskeleton is a wearable robot that assists human movement by providing external power, enhancing strength and stability. It can improve human movement, restore mobility for people with disabilities, and significantly improve people's health and quality of life.

"Our philosophy is that science and technology should be people-oriented, serve people, and solve real-world problems," said Su Hao, the corresponding author of the paper and a professor at North Carolina State University and the University of North Carolina at Chapel Hill. "Existing exoskeleton control algorithms usually require several hours of human experiments and parameter adjustment, which is a time-consuming and labor-intensive process that hinders the widespread use of exoskeletons. This innovative artificial intelligence framework breaks the gap between simulation and reality. Through pure computer simulation, or the digital twin approach, this model- and data-driven reinforcement learning algorithm enables the exoskeleton to provide effective assistance for a variety of actions such as walking, running, and climbing stairs, reducing human energy consumption, which is equivalent to losing 11.9 kilograms of body weight."

To explore the feasibility of this technology, the first author of the paper, Dr. Luo Shuzhen (formerly Professor Su Hao's postdoctoral fellow and now an assistant professor at Embry-Riddle University), conducted a four-year research. She said: "We first created a high-fidelity musculoskeletal model and designed a closed-loop simulation training method for three deep neural networks. This method integrates the human model (including the motion imitation network and the muscle coordination network) and the exoskeleton controller (the control strategy of the neural network) to accurately simulate the human-computer interaction process by exchanging state information. Through this "learning-in-simulation" method, the trained controller can generate assistance adapted to different motion patterns in real time without any human experiments or debugging."

Professor Su Hao's team conducted experiments on the controller learned through simulation on three activities (walking, running and climbing stairs), each of which involved 8 healthy subjects. The experimental results show that the power curve generated by the controller can be adjusted autonomously according to different types of activities without any human intervention. For example, as the speed of travel gradually increases, the size of the power generated by the controller will also increase, and the shape of the curve will also change to adapt to the power requirements of different activities. The key to achieving this ability is that the controller relies entirely on the thigh kinematic information provided by the inertial measurement unit sensor worn on the subject's thigh as input, and the controller has mastered the ability to directly output the appropriate power through the input kinematic signal through millions of rounds of simulation training. "Our controller can generate appropriate power end-to-end. Most existing methods require many additional intermediate steps, which will bring about a tedious process of manually adjusting parameters and will also affect the generalization ability of the controller," said Professor Su Hao. At the same time, due to the different ways of action of different subjects, even for activities at the same speed, the controller will generate slightly different power curves for different subjects. "A highlight of our work is that the power generated by the controller is different for different people. It will adjust autonomously instead of giving a fixed shape of power."

Further experimental results showed that the use of the controller reduced the average metabolic rate of the subjects by 24.3% when walking, 13.1% when running, and 15.4% when climbing stairs. These reductions exceeded the energy reductions achieved by any portable lower limb exoskeleton in previous studies. The above results show that the exoskeleton controller obtained through simulation learning does provide great help for different human activities.

Figure ｜ Experiment-free optimization of exoskeleton assistance through simulation learning

No need for human experiments, exoskeleton development enters the fast lane

The core of this research is to use simulation learning to bridge the gap between computer simulation and real-world applications. Through millions of rounds of simulation training, the research team enabled the controller to generate effective assistance in different activities. This method not only improves development efficiency, but also reduces dependence on expensive and time-consuming human experiments, providing a feasible path for the rapid development and widespread application of exoskeletons.

Figure｜Simulation Learning Framework

Support continuous multi-action to achieve efficient assistance

Dr. Zhang Sainan (former PhD student of Professor Su, now his postdoctoral fellow and one of the authors) said: "This is a study of embodied intelligence. Our algorithm is based on an exoskeleton we developed ourselves, which is the lightest powered wearable robot. A highlight of this study is that the exoskeleton can provide continuous support for multiple actions. In the experiment, subjects wearing the exoskeleton can seamlessly connect multiple actions. For example, the user gradually accelerates from slow walking to running, and then quickly switches to climbing stairs. The exoskeleton can provide stable and effective assistance throughout the process. Through the control strategy obtained by reinforcement learning, the exoskeleton can adjust the size and timing of the assistance in real time to ensure that each action receives the appropriate assistance." This ability to provide assistance for continuous actions significantly improves the practicality and user experience of the exoskeleton, and demonstrates the potential of exoskeleton technology in diverse sports.

Figure | Power curves during various activities and sports transitions

Broad application prospects

This research result is an important milestone in the development of exoskeleton technology. Exoskeleton devices can not only significantly improve the athletic performance of ordinary people, but also help disabled people regain their mobility. Dr. Luo Shuzhen, Professor Su Hao and Professor Zhou Xianlian of New Jersey Institute of Technology collaborated with Professor Yue Guanghui of Kessler Foundation, one of the largest rehabilitation research centers in the United States, to study the use of simulation learning to control rehabilitation robots. Professor Zhou Xianlian said: "I think simulation learning has very good application prospects in the control of rehabilitation robots. Patients who need motor rehabilitation have different conditions and therefore have different assistance needs. Our simulation learning technology has good adaptability and it is also possible to further realize customized auxiliary control." The research team believes that by further optimizing and promoting this simulation learning framework, wearable robots will play a more extensive role in medical, industrial and daily life in the future.

The authors of the study include Professors Hao Su, Shuzhen Luo, Menghan Jiang, Sainan Zhang, Junxi Zhu, Shuangyue Yu, Israel Dominguez, Tian Wang, Xianlian Zhou, Professor Elliott Rouse of the University of Michigan, Professor Bolei Zhou of the University of California, Los Angeles, and Dr. Hyunwoo Yuk of the Korea Advanced Institute of Science and Technology.

Related paper information:

https://doi.org/10.1038/s41586-024-07382-4