"Tianmo chip" boosts the "acceleration" of intelligence! Another major breakthrough in Tsinghua chip!

Recently, the brain-like computing research team of the Department of Precision Instruments of Tsinghua University developed the world's first brain-like complementary vision chip "Tianmo Core" . The achievement was published on the cover of Nature magazine on May 30. This is the second time that the team has appeared on the cover of Nature magazine after the heterogeneous fusion brain-like computing "Tianji Core", which marks that China has made fundamental breakthroughs in two important directions: brain-like computing and brain-like perception.

The cover of Nature magazine on May 30, 2024. Image source: Nature magazine website

The research team focused on brain-like visual perception chip technology and proposed a new paradigm of complementary dual-pathway brain-like visual perception based on visual primitives. Drawing on the basic principles of the human visual system, the open-world visual information is decomposed into information representation based on visual primitives. By organically combining these primitives, the characteristics of the human visual system are imitated to form two visual perception pathways with complementary advantages and complete information.

On this basis, the team developed the world's first brain-like complementary vision chip "Tianmouxin", which achieves high-speed 10,000 frames per second, high-precision 10-bit, and high dynamic range of 130dB visual information acquisition at extremely low bandwidth (reduced by 90%) and power consumption. It not only breaks through the performance bottleneck of the traditional visual perception paradigm, but also can efficiently cope with various extreme scenarios, ensuring the stability and security of the system.

With the rapid development of artificial intelligence, unmanned systems such as autonomous driving and embodied intelligence are being promoted and applied in real society. Visual perception, as the core way to obtain information, plays a vital role. However, in a complex, changing and unpredictable environment, achieving efficient, accurate and robust visual perception (i.e. the ability of the system to survive in abnormal and dangerous situations) is still a daunting challenge.

In the open world, intelligent systems not only have to process huge amounts of data, but also need to deal with various extreme events, such as sudden dangers during driving, dramatic light changes at tunnel entrances, and strong flash interference at night. Traditional visual perception chips are limited by the "power consumption wall" and "bandwidth wall" and often face distortion, failure or high latency when dealing with these scenarios, which seriously affects the stability and security of the system.

The "Sky Eye Core" achieves visual information acquisition at a high speed of 10,000 frames per second, a high precision of 10 bits, and a high dynamic range of 130 dB at extremely low bandwidth (reduced by 90%) and power consumption. It not only breaks through the performance bottleneck of the traditional visual perception paradigm, but also can efficiently cope with various extreme scenarios and ensure the stability and security of the system.

Based on the "Tianmo chip", the team also independently developed high-performance software and algorithms, and verified their performance on an open environment vehicle platform. In a variety of extreme scenarios, the system achieved low-latency, high-performance real-time perception reasoning, demonstrating its huge application potential in the field of intelligent unmanned systems.

The successful development of "Tianmo Core" is a major breakthrough in the field of intelligent perception chips. It not only provides a strong technical support for the development of the intelligent revolution, but also opens up new paths for important applications such as autonomous driving and embodied intelligence. Combined with the team's accumulated technology in the application of brain-like computing chips "Tianji Core", brain-like software tool chains and brain-like robots, the addition of "Tianmo Core" will further improve the brain-like intelligent ecosystem and strongly promote the development of artificial general intelligence.

This research was supported by the Ministry of Science and Technology's Science and Technology Innovation 2030 "Brain Science and Brain-like Research" major project and the National Natural Science Foundation of China, as well as by Tsinghua University/IDG-McGovern Institute for Brain Research.

Comprehensive sources: Science and Technology Daily, China Youth Daily, etc.