Why your phone needs a more powerful AI processor

Three years ago, AI became a new selling point for smartphones. Smartphones that support AI features quickly became popular, and AI performance became the most concerned parameter of mobile phone processors by consumers after CPU and GPU. Today, demonstrating AI performance has become an indispensable part of mobile phone and mobile phone processor launch conferences.

But have you ever thought about why you need a mobile phone with powerful AI functions? What can the AI ​​processor in the mobile phone do?

[[313329]]

What AI features does your phone have?

When it comes to AI functions on mobile phones, many people immediately think of intelligent voice assistants and AI photography. Undoubtedly, AI photography is the most commonly used function by ordinary consumers, including AI background blur, AI super night scene, AI beauty, AI noise reduction, etc. In addition, some common AI functions include face recognition, call noise reduction, translation, etc.

At present, these common mobile phone AI functions are becoming more powerful. For example, in the past, automatic speech recognition (ASR) and offline real-time voice translation functions were not available on mobile phones. With the advancement of AI technology and the deepening of cooperation between hardware and software companies, ASR and offline real-time voice translation can be realized on mobile phones, and can even respond in noisy environments.

In addition, in terms of photography, the number of mobile phone cameras has increased to three or even five. When switching between wide-angle, ultra-wide-angle, and telephoto lenses, users can easily feel lag. The optical zoom smooth switching solution jointly completed by Qualcomm and its partners can bring users an AI-based smooth zoom experience.

AI-based smooth switching of optical zoom

Compared with smart voice assistants and camera functions, some AI functions on mobile phones are not easy to find. For example, Douyin, a globally popular short video platform, actually receives 320P resolution videos on the user's mobile phone due to network bandwidth and other considerations. However, higher resolution means better experience. For this reason, ByteDance has cooperated deeply with Qualcomm to optimize based on Snapdragon 865, using the mobile phone's AI processor to run a special neural network (super-resolution convolutional neural network) to increase the resolution of the video received by the user from 320P per frame to 720P. Because this processing is achieved through terminal-side AI, users can also get a smoother experience.

Not only that, AI can also be used to solve the power problem that makes many users anxious. The iPhone uses AI capabilities to learn the user's usage habits, thereby optimizing charging to extend the battery life. Qualcomm also introduced AI algorithms into the entire system of fast charging and discharging in its Quick Charge fast charging technology. By learning the user's personalized usage habits, such as playing games and the duration of games, and then through training, Quick Charge AI can well adjust the CPU of the entire system, including charging efficiency.

In this way, Quick Charge AI can further improve the prediction accuracy of the remaining battery life of the mobile phone by up to 15%. In addition, it can also achieve better charging and heat dissipation management, extending the battery cycle life by up to 200 days.

Of course, AI also helps make mobile games, shopping, learning and other functions more interesting and practical. For details, please see previous articles on Leiphone.com (public account: Leiphone.com).

In fact, the powerful AI function of mobile phones requires a powerful mobile phone processor and AI performance to support it. Taking advantage of the recently released Snapdragon 865 mobile platform, let's analyze how hardware and software work together to achieve powerful AI functions.

Higher-performance AI processors, more powerful AI functions

At the beginning of last month, Qualcomm released its latest flagship mobile platform, Snapdragon 865. The new platform integrates 5G and AI well, can provide a peak rate of up to 7.5 Gbps, and is equipped with the fifth-generation Qualcomm AI Engine and Sensor Hub. In addition, the Spectra 480 ISP can achieve a processing speed of up to 2 billion pixels per second, and Snapdragon Elite Gaming can also support a series of new features such as end-game-level experience and extremely realistic graphics performance, which can bring users excellent shooting and gaming experience.

As the focus of the Snapdragon 865 upgrade, the performance of the new generation Kryo 585 CPU has been improved by 25%, and the overall performance of the new Adreno 650 GPU has been improved by 25% compared with the previous generation platform. The AI ​​Engine, which is most closely related to AI functions, is the fifth-generation AI Engine integrated in the Snapdragon 865, with a performance of up to 15 trillion operations per second (15 TOPS), which is twice the AI ​​performance of the previous generation Snapdragon 855 and 5 times higher than the AI ​​performance of the Snapdragon 845.

The significant improvement in Snapdragon 865 AI performance does not rely on the improvement of a single processor. The AI ​​Engine has always adopted a multi-core heterogeneous computing solution, completing AI tasks through the collaborative processing of the Kryo CPU, Adreno GPU, and Hexagon processor. The Hexagon processor at the core of the fifth-generation AI Engine has been completely upgraded, with a TOPS performance that is 4 times that of the previous generation tensor accelerator, and a 35% increase in operating energy efficiency. The AI ​​computing power of the Adreno 650 GPU of the Snapdragon 865 has also increased by more than twice compared to the GPU AI computing power of the previous generation Snapdragon 855, making it the most powerful mobile AI processor.

Of course, AI Engine's multi-core heterogeneous architecture can also achieve the best balance between performance and power consumption. For battery-powered smartphones, it is crucial to have both high performance and low power consumption. Similar to the idea of ​​improving AI Engine performance, Qualcomm's power consumption reduction is not limited to a certain hardware, but is optimized from a system perspective.

Deep learning algorithms are the most commonly used algorithms in AI. These algorithms bring a lot of convolution operations, but the most energy-consuming is not the convolution calculation, but the data movement. To this end, Qualcomm upgraded the bandwidth compression technology used on the Snapdragon 820 to the deep learning bandwidth compression technology on the Snapdragon 865 AI Engine, which can achieve lossless compression with a compression ratio of up to 50%. In this way, the access to memory is greatly reduced. In addition, the newly supported LPDDR5 of the Snapdragon 865 can bring a 30% bandwidth increase, which together can reduce power consumption.

It is also worth mentioning that in order to further improve the intelligence of the mobile phone processor, the Snapdragon 865 also integrates a sensor hub (Sensing Hub), which can be used for real-time perception of audio and video. In terms of power consumption control, the multi-keyword language voice wake-up of this sensor hub consumes less than 1 mA, and with the AI ​​Engine, the phone can perceive the surrounding environment at extremely low power consumption.

For example, for the ASR mentioned earlier, Qualcomm and Google have worked together to enhance the Android Neural Networks API and migrated the Google Assistant's speech recognition function from the CPU to Hexagon, reducing power consumption by 3 times and latency by 30%, enabling always-on voice translation throughout the day, and further elevating contextual awareness AI to a new level.

However, even the most powerful hardware needs software to unleash its magic.

More interesting AI features are coming soon

TikTok, Snapchat, Google Translate, Youdao Translation and other apps have exported AI functions to mobile phones, which is also the result of close cooperation between mobile AI software and hardware developers. However, there is a huge gap in the cooperation between the two sides. On the one hand, software developers do not understand the underlying hardware, and calling and optimizing hardware is a huge problem for them; on the other hand, hardware providers often do not understand the diverse needs of software developers well enough, and it is difficult to meet the needs of all developers.

Therefore, unlike traditional chips, AI chip providers need to provide a complete solution covering hardware, software, and tools to ensure that everything works together. From the top to the bottom, they are application, framework, runtime, library, and hardware accelerator.

In terms of frameworks, AI developers may use frameworks such as TensorFlow and PyTorch, and AI chip companies will try their best to support mainstream AI frameworks. In order to simplify the work of AI developers, Snapdragon 865 currently supports more than 160 operators, which refer to complex functions in the framework that can help tensors work better. It is also worth mentioning that developers can also use Open CL and Hexagon SDK supported by Adreno to create custom operators, so that developers can differentiate themselves on the framework and make full use of the powerful AI computing power provided by the Snapdragon platform.

At the runtime layer, in order to support AI performance on mobile terminals, Qualcomm is the first in the world to launch an AI software toolkit for mobile platforms. The collaboration between Qualcomm Neural Processing SDK and Android Neural Networks API can provide non-parallel developer access to first-party and third-party applications on Snapdragon mobile platforms, better combining AI hardware and software.

It is reported that Qualcomm's neural processing SDK focuses on improving energy consumption, performance, access and other aspects. It also provides monthly version updates and works closely with partners, especially OEMs, to provide top AI solutions that support more networks and higher performance. Qualcomm's AI R&D team also works closely with Google to jointly optimize the Android Neural Networks API, achieving a 3-5 times performance improvement and making access easier.

It should be added that Qualcomm has also launched Hexagon NN Direct. For example, through cooperation with Google, TensorFlow Lite developers can directly access the terminal without running the Runtime, and the library running directly on the Hexagon kernel can also provide the same access rights to other solution providers.

This feature can bring significant improvements. By leveraging Hexagon NN Direct, Snapchat can increase video from 10 frames per second to over 40 frames per second.

In addition, Qualcomm has also launched a new AI model efficiency toolkit, which can achieve 3x model compression with less than 1% accuracy loss by removing redundant layers in the network, which is more than enough for most use cases. In addition, this toolkit can also compress 32-bit AI models to 8 bits, increasing performance per watt by more than 4 times.

Based on high-performance hardware and easy-to-use software, Snapdragon's AI technology has enabled more than 1 billion terminals worldwide. With the further improvement of hardware performance and the iteration and innovation of software, there will be more and more AI applications in more fields such as XR, work, and social interaction.

Leifeng.com Summary

Mobile phones are the first smart devices to popularize AI technology, but two years ago, the AI ​​functions available to consumers were very limited. With the continuous improvement of mobile phone AI performance and the continuous iteration of AI algorithms and software, AI photography has become the most popular AI function among consumers in 2019. In addition, thanks to the powerful AI performance of mobile phones, AI tasks that could only be achieved in the cloud before can also achieve good results on terminals. While mobile phone AI functions are more practical, they have also spawned many more interesting and novel AI functions.

Qualcomm, the leader in this market, has achieved excellent AI performance in its latest flagship mobile platform Snapdragon 865, thanks to its long-term accumulation. As early as 2007, Qualcomm Research launched its first artificial intelligence project, and has since continuously strengthened its research and development in the field of AI, and established Qualcomm AI Research in 2018.

Of course, the success of chips, especially AI chips, is more important to the better integration of software and hardware and the prosperity of the ecosystem. With its strong technical strength and appeal, Qualcomm already has many AI partners around the world, and its performance in the field of AI is therefore worth looking forward to. Especially for Qualcomm, which is also a leader in the 5G field, the integration of 5G and AI will collide with unexpected surprises.

This article is reproduced from Leiphone.com. If you need to reprint it, please go to Leiphone.com official website to apply for authorization.