From Olympic live broadcasts to blockbuster movies in theaters, can naked-eye 3D become the mainstream of visual interaction in the future?

Vision is one of the main ways for humans to obtain information, accounting for about 80% of all information obtained by human senses. Display technology is a key technology for terminals to provide users with visual experience, an important way to present communication transmission content, and has a great impact on the network's carrying capacity and business characteristics. Naked-eye 3D technology is a display technology that provides users with a three-dimensional visual experience without the help of auxiliary equipment. It is an important technical route for the evolution of terminals from two-dimensional to three-dimensional. Many naked-eye 3D mobile phones and Pads have been produced and launched. Compared with traditional 2D display, naked-eye 3D display not only enhances the user's sense of immersion and impact, but also has no problems such as strong weight and limited battery life of head-mounted devices. Users can easily obtain naked-eye three-dimensional visual experience. In addition, due to its rich depth information and strong immersive experience, naked-eye 3D technology can be combined with other technologies to develop a variety of scenarios and applications such as 3D viewing, live broadcasting, calls, and conferences.

1. What is naked-eye 3D technology?

There is no completely unified standard definition of Glasses-Free 3D in the industry. Generally speaking, the broad definition of Glasses-Free 3D covers all technologies that achieve three-dimensional visual effects without the help of external tools, such as the Chengdu Chunxi Road Glasses-Free 3D large screen that uses a folded screen with content special effects to create a 3D experience, and the 3D image of Jay Chou singing Teresa Teng in 2013 formed by using a semi-transparent and semi-reflective film projection combined with stage design; the narrow definition of Glasses-Free 3D specifically refers to the technology that achieves a naked-eye three-dimensional effect by projecting different images to both eyes based on the principle of binocular parallax, such as ZTE Nubia series Pads and mobile phones. This article uses the narrow definition of Glasses-Free 3D as a definition for the sake of technological development and industry consensus. The binocular parallax principle in this definition uses the mechanism that the brain synthesizes a three-dimensional experience when the human left and right eyes input corresponding parallax images to achieve a three-dimensional effect. The so-called parallax image can be understood as the two left and right two-dimensional images of the target object with slight differences when the two human eyes (the average distance between the pupils of an adult's eyes is 65mm) observe the same object from different directions.

Through the coordination of content and display software and hardware technology, naked-eye 3D technology can achieve impactful and immersive visual effects, with strong three-dimensional sense and rich spatial information. The integration of naked-eye 3D technology with intelligent media networks, digital humans, artificial intelligence generated content (AIGC) and other technologies can bring immersive and virtual-real integration experience, break the dimensional wall inside and outside the screen, enable a series of scenarios such as education, games, entertainment, and calls, and shape a future-oriented immersive business system to promote the evolution of the industry to the 3D era. For example, in the medical field, remote surgery training can be achieved, and doctors can interact with experts in an immersive environment to improve their medical professional skills; in education, students can fully experience historical events through naked-eye 3D, making learning more vivid and interesting. In addition, naked-eye 3D will also promote innovation in the entertainment industry. Naked-eye 3D games and sports events will become more fascinating, attracting audiences to participate more deeply in the content, and creating a more immersive entertainment experience. The widespread application of this technology will enable people to connect more closely in social networks, and share richer and more interesting experiences no matter where they are.

Glasses-free 3D technology can not only bring an immersive stereoscopic visual experience beyond two dimensions, but also has the potential for application in many scenarios due to its special technical principles and data formats. It is expected to bring new experiences to users and promote the upgrade of business from two dimensions to three dimensions. Glasses-free 3D technology has the following characteristics:

① Outstanding 3D experience: Naked-eye 3D can project different images to the left and right eyes of the target user to simulate human vision and produce 3D images. Compared with 2D projected with a single screen, it can bring a strong immersive experience. In addition, rich spatial information provides natural convenience for the access and display of virtual content such as 3D models.

② Low cost of use: Low cost includes low wearing burden and low hardware implementation cost. First of all, naked-eye 3D users do not need to wear other auxiliary equipment, and there is no discomfort caused by the weight of batteries and display computing devices of VR and other head-mounted devices. Moreover, compared with head-mounted devices, it is not easy to produce motion sickness caused by visual convergence adjustment because its screen is far away from the eyeball. Second, compared with the current 2D display, the hardware spectrophotometry of naked-eye 3D display can support the projection of light by adding a layer of grating film. The cost of the film is controllable and the marginal effect is obvious. In addition, the upgrade method of external cylindrical lens film similar to mobile phone film is gradually mature, and the naked-eye 3D upgrade of existing 2D terminals can be supported without changing the equipment. Third, naked-eye 3D supports compatibility with 2D display. By adjusting the content and encoding method, hot switching of 2D/3D display can be completed.

③ Software and hardware system collaboration: The realization of naked-eye 3D requires the system collaboration of hardware spectrometry, software encoding and rendering, and 3D content. As a commonly used 3D content format in the industry, the SBS (Side-By-Side) format is characterized by merging the left and right viewpoints into a single-channel transmission, such as horizontally splicing the left and right viewpoint images into an SBS format image. The 3D content is first encoded and rendered by the software to form raster data, and then the display hardware is used to split and project the target to produce a 3D visual effect.

④ High traffic bandwidth: The realization of naked-eye 3D effects requires the cooperation of 3D content. This type of content contains both left and right eye views, so the required resolution is higher than that of 2D effects of the same level. Therefore, for the same material, the transmission bandwidth and traffic requirements of naked-eye 3D content are about 1.5 to 3 times that of 2D content, which has obvious driving value for operators' business revenue.

⑤ Endogenous AI: The 3D content required for naked-eye 3D contains image information from the left and right eye perspectives. Direct acquisition requires dual-channel equipment to work and the equipment spacing has certain requirements, which is difficult to achieve for existing highly integrated and miniaturized equipment. Therefore, it is necessary to use AI technology to convert the collected single-view 2D information into dual-view 3D information. In addition, the demand for virtual 3D content is huge. If it is produced by manual design and modeling, the cost is very expensive and it is difficult to meet business development. It is necessary to introduce AI automated intelligent generation technology to quickly form massive, high-quality 3D content.

2. Principle of naked eye 3D technology

After the naked-eye 3D technology renders the SBS (Side-by-Side) content, it projects different views to the left and right eyes of the target user to form a 3D effect. The core of the naked-eye 3D technology is the 3D encoding algorithm and the grating hardware, which are responsible for the rasterization encoding and subsequent splitting and projection of the SBS content respectively. The two work together to form a three-dimensional effect. The overall process is: after the naked-eye 3D system obtains the SBS content, the 3D encoding algorithm first obtains the binocular position of the target user through the identification and positioning algorithm, and encodes the SBS content to form rasterized data based on the position information, grating hardware parameters, etc. After that, the grating hardware splits the coded data according to its distribution, controls the light to be projected to different positions, so that the corresponding image enters the left and right eyes of the target user to form a 3D visual experience (Figure 1).

Figure 1 Schematic diagram of the naked eye 3D technology principle

According to different grating hardware implementation methods, there are currently three specific technical implementation routes in the naked-eye 3D industry: light barriers, cylindrical lenses, and directional light sources:

① Light barrier technology: Its principle is to project information from different viewpoints to the left and right eyes respectively by blocking light (Figure 2). Its advantages are simple, easy to implement and low cost, and its disadvantages are that the grating is opaque and blocks light, resulting in a significant decrease in overall brightness. Currently, this technology is generally used in the industry with liquid crystal electronic gratings, which can achieve lossless switching between 2D and 3D by controlling the liquid crystal to pass/block light. The industry currently uses it less due to its low brightness.

Figure 2 Schematic diagram of the principle of light barrier technology

② Columnar lens technology: The principle is to use a micro-nano columnar lens structure to refract light to achieve the splitting effect for the left and right eyes (Figure 3). Its advantages are low brightness loss and low cost (physical columnar crystal grating), and its disadvantage is that 2D display will have a certain sawtooth effect (physical columnar crystal grating). At present, there are two ways to realize columnar lenses in the industry: physical columnar crystal grating and liquid crystal lens. Among them, the liquid crystal lens uses liquid crystal to form a columnar lens with adjustable curvature, which not only retains the advantages of the columnar crystal grating but also can switch 2D/3D effects without loss, but the disadvantage is that the cost is relatively high. Overall, the columnar lens route is more widely used in the industry.

Figure 3 Schematic diagram of cylindrical lens technology principle

③ Directional light source technology: Its principle is to add a layer of directional backlight, cooperate with fast-response LCD panel and driving method, so that 3D content can enter the viewer's left and right eyes in a sequence to exchange images and produce parallax (Figure 4). Its advantages are high brightness and lossless switching of 2D/3D effects. Its disadvantages are that the additional layer of backlight makes the device thick and costly. In the current industry, some companies use directional backlight technology to design and manufacture naked-eye 3D devices.

Figure 4 Schematic diagram of the principle of directional light source technology

3. Application scenarios of naked-eye 3D technology

① Naked-eye 3D live broadcast: Based on the real-time transcoding capability of the 2D to 3D platform, 2D images are collected and encoded and transmitted through an ordinary monocular camera. After the platform transcodes, 3D content in SBS format is formed and sent to a terminal with naked-eye 3D display capability to complete positioning, encoding and spectral projection, and then present a 3D effect to the user. Naked-eye 3D live broadcast does not require upgrading the acquisition terminal equipment, and can provide users with a real-time, immersive naked-eye 3D live broadcast experience. Migu Company used naked-eye 3D live broadcast for the first time in the live broadcast of the European Cup in June 24, and further optimized and upgraded it in the live broadcast of the Paris Olympics in July.

Figure 5 Naked-eye 3D Olympic Games live broadcast

② Naked-eye 3D viewing: The SBS videos produced or converted are stored in the cloud or terminal. When the user issues a video-on-demand command, they are transmitted, decoded, and rendered and displayed on the terminal to provide users with a 3D experience. Naked-eye 3D viewing provides users with a new video viewing experience. Its strong immersion and impact are highly compatible with various types of videos such as records, action, and science fiction. Migu Video's Aikan version has launched a naked-eye 3D video area, where users can watch pre-made 3D video sources through naked-eye 3D dedicated equipment.

Figure 6 Naked-eye 3D video

③ Glasses-free 3D communication: Glasses-free 3D communication includes video playback services such as ringback tone and screen lighting, and real-time 3D call services. 3D video ringback tone and screen lighting can bring a business experience with visual impact, effectively highlighting the product characteristics of ringback tone and screen lighting in advertising and personalized display. Glasses-free 3D calls are based on VoLTE/VoNR video calls, integrating high-performance 2D to 3D conversion technologies to bring users an immersive three-dimensional call experience, providing space for expanding new scenarios for call services.

Figure 7 Naked-eye 3D ringback tone and call

4. Conclusion

Naked-eye 3D uses the system coordination of software, hardware and content to complete the accurate delivery and response control of left and right eye images, and uses binocular parallax to form three-dimensional vision. Based on naked-eye 3D technology, users can get 3D effects without wearing additional equipment, bringing a novel impact and immersive experience. Since the currently commonly used grating hardware can only split and project a pair of left and right eye images at the same time within a certain angle range, and the accuracy is insufficient, naked-eye 3D still faces problems such as only supporting single-person use, limited viewing angle, and virtual images. However, with the continuous development of naked-eye 3D related technologies, these problems will be gradually solved and optimized. It can be foreseen that naked-eye 3D will be gradually promoted and applied in many scenarios such as live broadcast, movie watching, calls, games, and meetings. With the large-scale application of naked-eye 3D, its high-bandwidth traffic, endogenous AI and other characteristics will greatly drive the growth of network traffic and computing resource demand, becoming a new driving force for the evolution and development of the network.

Author: Yin Xinghua, Yang Benzhi, Liu Kun, Ji Hongfei

Unit: China Mobile Research Institute