With a valuation of $3.7 billion, it may dominate the future development of VR/AR hardware!

There is no doubt that virtual reality/augmented reality (VR/AR) is the hottest initial field nowadays.

The concentration of capital is a perfect example, and we have previously counted investments. Surprisingly, the company that topped the list in terms of total investment is a company with almost no presence and actual products - Magic Leap.

Just at the beginning of last month (already last year), Magic Leap completed a new round of financing with a total of US$827 million, bringing the cumulative financing amount to US$1.4 billion, and the total valuation also rose to US$3.7 billion.

Magic Leap has only released a video demo so far (which we have reported before), so what is its potential value? Recently, a question posted by Botao Amber Hu on Zhihu in China seems to reveal the secret.

Please look carefully, this Demo has a hidden secret!

Let's get back to the topic. What is Magic Leap's unique skill? In one sentence, it integrates light field display into AR devices.

Take the most common VR equipment manufacturers such as Oculus as an example. The basic principle of the equipment is actually to display a 2D image to each eye, and then the brain analyzes the 2D image to form a sense of space. This leads to a very simple conclusion: there is no 3D effect when using only one eye.

[[161353]]

Is this really the case? If you cover one of your eyes, will the world you see become a blank sheet of paper? And from a medical point of view, there is a primary eye in both eyes (your vision is mainly provided by one eye), and the other eye just provides more depth of field information. From this perspective, although binocular 2D can achieve partial immersion, it cannot reach the level of reality after all.

Next is the light field technology: the propagation of light in free space can be uniquely represented by a four-dimensional light field. Each pixel on the imaging plane contains information about all directions of light to this pixel. For the imaging plane, the direction is two-dimensional, so the light field is four-dimensional. When we usually image, we discard two of the dimensions and it becomes two-dimensional image information.

Having said so much, what exactly is the advantage of doing this? That's right, the human eye can directly choose to focus (active selective focus)! Because the light field it projects restores all the information, the angle and position of the light of objects near and far in the picture will directly give people a sense of space. This is the direct reason why when the human eye looks at nearby places, distant places will be blurred.

[[161354]]

This effect has been demonstrated in the official demo video. In the video of the virtual solar system, there is a process of losing focus and then focusing again, but this has not been processed later. This method of achieving the closest to reality is the most amazing thing about Magic Leap, and they call it Cinematic Reality.

In addition to better effects, this principle can also fundamentally eliminate the dizziness of virtual reality. This is also because of the basic imaging principle of the human eye mentioned above. The imaging changes during the focusing process of the human eye are also an important experience of spatial sense. You may say that it can also be achieved with eye tracking technology, but whether it is possible is the ultimate key.

Solution: Fiber optic projection, plus micro-lens array

The next question is feasibility. Eye tracking feedback technology works in principle, but even if feedback and complex system processes are implemented, latency is extremely difficult to solve. So what about the feasibility of Magic Leap's solution?

Since light field display is to be achieved, the traditional LCD solution cannot be used. Magic Leap should use the fiber optic projector method, but the technical details are still a secret. We only know that the corresponding person in charge has achieved the projection of several-inch high-definition butterfly images with a 1 mm wide and 9 mm long optical fiber 6 years ago.

[[161355]]

After the projection is solved, the next thing to solve is the video source. This relies on the microlens array to generate a four-dimensional light field. This has also been produced in real life, from NVIDIA's research project, called near-eye light field display. The main problem with it, which was born in 2013, is the resolution, and the effect can only be close to mosaic.

So how to further improve the resolution? The solution given by Botao Amber Hu in the article is to achieve the effect of time-sharing display by increasing the display frame rate, which is similar to synchronous 3D display.

*** Talk about the difficulty: data & processing capabilities

As for light field, games are easier to achieve than reality. Because the entire imaging process of the game itself includes ray tracing and rendering of the entire space, but it is only presented on a two-dimensional plane. If you want to transform it to support the effect of light field projection, you can just modify the game engine and driver slightly.

[[161356]]

[[161357]]

The real difficulty lies in the real environment. In correspondence with the output, some people have also started to collect light field information. Lytro, the current leader, is planning to launch its own light field camera product next year. However, in addition to the camera body, a dedicated cabinet is required for complete use, which includes the processing and storage of light information.

So do we have to drag such a big oil bottle when playing? At present, this is the only way. Even if high-speed networks are expanded and remote processing and storage capabilities are provided, latency issues will arise.

So it is not difficult to understand why so much money is concentrated on Magic Leap. Even though it is currently mainly engaged in the AR field, its potential to expand into the VR field is obvious at a glance. Gary Bradski, senior vice president of perception research at Magic Leap, once mentioned in an interview with other media:

Fundamental, world-changing research takes time and money. MagicLeap hopes to get enough money in early financing to support this project to the end, and does not want to create short-term milestones that hinder the development process.