Magic Leap finally officially announced its first headset! Can $2 billion plus six years of research and development rebuild market confidence in AR?

Finally, Magic Leap, an augmented reality startup that has raised nearly $2 billion before officially releasing any products, has really begun to unveil its products. On its newly launched website, Magic Leap released a picture showing a pair of retro-looking black smart glasses, Magic Leap One, which look very much like Google Glass and are expected to be shipped in 2018.

Figure: Magic Leap One

Magic Leap claims the product mixes digital images with reality in a way that's realistic and comfortable to use. However, as in the past, the company didn't get too specific about how this works, other than to say it's using technology that involves the reproduction of light fields, which are the patterns created when light bounces off objects.

Magic Leap founder and CEO Rony Abovitz has spent years working on an AR headset and raised nearly $2 billion in funding.

Figure 丨Magic Leap founder and CEO Rony Abovitz

It's unclear how big or heavy the Magic Leap One will be, but it won't be tethered to a computer, and will instead need to be connected to a device called a Lightpack to do the processing.

The company says Magic Leap will have sensors so it can inventory the world around it and properly place digital objects, such as a virtual pet on a tabletop — which sounds similar to what Apple and Google developers are doing with their AR tools for iPhones and Android smartphones.

Magic Leap also says the product will remember the physical details of the user's environment, such as walls and objects, so that a digital object that was in a specific place when you put on the headset (such as a computer monitor on your desk) will still be in the same room the next time you put on the headset.

The company says users will be able to interact with Magic Leap One using their voices and gestures, and the product will also track head poses and eye positions, and there will be a handheld remote control. It uses force control and haptic feedback mechanisms to create a smooth sensory experience. With six degrees of freedom, movement feels smoother and more natural. In short, it senses your every move.

Figure 丨Magic Leap One handheld remote control

In addition to announcing new products, Magic Leap has also prepared a surprise for developers. It will open the relevant SDK and all tools, documents, learning resources and other support as much as possible in early 2018 so that developers can start making applications for the headset early next year.

Although still niche products, the augmented reality and virtual reality fields have evolved a lot since Magic Leap got its start. Some AR headsets have been officially released, but they are still mainly aimed at developers, such as Microsoft's HoloLens and Meta's Meta2, and their functions are still quite limited.

Figure 丨Microsoft HoloLens

Magic Leap is generally reluctant to talk to reporters, and its spokesperson Julia Gaynor did not comment on the details that appeared online. Abovitz seems to prefer to reveal some details about Magic Leap through social media - for example, on December 12, he said that there might be some "fun and cool stuff" in the coming week.

Magic Leap One Platform Technical Features

1. Digital Lightfield

Magic Leap One's light field optoelectronic devices can emit light of different depths of field and combine seamlessly with natural light to generate simulated digital images that coexist with real objects. This special technology actually "cheats" our eyes and brains, making them unable to distinguish the images generated by digital light when processing images, so that digital images and real-world objects are perfectly integrated. Perhaps you will ask what is the use of this? This technology solves the dizziness caused by long-term viewing of virtual images, allowing you to immerse yourself in a mixed world of virtual and real for a long time, just like watching real-world scenes.

2. Visual Perception

The sensor suite on Magic Leap One reconstructs the physical environment around you by detecting surfaces, levels, and objects themselves. The end result: a digital system that replicates the way the human eye perceives objects, making virtual light field objects not only appear to exist in the physical world, but also allowing users to interact with them. Whether it's a virtual display that appears out of thin air next to your computer monitor or a cute panda that crawls across your bedroom sofa, visual perception technology has fully unleashed the power of spatial computing through machine learning.

3. Persistent Objects

Magic Leap's visual perception and room mapping technology can completely replicate your physical environment, detecting and locating the exact position of walls, various planes, and other types of physical targets. Virtual light field objects are exactly the same as objects in the physical world, and you can place them anywhere. Imagine putting a virtual big-screen TV on the fireplace. Isn't this a great experience? You know, no one would put an expensive new TV on the fireplace... What's even better is that when you return to the fireplace, the virtual TV is still there, ready for you to use at any time.

Figure 丨 Magic Leap One wearing effect

4. Soundfield Audio

To feel real, it has to sound real. The Sound Field Audio system simulates the sounds you hear in the real world while delivering very high density sounds over long distances. This allows users to hear exactly where the sound is coming from, meaning users can even tell how close a virtual T-Rex is when it's very close to you.

5. High-Powered Chipset

Magic Leap's integrated processing unit delivers high-fidelity, gaming-quality graphics with the power and performance of a laptop. From editing sophisticated 3D models to playing first-person shooters in your living room, Magic Leap One generates light-field objects in intricate detail, all in a highly responsive, self-sufficient wearable device.

6. Next Generation Interface

As we all know, we live and think in a 3D world, not on a flat screen. Magic Leap's spatial interface includes multiple input modes, including voice, gesture, head pose, and eye tracking. This comprehensive input system includes the tools needed to break through the outdated conventions of point-and-click interfaces and provide a more natural and intuitive way to interact with technology.

Blooming Imagination

Of course, as a highly imaginative company, Magic Leap certainly wants to do much more than that. They also hope that users can use the functions of Magic Leap One to create richer and more interesting experiences such as the following.

1. Web pages “jump out” of the screen

Magic Leap One will allow web developers to optimize for content extraction and spatial browsing, enabling new experiences for shopping and exploration in 3D environments.

2. Display on demand

With Magic Leap One, users can open multiple screens of different sizes at any time and any place, so that they can operate freely anytime and anywhere, whether for work or entertainment.

3. Truly immersive gaming

Magic Leap One will revolutionize the gaming experience, which is something that no other good game can currently achieve. Imagine what a magical experience it would be if you were completely surrounded by all kinds of characters and people in the game.

4. You will be everywhere

With Magic Leap One, you no longer have to go to a specific place to meet with friends or colleagues. Communication and sharing can be carried out anytime and anywhere, and a new multi-dimensional collaboration method is born.

5. Imagination is everywhere in life

Remember Minecraft? This highly free sandbox game was once popular among teenagers, but with Magic Leap One, creativity will be more fully utilized and imagination will be unfettered.

According to official information, in addition to wearing a pair of AR glasses, Magic Leap One also requires a device called a "light field backpack" (lightPack) to provide the computing power required for drawing and the power required for the entire set of equipment.

Based on its appearance design and the ratio of the displayed image when worn on the model's head, it should be lighter than the VR glasses currently on the market. The front end includes an environmental perception camera and a depth camera. The depth camera is likely to use a VCSEL component similar to the current iPhone X to better depict the appearance of the entire real world as the basis for superimposing virtual images.

Compared to the bulky beta version in 2014, the Magic Leap One is very compact and solid. In addition to the possible technical breakthroughs in optical components, the relevant computing components have been moved to an independent light field backpack, which seems to be of great help. Of course, because of this, the backpack and glasses must be connected via a wire, which is a flaw in the ointment.

As for the mobile solution of glasses with backpack, Qualcomm should be the solution with the highest performance graphics capability and low power consumption, as well as AI computing. So is it possible that Magic Leap One is equipped with Qualcomm's SnapDragon 845/835? But if it is not 845, what other combinations are possible?

At present, there are many low-power image recognition chips to choose from on the market. Is it possible to maintain the NVIDIA solution rumored in the past, but add an additional independent image recognition chip?

There have been rumors that Magic Leap One may be equipped with NVIDIA Parker processor, which is the latest processor of NVIDIA Tegra system. However, there is no applicable software for Magic Leap One at present. The development environment is the most important part of a new hardware, even more important than the specifications of the hardware itself. Since it is a Creator platform, the most important thing to consider is whether the development kit can be prepared in the first place.

Continuing with the previous inference, if Qualcomm's platform is used, then no matter in AI computing or even AR environment development, there will be the fastest and most complete ready-made development environment. It is not impossible to launch a developer version with a complete SDK in 2018, which increases the possibility of Magic Leap using Qualcomm's solution. Of course, if other solutions are used, the SDK solution may not meet the schedule requirements.

MIT Technology Review was invited to experience the "monster" version of Magic Leap in 2014. The effect was indeed very shocking, but what was even more shocking was the size of the "monster". If you have seen the real machine, you will never associate it with words such as portable, head-mounted display, and low power consumption.

Therefore, even with Magic Leap's strong financial support, miniaturizing the device and making it wearable and mobile is still a very difficult task. At least from the current appearance design, Magic Leap one has made a great breakthrough, but the real challenge is whether the effect presented by the hardware is good enough and whether it can attract enough developers to join in the fun.

Judging from the multiple possible application directions and scenarios provided by Magic Leap, it can be expected that industry partners will be the focus when the next wave of Magic Leap One product solution details are released. Whether they are from consumer retail, Internet e-commerce, game entertainment, industrial process design tools, or enterprise collaborative operation platform solution manufacturers, they are all possible industry partners.

But it is worth noting that, just like all the innovative hardware in the past, everyone will ask: how many software applications can be supported? Therefore, although Magic Leap said that Magic Leap One will start shipping in 2018, the timing of shipment will be the next big problem, because this involves not only the production and yield rate of hardware products, but also how large the scale of the application content symbiosis system will be after Magic Leap is established before officially launching Magic Leap One for sale. From this, we can further judge how high the success rate of Magic Leap's first step in commercialization will be.