The starting price was 25,000 and it was sold out in an instant, causing the server to crash! Will Apple's head-mounted display be the next iPhone?

At the Apple Worldwide Developers Conference on June 6, 2023, the long-awaited "One More Thing" session reappeared. This time, Apple CEO Tim Cook brought out his most ambitious product since he took office: Apple Vision Pro - a groundbreaking product that Apple calls "the beginning of the era of spatial computing."

On January 19, Apple Vision Pro officially went on sale. At 9 p.m., just five minutes after the sale started, the server was overwhelmed. Half an hour after the pre-sale, the physical store was sold out.

Currently, Vision Pro is only sold in the United States. The domestic "scalper price" ranges from RMB 39,000 to RMB 90,000, which is at least 50% premium. Some merchants cannot even guarantee the delivery date, which shows how popular the pre-sale is.

Image source: Internet screenshot

What exactly is Apple Vision Pro, and why has it caused such a sensation? What's so great about it?

Let us first briefly introduce Apple Vision Pro.

Apple Vision Pro is a MR (Mixed Reality) device - it allows users to see part of the real world while also allowing users to be fully immersed in the virtual world.

Image source: Apple official website

It looks like a large ski goggle, filled with complex and expensive components. It costs nearly 25,000 yuan, seems to have no new technology, and seems not to be fully mature. Various technology media and popular media have mixed reviews on it, ranging from "Apple's driving force for the next decade" to "Apple's biggest failure in the past 20 years."

At first glance, Google tried this type of device ten years ago. Later, Microsoft did it, and Facebook did it. Microsoft is still doing this kind of thing, and it has already had some applications in the enterprise market. Facebook has even changed its name to Meta and invested heavily in this type of device. We all remember the "Metaverse" wave last year.

None of these products have made much of a splash this year, with generative AI being the brightest star of the year. However, breaking the boundaries between the virtual and real worlds may be a new answer that Apple has provided to the market after its long-term exploration of technology and full exploration of user scenarios.

Merging the real and virtual worlds

Apple's plan to connect the two worlds can be traced back to at least 2015. At that time, Apple introduced Mike Rockwell, the head of product and technology at Dolby Laboratories, to start researching the next generation of mobile computing devices. They hope that users can spend more time on such devices, preferably "as long as they are awake, they are using them."

According to the vision of Facebook founder Zuckerberg, the future of mankind is in the virtual world. It will be a completely artificial, immersive digital space where people can appear in this virtual world while sitting at home. Spatial distance is no longer a distance, and reality is not important - just like what is depicted in the movie "Ready Player One".

Apple CEO Tim Cook doesn't think so. He believes that immersive virtual space has its value, but the interaction between people and the world is still important. Apple wants people to live better in the real world with the help of mobile devices.

The core difference between these two approaches is the difference between virtual reality (VR) and augmented reality (AR). In other words, it is the difference between creating a virtual world that is difficult to distinguish between real and fake, and adding a layer of data filter to the existing world.

Stunning panoramic photos taken with iPhone expand and surround the user, creating the feeling that the user is at the scene of the photo shooting. Image source: Apple official website

The concept of virtual reality appeared earlier. In 1965, Ivan Sutherland, the founder of computer graphics, envisioned such a device. In an article called "The Ultimate Display", he proposed a new theory of human-computer collaboration that felt real and interacted realistically.

A few years later, Sutherland built a display system that was close to the ideal. It was a huge and heavy helmet-mounted display that could only be hung on a shelf. It had a very ironic name: "Sword of Damocles".

A few decades later, with the development of computing and display technologies, the sword of Damocles finally became a helmet that can be worn on the head. It can deceive our vision and hearing, making us feel that we are in another space. Although it cannot completely replace the real world, the brain does not seem to mind; some studies have shown that even rough virtual reality environments can bring real emotions. The brain will think they are real.

In 1992, Thomas Caudell of Boeing proposed the concept of "augmented reality". Soon, the US Air Force and Columbia University followed suit and used augmented reality for the maintenance of complex machinery - engineers wore helmets with screens, and when they looked at the machine that needed maintenance, the screen in the helmet would superimpose maintenance instructions at the corresponding position.

In 2012, Google released Google Glass. Later, Microsoft released HoloLens. Now we can buy many kinds of augmented reality glasses. However, most of them can only display a huge screen in front of the user.

Ultimately, the real meaning of augmented reality is to combine the two worlds - the real world and the digital world. Apple believes that being able to switch between the two worlds at will, or to freely superimpose the two worlds together, will be the way of life for people in the next era.

In 2018, Rockwell, who had joined Apple for three years, developed a base station that allows users to wirelessly receive virtual or augmented reality content as long as they are within the coverage area of ​​the base station. This solution looks good: the headset will not be too heavy, the computing power can be very strong, and battery life and heat dissipation are not even a big problem.

But it has a major flaw: users are limited to the range that the base station can support.

Apple rejected this plan. Apple wanted a more liberal plan.

Where does the future come from?

In 1945, Vannevar Bush, then the US President's science advisor, published an article titled "As We May Think" in The Atlantic Monthly. This is often considered the beginning of the Internet.

In that article, the author envisioned a grand blueprint: through a system called "Memex", people's knowledge can be connected and easily expanded. The Memex he envisioned has a screen and a keyboard, and can be held lightly in the hand, but can accommodate a huge amount of information.

Due to the limitations of the times, Vannevar's idea at the time was not necessarily the perfect form of technology. The best technology should make users unaware of its existence, and it should seem to be born that way. Screens and keyboards are just the result of technological limitations. Line of sight, gestures and natural language should be better choices. Or to put it in a more technical way: eye tracking, gesture tracking, and voice control are the most suitable ways to interact with the three-dimensional world.

Image source: Apple official website

AVP does a good job in this regard. According to users, it can accurately locate the line of sight and recognize the finger movements very well. It is not easy to achieve such natural tracking and display effects.

Apple has crammed a lot of things into this ski goggle-sized device: in order to perceive the user's behavior and surrounding environment, it uses 12 cameras, including 2 sets of RGB main cameras, 4 sets of downward-view cameras, 2 sets of outer-view cameras, and 4 sets of eye-tracking infrared cameras; in addition, there are 5 sensors and 6 microphones.

In order to present the most realistic visual effects, the user is presented with two 23-megapixel micro OLED displays. In order to complete a large amount of data processing, two chips are even crammed into the already limited space: an M2 chip used in laptops or iPads, and a specially developed R1 chip - it is responsible for processing input from cameras, sensors and microphones, and can reduce latency to 12 milliseconds, which is only one-eighth of the time required to blink.

"Today marks the beginning of a new era in computing," Cook said at Apple's Worldwide Developers Conference. " Just as the Mac brought us into the era of personal computing and the iPhone brought us into the era of mobile computing, Apple Vision Pro will bring us into the era of spatial computing. "

Cheetah Mobile founder and CEO Fu Sheng said on TikTok that this was "Apple dropping a nuclear bomb late at night." Fu Sheng believes that it will be an epoch-making product, powerful enough to suffocate people, and advanced enough to make peers despair. It may really usher in a new era.

Apple's long-term approach to technology

In the "One More Thing" segment of Apple's press conference in 2007, Jobs took out the iPhone from his jeans pocket. At that time, the iPhone software development was not yet completed. In order to avoid problems during the demonstration, he had to prepare several mobile phones, and he changed the phone every time he demonstrated an application. A few years later, the iPhone became the most watched mobile phone and completely brought mankind into the touch screen era.

The newly born AVP may have various problems. Its functions are far from perfect. Its processing difficulty and price will also be an obstacle. Compared with the iPhone, the structure of AVP is more complex and carries greater ambitions. But the AVP we see now is still a little far from this vision.
Of course, as mentioned above, the concept of the Internet precedes the Internet, and the concept of augmented reality precedes augmented reality devices. Regardless of the final product, we need to go through the creation and accumulation of theories, as well as the long-termism of technology that comes with it. After a series of long, sometimes even "aimless" explorations, we will suddenly find the exact path to our goal one day.

The story of Apple may have started with the acquisition of augmented reality software company Metaio in 2015. Since then, Apple has made more than 20 acquisitions disclosed in public documents, covering technology companies in the fields of chips, sensors, and displays related to hardware production, software and content companies that build an augmented reality ecosystem, such as German eye tracking company SMI, Japanese screen technology company JDI, and American virtual reality live broadcast company NextVR. Apple has been preparing for this product for eight years and has invested more than billions of dollars.

After several years of polishing and continuous high investment, AVP has set the standard for MR products. With its own technical strength and sufficient cash reserves, Apple is good at making the user experience perfect, so that users can never give up once they use it.

Unlock the magical experience of work and home with AVP Image source: Apple official website

Perhaps only Apple dares to make products in this way - it has money, people and time. It can practice true long-termism, and it is this long-termism that has turned Apple into a player capable of setting standards in the red ocean of electronic consumer products.

Continuously cultivating a field and continuously investing in a vision is long-termism. Apple is very different from other consumer product companies: it does not rely on specific resources and has never monopolized a product category. Its value depends on the release of new products every year, from user recognition, from continuous investment in technology, and from insights into the market and marketing.

In a market full of free competition, in a huge cycle consisting of researchers, various enterprises, technology development and consumers, with a lot of investment and a long time, our lives and the world are becoming more and more convenient. AVP may be just the latest in a long list of products that have been continuously improved in the past century.

Planning and production

This article is a work of Science Popularization China-Starry Sky Project

Produced by: Science Popularization Department of China Association for Science and Technology

Producer｜China Science and Technology Press Co., Ltd., Beijing Zhongke Xinghe Culture Media Co., Ltd.

Author: Mammoth Harbin University of Science and Technology

Review｜Yu Yang, Head of Tencent Xuanwu Lab

Planning｜Xu Lai

Editors: Wang Mengru and Yang Yaping