Can AR glasses allow Cook to break through the influence of Jobs?

It has long been an open secret that Apple has deeply invested in the AR/VR industry.

During the Steve Jobs era, the iPhone revolutionized an era. But since Jobs' death, every new iPhone released by Apple has failed to wow the public.

According to a new report from Counterpoint Research, in the second quarter of 2018, Apple's share of global mobile phone profits still exceeded the total of all other mobile phone manufacturers in the world.

However, Apple failed to maintain its advantage in terms of sales volume and sales growth rate.

According to the Q2 2018 global smartphone sales data released by IDC, the top five brands in terms of sales in the second quarter were still Samsung, Apple, Huawei, Xiaomi and OPPO, but Huawei surpassed Apple with sales of 54.2 million units to 41.3 million units in the second quarter, and jumped to second place with a market share of 15.8%. This result was also confirmed by another research organization, Canalys.

Meanwhile, among smartphone brands, Samsung ranked first with a market share of 20.9%, Huawei's market share was 15.8%, surpassing Apple (12.1%), and OPPO's market share was 8.6%. In terms of sales growth, Apple's mobile phone sales increased by only 0.7% year-on-year and fell by 21% month-on-month.

It can be seen that although Apple's mobile phone is still leading in terms of profit, its market share and sales are far less than before. For this reason, Apple needs to find and create the next industry and product that can make it "leading" again.

And this is most likely AR/VR.

In recent years, news about Apple in the AR/VR field has been constantly revealed. Wangchao Technology has uncovered the full picture of Apple's AR/VR development through the clues behind the layers of revelations, combined with its recent series of investments, mergers and acquisitions, patents, etc.

Apple’s AR/VR M&A Spree

In addition to its products, Apple's acquisitions are also talked about. Recently, Apple was exposed for its latest acquisition.

Danish media Brsen reported that Apple has acquired a Danish startup called Spektral. Spektral focuses on separating objects in photos and videos in real time to achieve a "green screen" effect. It was revealed that the acquisition was actually completed at the end of last year, but it was not discovered until now.

In August, Apple also confirmed the acquisition of Akonia Holographics, a startup company that makes augmented reality glasses lenses. At first, Akonia focused on the data storage of holographic images, and later began to build lenses for AR devices. In the latter stage, Akonia's main product is HoloMirror, which can display dynamic, full-color, and wide-viewing-range images on a thin, transparent lens.

In fact, AR has been a serious M&A target for Apple in recent years.

If Apple had a premeditated plan for the AR/VR industry, then by observing Apple's acquisitions, we can speculate that this time point can be traced back to 2013.

This was the year that Apple had completed its acquisition of Prime Sense, whose infrared (IR) technology first made waves in Kinect, the body-sensing add-on for Microsoft's Xbox.*** The "sensor" in the iPhone is basically a shrunken Kinect to enable FaceID.

While it's primarily used to map faces right now, the infrared camera in the iPhone X(S) could be used by future AR apps to enable front-facing AR apps. For example, a user could point the phone at a vacant lot and the phone would spray thousands of infrared dots into the outline of the area to create a rendering of what a custom house would look like in that area.

Since then, Apple's acquisitions in the AR/VR field have never stopped.

In 2015, Apple announced the acquisition of Metaio, a German augmented reality startup whose "Thermal Touch" system enables users to transform any physical surface into a virtual electronic touch screen by using augmented reality glasses or ordinary cameras. However, the technology was not yet mature at the time and still needed 5 to 10 years.

In 2016, Apple acquired augmented reality startup Flyby Media. Both parties refused to disclose the acquisition amount, but the Flyby Media team will join Apple's secret research department and work with the Apple team to develop VR and AR headset prototypes.

Last June, Apple acquired Sensomotoric Instruments, a computer vision company that makes AR headsets. It is reported that Sensomotoric Instruments focuses on developing eye tracking hardware and software products, and its business covers virtual and augmented reality, in-vehicle systems, clinical research, cognitive training, linguistics, neuroscience, physical training and biomechanics, and psychology.

The company's eye tracking technology solves the common shortcomings of VR. It can record the user's natural gaze behavior in real time at a sampling rate of 120Hz, analyze the wearer's gaze, and help reduce dizziness.

However, "strangely, Apple has not officially confirmed the deal, but the company did give Axios a statement on the report," said the person involved.

It is worth noting that in November last year, Apple acquired Vrvana, a team dedicated to the development of AR glasses, for US$30 million.

In fact, Vrvana has an actual product when it comes to AR glasses. Its Totem glasses, which never shipped, received glowing early reviews. So far, it appears to be Apple's first AR hardware (not AR software).

Until this year, two consecutive acquisitions fully demonstrated Apple's serious desire in this field.

Apple AR/VR Patents

Apple's acquisition of startups is nothing new. Through acquisitions, Apple's ultimate goal is to obtain technology and patents.

If we can guess from the Apple acquisition that Apple had planned to enter the AR/VR field in 2013, then a patent titled "Interactive Three-Dimensional Display System" in 2012 can move the time point forward by one year.

The patent application details a system for projecting hologram-type images that can be manipulated by users in mid-air without the need for special glasses.

The patent application was filed in October 2012 and could be part of a larger AR capability, with possible applications in education, medical diagnostics, and biomedical engineering.

AR in the form of holograms is undoubtedly the future direction that augmented reality may take. AR via holograms means that projectors can send images into physical environments (just like in movies like Star Wars). In this way, holograms can eliminate the need for bulky headsets, which is a big barrier to consumer adoption of AR.

In recent years, there have been constant reports about Apple's AR patents. Recently, foreign media PatentlyApple discovered a new Apple patent in Europe, which is closely related to the legendary Apple glasses.

It is reported that the head-mounted device in the patent not only supports Apple's main AR, but also supports VR and holographic content display. The patent describes Apple's optical waveguide solution in detail. Compared with MagicLeap's two-layer optical waveguide solution, Apple can display three layers of image planes: far, medium and near.

The display system may have one or more pixel arrays, such as liquid crystal on silicon (LCOS) pixel arrays. The image from the display system may be coupled to one or more optical waveguides via a coupler system, and it is also possible to couple into optical waveguides in multiple image planes using an output coupler system.

Input and output coupler systems may feature single couplers, coupler stacks, and tiled array couplers. These couplers may be thin planar volume holograms or other optical couplers that couple light into or out of the upper and lower surfaces of an extended ribbon waveguide.

Image signals such as wavelength, polarization, time division and angle multiplexing techniques all have different angular orientation ranges, and lens tunability techniques can also be used to display images on multiple image planes to the user.

Apple AR/VR Products

Currently, in the entire AR/VR industry, technology giants such as Google, Microsoft, Facebook, and Samsung have already taken the lead and launched their own hardware products. In comparison, although Apple already has the AR application ARKit, it is already a step behind in hardware products.

However, both Bloomberg and well-known Apple analyst Ming-Chi Kuo have predicted that Apple will launch its hardware product AR glasses in 2020, two years from now.

In April this year, foreign media CNET also reported that people familiar with the matter revealed that Apple is developing a head-mounted device that integrates AR/VR experience. It will be connected to a dedicated box via short-range wireless technology, and the single-eye display will reach 8K. The device is code-named "T288" and will be shipped in 2020.

It is understood that the T288 will use high-speed short-range wireless technology to connect to the dedicated box. This high-speed short-range wireless technology is the 60GHz WiGig wireless technology launched by Intel at the end of 2016.

In 2017, the revised version 802.11ay, based on the first generation 802.11ad, was born with stronger performance. The target rate is as high as 176Gbps, which is 22GB/s. In theory, a Blu-ray movie can be downloaded in 1 second. However, ordinary computers can hardly support such a rate. The practical significance of 802.11ay is expected to be in scenarios such as high-definition transmission and wireless screen projection.

Therefore, it seems that the box connected by wireless technology is similar to a PC case, not a real Mac computer. The box will be equipped with Apple's custom Apple processor as the computing center of T288. The size of the processor is only 5 nanometers, while the iPhone X currently uses a 10 nanometer processor.

In addition, unlike HTC's new Vive wireless upgrade kit, the T288 does not require an external positioning base station and will have inside-out tracking.

Apple did not respond to this news.

Last November, Bloomberg published a report detailing the configuration of Apple's AR glasses. The most important thing was that Apple was developing its own chips for AR headsets.

Conceptually, this chip will be highly integrated, just like the chip in the Apple Watch. Such a chip can concentrate more components, such as graphics processors, AI chips, CPUs, etc., in a small area, and energy consumption will also be reduced.

In terms of operating system, Apple internally calls it “rOS” (reality operating system), which is based on iOS. Just like the operating system of Apple TV is tvOS, the operating system of Mac computer is macOS, and the operating system of Apple Watch is watchOS, rOS will be the operating system of Apple AR headset.

In addition, AR headsets will also use the existing APP Store, allowing users to download content themselves, just like iPhone, Watch, Apple TV and Mac.

From previous news, we can see that Apple's hardware products focus on AR experience. Although Apple has not released any news about VR, Bloomberg reported that the T288 project team is also testing VR.

At present, Facebook and Google are making great efforts to develop VR, so there is no reason for Apple not to make some progress in VR. HTC's Wang Congqing has said more than once that the future trend is to integrate the functions of AR and VR into the same device.

Conclusion: Can Cook break through Jobs' influence on Apple?

Deeply deploying the AR/VR industry and independently developing hardware products are of great significance to both Apple itself and its CEO Cook.

Apple needs new products to iterate. As mentioned above, Apple's mobile phone sales and sales growth have shown signs of weakness, and products such as iPad and iWatch are also difficult to impress users.

If Apple's AR glasses are shocking enough, perhaps this is its chance to continue its glory.

For Cook, Jobs' influence on Apple is profound. Therefore, it is very important to break through Jobs' influence and create products that do not belong to the Jobs era.

If AR glasses are successful, perhaps this will make Cook one of the best CEOs in Apple's history.

However, at this stage, everything Apple has done in the field of AR/VR hardware is still in the conceptual stage. In addition, Apple faces many difficulties in product development, technical difficulties, market promotion, and user acceptance.

But there are still two years until 2020, which is enough time for Cook and Apple to think about the AR/VR field. All we have to do is look forward to the arrival of Apple AR glasses as scheduled in 2020.