The AR technology strongly promoted by Apple iPhone X is 20 years old this year, and BAT has also played with it

1. Combining virtual and real

2. Real-time interaction

3. Present in 3D

Ronald Azuma (1997). A Glimpse into Augmented Reality.

This is the earliest widely accepted definition of AR (augmented reality). Ronald Azuma, then a professor at the University of North Carolina, summarized several application scenarios of AR in this paper, including actual cases in medical, manufacturing, visualization, navigation, and even military applications. If it weren't for the journal information clearly written in the header, it would be hard to believe that such a thing was the content of a paper in 1997.

From this article on, the concept of AR in the modern sense was officially born.

Big toys in the ivory tower

1997 is centuries ago in tech terms, but many experimental AR products appeared even before that. Although the graphics were shoddy and the bodies were heavy, they were indeed real AR devices.

For example, this KARMA.

Don't get me wrong, this KARMA is not a GoPro drone. Its full name is: Knowledge-based Augmented Reality for Maintenance Assistance. Its origin is as impressive as its name: two of the three authors of the paper are from Columbia University, and the other is from the famous Bell Labs.

The main part of KARMA is a transparent head-mounted display, but the real heart runs on a HP 9000 workstation, which also needs a Trobo SRX graphics accelerator card (not called a graphics card at that time). The specific price of such a set of equipment at that time is unknown, but it must be much more expensive than the current iMac Pro.

So, what can such a powerful device do?

As you can see, all that's shown are these red lines. This is actually a tutorial on how to remove the printer's paper tray. These rough lines are anchored to the device in your field of vision. Don't be fooled by its simplicity, if you always get confused when reading manuals, this is definitely a good thing.

Do you think this scene looks very familiar?

At the Microsoft Build 2017 conference in May this year, HoloLens demonstrated almost the same functions. The difference is that the maintenance suggestions are not rigidly built-in, but provided by a second person in real time. The field of vision is no longer just simple lines, but smooth prompt animations. In similar scenes, the experience is very different. The 20-year history of AR development has drawn a spiral upward development trajectory here.

Newcomers

As technologies in various fields gradually mature, AR technology has gradually emerged from universities and laboratories and begun to move towards the general public.

A milestone application is the emergence of AR Toolkit, a software project led by Hirokazu Kato of Nara Institute of Science and Technology in Japan. Its main function is to track objects. It allows you to create a pattern with features and then display content on it. It is basically an ancient version of Onmyoji's summoning.

Later, AR Toolkit actually ran on mobile phones. In 2005, it was ported to the Symbian platform, and then landed on iOS and Android.

As early as 2003, an AR mobile game appeared on Symbian: Attack of the Killer Virus. In this game, you need to raise your phone to kill the viruses around you, otherwise you will be attacked by these shining square octahedrons and die. This game not only integrates 3D objects with the surrounding real scene (although the integration is quite rough), but also can sense the movement of the camera's perspective. If you walk towards the direction of the virus in your field of vision, the virus on the screen will be enlarged, which can be said to be quite scientific. In the forums at that time, it was called the "camera game".

In the following years, AR applications have been developing slowly. New hardware devices have been entering the market, but none of them have made a big splash. However, the huge wave of mobile Internet is about to hit. The AR applications we know today and their predecessors will also make their debut.

Evolution

The principle of the earliest AR games on mobile devices is similar to the AR Toolkit mentioned above, which requires the use of special marked cards to calibrate the real environment for gaming.

Nintendo's 3DS has built-in AR games since its launch, and it must be used with its own AR cards. By scanning different cards, you can see many classic Nintendo animation characters, as well as several mini-games to choose from.

(Scan the question mark card to start the game)

There is also an AR battle game called "Invisible Monster" on PSP. Because of its novel gameplay, IGN gave this creative score of 8 points. However, this game not only requires AR cards, but also requires a dedicated camera for PSP. Most domestic players may not have played it.

The next successful application of mobile AR appeared on the now-defunct Nokia Lumia series.

This is Nokia's "City Kaleidoscope". The name is confusing at first glance. In fact, it is an AR real-life navigation application. Based on LBS data and map data, when you open the application and hold up your phone, the image in the viewfinder will be overlaid with a layer of landmark information, so you can see what places are worth visiting around you and how far away they are. This reality scanner-like experience is really pale in description, so I suggest you watch the video.

Yes, after watching the video you will realize that this thing is actually useless...

The problem is that when City Kaleidoscope appeared, the processing power of mobile phones could not provide sufficient frame rates. You can see obvious frame drops and freezes in the video. The location cards are not 3D as expected, but always parallel to the screen, even in the official version. City Kaleidoscope is a great idea, but it appeared too early.

But soon, the performance of mobile chips saw explosive growth in just a few years.

On April 1, 2014, Google, which always loves to play, released an April Fools' Day Easter egg related to Pokémon: you can catch Pokémon on Google Maps. While users are browsing the map, Pikachu will randomly appear, and users can catch it with one click.

They even released a video at the same time, showing the story of Google employees fighting against the sky, the earth, and Pokémon, and working hard to become Pokémon masters.

This is of course false, but everyone knows the rest of the story.

(Pictured is Tsunekazu Ishihara, CEO of The Pokémon Company and a loyal Ingress player)

The April Fools' Day Easter egg video that year caused a strong response on the Internet, and Google finally decided to turn this April Fools' Day joke into reality. They first obtained the authorization of the first 150 Pokémon from Nintendo and Pokémon Company, and then handed the development work to Niantic Labs Balloon Club, which had earlier successful experience in Ingress. Balloon Club lived up to expectations and successfully brought this idea to Android and iOS platforms in 2016.

This is the popular Pokémon Go.

Pokémon Go, which has a strong mass base, quickly spread faster than a virus. Even in mainland China, thousands of miles away and unable to play the game, it was popular on social media for weeks. For a time, the concept of LBS games and AR swept the world. After that, some AR and LBS imitators also appeared in China, including the relatively excellent and steady "Travel: Zero" and IP piggybacking "Pig Man AR Virtual Mission".

Unfortunately, due to the lack of gameplay, this batch of AR games, even including Pokémon Go itself, failed to really retain players, repeating the mistakes of "Invisible Monster" several years ago. However, AR games at this time have powerful performance support, and the experience has left the era of city kaleidoscope far behind.

(Pokémon Go active users dropped sharply)

Most of the subsequent domestic AR applications were marketing-oriented, such as the mini-games launched by Baidu Maps and Tmall, but they were short-lived. The most successful AR marketing should be the torch relay activity of QQ AR, which attracted more than 1 million participants within 24 hours, which was truly a "million strong men spreading the torch online".

The only AR application that still maintains a certain degree of popularity seems to be the Onmyoji who can be "summoned in the world". However, apart from bringing more metaphysical legends to African players, it is difficult to say how strong the connection between AR and Onmyoji is.

Another large-scale explosion of label-based AR was Alipay's Lucky Blessings event during the Spring Festival this year. The label used the common Chinese character "Fu" (blessing) and reversed the reputation of the previous year's "Five Blessings" event. Interestingly, the two major elements of Pokémon Go, AR, were promoted by Alipay, while the LBS game was used by Tencent QQ to play with red envelopes. Global trends always have different implementation ideas in China.

So far, there are roughly two types of AR on the market: Pokémon Go-style or tag scanning-style. Has the imagination of AR really reached its limit?

The future is here

Google's interest in AR began even earlier than Pokémon Easter eggs.

As we have mentioned before, one of the key points of AR games is to be able to mark the real space. Many games use pre-designed special markers to do this, but to expand more AR applications, markers alone are far from enough. Therefore, Google officially announced the Tango project in early 2014 to give mobile phones real-time 3D modeling capabilities and the ability to determine their own position in space.

These two sentences may seem boring, but they actually point directly to the core definition of AR applications. Presenting 3D objects has long been possible, but integrating virtual and reality and responding in real time requires machines to be able to recognize the real world. Previous computer vision was limited by cameras and other hardware, and could only understand two-dimensional images and could not locate three-dimensional space. Project Tango solves this problem with the help of a dedicated computer vision processing chip and two cameras with different purposes. By scanning the room with it, it can quickly recognize the surroundings and locate itself, and subsequent model display and interaction will naturally not be a problem.

In September 2016, the first Tango-powered phone, Lenovo Phab 2 Pro, went on sale in China, but it was a failure like a punch on cotton. There are less than 40 Tango apps in the Play Store, and they can't run on ordinary Android phones.

The second Tango hardware ZenFone AR was launched in June this year, but it did not arouse much response. Hardware barriers plus software limitations ultimately made the Tango project become the embarrassing situation it is today, with good reviews but poor sales.

AR has become popular again this year thanks to Apple, which even made headlines when Cook went to participate in the gay parade in San Francisco.

At WWDC in June, Apple announced a bunch of hardware that Apple fans couldn't afford even if they saved up money, and a lot of new features for iOS 11. The most eye-catching one was ARKit. With system-level support, iOS became the world's largest AR device platform overnight. Not only that, the actual machine demonstration effect of ARKit was also amazing:

With ARKit, the Pikachu on the screen is now firmly fixed on the ground instead of floating in the air like before. The surrounding shadows and lighting effects on Pikachu also appear more natural.

After WWDC, ARKit quickly became popular in a small circle, and various demos on social media were eager to prove the potential of this platform. Recently, at the iPhone autumn conference, Apple once again demonstrated a nearly final version of ARKit. The smooth picture and realistic lighting made various media feel:

It's stable.

Naturally, Google was not willing to be outdone. Since the public was not buying into the professional-level Project Tango, Google came up with ARCore, which is common to all Android devices, to try to compete with Apple.

However, in terms of the demonstration effect, ARCore's lighting effect and the number of polygons of objects in the demonstration are not as good as Apple's ARKit. What will the finished products on the two platforms look like, and whether AR can set off a new round of enthusiasm in the future, this will be judged by time.

At least now, the door to the future has been opened.

As a winner of Toutiao's Qingyun Plan and Baijiahao's Bai+ Plan, the 2019 Baidu Digital Author of the Year, the Baijiahao's Most Popular Author in the Technology Field, the 2019 Sogou Technology and Culture Author, and the 2021 Baijiahao Quarterly Influential Creator, he has won many awards, including the 2013 Sohu Best Industry Media Person, the 2015 China New Media Entrepreneurship Competition Beijing Third Place, the 2015 Guangmang Experience Award, the 2015 China New Media Entrepreneurship Competition Finals Third Place, and the 2018 Baidu Dynamic Annual Powerful Celebrity.