In 2017, AI beat humans at every game designed by humans

For much of the 20th century, chess was a benchmark for AI researchers. John McCarthy, who coined the term “artificial intelligence” in the early 1950s, once called chess “the fruit fly of AI,” a reference to the influence of early studies of fruit flies on the field of genetics.

In the late 1990s, IBM's Deep Blue played a series of chess matches against world champion Garry Kasparov.

In 1997, Deep Blue finally defeated Kasparov, marking the first time a machine had beaten a world champion in a match, and by the early to mid-2000s, the technology had improved to the point where machines were consistently beating chess grandmasters in nearly every variation of the game.

Naturally, AI developers have turned to other, more complex games to test their increasingly sophisticated algorithms. In the past 12 months, AI has crossed a series of new thresholds, ultimately beating human players in a variety of different games, from the ancient game of Go to the dynamic interactive card game, Texas Hold'em.

From Chess to Go

In the late 1990s, after machines finally and resoundingly beat chess grandmasters, an astrophysicist from Princeton commented that “it may be a hundred years before computers beat humans at Go—maybe even longer.”

So computer scientists turned their attention to Go, an ancient strategy game from China that is very easy to learn but difficult to master.

Over the past decade, advances in machine learning have created truly competitive AI Go players. In 2014, Google began developing a deep learning neural network called AlphaGo. After a few years of near-success, the development team tried something different.

In late 2016, a mysterious online Go player named "Master" appeared on the popular Asian game server Tygem. Over the next few days, the mysterious player dominated matches against many world champions. By January 4, 2017, it was confirmed that "Master" was actually the latest version of DeepMind's AI AlphaGo.

In May 2017, AlphaGo “Master” defeated Ke Jie, the world’s top-ranked Go player. In the three games AlphaGo and Ke Jie played against each other, the machine was always in the lead, but the most surprising thing was that in October, Google had developed a more advanced version of AlphaGo than “Master”.

According to an article in Nature, AlphaGo Zero is a revolutionary algorithm that was able to rapidly teach itself to play Go. The system played against itself over and over again, mastering how to play in different situations. After 21 days of learning, AlphaGo Zero had reached the level of "Master". After 40 days, it had surpassed the skill level of all previous versions.

By December 2017, DeepMind had unveiled a more advanced version of the system. This new AI, called AlphaZero, could master multiple games in a matter of hours. After eight hours of self-training, the system could not only beat the previous AlphaGo Zero, but also fully master chess and shogi.

Master the art of cheating in poker

Although the complexity of the game of Go is very high, for artificial intelligence, Go and poker are two completely different models. To win in poker, you need to master certain cheating skills. Cheating and identifying others' cheating are key skills to win in card games.

After more than a decade of trying, two independent studies showed in 2017 that artificial intelligence had finally beaten top poker professionals. Researchers from the University of Alberta in Canada unveiled an AI system, DeepStack, that could outperform human poker players using an AI form of "intuition."

After a grueling 20-day marathon, the machine has comprehensively thrashed all four professional poker players

A Carnegie Mellon University research group staged a more public event in January 2017, when its Libratus AI system spent 20 days playing 120,000 rounds of no-limit Texas Hold’em against four professional poker players. While the professionals debated each night which weaknesses they could exploit in the AI, the machine improved itself every day, patching holes in its gameplay and refining its strategies.

The human brain was no match for the machine, and after nearly a month of nonstop play, the machine had won a total of $1.7 million, while each of the four professionals lost thousands of dollars in play money. "During this intense competition, I felt like I was playing against a cheater, as if my cards could be seen by him," one of the professional players told Wired magazine. "I'm not accusing him of cheating. This is actually a good thing."

Elon Musk's AI Research

In 2015, Elon Musk and a small group of investors founded a project called OpenAI. The project was designed to explore the development of artificial intelligence systems, especially reinforcement learning, in which machines can teach themselves how to improve their abilities at specific tasks.

In August 2017, the OpenAI team set its sights on conquering Dota 2, the core game in a major esports tournament called “The International.” Dota 2 is a hugely popular and complex multiplayer online battle game that is a serious competition in the world of competitive gaming.

After just two weeks of learning, the OpenAI bot joined the tournament and subsequently defeated several of the world's top players. The AI ​​system has only been trained on the simpler one-on-one version of the game, but the OpenAI team is studying how to get the AI ​​to master the five-on-five "team" game.

AI plays the game of "Pac-Man" by dividing the work

A few years ago, Google DeepMind trained its AI on 49 Atari 2600 games. Given the same inputs as human players, the AI ​​figured out how to play and win at those games. It turns out that some games are harder to master than others, and among these classic, notoriously difficult games, one 1980s video game, Pac-Man, was particularly challenging.

In 2017, Google acquired a deep learning startup called Maluuba and merged it into DeepMind. Maluuba's novel machine learning method is called "hybrid reward architecture" (HRA). Applying this method to the Pac-Man system, the system created more than 150 individual agents, each with a specific goal - such as finding a specific bean, or avoiding ghosts.

The HRA method generates a high-level agent, similar to a senior manager. This high-level agent will evaluate all suggestions from subordinate agents before making the final decision at each step. This approach is euphemistically called "divide and conquer," which means breaking down complex tasks into smaller parts.

After applying this method to Pac-Man, the AI ​​quickly mastered how to get a high score of 999,990, which no human or artificial intelligence had ever achieved before.

AI will start designing games

If AI can beat us in almost every game, what should we do next?

A researcher at Falmouth University has recently unveiled a machine learning algorithm that he claims can create our own games for us, that we play from scratch. The AI ​​system, called Angelina, continues to improve itself every day, but it can currently make games using datasets from a variety of sources, from Wikipedia Commons to online newspapers and social media.

So what does all this mean?

Perhaps the biggest, most terrifying development of 2017 was the huge advances in reinforcement learning systems. These programs can effectively teach themselves how to master new skills. For example, the most recent iteration of AlphaZero can achieve super-competence in some games after just a few days of autonomous learning.

A massive survey of more than 350 AI researchers suggests that AI isn’t good enough to beat us yet. The survey predicts that within 10 years, AI will be better than us, by 2049 it will be able to write a bestselling novel, and by 2053 it will be better than humans at performing surgery. In fact, the survey concluded that by 2060 there’s a 50-50 chance that AI will be able to do everything we can do, only better.

2017 was undoubtedly a landmark year for AI beating humans at increasingly complex games, and while this may seem like a modest achievement, its implications are huge. Many of these AI development companies are quickly turning their sights to real-world challenges.

Google DeepMind has applied AlphaGo Zero's system to other fields and has conducted a comprehensive study of protein folding in the hope of revealing treatments for diseases such as Alzheimer's and Parkinson's.

“Ultimately, we hope to use algorithmic breakthroughs like this to help solve pressing problems in a wide range of real-world problems,” said Demis Hassabis, co-founder and CEO of DeepMind. “If similar techniques can be applied to other structural problems, such as protein folding, reducing energy consumption, or finding revolutionary new materials, the breakthroughs achieved will have the potential to enhance humanity’s understanding of the world and positively impact all of our lives.”

As a winner of Toutiao's Qingyun Plan and Baijiahao's Bai+ Plan, the 2019 Baidu Digital Author of the Year, the Baijiahao's Most Popular Author in the Technology Field, the 2019 Sogou Technology and Culture Author, and the 2021 Baijiahao Quarterly Influential Creator, he has won many awards, including the 2013 Sohu Best Industry Media Person, the 2015 China New Media Entrepreneurship Competition Beijing Third Place, the 2015 Guangmang Experience Award, the 2015 China New Media Entrepreneurship Competition Finals Third Place, and the 2018 Baidu Dynamic Annual Powerful Celebrity.