Google's machine learning has made a major breakthrough and its Go program will compete with Lee Sedol

On the morning of January 28, Google held a global conference call today. Demis Hassabis, founder of Deep MInd, announced important progress in Google's artificial intelligence: the development of a program that can beat professional players in Go - AlphaGo, which can master game skills through machine learning.

Computers and humans have been competing against each other in chess games for a long time. In games such as three-in-a-row, checkers and chess, computers have successively defeated humans. However, in Go, which has a history of more than 2,500 years, computers have never defeated humans before. Go looks simple and its rules are not difficult. The board has 19 parallel lines that are equidistant and perpendicular to each other, forming a total of 19×19 (361) intersections. The two players take turns to place their pieces, with the goal of occupying as much space as possible on the board.

Under the minimalist appearance of the game, Go has incredible depth and subtlety. When the board is empty, the first player has 361 options. During the game, it has far more options than chess, which is why developers of artificial intelligence and machine learning are always hoping to make breakthroughs in this area.

From the perspective of machine learning, the computational oracle of Go has 3361 positions, which is roughly 10170, while the number of atoms in the observed universe is only 1080. The computational oracle of chess has only 2155 positions, which is called the Shannon number, which is roughly 1047.

The traditional artificial intelligence method is to build a search tree for all possible moves, but this method is not applicable to Go. AlphaGo launched by Google combines advanced search trees with deep neural networks. These neural networks pass the description of the chessboard through 12 processing layers, and the processing layers contain millions of neural-like connection points.

One of the neural networks, a “policy network,” chooses the next move, while the other, a “value network,” predicts the winner of the game. Google trained the neural networks with 30 million moves by human Go masters, while AlphaGo also worked on new strategies on its own, running thousands of games between its neural networks and adjusting the connections through trial and error, a process also known as reinforcement learning. Much of the research was done through extensive use of the Google Cloud Platform.

Schematic diagram of the neural network structure used by AlphaGo

Conquering Go is of great significance to Google. AlphaGo is not only an "expert" system that follows artificial rules, but it also learns how to win the game of Go by itself through "machine learning". Google hopes to use these technologies to solve the most serious and urgent problems in real society - from climate modeling to complex disaster analysis.

In terms of specific machine training, the decision network is fed with human Go expert games until the system can predict 57% of human actions, compared to 44% previously. After that, AlphaGo began to learn to explore new Go strategies autonomously by playing games inside the neural network (which can be simply understood as playing Go with itself). Currently, AlphaGo's decision network can defeat most of the most advanced Go programs with huge search trees.

The value network is also trained by playing chess with itself. Currently, the value network can evaluate the chances of winning each move. This was previously thought to be impossible.

In fact, AlphaGo has become the best artificial intelligence Go program. In the game with other programs, AlphaGo has won 500 games with a single machine, and even had a record of winning after letting the opponent win 4 moves. From October 5 to October 9 last year, Google arranged a closed-door match between AlphaGo and European Go champion Fan Hui (Fan Hui: head coach of the French national Go team), and Google won 5-0.

AlphaGo vs. European Go Champion Fan Hui in 5 games

The public competition will be held in March this year. AlphaGo will compete with Korean Go player Lee Sedol in Seoul, South Korea. Lee Sedol has won the most world champion titles in the past 10 years. Google has provided a prize of 1 million US dollars for this. Lee Sedol said he is looking forward to the match and is confident of winning.

It is worth mentioning that the last famous man-machine game dates back to 1997. At that time, the supercomputer "Deep Blue" developed by IBM defeated the chess champion Kasparov. However, the algorithm of chess is much simpler than that of Go. In chess, winning only requires "killing" the king, while in Go, the victory or defeat is calculated by counting pieces or comparing goals, not simply killing the opponent's pieces. Previously, the designer of the "Deep Blue" computer published an article in 2007 stating that he believed that a supercomputer would be able to defeat humans in Go within ten years.

In addition, the release of AlphaGo is also the first voice of DeepMind since it was acquired by Google in January 2014. Before being acquired, the London-based artificial intelligence company also received investment from Musk, the founder of Tesla and SpaceX.