DeepMind: Combining artificial intelligence and neuroscience to achieve a virtuous cycle

Recent advances in artificial intelligence are remarkable. Artificial systems have outperformed expert human players at Atari video games, the ancient board game Go, and poker. They can also generate indistinguishable human handwriting and speech, translate between multiple languages, and even restyle your vacation photos in the style of Van Gogh.

These advances can be attributed to several factors, including the use of new statistical methods and the growth of computer power. But as we recently published in Neuron, we argue that one factor that is often overlooked is the contribution of both experimental and theoretical neuroscience.

Psychology and neuroscience have played a key role in the history of artificial intelligence. The founders of artificial intelligence, such as Donald Hebb, Warren McCulloch, Marvin Minsky, and Geoff Hinton, were initially motivated by the desire to understand how the human brain works. In fact, throughout the late 20th century, most of the key work in developing neural networks did not take place in math or physics labs, but in psychology and neurophysiology departments.

Because of this importance, the need to combine the fields of neuroscience and artificial intelligence is more urgent than ever.

At DeepMind, we believe that despite the rapid progress in both fields, researchers should not lose sight of this point of view. We urge researchers in neuroscience and AI to seek a common language that allows knowledge to flow freely, so that the free flow of knowledge can drive both fields forward.

We think it is important for AI research to draw inspiration from neuroscience for two reasons: First, neuroscience can help validate existing AI techniques. In short, if we find that one of our artificial algorithms can mimic a function in the human brain, this suggests that our approach may be on the right track. Second, neuroscience can provide a rich source of inspiration for new algorithms and structures when building artificial brains. Historically, traditional AI approaches have been dominated by logic-based methods and models based on theoretical mathematics. We think neuroscience can complement this by identifying classes of biological computations that may be critical to cognitive functions.

Take, for example, an important recent discovery in neuroscience: the discovery of “replay” of offline experiences. During sleep or quiet rest, biological brains “replay” neural activity generated by earlier periods of activity. For example, as a mouse moves through a maze, “place” units activate as the mouse moves. During rest, the same sequence of neural activity was observed in the mouse’s brain, as if the mouse were mentally reimagining its previous activities and using them to optimize future behavior. In fact, interference with the replay impaired their performance on the same task later.

[[199050]]

At first glance, it may seem counterintuitive to build an AI that needs to "sleep"—after all, they should spend as much time as possible on a computable problem after their programmers have gone to bed. But this principle is a key part of our deep-Q Network (DQN), an algorithm that can learn to play Atari 2600 games to superhuman levels using only raw pixels and scores as input. DQN mimics "experience replay" by storing a subset of its training data that it can "review" offline, allowing it to learn new things from past failures and successes.

Such successes give us confidence that neuroscience has long been an important source of ideas for artificial intelligence. Looking ahead, we believe that neuroscience will become indispensable in helping us deal with some of the still unsolved problems, such as efficient learning, understanding the physical world, and imagination.

Imagination is a critical function for humans and animals, allowing us to plan for the future before it happens, albeit at a cost. A simple example is planning a vacation. To do this, we draw on our knowledge or “model” of the world and use it to advance or assess future states in time. This allows us to calculate the path we need to take, or pack clothes for a sunny day. While cutting-edge research in human neuroscience is beginning to reveal the computational systems and mechanisms that underpin this kind of thinking, much of this new understanding has yet to be applied in artificial models.

[[199051]]

Another important challenge in current AI research is transfer learning. In order to effectively respond to new situations, AI agents need to build on existing knowledge to make smart decisions. Humans are already good at this: they can drive a car, use a laptop, or host a meeting, and they are usually able to respond effectively when faced with unfamiliar vehicles, operating systems, or social environments.

Researchers are now taking the first steps toward understanding how this happens in AI systems. For example, a new network structure called a progressive network can use knowledge learned in one video game to learn another video game. The same structure is also used in the scenario of transferring knowledge from a simulated robot arm to a real arm, which greatly reduces training time. Interestingly, these networks have some similarities with human models of sequential task learning. These connections show that future AI research has great potential to learn something from work in neuroscience.

But this exchange of knowledge cannot be one-way, and neuroscience can also benefit from artificial intelligence research. Take reinforcement learning as an example - reinforcement learning is one of the core methods of current artificial intelligence research. Although the original idea of ​​reinforcement learning comes from animal learning theory in psychology, it was developed and elaborated by machine learning researchers. These ideas feed back into neuroscience research, helping us understand neurophysiological phenomena, such as the firing characteristics of dopamine neurons in the basal ganglia of mammals.

AI researchers draw ideas from neuroscience to build new technologies, and neuroscientists learn from the behavior of artificial agents to better interpret the biological brain—this back-and-forth is necessary if the two fields are to continue to build on each other’s ideas and create a virtuous cycle. Indeed, recent advances such as optogenetics, which allow us to precisely measure and manipulate brain activity, generate large amounts of data that can be analyzed using machine learning tools.

So we think it is now critical to translate intelligence into algorithms and compare them to the human brain. Not only will this strengthen our quest to develop artificial intelligence, a tool that has the potential to create new knowledge and advance scientific discovery, but it may also help us better understand what is actually going on inside the human brain. This could shed light on some of the mysteries of neuroscience, such as creativity, dreams, and even the nature of consciousness. Because of this importance, the need to bring the fields of neuroscience and artificial intelligence together is more urgent than ever.