DeepMind's first step in decrypting the black box: It turns out that the cognitive principles of neural networks are the same as those of humans!

Humans have taught deep neural networks to do amazing things, from recognizing and reasoning about objects in images to playing Atari games and Go at superhuman levels. As the structure of neural networks and the tasks they perform become more complex, the solutions they learn become increasingly difficult for humans to understand.

People call this problem the “black box.” As neural networks are increasingly used to solve real-world problems, solving this black box problem becomes increasingly important.

In order to understand and explain these neural network systems, DeepMind researchers have been exploring new tools and methods. Recently, ICML included a paper from DeepMind, in which they proposed a new method from cognitive psychology to understand deep neural networks. Cognitive psychology infers the mechanism of cognitive processes by measuring behavior. There are many cognitive psychology papers that explain these mechanisms in detail and introduce many experimental methods to verify the mechanisms. As the latest neural networks reach human levels in specific tasks, cognitive psychology methods can help solve the black box problem.

[[195594]]

Black Box

DeepMind studied a specific case in the paper to illustrate its point. They designed an experiment to illustrate the human cognitive process, which was used to compare and help understand how deep neural networks solve image classification tasks.

The result is that behaviors that cognitive scientists observe in humans are also observed in these deep neural networks. Furthermore, these results provide a useful and surprising understanding of how neural networks solve image classification tasks. Overall, the success of this case study demonstrates the potential of using cognitive psychology methods to understand deep learning systems.

Measuring shape preference in a one-shot vocabulary learning model

In DeepMind's case study, they pondered the question of how human children recognize and classify objects, a question that has also been extensively studied in developmental cognitive psychology. Children have the ability to guess the meaning of words from a single example, which is called "one-shot vocabulary learning". This ability is very easy to acquire, and people often think that this process is very simple. However, philosopher Willard Van Orman Quine proposed a classic thought experiment that shows how complicated this process is:

A field linguist goes to experience another culture, and the language used in this culture is completely different from what he is used to. The linguist needs to find a local who is willing to help teach him some words. When a rabbit runs by, the locals say "gavagai", and the linguist has to guess what the locals mean by this word. To the linguist, this word can refer to many things, it could be a rabbit, an animal, a white thing, a specific rabbit, or even a single part of the rabbit. In fact, there are infinite images that this word can refer to. How do humans choose the right one among them?

[[195595]]

"gavagai"

Fifty years later, the same problem is being faced again with deep neural networks that can do one-shot vocabulary learning. Take the “matching network” developed by DeepMind, for example, which uses recent advances in attention and memory models to achieve state-of-the-art classification of ImageNet images from a single classification example. But we don’t know what assumptions the network makes when classifying images.

To explore this question further, DeepMind researchers referred to some research in developmental psychology. These psychologists found evidence that children have an inductive bias. This bias can eliminate many incorrect references and allow them to find the correct reference. This preference includes:

• Whole object preference, where children assume that a word refers to the whole object rather than its parts (eliminating Quine's concerns about referring to a single part of the rabbit)

• Category bias, where children assume that a word refers to a basic category to which an object belongs (eliminating Quine's concern about the basic meaning of referring to "all animals" rather than "rabbits")

• Shape preference, where children assume that the meaning of a noun is determined by the shape of an object rather than its color or texture (eliminating Quine's concern about referring to all white things rather than the specific object "rabbit")

The DeepMind researchers measured the shape preferences of their neural networks because human shape preferences are particularly well-studied.

Examples of stimuli from cognitive psychology that DeepMind uses to measure shape preferences in deep neural networks. These images were generously provided by Linda Smith of the Cognitive Development Lab at Indiana University.

The classic shape preference experiment used by DeepMind researchers is conducted as follows: a deep neural network is shown photos of three objects: a test object, a shape-matching object (same shape as the test object), and a color-matching object (same color as the test object but different shape). Shape preference is then measured by defining it as the proportion of cases in which the test object and the shape-matching object are classified as the same category by the network.

The images used in the experiment are the same images used in human experiments at the Indiana University Cognitive Development Laboratory.

This outline of a cognitive psychology experiment uses a matching network. The matching network matches the test image (left) with image A or B (top center or top right). The output (bottom right) depends on the shape preference of the matching network.

The DeepMind team experimented with their deep neural networks (matching networks and a baseline Inception model) and found that their networks have a much stronger preference for object shape than color or material, just like humans. In other words, they do have a "shape preference."

This result suggests that both Matching Networks and Inception classifiers use an inductive preference for shape to eliminate false assumptions, giving the researchers a clear understanding of how these networks solve one-shot vocabulary learning.

In addition to shape bias, the DeepMind team also found some interesting things:

• They found that shape preferences emerged early in the network's training. This is reminiscent of the emergence of shape preferences in humans: psychologists have found that younger children have weaker shape preferences than older children, and adults have stronger shape preferences.

• They found that using different random seeds for initialization and training would result in different preferences of the network. This shows that when studying deep learning systems and drawing conclusions, the number of samples studied needs to be large, just as psychologists have known that they cannot draw conclusions by studying only a single subject.

• They found that even though the networks had very different shape preferences, their one-shot vocabulary learning performance was similar, suggesting that different networks can find many equally effective ways to solve complex problems.

This preference exists in standard neural network architectures, but no one has recognized it before. The discovery of this preference shows the potential of using cognitive psychology created by humans to explain neural network problem-solving. Research in other fields of psychology may also be helpful. Ideas in the episodic memory literature may help understand episodic memory architectures, and methods in the semantic cognition literature may help understand recent concept formation models. Many fields of psychology, including the above, have rich literature and are likely to provide humans with powerful new tools to help solve the "black box" problem and allow humans to better understand the various behaviors of neural networks.

Paper address: https://arxiv.org/abs/1706.08606

via DeepMind Blog, compiled by Leifeng.com AI Technology Review