ChatGPT is equivalent to a 9-year-old human child. Does it really think of itself as a human being?

If you ask what the hottest technology terms on the Internet are recently, they must be "ChatGPT" and "artificial intelligence". ChatGPT, an intelligent chatbot developed by OpenAI, can answer various questions raised by people, generate articles according to people's requirements, translate foreign languages, and write code.

In fact, ChatGPT is not the first AI with these functions, but its ability is almost a qualitative leap compared with previous AIs - it can not only answer questions with humans, but also write articles that are quite realistic. Many people speculate that ChatGPT marks another major breakthrough in the field of artificial intelligence .

At one time, many researchers began to pay attention to ChatGPT and carefully studied its various capabilities.

On February 11, Michal Kosinski, a computer scientist at Stanford University, submitted a preprint of a paper titled "Theory of Mind May Have Spontaneously Emerged in Large Language Models," which further increased the popularity of ChatGPT.

The so-called "big language model" refers to a type of artificial intelligence represented by ChatGPT. Using psychological tests, researchers found that the GPT-3.5 version of artificial intelligence (the currently popular ChatGPT is derived from the GPT-3 version) has developed a certain "theory of mind" (also often translated as psychological theory). In just a few days, various interpretations appeared on the Chinese Internet. Some people say that this artificial intelligence already has the empathy of a nine-year-old child; some people say that this artificial intelligence already has a mind; and some people say that GPT-3.5's intelligence is already equivalent to that of a nine-year-old child...

Wait a minute, "theory of mind", "empathy", "mind", "intelligence", these words seem familiar, but their meanings seem a little different. Compared with GPT-3.5 and a nine-year-old child, who is better? In order to answer this question, we need to do a word meaning analysis and see what this paper actually says.

01

What is Theory of Mind?

Many people may have heard of the term theory of mind for the first time. Don't be misled by the literal meaning. Theory of mind is not a theory, but an ability. Simply put, it is the ability to understand and infer the psychological state of others. For example, you carefully hid a gift when your friend was away. When he came back and found the gift, how would he feel? Most people would instinctively think that he would be surprised and delighted. You can think of this because you know that the other person doesn't know about the gift, and you think that the other person will feel happy when he receives the gift. This shows that you have the ability to infer the psychological state of others.

Seeing this, some people may ask, isn't this normal thinking of normal people? How can it be considered an ability? In fact, when psychologists first studied this issue, the target of their research was not humans. In 1978, psychologists Premark and Woodruff published a famous paper called "Do Chimpanzees Have a Theory of Mind?" The authors pointed out that humans can only observe the behavior of their own kind and cannot see what the other person is thinking, but they can infer the other person's intentions, knowledge, beliefs, thoughts, suspicions, and even disguises . This is human instinct. So, can chimpanzees do this? If they can, it means that chimpanzees, like humans, have the ability to infer the inner thoughts of others. Psychologists have found that chimpanzees can be aware of the thoughts of others and do have a certain theory of mind.

Caption: In a test used by psychologists, chimpanzees need to observe the experimenter's predicament and try to infer how to help him.

As soon as this paper appeared, it immediately inspired many scholars. They asked, is theory of mind really a human instinct? How do humans acquire theory of mind? So, psychologists did a series of studies and found that humans are not born with theory of mind. Infants and young children can first switch their attention targets with others, and gradually realize that other people have their own ideas. Then, young children begin to understand and infer the mental states of others. Next, this inference ability will become more and more accurate until it reaches the same level as adults, and can understand others without thinking.

How do psychologists test this? We can look at two classic experiments. The first one is called the false belief test . Psychologists show a group of pictures to children. In front of a child is a bag with a chocolate label on it, but inside is popcorn. At this time, the psychologist asks: If a child just came over and saw this bag, what would he think was inside? Adults would think that this child did not see the popcorn in the bag and would think it was chocolate based on the label. But children whose theory of mind is not yet sound are different. They may not be able to distinguish what they know and what others know, so they infer that this child thinks it is popcorn. As children grow older, psychologists will increase the difficulty and ask children to infer what the child in the picture thinks the other child thinks. This requires children to be more proficient in inferring the psychological states of different people.

Another experiment is called the Sally-Anne Test , where psychologists show children a video or picture of Sally and Anne in a room together. At this time, Sally puts a ball in a basket and then leaves the room. Anne waits for her to leave and hides the ball in a box. After a while, Sally returns. Where will she look for the ball? Adults can infer that Sally didn't know that Anne had hidden the ball, but she will still look for the ball in the basket. But children are different. Their theory of mind is not mature yet, so they may think that Sally will look for the ball in the box.

Both tests found that children cannot answer questions correctly and pass the test until they are four years old, and their performance is very unstable at first. It takes until they are nine years old to be able to flexibly infer the mental state of any other person.

Note: The Sally-Anne test diagram is from Wikipedia

In fact, each of us has experienced this stage of "not being able to guess" or "not understanding" others, but as we grow older, most people will acquire a theory of mind, and childhood memories gradually become blurred. If it weren't for psychologists doing experiments, people really wouldn't know that children's psychology is like this.

02

Does ChatGPT have a heart?

After looking at these two experiments, we will have a better idea when we look at this controversial paper. The large language model processes natural language, so scientists wrote the two tests just now into stories, input them into GPT-3.5, and then asked it questions. As a result, in the false belief experiment, GPT-3.5 was able to correctly answer 17 of the 20 questions; in the Sally-Anne test, GPT-3.5 was able to answer all 20 questions correctly. In order to prevent GPT-3.5 from guessing based on keywords, scientists also changed different expressions and asked questions repeatedly, and even deliberately added some logically confusing "lie detector questions." As a result, GPT-3.5 also passed the test and was still able to answer the correct questions, but was also confused when encountering logically confusing questions.

Figure 1: Stories and test scores input in false beliefs by ChatGPT based on GPT-3.5

Don’t underestimate this result. Previous artificial intelligence could not pass the theory of mind test. Not only that, scientists also tested 8 other artificial intelligences that have appeared recently, including previous versions of similar models. It turned out that only the GPT-3.5 version could achieve this result. The results of the GPT-3 version dropped significantly. Several artificial intelligences could not answer a single test question correctly. It can be said that the latest version of GPT-3.5 passed this theory of mind test and greatly surpassed its previous versions. Moreover, its accuracy rate is close to the results of a nine-year-old human child, which is why there are so many magical interpretations on the Internet .

Figure 1: Comparison of the accuracy of different artificial intelligences, GPT-3.5 is far ahead

Now that we understand the concept of theory of mind and have read the methods of the paper, let’s talk about the interpretations on the Internet. Which of these statements about a nine-year-old’s “theory of mind”, “empathy”, “mind”, and “intelligence” is more reliable?

"Theory of mind" is obviously the most accurate interpretation , and the ability of theory of mind is tested in the paper.

"Empathy" is a little bit biased . Psychologists believe that theory of mind and empathy are two similar abilities, but empathy is more focused on the experience of emotions and feelings. Large language models represented by ChatGPT do not have emotional capabilities, nor do they have what is commonly known as empathy.

The deviation of "mind" is even greater . In the field of psychology, mind is a "big word" that refers to people's thoughts, imagination, memory, motivation, feelings, etc. Scientists are still arguing about what the human mind is. Theory of mind and mind share the same words, but the meanings are very different. Obviously, the current large language model does not need to have a mind, but it has no problem passing the theory of mind test.

Finally, the interpretation of "intelligence" is also inaccurate . Scientists do not conduct intelligence tests. Intelligence and theory of mind are two completely different concepts .

03

ChatGPT has no mind, so isn’t it powerful?

It seems that although GPT-3.5 has made a series of breakthroughs, it should not be over-interpreted. There is a huge difference between the improvement of a particular ability and the mind of a nine-year-old child. Moreover, there are many debates about models such as ChatGPT. Yann LeCun, a well-known artificial intelligence expert, believes that ChatGPT has no essential breakthrough compared to artificial intelligence in previous years. Another well-known artificial intelligence expert, Douglas Hofstadter, is more radical. He denied the research routes of a series of artificial intelligence such as ChatGPT many years ago. He believes that this type of artificial intelligence uses mathematical models to obtain good test results, but no matter how it evolves, it is impossible to obtain a mind like a human .

There are many similar debates. Scholars from different fields and many investors are very concerned about how much progress the subsequent versions of GPT-3.5 can make and whether they can produce mind-like abilities. At least it is still unknown at present. However, I would like to remind everyone that there is another keyword in the title of this paper that is worth studying, that is, "spontaneous emergence". This word means that after many elements form a system, characteristics that were not originally there spontaneously appear. People often write artificial intelligence for specific tasks, such as artificial intelligence for playing chess, artificial intelligence for driving, and so on. The large language model represented by ChatGPT processes natural language. It was not designed to cope with the theory of mind test, but it has the ability to pass the test by language.

This result may be more important than how old a child is compared to ChatGPT. In the past, many AI experts hoped to understand the human brain and mind first, and then simulate it with machines. However, the human brain is too complex to be understood by the current human mind. However, the large language model tells us that even if we can't figure out how humans speak, we can still teach computers to speak. In fact, when computers learn to speak, they learn other skills by themselves.

In the eyes of many scientists, today's artificial intelligence is like a nine-year-old child. The future is really promising. No, it is really exciting.

References:

[1]Kosinski, M. (2023). Theory of Mind May Have Spontaneously Emerged in Large Language Models (arXiv:2302.02083). arXiv

[2]Premack, D., & Woodruff, G. (1978). Does the Chimpanzee Have a Theory of Mind? Behavioral and Brain Sciences, 4(4), 515–629.

[3]Gopnik, A., & Astington, JW (1988). Children's understanding of changes in their mental states. Child Development, 62, 98–110.

[4]Baron-Cohen, S., Leslie, AM, & Frith, U. (1985). Does the autistic child have a “theory of mind”. Cognition, 21(1), 37-46.

Author: Chen Chao, Master of Cognitive Neuroscience, Beijing Normal University

Review | Tang Yicheng, Deputy Director of Beijing Zhongke Popular Mental Health Promotion Center

Produced by: Science Popularization China-Creation and Cultivation (Starry Sky Project)

The cover image and the images in this article are from the copyright library

Reproduction of image content is not authorized