Guess what? It’s not mind reading, but predictive processing of the brain

Produced by: Science Popularization China

Author: Li Xiaoqing Research Group (Key Laboratory of Behavioral Sciences, Institute of Psychology, Chinese Academy of Sciences)

Producer: China Science Expo

In daily language communication, people often predict what the other person is about to say. For example, when a friend says to you, "I had soy milk and..." for breakfast, before he finishes speaking, you have already predicted that what he is about to say is "dough dough sticks" based on the information such as "soy milk".

If he really said "youtiao", you would understand it faster and easier than if he said "mantou".

This kind of predictive processing is common in the process of language comprehension and is one of the important reasons why the human brain can achieve efficient language comprehension . However, previous scientific research has mainly focused on how people process the target information after the predicted target information (the "dough sticks" in the example) appears. However, a more worthy and challenging research question is: How is this prediction generated?

Working Memory: A Temporary Memory Workstation

In his book Thinking, Fast and Slow, Daniel Kahneman, Nobel Prize winner in Economics and psychologist, systematically explained the two systems involved in the thinking process: System 1 is the "fast system", which is automatic and unconscious ; System 2 is the "slow system", which requires cognitive effort, is logical and conscious .

He believes that the associative processing between concepts such as hearing soy milk and thinking of fried dough sticks depends on System 1. So, can the prediction process in language comprehension be completed by relying only on System 1? To answer this question, let us first understand "working memory".

Working memory refers to a memory system that temporarily stores and processes information . Because it is a temporary "workstation", the information capacity that can be stored and processed is limited.

Working memory capacity is affected by many factors, including age. It increases during childhood and adolescence, and gradually decreases in old age.

In 1956, Miller proposed that the capacity of working memory is 7±2 "chunks" , that is, 5-9 chunks of information are generally stored in working memory. For different people, the content of each "chunk" is different, which leads to great individual differences in the amount of information that can be stored and processed in working memory.

Working memory capacity (Image source: author)

For example, it is difficult for people to remember a string of 18 digits, but if this number is an ID number, it will be easier to remember after dividing it into several "blocks" such as province, city, district (county) code, and date of birth.

What is the prediction process in language comprehension?

Let’s go back to the prediction process in language comprehension. Given the limited capacity of working memory, if we only rely on the automated System 1, then the prediction process will not be affected by the capacity of working memory. If System 2 is also involved in prediction processing, then individuals with different working memory capacities will also perform differently in prediction processing.

In order to verify different hypotheses, Li Xiaoqing's research group at the Institute of Psychology, Chinese Academy of Sciences recently conducted an EEG study.

Example of EEG experiment process (Image source: Pixabay)

The researchers recruited 160 college students and screened out two groups through a working memory capacity test, one group containing 24 individuals with high working memory capacity and the other group containing 24 individuals with low working memory capacity.

Two groups of college students read and comprehended a series of sentences presented on a computer screen. The sentences were of three types: highly restrictive, moderately restrictive, and less restrictive.

The limitation here refers to the probability that people can predict the same word based on the information in the sentence context.

For example, if 9 out of 10 people can predict that the word "hammer" will appear in the context "In order to drive the nails into the board, he bought...", then we say that the restrictiveness of this context is 90%.

Accordingly, the predicted noun "hammer" has a prediction probability of 90%. Similarly, "hammer" has a medium prediction probability in the medium-restrictive context "In order to crack the hard walnut, he bought..."; "books" has a low prediction probability in the low-restrictive context "In order to complete this task well, he bought..."

Brain wave concept map (Image source: veer library)

The study recorded the brain waves of individuals while reading sentences, and mainly analyzed the amplitude of brain waves. In simple terms, the larger the amplitude of brain waves, the more difficult cognitive processing is.

Through research, researchers have made the following discoveries:

The first finding was that when reading the verb before the target noun (such as "买" in the example), college students had larger EEG waves when reading "买" in a high-restrictive context compared to "买" in a low-restrictive context. This phenomenon was only observed in the high working memory capacity group.

The researchers further analyzed through mathematical models and found that the EEG amplitude at the verb in a highly restrictive context is positively correlated with the predicted probability of the subsequent target noun, that is, the greater the predicted probability of the noun, the greater the EEG amplitude caused by the verb.

For example, the prediction probability of "cake" in the highly restrictive context "To celebrate her mother's birthday, she bought..." is 100%, so the brain waves induced by "bought" in this context will be larger than those induced by "bought" in the previous highly restrictive example, because the prediction probability of "hammer" is 90%.

This result suggests that people with high working memory capacity are now working hard to activate the nouns that are about to appear and store them in working memory.

The second finding is that after the target nouns, such as "hammer" and "book" in the example, appeared, the EEG amplitude evoked by the nouns in the high working memory capacity group gradually decreased as the prediction probability of the nouns increased, that is, "hammer" in a high-restrictive context < "hammer" in a medium-restrictive context < "book" in a low-restrictive context, while the low working memory capacity group only showed the EEG effect of "high restrictive < low restrictive".

This suggests that highly predictive nouns have better semantic understanding effects because they are the same as nouns stored in working memory, and that individuals with high working memory capacity have certain language comprehension advantages.

The researchers further analyzed the mathematical model and found that the greater the difference in the predicted probability of the target noun, the greater the difference in the induced EEG wave amplitude, indicating that the EEG wave amplitude induced by the target noun under different conditions is indeed related to its predicted probability.

EEG waves at the Cz electrode point during the prediction generation (verb position) and prediction integration (noun position) phases of individuals with high working memory capacity (left) and low working memory capacity (right) (Image source: Reference [1], adapted)

In summary, the differences in predictive language comprehension shown by individuals with different working memory capacities indicate that predictive processing in language comprehension can indeed promote people's understanding process, but prediction itself is a meaning-computation construction process that requires cognitive effort , and does not simply stem from the automatic associative activation between concepts as the example given by Daniel Kahneman.

We can call it the "duck paddling" phenomenon in language comprehension. The duck on the water looks relaxed and comfortable, but in fact its webbed feet never stop moving underwater. It seems that we understand language easily and quickly, but in fact our brain is constantly performing predictive processing.

Ducks paddling (Image source: Pixabay)

How to improve the brain's information capacity and predictive ability?

Since this study found that working memory capacity is closely related to semantic predictive processing, is it possible to promote language comprehension by improving working memory capacity?

Future research can further investigate this question. But before answering this question, it is necessary to clarify whether working memory capacity can be improved through training.

Existing scientific research has given a positive answer. Researchers divide working memory into domain-specific working memory and domain-general working memory .

The former refers to working memory related to specific information types , such as verbal working memory, digital working memory, etc. The latter is not limited to specific information types and can be called a control system , including processes such as controlling attention, controlling the flow of information in and out of working memory, and controlling the interference of irrelevant information.

Various Chinese characters (Image source: veer gallery)

There are different training methods for specific and general working memory.

Domain-specific working memory generally adopts strategic training . For example, through repeated practice, the amount of information contained in each "chunk" can be increased. For example, in the "ID card" example, multiple digital "chunks" can be put together to increase the digital capacity stored in working memory.

Domain-general working memory uses core training , which usually includes multi-modal tasks or stimuli, sets strong interference, or requires high-intensity cognitive participation, etc. For example, general working memory ability can be improved through attention training, cognitive training and other methods.

Illustration of artificial intelligence (Picture from Pixabay)

Conclusion

Now we have a general understanding of the cognitive mechanism of predictive processing in language comprehension, that is, language comprehension is not as simple and easy as it seems.

At present, various large language models such as ChatGPT have been put into use on the market. Although these models can quickly conduct human-computer interaction and can simulate the human language comprehension process to a large extent, the algorithms behind them still lack the support of human cognitive neural mechanisms.

If future research can further reveal the neural mechanisms of predictive processing in language comprehension, it will surely play an important role in promoting the development of general artificial intelligence.

References:

[1]Ding, J., Zhang, Y., Liang, P., & Li, X. (2023). Modulation of working memory capacity on predictive processing during language comprehension. Language, Cognition and Neuroscience.

[2]Ding, J., Wang, L., & Yang, Y. (2020). The influence of emotional words on predictive processing during sentence comprehension. Language, Cognition and Neuroscience, 35(2): 151–162.

[3]Zheng, Y., Zhao, Z., Yang, X., & Li, X*. (2021). The impact of musical expertise on anticipatory semantic processing during online speech comprehension: An electroencephalography study. Brain and Language, 221, 105006.

[4]Zheng, Y., Gao, P., & Li, X*. (2023). The modulating effect of musical expertise on lexical-semantic prediction in speech-in-noise comprehension: Evidence from an EEG study. Psychophysiology.