Why do foreigners have strange intonations when speaking Chinese?

Say it with me: "mā má mǎ mà ......"

Feeling stress-free?

But for foreigners, it is not that easy.

Image credit: Liang Pak Sun, illustrated by Jiang Xintong

There was a British teacher who was a high school teacher in China. He usually got along well with his students, but one day a group of overactive children were too noisy during a self-study class. The teacher who was watching the self-study class finally couldn't contain herself and stood up excitedly, slapping the desk:

"I don't know what you guys are making so much noise about!"

The students realized that the head teacher was angry, but they couldn't help laughing. The head teacher had no choice but to cover his mouth and laugh when he saw that the situation was out of control...

Image credit: Liang Pak Sun, illustrated by Jiang Xintong

In fact, he was an excellent student at the Oxford Faculty of Arts who was familiar with Chinese. At the first class meeting after enrollment, he took the roster and read out the names of his classmates one by one in a clear and correct voice, including the tones.

Before the school started, he called the parents of his students, but they didn't recognize his "foreign" accent. Later, they found that he had marked the pinyin and tones of each name in the roster. However, during the self-study class that day, the students were too noisy, and he was very angry and "revealed his true colors".

It is not just this teacher from the UK. There are many foreigners who can master several languages ​​but still cannot speak Chinese well. If you ask them what is the most difficult thing to learn in Chinese, they will tell you without hesitation - "tone".

Image credit: Liang Pak Sun, illustrated by Jiang Xintong

The difficulty of intonation is more difficult than...

Chinese is profound and extensive, and tones can be said to be one of the essences of Chinese. Another meaning of essence is difficult, just like the Goldbach conjecture is the crown jewel of mathematics, which took mathematicians so many years to learn, let alone the tones of our language? So, can we really blame the diligent and studious "foreigners" for the difficulty of learning tones and the inaccurate pronunciation of tones?

First, let's look at what intonation is. The speech we speak in daily life sounds very simple. We can record it with a tape recorder and draw a picture, that is, some waveforms:

Image source: Drawn by Liang Pak Sun

But there is something hidden in these waveforms. Please see, the bulging little bag in the picture below is a syllable. This is the Mandarin word "mama". The front and back parts of this "mama" represent /m/ and /a/ respectively.

Image source: Drawn by Liang Pak Sun

So where is the tone of "Ma" that has stumped us? In order to see the tone clearly, we need to perform a windowed Fourier transform on "Ma".

Image credit: Liang Pak Sun, illustrated by Jiang Xintong

Simply put, Fourier transform decomposes the sound signal into many simple harmonic waves of different frequencies (that is, they sound different pitches) - these waves can be superimposed to form the sound signal we hear.

This is like a large ensemble. The original waveform is like the ensemble itself, and the Fourier transform allows us to clearly see the various instruments that make up the ensemble, such as violins, cellos, flutes, etc.

Windowed Fourier transform refers to the Fourier transform performed after expanding the signal within a selected period of time, just like selecting a section of a piece of music to see how each instrument performs.

Let’s talk about frequency - let’s look at the picture below.

Image credit: Liang Pak Sun, illustrated by Jiang Xintong

A small part of the sound of "Ma" is decomposed into many simple harmonic waves of different frequencies. The frequency of the wave with the lowest frequency is called the "fundamental frequency" (F0), and each wave with a frequency that is an integer multiple of the fundamental frequency is called "harmonics". The pitch of this small sound is determined by the fundamental frequency.

Each segment can calculate a fundamental frequency value, which represents the pitch of this segment. As shown in the figure below, connecting the values ​​of each segment forms a curve representing the pitch change.

Image credit: Liang Pak Sun, illustrated by Jiang Xintong

What is tone? For a single syllable, tone is the pitch curve (pitch contour). Physically, tone is the fluctuation of fundamental frequency F0.

So why is it so difficult for foreigners to learn tones? Let's take a look at how the fundamental frequency (F0) of English and Mandarin changes.

This is English:

Image source: Drawn by Liang Pak Sun

This is in Mandarin:

Image source: Drawn by Liang Pak Sun

See the difference?

As a non-tonal language, the pitch of a sentence in English is basically smooth. It will only rise if it is a question tone.

But Mandarin is different. When individual syllables are combined into sentences, the pitch change of each syllable is the superposition of tone and intonation.

In other words, Mandarin has sentence-length pitch changes that indicate intonation, just like English, and there are also "small waves" superimposed on the "big waves" of the sentences - tones!

Image credit: Liang Pak Sun, illustrated by Jiang Xintong

Just like driving a car, speaking English is like driving on a flat road. Although there are ups and downs, they are all gentle ups and downs.

Mandarin is different. Not only do you have to go up and downhill, you also have to face the sudden and dense vibration of syllable length. It's like a driver who is used to driving on a flat road suddenly encounters a bumpy mountain road, which requires you to use your hands and feet to change gears and step on the brakes and accelerator. If your voice (brain) has not been trained in all kinds of skills since childhood, how can you cope with it quickly?

However, is it not just difficult to learn the tones?

Because tones have such a unique linguistic status, researchers have conducted a lot of research on them. For example, Mr. Zhao Yuanren, the father of Chinese linguistics, created a research method for marking tones:

Image credit: Liang Pak Sun, illustrated by Jiang Xintong

For example, we can create a series of "tones" between the first and second tones (which do not exist in the human world) so that people can judge whether it is the first or second tone after listening (category perception):

The first and second tones in our daily speech usually have a fixed range of fundamental frequency rise and fall - for example, the first tone is almost unchanged, and the second tone is between 100~200Hz. So, if we artificially create a tone with a change amplitude lower than the normal second tone change amplitude (such as 2~5 in the figure), it will make it difficult to judge the tone. (Image source: Liang Baishen, drawn by Jiang Xintong)

Or maybe some brains that are not afraid of jamming (or are old jammers but get braver with jamming) are trying to establish the relationship between the brain and intonation - how does our brain process intonation?

To figure this out is N times more difficult than for a foreigner to learn tones. However, some people still rushed to the front and took a few bites of this crab.

For example, twenty years ago, Gandour and other researchers asked Thai, Chinese, and English speakers to lie in an MRI machine (Gandour, Wong, & Hutchins, 1998). Each time, the researchers played them a pair of syllables that differed only in tone (e.g., /khaa/ and /khàa/), and they had to judge whether the tones of the syllables were the same.

Throughout the experiment, they will hear many pairs of syllables like this and make judgments. They hope to study the impact of native language background on the way the brain processes tones by recording and comparing the similarities and differences in brain areas activated by people from three native language backgrounds when judging tones.

Image credit: Liang Pak Sun, illustrated by Jiang Xintong

Since then, more and more brain imaging studies have been conducted on how the brain processes language—including no fewer than twenty studies on intonation.

So here comes the question.

Are all the results of so many studies on intonation the same?

Here we need to mention one of the cornerstones of the existence and development of science - the repeatability of conclusions. For example, Xiao Ming observed that the sun rises from the east today and proposed the "sun rises in the east theory". Xiao Hong also observed that the sun rises from the east the next day and repeated the verification of Xiao Ming's "sun rises in the east theory". If Xiao Ming's conclusions are repeatedly verified by independent observations, then what he discovered should be the truth.

However, when it comes to brain research, things are not that simple.

Image credit: Liang Pak Sun, illustrated by Jiang Xintong

Different people's brains are different: some are round, some are flat; different models of equipment also produce different results. Therefore, it is not practical to require that subsequent studies strictly replicate the results. However, if people lie in the scanner and do similar tasks, then their brain activation patterns should also be very similar.

In order to find the most likely activation points in the brain processing tone (points that are activated in different studies), we conducted a literature review (scientific name: meta-analysis) of existing brain imaging studies on tone processing (Liang & Du, 2018).

In this work, we also collated brain imaging studies of phonemes (such as vowels and consonants in English) and rhythm (such as question intonation), and compared the similarities and differences in the brain areas activated by them and intonation.

Image credit: Liang Pak Sun, illustrated by Jiang Xintong

In order to compare foreigners (people who don't speak tones) with people who speak tones, we divided the tone brain activation results into two categories according to the volunteers' native language background. And because rhythms can be long or short, we divided the rhythm processing tasks into two groups according to the length of the rhythm.

In other words, we collected five groups of studies: tonal tone perception in tonal speakers, non-tonal tone perception in tonal speakers, phoneme perception, word prosody perception, and sentence prosody perception. We meta-analyzed the brain activation results for each group separately.

The results showed that the brain processes intonation in terms of what it sounds like (acoustic analysis), what it sounds like when spoken (articulatory simulation), and what language function it has.

Activation result diagram of meta-analysis: red represents the tone perception of tone native speakers, blue represents the tone perception of non-tone native speakers, green represents phoneme perception, purple and yellow represent the rhythmic perception of word length and sentence length respectively (Image source: Liang & Du, 2018).

Specifically, the activation area for tones was more biased towards the right side in the auditory cortex (the brain area near the ear), and only native tone speakers showed activation on the left side when processing tones, suggesting that the role of determining meaning gives tones more language functions in the brains of native tone speakers (because the language area is biased towards the left brain).

In the left auditory cortex, the tone activation area overlaps with the phoneme activation area and is located in front of the sentence length rhythm, which also reflects the linguistic function of tone. In the right auditory cortex, the tone activation area is located behind the phoneme and in front of the rhythm, which further reflects the acoustic property that tone length is sandwiched between the phoneme and the rhythm.

Another, more mysterious area of ​​activation was in the area responsible for speech. We found activation for phonemes, intonation, and rhythm in the left motor cortex.

Moreover, the tone and rhythm (both controlled by the larynx) overlap and are located below the phonemes (controlled by the lips and tongue), which is consistent with the topological distribution of the brain's motor cortex (different small areas of the motor cortex are responsible for conveying motor commands to different areas of the body, and are drawn in human form according to the proportion of the area, which is the "motor homunculus", as shown in Figure 1 below). The involvement of the pronunciation system is actually a unique way for people to perceive speech. Listeners will assist in speech understanding by reconstructing and predicting the speaker's pronunciation movements (as shown in Figure 2 below).

Figure 1 (Photo credit: Liang Boshen, drawn by Jiang Xintong)

Figure 2: Brain mechanism model of tone perception: (A) Spectrogram and pitch contour of the Mandarin “Eh? Are you eating?”; (B) Ventral stream (in the auditory cortex, acoustic analysis and semantic recognition of tones) and dorsal stream (in the articulatory motor area, articulatory motor simulation of tones) of tone perception (Image source: Liang & Du, 2018)

All in all, this literature review has helped us better understand how the brain processes intonation. However, there is still a long way to go.

For example, how can we make it less painful for foreigners to learn Mandarin? It is like the Big Dipper twinkling in the distant night sky, indicating the way forward for us.

Image credit: Liang Pak Sun, illustrated by Jiang Xintong

References:

1.Du, Y., Buchsbaum, BR, Grady, CL, & Alain, C. (2014). Noise differentially impacts phoneme representations in the auditory and speech motor systems. Proceedings of the National Academy of Sciences, 111(19), 7126–7131. https://doi.org/10.1073/pnas.1318738111

2.Du, Y., Buchsbaum, BR, Grady, CL, & Alain, C. (2016). Increased activity in frontal motor cortex compensates impaired speech perception in older adults. Nature Communications, 7, 12241. https://doi.org/10.1038/ncomms12241

3.Gandour, J., Wong, D., & Hutchins, G. (1998). Pitch processing in the human brain is influenced by language experience. NeuroReport, 9(9), 2115–2119. https://doi.org/10.1097/00001756-199806220-00038

4. Liang, B., & Du, Y. (2018). The functional neuroanatomy of lexical tone perception: An activation likelihood estimation meta-analysis. Frontiers in Neuroscience, 12, 495. https://doi.org/https://doi.org/10.3389/fnins.2018.00495