A congenitally blind man can "see" the flying ball without using his eyes

Produced by: Science Popularization China

Produced by: Zhang Shuyu

Producer: Computer Network Information Center, Chinese Academy of Sciences

Can you still see after your eyes have been irreversibly damaged?

Neuroscience has discovered that the brain has a special talent that allows people born blind to learn to "see things" without relying on their eyes.

Image source: veer gallery

Lying in a "massage chair", a congenitally blind person can also "see" the flying ball

In 1969, Nature published a science fiction paper that described in detail a very strange instrument that could enable blind people to "see" by "massaging" their backs.

Let's restore the experimental scenario:

A man who was born blind and had no visual experience was lying in a dental chair. Next to him was an old-fashioned camera with a zoom lens.

The therapist moves the camera by hand to scan the scene in front of the blind person. The generated image is transmitted to the instrument behind him, and the instrument transmits the processed visual signal to the stimulation point matrix on the back of the treatment chair.

Specifically, the matrix stimulation points corresponding to the weak light parts of the scene will vibrate, while the strong light parts will not vibrate. These contact points will directly stimulate the skin on the back of the blind person, just like a massage chair.

The "vision aid" invented by Bach-Rita. The position of the lens in the picture simulates the perspective of the human eye, which enables the blind to perceive objects in front of them (Image source: Reference 2)

After 20-40 hours of intensive training, something magical happened - the blind people were not only able to distinguish different lines such as vertical lines, oblique lines and curves, but also recognize common geometric shapes such as circles, rectangles and triangles.

After learning to operate the camera, the blind person can aim at different locations in the room through the zoom lens to identify various objects such as telephones, chairs, cups, etc. (even if they are partially obscured) and describe their positional relationships.

Gradually, they began to perceive the three-dimensional space in front of them:

The distance of an object can cause the size of the image to change;

When you look at an object from different perspectives, its shape will be distorted;

The backlit side of an object will cast a shadow, etc. If someone throws a ball towards the camera, the blind person will naturally avoid it.

With the help of this "tactile-vision" device, blind people have even learned to recognize faces (such as those of supermodels).

Even more incredible is that they can also "observe" changes in the appearance and behavior of the characters.

For example, they describe a woman: "She has her long hair down today and is not wearing glasses. She is moving her right hand behind her head."

The vibration pattern of the stimulus dot matrix projects a two-dimensional image on the monitoring oscilloscope.

The image shows a human face. After long-term training, blind people can learn to recognize tactile patterns of similar complexity (Image source: Reference 2)

Why can the brain "see" by stimulating the back?

Paul Bach-y-Rita, the main person in charge of this study, has conducted detailed observations and studies on blind people who use canes. When walking, blind people sweep their canes back and forth, and the tip of the cane tells the blind people about road conditions through the tactile receptors on their skin.

Bach-Rita was deeply inspired: the cane can be seen as an "interface" between the blind and objects. Through the pressure and touch of the cane on the hand, spatial information such as the layout of the room is formed.

The skin on the hand and its touch receptors are thus like an information collection station. They can serve as a replacement for the retina, allowing images to be formed in the brain.

In a similar way, the "massage chair" allows the blind to "see". Simply put, it is the brain that sees, not the eyes.

Image source: Pixabay

The Adaptable Brain

The results of Bach-Rita's prospective experiment confirmed the theory of "sensory substitution".

Specifically, this refers to the important neural pathways responsible for visual function. Once they are broken or blocked, the brain will take a detour.

The neural pathways that control tactile sensations, which were rarely used in visual perception, can now be used as substitutes to play the role of "seeing things". It seems that the brain also understands that "all roads lead to Rome".

In fact, the brain is like a decoder in the dark skull. When various sensory information from the outside world is transmitted, whether it is photons, air compression waves, molecular concentration, pressure, texture or temperature, it will be uniformly converted into the universal language of the brain: electrochemical signals.

It is the biochemical reactions in this dark theater that form all our perceptions of reality.

Image source: Pixabay

Even if the perceived signals come from unusual sensory nerve pathways, the brain will rise to the challenge and reorganize sensory perception by constantly learning and understanding new signals.

This is thanks to millions of years of biological evolution, which has made the brain a "master" that can adapt to changes. The super learning ability that can turn decay into magic is derived from the flexible plasticity of human brain nerves.

Bach-Rita is a pioneer in applying the plasticity of brain neurons to rehabilitation medicine.

After the "massage chair", some more modern designs have emerged in the world. For example, by transmitting sound to the ears, or using small vibrations to stimulate the forehead or tongue, visual information is transmitted to the brain. You can "see" without using your eyes. At first glance, this seems like a special ability. But if you think about it carefully, it is also the result of the normal functioning of the brain.

Coincidentally, hearing does not have to rely on the ears.

Alternative ways to "listen" to the world

Neuroscientist David Eagleman and his team have carefully designed a hearing aid for the hearing-impaired - the Variable Extra-Sensory Transducer (VEST, commonly known as the "vest").

The "vest" has its own microphone, which can perform real-time perceptual coding of external sounds.

The coded information is then mapped to some tiny vibration motors on the "vest". The motors can activate dynamic vibration patterns based on the frequency of the sound and then transmit them to the entire torso.

How the vest works (Image source: eagleman.com)

This wearable technology can also run on mobile devices such as phones and tablets. Once the device captures the sound in the environment, it will map the signal to the motor via Bluetooth.

The working principle of the “vest” under the conditions of mobile equipment (top); the front and back of the “vest” (middle); Eagleman shows the wearing effect of the “vest” (bottom) (Image source: Reference 3)

Adapting to the vibration signals from the vest is like learning a new language. At first, these foreign signals are elusive. But after enough training, the brain will cross-check the different touches on the back and gradually learn to extract patterns from them, converting the language of the vest into understandable information. For example, the brain can match a word with a specific vibration pattern.

Experiments show that by wearing the "vest" for two hours a day, hearing-impaired people can correctly spell words spoken by others in less than a week.

Eagleman said that this patented technology can not only save patients from the invasiveness of cochlear implant surgery, but also allow them to gain direct auditory experience, which is similar to the blind understanding the meaning of Braille by touching it.

The story is not over yet, as the engineers came up with another brilliant idea! They condensed the core technology of the "vest" into a small bracelet and gave it a vivid name - "Buzz", which means "buzzing".

The wristband's interface is equipped with a power switch, user settings buttons, a microphone, and a microcontroller. (Image source: Neosensory.com)

Although the bracelet is small, it is fully functional. It is equipped with a microphone to capture ambient sound, four vibration motors and a sophisticated signal processing system. Like a "vest", the system can encode external signals through a "sound-to-touch" algorithm and convert them into dynamic vibration patterns output by the motor.

The sound signal captured by the microphone is processed by the microcontroller and converted into different vibration patterns output by the four motors. Each vibration point is a rectangular area of ​​8.2mmX8.5mm (Image source: Reference 1)

Can a bracelet with only four vibration stimulation points transmit enough tactile signals?

The research team monitored 18 patients with severe hearing loss for one month. During this period, the patients wore the bracelet for at least 4 hours a day in their daily lives. From the first day of wearing the bracelet, the patients underwent a round of testing every two weeks.

The data showed that patients were able to learn to distinguish between the different vibration patterns generated by the audio of words, and they were also able to recognize vibration patterns from similar but different words.

Further research found that patients can also learn to recognize the sounds of daily life. Their learning materials include 14 categories such as babies crying, car horns, alarm clocks and clapping.

It is gratifying that their recognition ability improves with the number of wearing days.

Image source: Pixabay

It turns out that as the wearer gets used to the bracelet's functions, they gradually become better able to perceive the sonic world around them.

The beautiful story continues. I believe the bracelet can bring hope to more hearing-impaired people.

This learning ability unique to the human brain, which can turn decay into magic, brings entirely new possibilities to limited sensory perception.

References:

1. Perrotta, MV, Asgeirsdottir, T., & Eagleman, DM. (2021). Deciphering sounds through patterns of vibration on the skin. Neuroscience (4).

2. Bach-Y-Rita, P., Collins, CC, Saunders, FA, White, B., & Scadden, L. (1969). Vision substitution by tactile image projection. Nature, 221, 963–964.

3. Novich, SD, & Eagleman, DM. (2014). A vibrotactile sensory substitution device for the deaf and profoundly hearing impaired. 2014 IEEE Haptics Symposium (HAPTICS). IEEE.

4. Norman Deutsch, Reshape Your Brain, Reshape Your Life (2015), China Machine Press

5. The Story of the Brain by David Eagleman (2019) Zhejiang Education Publishing House

6. Can we create new senses for humans? | TED

7. https://www.ted.com/talks/david_eagleman_can_we_create_new_senses_for_humans/transcript?language=zh-cn