Can people speak without vocal cords? A new AI-assisted pronunciation system is born!

With the help of artificial intelligence (AI), a new flexible patch is expected to help some patients with voice disorders achieve effective communication .

Jun Chen, an assistant professor in the Department of Bioengineering at the University of California, Los Angeles, and his team have developed a "flexible patch" voice-aiding device that can convert laryngeal muscle movements into speech with the assistance of machine learning.

The device is based on the magnetoelastic effect and enables patients with vocal cord disorders to generate speech signals through muscle movement, thereby assisting their speech function.

Specifically, this voice-aiding device first collects data on the patient's laryngeal muscle movements, then uses a machine learning algorithm to analyze and classify these signals, and finally identifies semantic information and selects corresponding speech signals for output .

Figure ｜ Schematic diagram of the wearable flexible patch attached to the throat.

According to reports, this sound-enhancing device uses magnetoelastic materials, which are light, highly stretchable, and have a high signal-to-noise ratio, ensuring the comfort and wearability of the device. In addition, this sound-enhancing device has also achieved self-power supply .

The research team tested eight subjects without voice disorders and asked them to say and whisper words and phrases such as "Merry Christmas" or "I love you" while standing, walking and running.

The results showed that the accuracy of the sound-aiding device can reach 95%.

The related research paper, titled "Speaking without vocal folds using a machine-learning-assisted wearable sensing-actuation system", has just been published in the scientific journal Nature Communications.

The research team believes that the voice-aiding device uses a new technology that does not rely on traditional vocal cords , providing a new way of communication for patients with speech disorders.

After further testing in patients, the voice-aiding device may enable individuals with voice impairments to communicate more easily, thereby improving their overall quality of life .

Flexible patch that makes sound

Speaking is the most important part of interpersonal communication, but for people with vocal cord dysfunction, speaking is a difficult task. About 30% of people experience voice disorders at least once in their lifetime.

Existing solutions, such as handheld electronic laryngeal devices or surgery, more or less affect daily life and are uncomfortable to use. Therefore, we urgently need a wearable, non-implantable medical device that can assist patients in communication to improve the quality of life of patients with vocal cord dysfunction.

In this study, the flexible magnetoelastic patch designed by Chen Jun's team can not only fit comfortably on the patient's throat, but can also change shape with the laryngeal muscles involved in speaking movements, and does not require functioning vocal cords.

At the same time, the movement of the patch can detect specific muscle movements and generate electricity, making the sound-enhancing device self-powered. These movements are then converted into electrical signals and processed by a machine learning algorithm that can recognize words and translate them into language signals.

Figure | Magnetoelastic layer structural parameters. Side length is 30 mm. 12 semicircular units, diameter 2.16 mm, in a serpentine copper coil, span 25.92 mm. Scale bar is 10 mm.

According to the paper, the whole process includes four steps: signal acquisition, feature extraction and compression, machine learning classification, and sound output .

Specifically, the sound-aiding device utilizes the interaction between magnetic materials and magnetic powder in a soft matter system, which can cause deformation or movement of the material, thereby realizing signal collection of laryngeal muscle movement.

Subsequently, the collected signals are subjected to feature extraction and compression, and principal component analysis (PCA) is used to compress each speech signal into an N-order matrix to reduce data redundancy and prepare for subsequent classification.

The feature-extracted and compressed signal is then fed into a machine learning algorithm for classification. Once the machine learning algorithm has classified the signal, the device selects the corresponding sound signal for output.

Finally, these pre-recorded sound signals are played out through the driving part of the sound-assisting device, thereby realizing the output of corresponding voice signals to help the user to speak assistedly.

Figure｜Silent wearable voice device assisted by machine learning

The test results show that even after 40 minutes of continuous operation, the sound pressure and temperature of the sound-aiding device did not show a significant drop or increase, which shows the durability and safety of the sound-aiding device in terms of sound output.

In the sweat resistance test, the research team used artificial sweat to simulate the real usage environment. The results showed that even in the presence of sweat, the performance of the sound-enhancing device was very stable, with no obvious signal attenuation.

The research team tested the sound output of the voice-assisting device at different normal conversation angles. The results showed that the voice-assisting device exhibited reliable sound performance at various angles, which enables it to provide assisted speech in a variety of real-life scenarios.

However, this type of voice-aiding device still has some limitations if it is to truly help patients with vocal cord disorders.

For example, the experimental sample size used in the study was relatively small and only included participants of a certain age and gender, which may not fully represent the entire population. Future research could expand the sample size and scope to include a wider population.

In addition, artificial sweat was used in the study to simulate the sweat environment, but this environment is still different from the real physiological environment. Future research can be closer to the real physiological environment to better evaluate the performance of the sound-enhancing device in actual use.

Wearable devices full of imagination

In recent years, the development of wearable devices has been constantly breaking through human imagination. For example, a washable hat can help the visually impaired perceive the changes in traffic lights, or a piece of clothing can act as a guide when the wearer browses a museum. The relevant research was published in Nature last month.

Last month, a study also published in Nature Communications introduced a wearable emotion recognition system that can accurately identify human emotions by simultaneously sensing the signals generated by facial and vocal expressions through specially designed sensor units based on the self-powered triboelectric principle.

Furthermore, a paper published in Science last year by a research team from Nankai University introduced a "winter heating magic tool" that achieves two-way temperature regulation powered by solar energy throughout the day.

Similarly, wearable devices can also be transformed into an interactive screen that can be deformed at will. The screen's display content and presentation form can change in real time according to your instant interactive content, and even give you an immersive sensory experience.

In the future, when imagination catches up with technological development, these wearable devices will make human life convenient and better in various senses.

Reference links: https://www.nature.com/articles/s41467-024-45915-7 https://samueli.ucla.edu/people/jun-chen/