Shengka launches the first AI dubbing production platform for audio novels, bringing a new immersive reading experience

Romain Rolland once said: "Live with books and you will never sigh." The love of reading is also an excellent tradition passed down from generation to generation among the Chinese people. At present, the country is also continuously promoting reading among all the people.

In recent years, with the rapid development of the Internet and new media, people's ways of acquiring knowledge have been unprecedentedly broadened, and the way of reading has also changed greatly. According to the "2020 China Digital Reading Report" released by the 7th China Digital Reading Conference, the number of digital reading users nationwide has exceeded 494 million, among which the average number of audiobooks read per person has reached 6.3, which is particularly rapid.

Audiobooks have the advantages of fast dissemination, convenient reading, and a wide readership. But on the other hand, the balance between production costs and user experience is still difficult to control, resulting in insufficient production capacity of high-quality audiobook content, which restricts the development of the industry.

At present, audiobook content production mainly includes two methods: manual recording and machine generation. Real-person reading has great advantages in artistic expression, but the production cost is very high, which is as high as 30 yuan per minute on some platforms. If the recorded work is long, the cost may be as high as hundreds of thousands of yuan.

Generating audiobook content through machines can reduce production costs by about 90%, which is a more efficient and cost-effective way. In addition, with the continuous maturity of technologies such as speech synthesis, the reading voice generated by machines is very close to the effect of human voice expression, so it has been widely used in the production of popular science audiobooks such as humanities knowledge, science and technology.

As for literary and expressive novels, the potential of machine-generated speech has yet to be explored. On the one hand, such works require higher expressiveness of generated speech, and on the other hand, there are often many characters in novels, and they need to be distinguished, which also requires advanced AI technology.

In this context, Tencent PCG AI Interaction Department's audio and video creation platform "Shengka" has relied on its deep AI technology accumulation to launch the first AI production function for audiobook dubbing. This function is currently free for a limited time, and one person can complete the production of an entire audiobook. After entering the text, the dubbing can be generated by AI, which greatly reduces the cost of audiobook production and greatly improves efficiency.

In addition, this feature also allows all users to create immersive audiobooks for free according to their preferences, thereby meeting more diverse audiobook needs.

The usage of Shengka is very simple. Just import text in common formats such as txt and doc, select the corresponding AI dubbing, and you can start audio reading. No matter which AI dubbing you choose, if you don't pay special attention, you might really think it's the effect of a real person reading.

  AI dubbing, voice style, dialect, etc. can be changed at any time

Of course, due to the rich expressiveness of Chinese, the effects generated entirely by AI are bound to have some flaws. For example, in the text we tested, there was a pause in the last two characters of the word "mao qian peng yu", and the "dai" in "wait for the next person to note the origin" was also pronounced as one tone. Shengka has made good optimizations for these problems, and users can easily adjust the audio through functions such as phrase chaining and polyphonic characters. In addition, functions such as inserting pauses, local speed changes, and word pronunciations make the effect more vivid and detailed.

  After selecting any area, you can fine-tune the reading effect.

For novels with many characters, Shengka also has a very unique audio novel creation function. After uploading the novel text, the system will automatically identify the characters in the novel through the NER algorithm, and then automatically divide the chapters through "regular expressions". Its recognition speed is also very fast. A million-word novel like "The Count of Monte Cristo" takes less than 30 seconds. After that, users can choose AI dubbing for it based on their understanding of the character. Shengka uses cross-speaker style transfer technology, allowing the same AI dubbing actor to interpret different emotions and even dialects.

Each AI voice actor is marked with the appropriate work style

After entering the editing interface, the layout of the chapters on the left and the characters on the right is clear at a glance. If a character has multiple names (such as Dantes, Edmond and the Count of Monte Cristo in this book), or repeated recognition due to the way of expression (such as Mr. Tengral and Tengral in the picture below), users can also easily and quickly select the same AI dubbing for it. In addition, if there are lines spoken by some unnamed characters, you can also manually add the character or select a single sentence to add a dubbing specifically.

The character recognition accuracy is very high, and all the characters that appear are basically covered.

Take the classic "Lin Daiyu Enters Jia Mansion" in "Dream of Red Mansions" as an example. We selected three AI dubbing voices for the three main characters, Daiyu, Jia Mu, and Wang Xifeng, respectively: gentle, mature, and kind, and the narration chose a relatively deep male voice. Among them, Wang Xifeng is known for her quick thinking and fluent speech, so we accelerated some of her lines. In this way, the text, which is already very expressive, becomes more vivid and impressive with the blessing of voice.

Each character's lines are highlighted to facilitate adjustment of individual sentences

Many novels have dozens or even hundreds of characters. It would be time-consuming and labor-intensive to manually select matching voices. Shengka's audio novel function can quickly distinguish different characters and deepen the user's impression of each character through different dubbing. This also reflects the innovation of AI technology in user reading experience and reading effect.

As lifestyles change, consumers' reading methods and reading scenarios have become more diversified, and digital reading has shown increasingly strong potential. Among them, audiobooks, which are more convenient and more emotional, have the broadest development prospects.

The new generation of information technology represented by big data, 5G, and AI is developing rapidly, and its application scenarios are constantly expanding, thus promoting the transformation, upgrading, and integration of all walks of life. If the audiobook industry wants to develop, it must also be empowered by technology. Shengka is an excellent example of technology empowering the industry. With the addition of AI technology, the production cost of audiobooks has been greatly reduced, and it is also convenient for mass production of content, which helps to quickly improve the economic benefits of the audiobook industry in a short period of time and form a scale effect.

For content creators, whether it is re-creation of famous works or "audioization" of their own works, Shengka provides an innovative solution. The content produced in this way not only meets the current public demand for audio novels, but is also more suitable for promotion to special groups such as the elderly, teenagers, and the visually impaired, improving their reading status.

Li Dongdong, former deputy director of the General Administration of Press and Publication, said: "In the face of new trends in digital development, we must vigorously promote the development of digital reading, establish a digital resource platform for national reading, and promote digital reading services." Tencent PCG AI Interaction Department's Shengka is an excellent application case of "reading + technology", which allows users to obtain professional and high-quality reading content anytime, anywhere, and at their leisure. This will definitely play a very positive role in promoting national reading.

As a winner of Toutiao's Qingyun Plan and Baijiahao's Bai+ Plan, the 2019 Baidu Digital Author of the Year, the Baijiahao's Most Popular Author in the Technology Field, the 2019 Sogou Technology and Culture Author, and the 2021 Baijiahao Quarterly Influential Creator, he has won many awards, including the 2013 Sohu Best Industry Media Person, the 2015 China New Media Entrepreneurship Competition Beijing Third Place, the 2015 Guangmang Experience Award, the 2015 China New Media Entrepreneurship Competition Finals Third Place, and the 2018 Baidu Dynamic Annual Powerful Celebrity.