From The Mandalorian to iQiyi’s The Bad Well, virtual production is integrated into the film and television industry

Recently, many LED manufacturers have announced that they will increase their investment in the XR virtual production market. On March 17, Unilumin Technology’s official Weibo account announced that it had built the largest XR virtual production stage in Australia and even in the Southern Hemisphere, The TDC Studio, for the famous Australian entertainment company Technical Direction Company. Aoto Electronics also announced that it had reached a strategic cooperation with an Indian film production and broadcasting company to provide its latest investment virtual studio with RM2.3 LED display screens to create a seamless large-scale immersive digital world.

Regarding the XR virtual shooting related market, relevant data show that the global XR film and television shooting related market size in 2021 was US$3.2 billion.

Behind the rising popularity of the LED screen market is the rise of a new video production method driven by virtual shooting. I believe that friends who watched the opening and closing ceremonies of the Winter Olympics still remember the big snowflakes rising from the ground and countless small snowflakes gathering together, as if they were in a world of ice and snow.

The Winter Olympics can achieve this visual effect thanks to XR virtual production technology, motion capture, and multiple huge LED screens. This method can better meet the design imagination of the creative team and is being widely used in stage performances and other content.

In fact, this more advanced and efficient production method has been adopted by some video content producers since last year, a typical example being iQiyi.

Laying the foundation for "next generation entertainment"

iQiyi's use of virtual production technology and LED screens is at the leading level in China, both in terms of starting time and proficiency.

Last March, iQIYI held the world's first film-level LED realistic virtual production XR live concert "City of Virtual Reality", allowing THE9 to "travel" through different scenes such as cities, forests, and the Sphinx. The entire concert had more than 15 scene changes, and each song had a background that matched its theme, making the performance look more like a Hollywood blockbuster.

In order to give the audience a more immersive experience, iQiyi used two solutions, realistic XR and stylized XR, and set up multiple virtual positions to render different perspectives. The background of the concert was rendered on the LED screen in real time. Whenever the camera position changed, the picture on the LED screen needed to calculate the new position in real time, and all this happened during the live broadcast, which was undoubtedly more technically challenging and demonstrated the strength of iQiyi's virtual production technology.

It can be said that this concert greatly unleashed the imagination of the creative team, turning the scenes imagined in the past into reality, thus creating the prototype of "next-generation entertainment" for the audience. In June 2021, iQiyi held an online product launch conference for the Qiyu 3 VR glasses XR. Subsequently, iQiyi applied this technology to film and television production.

In November 2021, iQIYI conducted a high-precision digital scan of the "Bad Well" set in the hit self-produced drama "The Wind Rises in Luoyang", and used the scanned model as the scene to shoot the first domestic 4K film-level virtual production sample "Bad Well: The Storm Rises Again". Subsequently, iQIYI produced the 4K MV "Heart Thoughts" of the virtual idol Xiao Moli in the virtual "Bad Well". This is also the first time in China that a producer has run through the complete production process of "real scene scanning - asset reconstruction - virtual shooting - refined storage - multi-business reuse".

From these two examples, we can see that iQiyi has reached a fairly sophisticated level in virtual production based on LED screens, and this can bring iQiyi far more than just improved visual effects.

Unleash your imagination and create digital assets

For many years, the film industry has been emphasizing "film and television industrialization", and to truly achieve this, it is inseparable from the use of intelligent tools and the optimization of production processes. Virtual production is a technology that can kill two birds with one stone.

In the past, film and television shooting often used green screen backgrounds, but this method relied heavily on the actors' "imagination", which might compromise the quality of the performance. After using LED screens, the shooting team can interact with the LED screens in a built 3D environment, and the actors can interact with the virtual scenes, and the performances can be more natural, thus achieving the director's ideal effect.

In addition, the addition of LED screens can greatly improve production efficiency. When shooting with a green screen, the production team needs to spend a lot of time on fine keying and repairing color overflow issues. After using LED screens, the shooting team can edit and correct the playback content in real time, and can also import pictures, models and other materials at any time to achieve fast scene switching.

Finally, this method allows the director to see more accurate lighting, color and other visual effects in real time on the set, which can not only create richer scenes according to the director's requirements, but also greatly reduce the workload of post-production and greatly improve the production efficiency of the film. It is understood that the shooting and post-production work of the American TV series "The Mandalorian", which has been very popular in the past two years, were all completed on the set, which is undoubtedly revolutionary for the film and television industry.

At the same time, iQiyi is also promoting the implementation of "digital assets".

When filming "The Wind Rises in Luoyang", iQiyi used a studio covering nearly 5,000 square meters to create the "Bad Well", which took hundreds of people 65 days to build. According to the traditional production process, this expensive scene will be abandoned after the filming is completed, which will undoubtedly cause a lot of resource waste. However, iQiyi used high-precision laser 3D scanning of the scene to turn the originally "one-time" set into a permanent, reusable digital asset.

In iQIYI's strategy, a very important idea is to "kill many birds with one stone". The so-called "kill many birds with one stone" means to transform IP into literature, comics, games, film and television, etc., to form brand value and user scale, and promote the value of each segment of the industry chain.

Taking the "Luoyang" IP as an example, in addition to film and television, iQiyi also plans to launch games, animations, documentaries, VR full-sensory movies, stage plays, cloud performances and other derivative content around it. Through virtual production technology, digital assets such as "Bad Well" can be used in a variety of content.

Imagine if a game based on The Lord of the Rings is released in the future, where players can travel and fight in Mordor, the Shire, Gondor, and Helm's Deep, which are exactly the same as in the movie. How attractive will this be to the audience of The Lord of the Rings movies and novels? Excavating more value from IP through the reuse of digital assets is what iQIYI is doing.

iQiyi's virtual production and digital asset strategy also paves the way for the coming "metaverse" era. Perhaps we cannot yet clearly explain what the metaverse is, or even determine that it will definitely be the future. But at least we can be sure that the metaverse will bring a virtual world with a strong sense of immersion. Every time iQiyi completes a virtual production, it can accumulate technology for the metaverse era; every digital asset created may also become a pavilion in the metaverse era.

As a winner of Toutiao's Qingyun Plan and Baijiahao's Bai+ Plan, the 2019 Baidu Digital Author of the Year, the Baijiahao's Most Popular Author in the Technology Field, the 2019 Sogou Technology and Culture Author, and the 2021 Baijiahao Quarterly Influential Creator, he has won many awards, including the 2013 Sohu Best Industry Media Person, the 2015 China New Media Entrepreneurship Competition Beijing Third Place, the 2015 Guangmang Experience Award, the 2015 China New Media Entrepreneurship Competition Finals Third Place, and the 2018 Baidu Dynamic Annual Powerful Celebrity.