The future of virtual reality: multi-sensory interaction technology

Virtual reality provides us with a virtual world without boundaries, but in reality we are still more bound to the real world. The size of the play space given by the virtual reality headset means that this virtual world is not a place where people can move around freely. We may exceed the sensor range or hit the real wall at any time. So how can we realize a virtual world without boundaries in a limited play space?

One approach is to readjust the user's visual perception by turning so that the user can walk in a straight line again. But constant turning makes people feel dizzy, which is not an ideal solution considering that virtual reality already has motion sickness problems, and this method will inevitably affect the experience and break the sense of game.

Researchers from the University of Tokyo think they have found a better way, which they named "Unlimited Corridor", and YiViAn has also briefly introduced it before. Unlimited Corridor uses visual feedback and tactile feedback to make users think they are walking in a straight line, but in fact they are walking in circles. In actual operation, one hand of the player will touch the real wall when walking, which will naturally adjust their walking direction, making the user mistakenly think they are walking in a straight line.

This project is led by Dr. Takuji Narumi of the University of Tokyo and Yohei Yanase, a researcher at Unity. We can see that this project actually requires a relatively large circular wall and a mobile virtual reality device (they are carrying a high-performance gaming laptop). The team said they have successfully reduced the size of the circular wall to a radius of 2.5 meters. In addition, the area can be further reduced through "visuo-haptic interaction" (visual and tactile interaction).

So what exactly is visuo-haptic interaction? Narumi said it is "an illusory effect in our brain" that can produce specific brain perceptions by combining sensory inputs in different ways.

"The combination of visual and tactile stimulation changes our proprioceptive sensations," he said. "It has long been assumed that different sensory modalities operate independently of each other. However, recent behavioral and brain imaging studies are overturning this view and showing that cross-modal interactions, in which one sense's perception of the world can be altered by stimulation of other senses, play a very important role in our perception of the world."

The research team is also developing a system called "Flavor Display", which also uses multi-sensory interaction to change our perception of food taste. Users can choose different flavors of cookies in the virtual world, while in fact they are just tasting ordinary cookies in the real world (this "taste enhancement" requires users to wear an olfactory capsule device on their heads, which sprays different odors into their noses during chewing to match the flavor selected by the user).

“In the Unlimited Corridor implementation, we were able to reduce the required footprint of the device by using a multi-sensory interactive VR system (vision and touch). Therefore, we believe that multi-sensory interactive effects will become a key technology for the next generation of VR,” added Narumi, who is also studying the perception of shape deformation using multi-sensory interaction.

The team's experiment basically consisted of two convex semicircular walls with a passage in the middle, which allowed users to turn corners in the game and accommodate multiple users at once.

The core technology of this device is to match the user's perception in virtual reality through visual and tactile stimulation. Narumi pointed out that the interval between visual and tactile appearance should be within 200 milliseconds, and they are studying the acceptable time interval between visual and tactile appearance through different experiments.

While the researchers call the illusion added to Unlimited Corridor "very powerful," they also admit that a small number of people will still detect the circular wall based on touch, so this technology is not suitable for everyone. Although the research team claims that this system causes "much less" motion sickness than general virtual reality or redirected walking systems, there is no absolute guarantee that motion sickness will not occur.

Although Unlimited Corridor is just a research project exploring how to control the brain's perception of reality, the research team believes that this technology can be applied outside the laboratory, such as entertainment, education, simulation, etc.