Throw away the controller! Humans are not far from gesture-controlled VR

The controller, as the most critical touch control method for VR experience, has been considered by some media as the winning factor in the VR battle. Recently, the startup company Leap Motion launched a new product Orion, which can help VR players control digital objects with their hands.

The advent of Orion brings new challenges to the previous conclusion: Can gesture control in VR replace handles?

It is undeniable that gesture control is more natural in interaction compared to handles. From the existing products, after putting on the VR helmet, the user's vision is completely isolated from the real world. It is difficult to operate the controller by observing first and then acting in daily life, especially input devices such as keyboards that have artificially defined high-density key actions. In terms of VR, taking the experience of HTC VIVE as an example, after putting it on, the player needs the help of the on-site instructor to pick up the handle, and it is difficult to accurately grasp the position of different buttons when using it for the first time. Once gesture recognition technology is introduced, these problems will be solved.

From this perspective, Orion's function is similar to a glove controller. However, while it eliminates the need for physical contact devices, it still cannot avoid the drawbacks of glove controllers, which is that it can only complete hand movements in the game, and these only account for a small part of the game commands. Game controllers can assist players in issuing various commands such as turning, running, jumping, and even stunts in different games, which requires additional definitions in gesture control, and the complexity is likely to increase.

From a technical perspective, the actual effect of gesture recognition in terms of accuracy and latency is far behind that of handle operation. According to experiencers, the grabbing action of the Orion beta version is not perfect, and it is difficult to complete games that require high finger movement accuracy. Earlier, some applications that used Leap Motion to replace the mouse function were criticized for being difficult to operate, low sensitivity, and poor user experience.

Oculus, PS, and HTC's VR projects currently use controllers as standard input devices. Does this mean that the application prospects of gesture control in the VR field are not as good as Orion expected?

In fact, Leap Motion itself is a company under Oculus. In addition, Oculus has also acquired similar companies Nimble VR and Pebbles Interfaces, both of which focus on gesture recognition technology. Sony has also acquired a Belgian sensor technology company SoftKinetic, which has launched a gesture recognition accessory for Oculus Rift. Valve, which cooperated with HTC, paid attention to gesture control technology as early as three years ago.

VR giants' enthusiasm for gesture control technology precisely reflects the compatibility between the two. VR itself is a major innovation in the field of human-computer interaction. The natural interaction with the digital environment that is closest to the real environment is the attraction of the immersive experience. In real life, people can mobilize various senses to perceive the world around them and interact with the environment through various actions. These seemingly simple interactions are difficult to achieve in the digital world. One of the key points is that machines do not have the ability to recognize and understand human actions, and therefore cannot make corresponding feedback. Various tracking technologies are equivalent to the organs of the machine, enabling the machine to directly perceive the player's body movements, thereby avoiding the use of third-party input devices as intermediaries for unified information rules.

Whether it is gesture control, eye tracking, body motion capture or voice control, the essence is to enable machines to actively receive and understand human language, including body language. In this sense, the progress of human-computer interaction technology can be seen as the gradual adaptation of machines to human expression and communication methods. In addition to these, brain wave control has also entered the list of VR interaction methods. As long as you think, the other party can understand, which may be the highest level of communication.

Judging from the current situation, in the short term, the handle will still be the mainstream input device for game interaction. However, with the continuous development of gesture recognition technology, projects like Orion will gradually take over some input commands. For a period of time, developers should choose different input peripherals according to the game category. Before all-round human tracking technology matures, external input devices will always play the role of a medium for communication between people and machines. But there is no doubt that one day we will throw away the mechanical handles and see everything we want by moving our fingers like in science fiction movies.

As a winner of Toutiao's Qingyun Plan and Baijiahao's Bai+ Plan, the 2019 Baidu Digital Author of the Year, the Baijiahao's Most Popular Author in the Technology Field, the 2019 Sogou Technology and Culture Author, and the 2021 Baijiahao Quarterly Influential Creator, he has won many awards, including the 2013 Sohu Best Industry Media Person, the 2015 China New Media Entrepreneurship Competition Beijing Third Place, the 2015 Guangmang Experience Award, the 2015 China New Media Entrepreneurship Competition Finals Third Place, and the 2018 Baidu Dynamic Annual Powerful Celebrity.