I made a pair of Metaverse gloves, costing 300 yuan

After being a VR game anchor for a month, in addition to becoming more and more addicted to "Half-Life: Alyx", I was also thinking about another question: What is missing from VR now?

As a "person in the middle", I will use a virtual image to explain during live broadcasts - I believe that if there must be a virtual world in the future, we and our digital doubles should be "deeply bound" together. This binding is not only in external cognition, but also in the experience of the real world and the virtual world should be common.

Players know that VR games are completely different from traditional games. In VR games, you don't have to bend over in front of the computer and fiddle with the mouse and keyboard. You can really use the handles in both hands as guns, aim at the enemy, and press the trigger with your index finger. Today's VR games can already recognize finger movements very accurately - but you can't pick up a dropped magazine in the VR world.

By chance, when I was browsing YouTube, I saw Lucas (founder of an American VR startup team) made a VR tactile glove. The key point is that the cost of all the parts only cost him $60.

I applied to repost this video on my account, and it received a lot of responses. Lucas's words, "You can really touch anything with it," heightened my curiosity. Following the clues, I found an open source tutorial on GitHub and prepared to make a pair.

Parts you need to buy | Photo provided by the author

With current technology, I don’t expect VR peripherals that can “touch everything” to appear. But this is exactly what impressed me. In the current VR market, there is nothing new except a headset and two controllers. I think any attempt to expand the user experience at a “popular price” is worth a try.

The cheapest way to touch virtual

In 1987, during the era of Nintendo’s Famicom, gloves, as peripherals for motion-sensing games, had already taken on the function of “input devices”.

Power Glove丨YouTube "Gaming Historian"

If you want to touch the gun in the game in real life, in addition to letting the computer recognize that your hand is moving and modeling the virtual hand to move synchronously with the real hand (signal input function), you also have to wait until the virtual hand holds the gun and returns a signal - this signal acts on the hand, which is what we often call force feedback.

Meta once released a video of tactile gloves, which used microfluidics technology and pneumatic actuators to create precise pressure by inflating and deflating bubbles. Meta said it was still building a high-speed microfluidic processor. When everyone was still gratified that Zuckerberg, who was "all in" the Metaverse, finally came up with something, another tactile glove developer HaptX said, "Meta copied us!"

Meta took seven years to develop, and HaptX took ten years to develop. We don’t know who copied whom, but one thing is certain: it takes many years for VR-related technologies to mature from their launch (at least at a price that is acceptable to ordinary users).

I was excited about how to achieve force feedback at a "very low cost". After reading the tutorial, I found that LucidVR used a particularly "tricky" method. If Meta and HaptX make every inch of your hand skin a force point, then LucidVR's gloves only give force feedback to the fingertips.

Fix the drawstring on your finger | Photo provided by the author

Simply put, when you are about to grasp an object, your fingers should be pulled immediately to prevent further bending. Different bending ranges mean that the size and shape of the objects "held" in your hand are different. The way to prevent the fingers from bending further is to give the fingertips a "reaction force".

Force on fingertips | Photo provided by the author

Therefore, the main part of the glove is a "main switch", with five drawstrings extending out and wrapped around the fingertips, so that you can stop at any time.

The servo is connected to the potentiometer. When the wheel shaft rotates to the (servo) limit, the steering wheel will block the screw on the wheel shaft, so that the wheel shaft cannot continue to rotate and the pull rope cannot continue to unfold. 丨Photo provided by the author

After I understood the principle of force feedback, I still needed to figure out when and how to "close the gate".

"When" is easy to understand, that is, when the virtual hand touches the object in the virtual world. Therefore, we need to create a medium so that the action signals of the organism and the electrical signals of the computer can be converted and "understood" by each other. This is also what I think is the most ingenious design of this pair of gloves.

The rotary potentiometer (whose principle is similar to the sliding rheostat in junior high school textbooks) is connected to the pull rope for use. When the finger is bent, the five pull ropes drive the five potentiometers respectively. The length of the pulled rope is converted into a change in the resistance value of the potentiometer and transmitted to the virtual world for analysis.

Sliding resistor in the textbook | Picture from the Internet

In modeling "data gloves", there are also other implementation methods such as those based on IMU (inertial measurement unit) or bending sensors. The so-called inertial sensor is a device that measures the three-axis attitude angle and acceleration of an object. It generally includes a three-axis gyroscope, accelerometer, and magnetometer to perform multi-sensor data fusion to reconstruct the three-degree-of-freedom direction of each finger. The bending sensor is easier to understand. When the finger is bent, physical deformation occurs to change the sensor resistance.

Comparison of the prices of bending sensors and potentiometers | Image from Taobao

The data transmitted by these two methods is obviously more accurate, but why not use them? There is only one answer: they are more expensive.

Use 3D printed axles to "connect" the potentiometer and the pull rope | Photo courtesy of the author

After understanding when to "close the gate", there is also how to "close the gate". Obviously, the pull rope needs a device to "jam" it during its extension and retraction. I followed the tutorial and bought the MG-90S, a common servo in model aircraft, which is also very cheap at 10 yuan each.

MG-90S servo with wide application range | Image from Amazon

After working for a month, I only picked up a bottle of wine in the game.

After reading the tutorial, I quickly grasped the principle and said to myself, this seems to be quite simple? Isn’t it just this first, then this, and finally that?

I immediately ordered all the parts on the e-commerce platform, which were all common tools in life. The only thing I couldn't buy was a ready-made part (the skeleton to assemble the modules), which I solved with a home 3D printer based on the open source parts parameters.

Of course, you can also find a universal Taobao agent | Photo provided by the author

Then, this took me a month... At the same time, I deeply understood that the other meaning of "taking shortcuts" is "not perfect yet".

When I put on the haptic gloves, adjusted the position of the Valve Index (VR headset), and started the game "Half-Life: Alyx", I was in the familiar "City 17". I took out a pistol from my backpack and aimed at the bottles scattered on the street. "Bang!" At the moment of firing, I imagined that there should be a recoil. In fact, only my index finger was pulled at the moment of pulling the trigger - this experience is more "plastic" than playing with balloons in the park.

But I think I succeeded! I changed a bottle that was scattered on the street, and I could roughly feel it in my hand. It was a cylinder.

Effects of use in games | Photo provided by the author

When I wanted to use them more flexibly, I found a problem.

The first is the problem of "recognition accuracy". We need to first understand that positioning tracking and local tracking are two concepts. The positioning tracking of the entire glove can be achieved by attaching Vive Tracker, Oculus handle, and other existing tracking technologies.

The local tracking of fingers is the key point of a force feedback glove. When I was assembling it, I found that the design of this glove was not equipped with sensors for each knuckle. The bending state of the finger can only be roughly modeled through the resistance signal converted by the pull rope (Lucas may have designed a software for this purpose), which is not accurate enough for each joint.

Hand tracking | Photo courtesy of the author

The second issue is the "recognition range". Also due to the simple structure of the gloves, my fingers can only be recognized when they move up and down, and other degrees of freedom such as left and right expansion and rotation are not possible. But Lucas told me that he has already solved this problem, and the next version of the gloves will increase the degree of freedom of left and right expansion by adding a potentiometer at the first knuckle.

Whether the recognition is accurate or not will affect the effect of force feedback. Another factor is the adjustment of the limit position. The different bending degrees of the fingers correspond to different limit positions, driving the steering wheel to turn to different angles, and then acting on the rope that is being extended and retracted.

In my understanding, this is possible because the LucidVR system has preset this "correspondence".

However, the condition of each person's hands is different, so the limit position needs to be calibrated "personally" before use. If it is adjusted too tight, the hand will be pulled before the object is grasped; if it is adjusted too loose, the object will be grasped just like making a fist with bare hands.

There is no easy way to do this, you can only feel the tightness when putting on the headset and "estimate" the limit position. After taking off the headset, unscrew the servo screws and adjust the steering wheel, repeating this process until you just happen to grab the object in the virtual world.

Continuously debugging the limit position | Photo provided by the author

In addition, long-term stretching will cause the drawstring and gloves to deform. Since the entire device does not fit completely on the hand but is glued to the glove, a slight shift of the glove will also cause errors in force feedback.

Optimizing the experience can never stop! You can either modify it or use your imagination

There is a small episode before the gloves are actually used.

After I debugged it many times, the servo still didn't move. I went to Discord to ask for help, and netizens told me that it wasn't that the servo cable was connected to the development board incorrectly, but that the current was too small to support the servo. I needed to connect a new power bank to the servo separately, instead of sharing the same power bank with the development board.

LucidVR's Discord discussion group has 14,000 followers. Someone improved the gloves by replacing the drawstring with a tougher plastic sheet to solve the problem of the drawstring being easily deformed.

Some people have directly changed the structure and made it into an exoskeleton shape, leaving space for adding more sensors. Many open source projects will not cover everything, on the contrary, they will leave a lot of "blanks" for technology enthusiasts to explore on their own.

Exoskeleton version | Photo provided by the author

In his demo, Lucas described the VR experience of wearing gloves as extremely exciting. In Half-Life: Alyx, you can pick up a can and throw it, walk to a car, open the door and get in, and even climb a wall to escape when you are in danger. However, this is more of a "brain supplement" to bring in other sensory experiences. At most, you can experience the resistance of a wall, and you won't feel the feeling of overcoming gravity and climbing up.

Knowing all this, when I put on this glove for the first time, I felt whether the object was big or small, round or square, even the rough "touch" from my fingertips still made me feel novel. Before this, I couldn't even imagine holding a glass bottle "empty".

Almost at the same time as making the gloves, I made my own SlimeVR full-body tracker. Similar to the gloves, I made five sensors that were tied to the ankles, thighs, and waist (because the SlimeVR software does not model the upper limbs). The five sensors first identified the absolute position through T-pose reset, and then identified the relative position through "each other", parsing the electrical signals into the movements of the virtual person. With this pair of gloves (handles) and the headset, "old ACG" can happily fight dance in VRChat.

SlimeVR also open sourced its code | Image from SlimeVR

A few months ago, I posted a video I made online and randomly gained a lot of followers. Of course, this is not what I am most excited about. Two months after the "SlimeVR Full Body Tracker" video was released, the most important component, BNO-080 (nine-axis sensor module), increased in price from 90 yuan to 300 yuan on Taobao. (Before this, no one taught how to make a SlimeVR full body tracker by themselves, so let's just say that I inspired the love of the old ACG people.)

Sensor module | Photo provided by the author

Of course, not all the compliments I received were “praise.” I 3D printed a model that could hold a small fan, connected it to the headset, and solved the problem of stuffiness.

Homemade VR headset fan | Photo provided by the author

But this time, because the fans used were too cheap, my solution was ridiculed for making too much noise - "like an airport."

Author: Lulu Mikuru

Editor: Shen Zhihan

This article comes from Guokr and may not be reproduced without permission.

If necessary, please contact [email protected]