This AR technology will "dissect" you alive for others to see

Generally speaking, if a physical therapist wants to learn a set of skills, he must master basic anatomy. However, if he does not deal with real corpses, it is not intuitive enough to learn and understand only from textbooks. However, this trouble will come to an end, because the new AR system can let you understand the detailed structure of the human body by projecting different levels of muscles and bones.

The technology, called Augmented Studio, was created specifically for physiotherapist training, allowing trainees to bridge the huge gap between theory and practice.

Under the Skin

Using tracking sensors mounted on a stand, the system projects images of muscles and bones directly onto volunteers. The images are also quite intelligent, they automatically adapt to the volunteers' body shapes and follow their movements, so that physical therapists in training can clearly understand how the human body works.

In addition, students have the opportunity to "hands on" and train teachers and they can "mark" on the projected images, so that they can have a deeper understanding of the relevant knowledge.

The tracking sensor can lock the volunteer after he makes a "T" shape

"What we've done is we've layered virtual models of the muscles and bones underneath the skin on top of it, and have those virtual models change as the body moves," said Thuong Hoang, a researcher working on natural user interfaces at Microsoft Research at the University of Melbourne.

The Augmented Studio is the result of a collaboration between Hoang, computer engineer Zaher Joukhadar, and doctoral student Martin Reinoso. In this system, the somatosensory and tracking technology used on Microsoft Kinect and the "RoomAlive" spatial projection play an important role (both technologies were originally designed for computer games).

Once you step into the projection area and raise your arms to make a "T" shape, the sensor will lock you and then project relevant images based on your body shape.

However, due to the immaturity of the technology, the current projection cannot follow the movement of the human body to reflect the contraction and relaxation of muscles. It can only track the movement of human joints. However, Hoang wants to achieve the original goal by adding "animation" in the future.

David Kelly, a lecturer in the medical school's physiotherapy course, said that in the trial lecture in 2016, students quickly adapted to the new technology, so AR technology will be used throughout this year's class. Kelly values ​​the real-time and interactive nature of this technology the most. Students can truly feel the movement of virtual limbs, which can help them better grasp the knowledge in anatomy books.

“At first, it was difficult for students to understand how the body works when they only had anatomy textbooks because it is not easy to visualize the human body. However, everything became simple when interactive real and virtual muscles and skeletons appeared,” said Kelly.

Hands-on

The "augmented studio" can also provide a more intuitive way of learning. Kelly believes that visual learning is more effective and simpler than traditional reading and listening. "Before the introduction of AR teaching, some students have always been limited by traditional learning methods." Kelly added.

Although the popularity of AR was due to last year's hit game "Pokémon Go", in fact, products such as AR glasses are already on the market, which can enhance what we see, feel and hear in the real world through 3D pictures, videos and holographic images.

This technology is highly interactive

In fact, the biggest advantage of the "augmented studio" is that students can touch and move the volunteers' limbs during learning, which makes the whole learning process full of interesting interactions. At the same time, students do not have to wear heavy head-mounted equipment, so the class size can be much larger.

"The most difficult thing to create in anatomy classes is a dynamic atmosphere. The addition of AR technology solves this problem. For example, students can ask volunteers to kick, so that they can observe from different angles what happens to the muscles and bones when kicking," said Kelly.

In the next step, in addition to adding animation to the entire system, Hoang also plans to develop a dedicated system for student interaction, which can automatically transfer the virtual models displayed in class to the students' tablets so that they can continue to digest the knowledge they have learned after class.

In addition, the researchers also want to improve the mobility of the "augmented studio" (the current studio is fixed) so that instructors can take the entire set of equipment to other locations to teach.

This technique can also be used in martial arts

The author learned that Hoang is now expanding the tracking projection technology to other health and fitness fields, and theatrical performance art is also one of his goals. Hoang said that this technology can be used to show students the changes in muscles during fitness, sports and dance and teach them how to handle their own body movements.

Hoang and his doctoral student Martin Reinoso have also developed a new prototype product that allows a martial arts teacher to remotely guide students' movements by wearing a VR headset. Using a body tracker, students can also imitate the teacher's every move in the VR world.

"Our innovation is not limited to the projection of fixed information. In the future, we also want to project dynamic information onto the user (such as heart rate and breathing) or any surface near the user," Hoang said. "And for sure, my dreams are possible."