Teach you to use your eyes to play with your phone

When you play with your phone for a long time, what should you do if your hand gets tired? Switch hands and continue to swipe?

What if the other hand gets tired too? Switch back to the other hand?

Perhaps, there is no need to go through so much trouble.

You can play with your phone with just a simple glance.

For example, select and open an application;

(Source: YouTube)

View a photo on your phone;

(Source: YouTube)

Open or close a music album by singer Taylor Swift;

(Source: YouTube)

Switch songs, check lyrics, and adjust volume;

(Source: YouTube)

For example, check and mark emails sent to you by your boss.

(Source: YouTube)

Wherever your eyes reach, corresponding real-time feedback will be displayed on the mobile phone screen.

Isn't it cool and handsome? But this is not science fiction.

The above scenario is implemented by a research team from the School of Human-Computer Interaction at Carnegie Mellon University, who developed a tool called EyeMU that allows users to control smartphones through gaze control and simple gestures.

The related research paper titled "EyeMU Interactions: Gaze + IMU Gestures on Mobile Devices" was published at the ICMI 2021 conference.

More natural human-computer interaction

Currently, as smartphones become more and more powerful, people can use their phones to do more and more complex things, not just watching movies and TV series, editing videos, reading news, socializing and playing games.

However, mobile phones that are always in people's hands still need people to use one or both hands and voice commands to operate them. In a sense, it is sometimes not very convenient.

So, is there a more natural way of human-computer interaction?

Andy Kong is a senior computer science student at Carnegie Mellon University. He has always been interested in eye-tracking technology, but because commercial versions are too expensive, he wrote a program to use the built-in camera of a laptop to track the user's eyes and move the cursor on the screen.

In Kong's view, today's phones only respond when we ask them to do something, whether it's through voice, tapping, or clicking a button. If the phone can predict what the user wants to do by analyzing the user's gaze or using other biometric technologies, it will be very useful.

It was this "cheap" program that laid the foundation for the final realization of EyeMU.

In subsequent work, Kong and his colleagues improved on this early prototype by using Google’s Face Mesh tool to study and map the gaze patterns of users looking at different areas of the screen.

They then developed a gaze predictor, a tool that uses a smartphone’s front-facing camera to lock onto what a user is looking at and label it.

Figure | Adding filters to photos through eye control (Source: This paper)

Finally, they improved the efficiency of EyeMU by combining the gaze predictor with the smartphone’s built-in motion sensors to implement commands.

Figure｜Some gestures of EyeMU (Source: the paper)

For example, users can mark a piece of content as a target by looking at it for a long time, and then swipe the phone left to close it, or swipe the phone right to respond; at the same time, users can also enlarge the image by pulling the phone closer.

Although EyeMU's features are cool, there are still some key limitations that need to be overcome before it can be launched to consumers, such as the insufficient accuracy of the gaze module (lower than current dedicated eye trackers), the need to test it on more devices, and the possible waste of batteries.

In addition, I can’t help but wonder, would such a cool operation be more tiring to the eyes?

However, with continued optimization and design in the future, this tool may really become available to ordinary people.

By then, you will be able to play with your mobile phone just by moving your eyes.

How harmful are mobile phones?

Nowadays, the excessive use of smartphones has become a problem that concerns the whole society and needs to be solved urgently.

With the widespread coverage of 4G and 5G networks and the continuous strengthening of smartphone functions, "one phone in hand, everything" has gradually become the norm, "mobile shopping, mobile photography, mobile query, mobile communication..." Smartphones have undoubtedly become an indispensable part of people's lives today.

However, many people often worry about forgetting to bring their mobile phones or being afraid that their mobile phones will run out of battery. Little do they know that this is a disease called "Nomophobia", which can cause negative health effects (such as poor sleep quality) and harmful psychological effects (such as a bad temper).

Other scientific studies have confirmed that excessive use of smartphones is related to reduced physical activity, obesity, neck pain, impaired performance and addictive-like behaviors. It not only affects a person's biological clock, vision and even intelligence, but also damages the skin, causing finger numbness, chronic strain, etc.

A recent study shows that reducing the daily use of smartphones will have a positive impact on a person's happiness. It recommends that people reduce the time they spend looking down and change their habit of keeping their phones in their hands. But it also says that it is not necessary to completely give up using smartphones in order to have a better, more satisfying life, and less anxiety.

Everything in moderation.

References:

https://dl.acm.org/doi/10.1145/3462244.3479938

https://techxplore.com/news/2022-04-eyes-smartphone-gaze-tracking-tool.html

https://www.youtube.com/watch?v=-HwcmWRAsaA&ab_channel=FutureInterfacesGroup

https://doi.apa.org/doiLanding?doi=10.1037%2Fxap0000430