Google Glass can be used to steal passwords with 90% accuracy

The latest research results from the University of Massachusetts in the United States show that Google Glass can not only be used to shoot hidden videos, but may even pose a threat to information security.

Researchers have developed software that can automatically recognize passwords entered during video recording, even when the camera is not pointed at the screen. The software determines which keys the user has hit based on the movement of the user's fingers. The software also works with regular video cameras, webcams and smartphones, but Google Glass may offer the most subtle way.

The findings suggest that stealing passwords and other data by watching how someone uses a device could pose a real threat to information security, given that digital cameras and powerful image processing software are now ubiquitous.

In the test, the subjects stood 3 meters away from the camera. The software had an accuracy rate of about 90% in judging the four-character strings typed by users through the iPhone's QWERTY keyboard. The researchers said that in theory, this method can recover short emails and text messages written by users.

"It will be possible to take secret photos while using Google Glass," said researcher Yue Qinggang, a graduate student at the University of Massachusetts, who worked on the study with his classmates Fu Xinwen and Ling Zhen.

Yue Qinggang presented his research Wednesday at the Black Hat information security conference, noting in the conference's newsroom that many people are typing on touch screens, making them vulnerable to such attacks.

He also showed a video demonstrating how to recover the password entered by the monitored person from a long distance. In a series of experiments, someone stood in a first-floor window with a camera and successfully captured the password entered by someone on an iPad 43 meters away. He said that if a telephoto lens is used, the distance can be even farther.

To capture the password, the software must determine the position and orientation of the device screen and the position of the user's finger. Yue Qinggang and other researchers used machine learning techniques to train the software to solve these problems. The software runs on a PC, so the video captured using Google Glass must first be copied to the PC before the password can be recognized.

The software automatically finds the device in the video, then determines the position of the four corners of the screen and tracks the speed of the monitored subject's finger movements.

The researchers are currently testing a number of ways to counteract this surveillance. One countermeasure is to randomly rotate the positions of the keys on a standard keyboard so that the software cannot correctly identify each click. Another measure is to make the keys drift on the screen instead of being fixed in one position.

As a winner of Toutiao's Qingyun Plan and Baijiahao's Bai+ Plan, the 2019 Baidu Digital Author of the Year, the Baijiahao's Most Popular Author in the Technology Field, the 2019 Sogou Technology and Culture Author, and the 2021 Baijiahao Quarterly Influential Creator, he has won many awards, including the 2013 Sohu Best Industry Media Person, the 2015 China New Media Entrepreneurship Competition Beijing Third Place, the 2015 Guangmang Experience Award, the 2015 China New Media Entrepreneurship Competition Finals Third Place, and the 2018 Baidu Dynamic Annual Powerful Celebrity.