Aipoly: Building a virtual reality for nearly 300 million people in the world

[[161600]]

Have you ever thought that while we are pursuing the rapid development of electronic products, there are still 285 million visually impaired people in the world. How do they use smartphones?

I have such a friend who has visual impairment, but he is proficient in using iPhone in daily life. He communicates with me through WeChat and posts on Moments. Before I met him, it was almost unbelievable that he was a person who could not see the real world.

"Just treat me as a normal person like you," I still remember him saying. All this is thanks to the Accessibility feature in the iOS system. For normal people, it may be a small dot, but for people with disabilities, it is a complete solution for using smartphones designed from the perspective of vision, hearing and physical movements.

[[161601]]

That friend told me that through VoiceOver and Speak Screen, he can use his phone normally like me, and it will tell you the specific location on the map and even the time, place and shooting environment of the photo through voice.

Accessibility helps the visually impaired understand the digital world, but who can help them see the real world? Recently, a word has frequently appeared in our field of vision, that is Aipoly.

Aipoly is a collaborative project between Alberto Rizzoli, Marita Cheng and a technical team from Singularity University. The former have many years of experience in using AI to solve social problems.

“We integrated what we learned about the neural network system into this app. Users face the image to be recognized, press a button, and they will receive a voice message. This is the simplest operation,” said Alberto Rizzoli.

[[161602]]

Aipoly works by analyzing the various parts of the image and determining the names of objects (such as "dog", "car") and adjectives (such as "silver", "shiny") through reverse image search. The most important thing is that AI will understand the information after decomposing the image, combine it and convert it into audio to convey it to the user (such as "shiny silver car").

This is still an experimental technology, although it can already recognize hundreds of objects and colors, continues to learn autonomously, becomes smarter, and is already available for public download on the App Store.

Alberto Rizzoli said that once Aipoly is perfected, the visually impaired can use it to identify what is in a plate of food, or interpret the clothing of a person in a photo.

[[161603]]

Aipoly has big ambitions. They envision using it to create a "Google Street View" designed for the visually impaired. "We can build a virtual model of the real world so that users don't have to constantly scan their surroundings." This sounds like a grand plan.

We have to admit that we have indeed ignored the electronic consumption needs of the visually impaired for a long time. In the past 10 years, we have made many breakthroughs in image recognition technology, and it will become more accurate in the next few years.

[[161604]]

Alberto Rizzoli believes that these 285 million people will inject more vitality into the smartphone and even VR markets in the next few years. Google has always been at the forefront in this field, and Aipoly is also applying for funding support from Google.