A new app by Facebook gives sight-impaired folks a way of “seeing” photos posted on social media. The app is known as Visual Q&A and was introduced at the Web Summit Internet Conference. Essentially, users can ask questions about a photo, and the app will respond with an audio answer describing what the image depicts.
So how does it work? Say for instance that that photo shows a cat. Users can ask questions like “What is the cat doing?” or “Where is the cat?” The app would respond by saying the cat is sleeping, or the cat is in the living room.
In a blog post by Mike Schroepfer, Facebook’s Chief Technology Officer, Visual Q&A was developed by AI Research, a subdivision of the social network giant. The group also gave the app “short-term memory” capabilities and an ability to answer questions the way a person would. The technology used is known as Memory Networks and is combined with image recognition to answer simple questions about a photo.
So what are Facebook’s motivations for creating Visual Q&A? AI Research is also in the process of developing its own intelligent personal assistant program known as Facebook M that would rival Apple’s Siri, Android’s Google Now, and Microsoft’s Cortana.
Behind the scenes, AI Research is working feverishly to make artificial intelligence into a concept with real-world applications. According to Schroepfer, the team is also teaching AI to make predictions. In one demonstration, a stack of blocks was shown to the AI, and it would guess whether the blocks would topple over.
Visual Q&A is just one of Facebook’s pending projects. Schroepfer announced that the social network has a 10-year plan in place in which it would release new innovations in the niche of virtual reality. One such project in the prototype phase is the Aquila, a drone that connects users to various parts of the globe without the need for Net access. The aerial craft is no lightweight project; it has the wingspan of a 737 jet, is solar powered, and can remain airborne for up to three months at a time.
As far as Visual Q&A goes, this is an app that could really redefine the way the visually impaired interact with social media. In the United States, an estimated 7,327,800 men and women over the age of 16 report some degree of visual disability, according to the National Federation of the Blind.