The world at your fingertips
Ever since William Gibson's fictional dystopia, cyberspace, became an inexorable part of our lives, digital technology has been seen as the scourge of man's association with the physical world. There is some truth to it, and it's especially hard to miss in the virtual age of text, tweet and Facebook. Most of our everyday interactions are intangible, slowly benumbing our five senses of perception that took millions of years to form. Massachusetts Institute of Technology researcher Pranav Mistry has set out to change all that with SixthSense, a wearable gestural interface that enables users to project their technology devices on any physical surface and use hand gestures to interact with digital information, in addition to recognising physical objects and sourcing information on them from the Internet.
To get a real sense of what this means, here's an overview of what you can do with SixthSense: you can take photos with the inbuilt camera by forming a 'frame' with your fingers and later scroll through them on any wall you happen to pass by; you can watch live video updates on a real newspaper, navigate a map with natural gestures of the hand, draw a circle on your wrist to display a watch and even play games that involve real-world objects. Neat, is it not? When Michael Benedikt wrote of "the tablet become a page become a screen become a world" in Cyberspace: First Steps, he was in fact chronicling the birth of SixthSense. The endless possibilities of SixthSense, aka WuW (Wear Your World), make the world your computer and augment your physical interactions with it.
"With computers and modern technology, we are more connected with information and with each other. But I still enjoy shopping at a real store with friends rather than buying stuff on Amazon. I think the new wave of digital technology has taken away the joy of our interaction with objects and surroundings and turned us into machines sitting in front of other machines," says Mistry, 27, an IIT-Bombay graduate who dreams of a "world that is more joyful, more tangible".
The gadget, which can be worn as a pendant dangling from a chain around your neck, is the culmination of Mistry's preoccupation with seamlessly connecting digital and physical interfaces. "At MIT's Media Lab, we like making things that we can touch, feel and take with us wherever we go," he says. His recent projects include Quickies, which transforms Post-It scribbles into digitally searchable sticky notes capable of sending reminders; Inktuitive, which translates pen drawings in real 3D space into digital data; and TaPuMa, a tangible public map that can display customised information.
A big leap towards integrating routine physical movements into digital technology, SixthSense—which comprises a projector, a camera and a mirror, and is connected to a smartphone in the user's pocket—recognises gestures of the hand by tracking the locations and arrangements of coloured markers at the user's fingertips. "Gesture is our most natural form of interaction with people and objects—in India, we greet each other with 'namaste'. SixthSense applies this to our interaction with the digital world and breaks the barrier between the real world and the world of information technology," Mistry says.
The prototype cost $350, which Mistry says is cheaper than Microsoft's surface computer and similar products. Much more than a computer or a phone, SixthSense redeems the natural world from the abstractions of technology and opens up enormous real-world avenues. Since it eliminates the need to learn how to use a mouse, keyboard and other complex input devices to access information, Mistry feels it can "radically improve the penetration of technology" in India. "We can use this technology to train someone to operate a complex machine or repair his own washing machine," says Mistry, who hails from Palanpur and visits his parents in Ahmedabad almost every year.
After Mistry and fellow MIT-ian Pattie Maes unveiled the innovation in a TED presentation in Long Beach, California, last month, he says he has been inundated with over 500 e-mails a day, most of which are corporate offers, interview requests and suggestions on improving SixthSense. "Many companies are interested in taking the technology to the market and at Media Lab itself, sponsor companies are in talks with me to turn this into a consumer product. Others have written to me expressing an interest in taking SixthSense a step further by adapting it for people with accessibility issues, so that it can be the eyes of the blind and the ears of the deaf," says Mistry, who has worked with Microsoft, NASA and CMU.
He is dismissive of allusions to science fiction. "I'm not a very big fan of science fiction. I think I'm a very big fan of living in the physical world. I'm good with digital technology, but I start to miss the physical world. I miss riding my bike, talking to friends," says Mistry, who enjoys playing ping-pong, driving and camping. "Whatever science fiction movies we watch now, we can make the technology real. What we can do is not important. What we should do is."
Mistry credits his family for his success. "I grew up in a very creative environment. In fact, if there is someone behind all my research and inventions, it is my parents. My dad is an architect—he is my hero. He is the one who taught me to see the world differently. In the '80s, when video games were rare to see in India, he built me one with electronic components," he says, warning that once you get him started on stories of his childhood, he won't be easy to stop. The impeccable design of his website (www.pranavmistry.com) is a testament to his creativity. "My Masters in design at IIT also taught me the human aspect of technology. I had the most memorable time of my life there—late night canteen meetings, preparing for the annual cultural skit, celebrating Holi, taking long walks in the green campus...," Mistry trails off.