Theme 5 - SixthSense Technology

Spearheaded in the United States, 'SixthSense' is a wearable device that enables users to interact with the physical world to obtain digital information by using natural hand gestures. It projects information onto any surfaces including walls, body parts and physical objects around us. It allows us to access information which is integrated with reality, making the entire world our computer. Supervisor of the research, Pattie Maes, talks about this new invention.




Task A: Matching Activity

Match the words on the left with its explanations on the right

Task B. Comprehension activities

Part I. Determine if the following statements are true:

Part II. Choose the correct answer (Multiple answers possible):

Task C. Thought-provoking task

It is not stated in the video. However, in your opinion, what are some of the disadvantage/short comings of this device?

Anything reasonable will do. The point is to develop our thinking and presentation skills.

Some answers might be:
  • We might be relying on the device so much that we would slow out our own reaction and independent thinking
  • People might be offended if you rely on the web to evaluate them instead of using your own judgment through your interaction
  • We are depending too much on one device: phone, calculator, dictionary, map contact book, clock, and computer. It is like putting all eggs in one basket. If it fails/is lost, we would lose a vast amount of information and becomes immobile and lost.
  • We are putting too much pressure on four fingers which could cause ergonomic diseases
  • It is not suitable for visually challenged people or people who have less than four fingers.

Hide answer.

Video ©


Pattie Maes: I’ve been intrigued by this question of whether we could evolve or develop a sixth sense - a sense that would give us seamless access and easy access to meta-information or information that may exist somewhere that may be relevant to help us make the right decision about whatever it is that we are coming across and some of you might argue well, don’t today’s cell phones do that already? But I would say no. When you meet someone here at TED, and this is the top networking place of course of the year, you don’t shake somebody’s hand and then say, “can you hold on for a moment while I take out my phone and Google you.” Or when you go to the supermarket and you’re standing there in that huge aisle of different types of toilet papers, you don’t take out your cell phone and open a browser and go to a website to try to decide which of these eh ... different toilet papers is the most ecologically responsible purchase to make. So ... we don’t really have easy access to all this relevant information that can just help us make optimal decisions about what to do next and what actions to take. And so my research group at the Media Lab has been um ... developing a series of inventions to give us access to information in a um ... sort of easy way without requiring that the user changes any of their behavior. And I’m here to unveil our latest effort and most successful effort so far which is still very much a work in process. I’m actually wearing the device right now and um ... we’ve sort of cobbled it together with um ... components that are off the shelve and by the way, it only cost 350 dollars at this point in time. Um ... I’m wearing a camera, just a simple webcam. A portable battery-powered projection system with a little mirror. These components communicate to my cell phone in my pocket which acts as the communication and computation device. And in the video here we see my student Pranov Mistry who is really the genius who’s been implementing and designing this whole system. And we see how this system lets him walk up to any surface and start using his hands to interact with information that is projected in front of him. The system tracks the four significant fingers. In this case, he’s wearing simple marker caps that you may recognize. But if you want a more stylish version you could also paint your nails in different colors and the camera basically tracks these four fingers and recognizes any gestures that’s making so he can just go to um ... for example a map of long beach, zoom in and out etc.. The system also recognizes iconic gestures such as the taking a picture gesture and then takes a picture of whatever is in front of you, and when he then walks back to the Media Lab he can just go up to any wall and project all the pictures he’s taken, sort through them and organize them and re-size them and etc., again using all natural gestures. So, some of you most likely were here two years ago and saw the demo by um ... uh Jack Han. Or some of you may think, “Well, doesn’t this look like the Microsoft Service Table?” And yes, you also interact using natural gestures, both hands, etc.. But the difference here is that you can use any surface you can walk up to any surface including your hand if nothing else is available, and interact with this projector data The device is completely portable and can be (Applause) ... So one important difference is that it’s totally mobile. Another even more important difference is that in mass production this would not cost more tomorrow than today’s cell phones and would actually not um ... sort of be a bigger packaging - could look a lot more stylish than this version that I’m wearing around my neck. Um ... but other than letting some of you live out your fantasy of looking as cool as Tom Cruise in “Minority Report”. Um ... the reason why we’re really excited about this advice is that it um ... really can act as one of the sixth sense devices that gives you relevant information about whatever is in front of you. So we see Pranav here going into the supermarket and he’s shopping for some paper towels. And as he picks up a product the system can recognize the product that he’s picking up using either image recognition or marker technology and give him the green light or an orange light. Um ... he can ask for additional information. So this particular um ... choice here is a particularly good choice given his personal criteria. Some of you may want the toilet paper with the most bleach in it rather than the ecologically responsible choice. (Laughter) Um ... but if he picks up a book in the book store he can get an Amazon rating. It gets projected right on the cover of the book. This is Juan’s book, our previous speaker, which gets a great rating, by the way, at the Amazon. And so, Pranav turns the page of the book and then see the additional information about the book-reader, comments maybe sort of information by his favorite critique etc.. If he turns to a particular page he finds an annotation by maybe an expert of a friend of ours that gives him a little bit of additional information about whatever it is on that particular page. Reading newspaper- it never has to be outdated. You can get video annotations of the events that you are reading about. You can get the latest sports scores, etcetera. This is a more controversial one. (Laughter) As you interact with someone at TED, maybe you can see a word cloud of the text, the words that are associated with that person in their blog or personal web pages. In this case, the student’s interested in cameras, etc. On your way to the airport, if you pick up your boarding pass it can tell you that your flight is delayed, that the gate has changed etcetera. And, if you need to know what the current time is, it’s as simple as drawing a watch ...(Laughter) eheh ... on your arm. So that’s where we are at so far in uh ... developing this sixth sense that would give us seamless access to all these relevant information about the things that we may come across. My student Pranav, who was really, like I said, the genius behind this (Applause)... eh he ... He does deserves a lot of applause because I don’t think he’s slept much in the last three months actually, and his girl friend is probably not very happy about him either. Um ... but it’s not perfect yet. It’s very much a work in progress and who knows maybe in another 10 years we’ll be here with the ultimate sixth sense brain implant. Thank you.

Pattie Maes and Pranav Mistry demo SixthSense (8:54) -
Copyright© 2012-2013 UGC ICOSA Project, Hong Kong. All rights reserved.