HCII, Disney Research Design System That Recognizes Objects Touched by User

A new technology developed by researchers in the HCII and Disney Research could enable smartwatches to automatically recognize what objects users are touching — for instance, whether the wearer is using a laptop, operating a saw or riding a motorcycle — creating new opportunities for context-aware apps.

The technique, called EM-Sense, takes advantage of the body's natural electrical conductivity to detect whether a person is touching an electrical or electromechanical device and, based on the distinctive electromagnetic noise emitted by such devices, automatically identify the object.

"This could be a great feature for smartwatches, making them much smarter than before," said HCII Ph.D. student Gierad Laput. A smartwatch equipped with EM-Sense would have a much more detailed understanding of what the user is doing than is possible with common mobile sensors, such as accelerometers or pulse monitors, he added.

The smartwatch could automatically start a timer when the wearer begins using an electric toothbrush, unlock a keyboard without a password when users touch their laptops or play the latest news when breakfast is being prepared. Linking a smartwatch with a smartphone or other mobile device would expand the possibilities even further.

"We are now able to gain a greater contextual understanding of user activities by recognizing what objects they are interacting with," said Alanson Sample, a research scientist at Disney Research. EM-Sense can differentiate between scores of objects based on the ambient electromagnetic noise they emit, so objects need not be modified or have RFID tags for the technique to work.

In addition to Laput and Sample, the research team includes Disney Research post-doc Chouchang Yang, HCII Assistant Professor Chris Harrison and HCII Ph.D. student Robert Xiao. They'll discuss EM-Sense at UIST 2015, the ACM Symposium on User Interface Software and Technology, Nov. 8–11, in Charlotte, N.C.

Read the full story on the School of Computer Science website, or learn more details on the project's webpage.