Three HCII projects were recognized with four of Fast Company’s Innovation by Design 2021 Awards.
Researchers from the HCII’s Future Interfaces Group (FIG Lab) and Smart Sensing for Humans (SmaSH) Lab collaborated on these interdisciplinary projects, which explore new applications of sensors to enhance smart environments.
“Current smart devices are oblivious to the user’s placement and pose, which can be extremely valuable to enable applications such as directional privacy-sensitive voice interfaces, full-body Animojis and embodied gaming interactions,” said Karan Ahuja, HCI PhD student who worked on two of the projects. “In our research, we aim to digitize the user’s body using sensor fusion techniques (Pose-on-the-Go) or sensing the acoustics of the environment (Direction-of-Voice).”
Learn more about the three projects below.
Pose-on-the-Go is a full-body pose estimation system that utilizes the sensors already found in today’s smartphones. This is in contrast to prior systems, which require additional worn or external sensors. The project uses extensive sensor fusion and leverages a phone’s front and rear cameras, the user-facing depth camera, touchscreen, and IMU.
Project details: Pose-on-the-Go
Student Design honorable mention
Modern homes and offices have a variety of IoT devices, many of which are activated by voice. In addition to receiving and processing spoken content, this work proposes that these devices can also infer the Direction of Voice (DoV). Such DoV estimation innately enables voice commands with addressability, in a similar way to gaze, but without the need for cameras.
Project details: Direction-of-Voice Estimation project
Experimental Design finalist
Although smart cities can provide rich data, it usually comes with the added costs of installing and maintaining new sensors. This project explores a novel application of laser vibrometry while utilizing the city’s existing retroreflective materials (commonly found as street signs, construction barriers, road studs and license plates) at very long ranges and use them as unpowered accelerometers for use in a wide variety of sensing applications.
Project details: City-Scale Sensing With Laser Vibrometry