HCI students win Bose Challenge, Designing for Augmented Reality with Sound
News
More than 100 Carnegie Mellon University students participated in an on-campus challenge last month, designing solutions for an emerging field so new that the technology, a pair of Bose augmented reality (AR) glasses, wasn’t even available to the participants.
The upcoming Bose AR technology will enable developers to superimpose sound on top of the real world based on location or head gestures, creating opportunities for novel experiences with learning, music, travel, and more.
Despite not having access to a pair of the Bose AR glasses, Rayna Allonce (MHCI '19), Ritu Parekh (MHCI '19), Michael Rivera (HCI Ph.D., ‘22) and Siyan Zhao (HCI Ph.D., ‘22), four human-computer interaction students, creatively prototyped and demonstrated a solution with the technology given to them, winning the Bose Challenge.
“Bose provided participating teams headphones (Bose QC 35 II) and a neck-worn speaker (Bose Soundwear Companion) that together can support most of the functionality and interactions enabled by their sound AR glasses,” said Rivera.
During the course of the two-week, extracurricular competition, participants were charged with exploring a problem or opportunity and building a solution using the Bose hardware. The team of HCI students identified the need for a more secure and efficient way of communicating patient information to healthcare professionals prior to patient interaction.
“In our first brainstorm session, we listed potential users (some examples are children, elderly, students, healthcare providers) for the technology and areas of interest (some examples are assistive technology, entertainment, transportation). We each had prior knowledge in some of the user groups and areas and jointly discussed what are the known pain points among the identities users and areas. From the brainstorming session, we narrowed down on healthcare as a general area to explore because it contains many critical issues that can benefit from Bose hardware,” Zhao said.
Team members researched issues in healthcare and narrowed their list to two problems: mishandling of medical information and fall prevention among the elderly. After competitive analysis of these two areas and semi-structured interviews with a doctor and a nurse, they turned their focus to preventing medical information mishandling.
According to the team’s project video (also embedded below), “We demonstrate an augmented reality medical assistant that leverages Audio Fencing to accurately localize medical professionals and relay patients’ information. With Bose Wearable technology, our solution enables timely access to key patient information, empowering healthcare providers to better and more securely serve their patients."
Similar to the virtual boundaries of geofencing, audio fencing will allow the glasses to deliver just-in-time patient information to the medical professional based on their precise location.
The students learned of the campus Bose Challenge, organized by Real Industry and Bose, via an email invitation from the CMU School of Computer Science.
“I had never heard of or designed for sound based augmented reality before so it really piqued my interest as a design challenge,” said Allonce. “So often when we think of interfaces we think of things you can touch and see. Designing for audio AR was a really cool experience.”
Zhao had not heard of sound AR either until it was introduced to her during the challenge.
“One selling point of sound AR is that it is hands free and unobtrusive (purely audio and open ear). I really enjoy its unobtrusiveness as nowadays, much of the technology relies on us seeing and interacting with them. And they are taking away a lot of our attention,” Zhao said. “Sound AR is different in that sense. It only delivers information to you when it's relevant (based on where you are or where you are looking at) and it doesn't require you to interact with it.”
Parekh added, “Augmented reality really excites me because it is an opportunity to blend the best of both our physical and virtual worlds. The most loved technologies integrate seamlessly into our lives, enhancing our abilities and experiences - in essence, augmenting our lives. In a world that bombards us with visual information, audio AR takes advantage of the comparatively underutilized auditory channel. Most auditory information is currently delivered passively, whereas audio AR takes context into account to provide information in a more active manner. Though it may require some creativity, I think audio AR presents opportunities to solve modern problems in new, perhaps more efficient ways.”
The glasses haven’t been released to the public yet, but the student winners will be among the first to have a pair.
“Bose and Real Industry will send us the sound AR glasses in January 2019 when they are released. The four of us will be one of the few (and first) to get them,” said Rivera. “I'm personally looking forward to using them, and have some ideas I'd like to explore. It's not very often you get the chance to play with a new technology before it becomes widely available.”