Impact: We developed rapidly deployable sensors using existing cameras

By tapping into facilities’ existing “CCTV” cameras, we demonstrated that sophisticated sensing could be instantly and flexibly deployed with minimal or no added infrastructure cost. 

In one system installed at the Pittsburgh International Airport in 2019, we use existing cameras as powerful general-purpose sensors to provide wait time estimates for the security line, which benefits almost 10M PIT travelers per year. 

We were early to demonstrate and apply general-purpose AI-camera sensing with Zensors.

 

This work led to...

  • A spin-off company, Zensors, which applies Visual AI agents to better understand spaces and scenes in a variety of industries.
  • Dozens of system deployments, some of which are user-facing. In addition to the security line at PIT, these sensors are currently deployed at other large facilities, international airports and NJ Transit rail stations.

Supported by:  National Science Foundation (#IIS-1149709), later by venture capital.

Timing:  This research started around 2015, with the paper Zensors.

Related work: 

Researchers:  Chris Harrison, Jeff Bigham and team

 

Looking at Additional HCII Impacts...

Nesra sits at a NoRILLA table and reaches for a tower of blocks

NoRILLA Interactive Mixed-Reality Science for Kids

Our novel, mixed-reality intelligent science stations bridge the physical and virtual worlds. Millions of children and families across the US are learning more science and improving their critical thinking skills after predicting, observing and explaining experiments with our patented system at their schools and museums.

screenshot of the game Anti-Phishing Phil, named after an orange fish in an underwater world that helps players to learn more about phishing

Protecting Millions of People from Phishing Scams

We studied the social aspects of phishing attacks and protected millions of people through education and training. This work led to new educational methods to raise awareness, effective anti-phishing warnings, and algorithmic detection of phishing attacks.

PrivacyGrade

Grading Your Smartphone Apps on Their Privacy Practices

We created a model that analyzed over 1 million smartphone apps on their privacy and data collection practices, and then assigned them a public-facing privacy grade. This easy-to-follow grading system raised public awareness and led to improved privacy practices from several app developers.

hands interacting with a black touch screen with labeled inputs of fingernail, knuckle and pad

TapSense Improved Touch Detection on Millions of Smartphones

We trained smartphones to reliably detect four different touchscreen inputs – for example, a tap by a finger pad as compared to a knuckle – which created opportunities for new interactions and features.

screenshot from PAT, a cognitive tutor for algebra

Millions Learned More Math with Our AI Tutor

Millions of middle school and high school students across the US have learned more math as a result of our decades of work with AI-based cognitive tutoring systems.

proposed data label for IoT devices, modeled after a nutritional label

Human-Centered Privacy for the Internet of Things

As more gadgets become “smart” things and the Internet of Things (IoT) expands to tens of billions of connected devices, we want consumers to be aware of what their devices are doing with their personal data.

decimal point game map

Improved Math Mastery with the Decimal Point Game

We explored the learning analytics behind our digital learning game to see how student learning responds to curriculum changes to the game. Over the past 10 years, over 1,500 students have benefited from the Decimal Point game and curriculum materials.

Zensors airport queue screenshot

Zensors Turns Cameras Into Smart Sensors

In one application installed at the Pittsburgh International Airport in 2019, Zensors use existing cameras as powerful general-purpose sensors to provide wait time estimates for the security line, which benefits almost 10M PIT travelers per year.