Impact: We invented the Intelligent Science Station

Our mixed-reality stations bridge the physical and virtual worlds.

As children do hands-on experiments with physical objects in the real world, a friendly on-screen gorilla character powered by AI and computer vision provides personalized interactive feedback as the learner makes discoveries. Millions of children and families across the US are learning more science and improving their critical thinking skills after predicting, observing and explaining experiments with our AI-based Intelligent Science Stations.

We invented Intelligent Science Stations powered by AI to improve children's inquiry-based STEM learning.

 

This work led to... 

  • Creating the intelligent science station, NoRILLA. Our Novel Research-based Intelligent Lifelong Learning Apparatus (NoRILLA) is a patented, mixed-reality system that improves children's inquiry-based STEM learning by 5 times compared to other screen-based technologies, while also increasing engagement.
  • New applications beyond the earthquake table. In one NoRILLA experiment, children build brick towers to see what style is less likely to fall in an earthquake, and then with interactive AI guidance and feedback from the system’s gorilla character, the table shakes the bricks to test the learner’s hypothesis. In addition to earthquakes, we are expanding the system to explore other content areas, such as ramps and balance scales.
  • Installations in many museums and schools. Millions of children and families have interacted with our installations at places such as the Carnegie Science Center, Children's Museum of Atlanta, Please Touch Museum in Philadelphia, Children's Discovery Museum in San Jose, Muse Knoxville, and after school programs.
  • An extensive partnering STEM curriculum. The curriculum we built for NoRILLA is being used in many schools and afterschool programs as well.
  • A feature in an international exhibit. NoRILLA was chosen as 1 of 6 exhibits from around the world to showcase the benefits of AI at the Caixa Forum Agora AI Museum (Valencia, Spain).

Supported by:  This work has been funded by multiple NSF grants including NSF AISL (Advancing Informal STEM Learning), NSF DRK-12 (Discovery Research Pre K-12), and NSF PFI: AIR-TT (Partnerships for Innovation: Accelerating Innovation Research - Technology Translation), as well as other foundation grants such as PNC Grow up Great, Grable Foundation, McElhattan Foundation, and EQT Foundation.

Timing:  The research started around 2012 and is ongoing.

Related work: 

Researchers:  Nesra Yannier, Ken Koedinger, Scott Hudson and team

Research Areas:  Learning Sciences and Educational Technologies, AR/VR/XR

 

 

Looking at Additional HCII Impacts...

Nesra sits at a NoRILLA table and reaches for a tower of blocks

NoRILLA Interactive Mixed-Reality Science for Kids

Our novel, mixed-reality intelligent science stations bridge the physical and virtual worlds. Millions of children and families across the US are learning more science and improving their critical thinking skills after predicting, observing and explaining experiments with our patented system at their schools and museums.

screenshot of the game Anti-Phishing Phil, named after an orange fish in an underwater world that helps players to learn more about phishing

Protecting Millions of People from Phishing Scams

We studied the social aspects of phishing attacks and protected millions of people through education and training. This work led to new educational methods to raise awareness, effective anti-phishing warnings, and algorithmic detection of phishing attacks.

PrivacyGrade

Grading Your Smartphone Apps on Their Privacy Practices

We created a model that analyzed over 1 million smartphone apps on their privacy and data collection practices, and then assigned them a public-facing privacy grade. This easy-to-follow grading system raised public awareness and led to improved privacy practices from several app developers.

hands interacting with a black touch screen with labeled inputs of fingernail, knuckle and pad

TapSense Improved Touch Detection on Millions of Smartphones

We trained smartphones to reliably detect four different touchscreen inputs – for example, a tap by a finger pad as compared to a knuckle – which created opportunities for new interactions and features.

screenshot from PAT, a cognitive tutor for algebra

Millions Learned More Math with Our AI Tutor

Millions of middle school and high school students across the US have learned more math as a result of our decades of work with AI-based cognitive tutoring systems.

proposed data label for IoT devices, modeled after a nutritional label

Human-Centered Privacy for the Internet of Things

As more gadgets become “smart” things and the Internet of Things (IoT) expands to tens of billions of connected devices, we want consumers to be aware of what their devices are doing with their personal data.

decimal point game map

Improved Math Mastery with the Decimal Point Game

We explored the learning analytics behind our digital learning game to see how student learning responds to curriculum changes to the game. Over the past 10 years, over 1,500 students have benefited from the Decimal Point game and curriculum materials.

Zensors airport queue screenshot

Zensors Turns Cameras Into Smart Sensors

In one application installed at the Pittsburgh International Airport in 2019, Zensors use existing cameras as powerful general-purpose sensors to provide wait time estimates for the security line, which benefits almost 10M PIT travelers per year.