Impact: Improving the livestream experience for audiences worldwide

Watching the livestream of a game on a service like Twitch can be entertaining and even educational, but viewers are limited in the ways they can actually participate.

We recognized that designing a better experience for livestream audiences has a variety of benefits, including: enabling blind and low-vision stream viewers to participate; helping the streamers to improve their audience engagement; supporting researchers who need better data on game streaming; helping game developers to create better streaming-centric designs; and supporting the teachers and learners who engage with educational game streams.

We created a toolkit to add new interactive capabilities to livestreamed games, and collaborated with the community to identify novel design opportunities using our technology.

This work led to... 

  • Developing and teaching “Game Design for Crowd and Cloud,” a new university course on game streaming. Students created prototypes to help streamers, viewers, and moderators interact more richly.
  • An uptake in use of our audience participation design principles by game designers, streamers, and audiences on Twitch and other platforms.
  • “Streamsourcing,” a new term coined by us to explain our strategy for crowdsourced data collection. We’re using our technology to help streamers and their audiences “donate” data to citizen science projects and other good causes.
  • Increased consulting opportunities with professional game streamers and national brands on how to increase engagement with educational game streams. With millions of Americans watching livestream gaming content each day, our work can contribute to improved streaming experiences.

Supported by:  Amazon seed funding (2016-2018), NSF CAREER award (2020-2025)

Timing:  2016-present

Related work: 

Researchers:  Jessica Hammer, Noor Hammad, Erik Harpstead, Nik Martelaro, and many other team members

HCII Impact Area: Games & Play, Tools & Programming

 

 

 

Looking at Additional HCII Impacts...

Nesra sits at a NoRILLA table and reaches for a tower of blocks

NoRILLA Interactive Mixed-Reality Science for Kids

Our novel, mixed-reality intelligent science stations bridge the physical and virtual worlds. Millions of children and families across the US are learning more science and improving their critical thinking skills after predicting, observing and explaining experiments with our patented system at their schools and museums.

screenshot of the game Anti-Phishing Phil, named after an orange fish in an underwater world that helps players to learn more about phishing

Protecting Millions of People from Phishing Scams

We studied the social aspects of phishing attacks and protected millions of people through education and training. This work led to new educational methods to raise awareness, effective anti-phishing warnings, and algorithmic detection of phishing attacks.

side by side screenshots of Tiramisu’s user interfaces for its 3 versions: V1 (left), V2 (center), and V3 (right)

Tiramisu Supported Public Transit Riders and Research

Years before the public transit buses in Pittsburgh had GPS location trackers, we were exploring how riders could be the co-creators of mobile transit info. During this 10 year research project, this app made it easier for travelers with disabilities to engage in opportunistic travel and benefited 75,000 unique transit riders.

screenshot of the Bloomwood stories game with enhanced overlay

Developing Novel Interfaces for Livestreamed Gaming

Livestream gaming viewers are limited in the ways they can participate, but this team found that designing a better experience for livestreams has a variety of benefits for audiences worldwide.

3 side by side images of a bar chart with improvements to each. Rather than the whole chart being 1 image, the middle segment of the first bar is highlighted for Building A, Product Category 2, Count 15

Creating Accessible Visualizations with Chartability

The Chartabilty framework has empowered people around the world to recognize the parts of a data visualization that produce barriers for people with disabilities. It has contributed to dozens of design languages, visualization guides, accessibility curriculum at universities, and the data journalism seen on news websites.

hands interacting with a black touch screen with labeled inputs of fingernail, knuckle and pad

TapSense Improved Touch Detection on Millions of Smartphones

We trained smartphones to reliably detect four different touchscreen inputs – for example, a tap by a finger pad as compared to a knuckle – which created opportunities for new interactions and features.

visual of the AI development lifecycle

Supporting Developers So They Can Build More Responsible AI

We conducted the first studies with industry practitioners to understand the challenges they face when building responsible AI systems. Our tools and methods are now supporting developers around the world.

PrivacyGrade

Grading Your Smartphone Apps on Their Privacy Practices

We created a model that analyzed over 1 million smartphone apps on their privacy and data collection practices, and then assigned them a public-facing privacy grade. This easy-to-follow grading system raised public awareness and led to improved privacy practices from several app developers.

decimal point game map

Improved Math Mastery with the Decimal Point Game

We explored the learning analytics behind our digital learning game to see how student learning responds to curriculum changes to the game. Over the past 10 years, over 1,500 students have benefited from the Decimal Point game and curriculum materials.

A handwritten sign in an office reads "Call Screening"

Understanding the Impacts of AI in Child Welfare

We conducted the first field studies to understand how child welfare workers incorporate AI-based recommendations into their decision-making, which had far-reaching policy impacts.

The point of view is from the back of a classroom while students are seated at 4 rows of computers while working on math problems. The teacher can see progress indicator icons above some students, such as a question mark when the student needs help

Designing Teacher-AI Collaboration Systems

Hundreds of elementary and middle school students across the US have learned more as a result of our work on real-time, teacher-AI collaboration systems in K-12 classrooms.

screenshot from PAT, a cognitive tutor for algebra

Millions Learned More Math with Our AI Tutor

Millions of middle school and high school students across the US have learned more math as a result of our decades of work with AI-based cognitive tutoring systems.

proposed data label for IoT devices, modeled after a nutritional label

Human-Centered Privacy for the Internet of Things

As more gadgets become “smart” things and the Internet of Things (IoT) expands to tens of billions of connected devices, we want consumers to be aware of what their devices are doing with their personal data.

Zensors airport queue screenshot

Zensors Turns Cameras Into Smart Sensors

In one application installed at the Pittsburgh International Airport in 2019, Zensors use existing cameras as powerful general-purpose sensors to provide wait time estimates for the security line, which benefits almost 10M PIT travelers per year.

Data Navigator offers developers a flexible foundation of structures, inputs, and toolkits for building inclusive data experiences

Accessible Visualization Tooling with Data Navigator

A visual labeled as “an image of a bar chart” or that requires a mouse hover to display important info does not support all users. We created the Data Navigator system to give developers a flexible foundation for designing and building more inclusive data experiences.