Research

ASSESSING CREATIVE PROBLEM SOLVING

New methods being added to crew members to go to space, but there's a question mark in whether they can problem solve

OUR GOAL

IDENTIFYING GAPS IN SOLVING ANOMALIES

Currently, NASA is researching new ways to aid astronauts in solving anomalies in space, but there is no standard method to assess readiness for their nearest furthest mission: going to Mars.

Our goal is to empower researchers to better identify gaps in anomaly resolution through evaluating teams’ performances in creative problem solving scenarios. To achieve this, we began researching extensively in relevant areas.

BACKGROUND RESEARCH

UNDERSTANDING THE PROBLEM SPACE

MHCI Team NASA looking at whiteboard of stickies

What are anomalies? What is creative problem solving? We started out with this project like a blank sheet, knowing so little about NASA and their missions and the problem at hand. We went through various stages of extensive background research to help us set a solid foundation of understanding.

Literature Review

Understanding NASA HSIA, training procedures, creative problem solving

NASA Interviews

Astronauts, spaceflight trainer, researcher, flight controller

Analogous Interviews

Pilot, new parent, game designer, scientist, designer

Anomalies

Historical understanding of anomalies and the qualities that made them difficult to solve

RESEARCH FINDINGS

KEY FINDING #1

Current CPS evaluation methods do not account for NASA’s high-risk, isolated, and low support environment.

NASA cannot fully rely on current studies to assess the effectiveness of their complex systems within these unique conditions. Therefore, they must develop their own standards and methodologies tailored specifically to their operational environment.

“Our lives are hours of boredom punctuated by moments of terror.”
- NYPD Police Officer

KEY FINDING #2

Crew members should rely on their breadth of knowledge to buy time for mission control to intervene when necessary.

We found that the ability to problem solve heavily depends on one’s expertise, and it’s not reasonable to expect 4 astronauts to have the same depth of knowledge as 85+ specialists in such a complex system. In these cases, astronauts must use their breadth of knowledge and just the right amount of depth to buy enough time for Mission Control to provide help.

“The goal is not to solve the problem, the goal is to gain more time”
-Kristina Holden, HRP Researcher

Mission control with many tables and screens

KEY FINDING #3

Complex, close to real-world scenarios can help researchers be more confident in their evaluations

Going into outer space is a risky endeavor. Therefore, researchers need a lot of confidence in their evaluations before sending people into the void. However, the most realistic evaluations come from being in the actual environment, which is not possible for NASA. The closest best option is to create close to real-word scenarios. However, in the current state, most studies on problem solving in NASA is conducted with procedural tasks.

“We need more realistic motivation and consequences”
- Kelli Pido

Cameras and computers monitoring people going through emergency training

FRAMEWORK

TASKS AND METRICS

How might we design cascading, scalable tasks that generate evaluative metrics?

Bringing everything together, we realized we need both task design and metrics to better understand crew members problem skills, so our system should include both. However, both the task and the metrics should be informed by how we define CPS to produce effective results.

A model where the creative problem solving model will help build tasks, and produce metrics

STEP #1

Defining the essential qualities of Creative Problem Solving for NASA - the ARRC model

In order to evaluate CPS, we needed to know what to evaluate. Through our background research, we were able to determine the human qualities that were essential to successfully problem solving.

Adaptability: The ability to be resilient to change.
Resourcefulness: Utilization of both your skills and the resources around you to deal with problems in creative ways.
Risk assessment: Predicting future scenarios and prioritizing the most urgent task.
Communication: Being able to be open on a team and communicate effectively.

STEP #2

Designing tasks through understanding anomalies

Tying in with our finding, the task should bear similar characteristics to real-world anomalies to enhance researchers’ confidence. We analyzed past anomalies in NASA to determine what makes them so deadly.

STEP #3

Preliminary metrics for play testing

Based on the ARRC model, we created several metrics for each of the qualities. Using the game Keep Talking and Nobody Explodes, we conducted our initial play tests to see how these metrics reflect creative problem solving.

Team members looking at screen observing a play testing session