/ Approach

Building off of our generative research, we designed and iterated on prototypes that were evaluated through usability testing and feedback sessions with experts.

We used the GV design sprint structure as a base for our testing to help our team with fast and dirty iterations and generating more ideas faster. Each sprint was rooted in exploring a hypothesis, which led to feature ideation and sketches. Selected ideas where transformed into a prototype and tested with Subject Matter Experts and external participants.

Each sprint iterated on findings, from the previous sprint, to drive the next round of ideation and prototyping.

/ Situational Awareness

We broke down Situational Awareness into three parts:

Level 1: Perception of elements in the environment

Perception of elements in the environment, refers to the ability of the operator(s) to perceive the values, functioning and status of the system. The kinds of information that needs to be mediated vary between fields, and the means of delivering this information may use different modalities.

Level 2: Comprehension of the current situation

Comprehension of the current situation, addresses the understanding and interpreting of the data that the system mediates with the operator. Level 2 Situational Awareness (SA) is determined by the union of level 1 SA and the user’s goals and mental model. Providing good level 2 SA means allowing the user to understand the importance of the pieces of data, and consequently to map the information with the current goals the operator is pursuing.

Level 3: Projection of future status

Projection of future status, refers to the operator being able to predict how the elements of the system will vary in the near future. This ability allows for the operator to be aware of what will happen if changes are introduced or not in the system.

/ Focus Areas

What makes up to situational awareness during an Extra-vehicular Activity on Mars?

Our team determined the different pieces that contribute to situational awareness, and decided to focus on specifically addressing Time Math, Data Triangulation, and Conversational Context.

Time Math

Understand time as it relates to the communication delay. This includes calculating when a message will be delivered or was sent.

Data Triangulation

Quickly getting all necessary and related information from disparate areas of the interface to assist in understanding context.

Conversational Context

Understand what a message is referring to and its context.

Crew Needs

Understand and predict what crew needs to assist in task execution.
Specific to Mission Control Center

Crew Status

Know what crew members are doing and their current status.

Task Progress

Quickly determine task status to determine changes and assistance needed.

If a Message is understood

Know if a message is read, and content and context is understood.

Immediate Actions Needed

Know when certain messages need immediate actions.

Changes Affecting current status

Understand overall situational changes that may affect the current status of the crew or tasks.

Sprint 1

Hypothesis

Situational awareness in communication can be improved by designing for topic clustering which improves message comprehension, improves task success (decreases task error), time to complete task, depth of task completion, etc.

Goal

Assess how we might increase situational awareness in communication through topic clustering using high breadth or high depth prototypes.

Sprint 2

Hypothesis

An increase in effective, timely, and usable "time" features in Playbook and Mission Log will increases the crew's confidence that they have enough information to move forward with a decision about a task and help in their decision to send (or not send) messages to MCC.

Goal

Test multiple designs for assisting in time math and assess how effective they are in informing decisions about whether to send a message or not in the context of an EVA and its constraints.

Sprint 3

Hypothesis

(1) Message Details can provide a second layer to understanding when messages are sent and received so you know what context the receiver is getting.
(2) Feedforward of Time Math will help crew not need to do time math
(3) Message status can help crew understand the context of where there message is (Sending vs Sent)
(4) As sent and as received options can provide crew with options to view coupled information

Goal

Understand how time assists in understanding of how their relationship with messages in a time-delayed environment.

Sprint 4

Hypothesis

If we match the users mental model (by using a topic-based organization method for asynchronous conversations, instead of chronological method), the interface will allow for the user to understand context more quickly through lower number of clicks/decreased "lostness", decreased frustration/negative emotions, less questions or confusions (about the prototype), less errors?

Goal

Assess what is mental model for hierarchy that matches most closely (lostness/navigation) with users for asynchronous communication.

Sprint 5

Hypothesis

If we use a centralized chat format, then it will improve understanding of the conversation and time details more than a traditional side-to-side interface.

Goal

Create a two versions of a chat interface (or two versions of a feature within a chat interface (middle or side-by-side)), and assess which one better assists in a crew/MCC member's understanding of how their relationship with messages in a time-delayed environment

Sprint 6

Hypothesis

Our updated chat interfaces that parallels the mental model of asynchronous chat communication will assist in lowering cognitive and emotional burden of time math and message context.

Goal

Create a working prototype that features key functionalities of our new design and a working prototype that reflects existing chat interfaces seen in synchronous chat platforms. By comparing both versions we can assess how they may perform in context through a Simulated Mission.

/ Sprint Findings

Sprint 1

Coupling tasks and conversation is helpful in understanding context.

While crew may look for specific information, information needs to be coupled with additional context information for future prediction as well.

Sprint 2

Coupling of EVA tasks/procedures and EVA timeline was very useful for assessing level 2 and level 3 Situational Awareness (SA) questions.

Combining status and timeline was useful, and greatly improved the level 1 SA.

Sprint 3

Understanding time-math features is difficult - feedforward is essential for participants to be able to understand and utilize these features.

Having “Replied to:...” is useful for understanding context of messages.

Sprint 4

Consistent with previous research, participants seek to sort by thread or task first (topic), then by time or another factor (like subthread / subtask).

“Who is on which side of the time delay, visually?” is an important distinction.

Sprint 5

Message "Aether" helped differentiate which messages were in-transit and which were delivered and placing a message into the future is understandable.

An as-delivered set up was widely understood by participants.

Sprint 6

Prototype A (baseline) seems overall more successful than B (our design). This may be caused by time needed to learn and understand a new interface and lack of previous exposure to prototype B (our design) prior to the Simulated Missions.

Prototype B (our design) was more successful in reducing cognitive load over a standard chat interface.

See the final features we implemented ⟶

site blueprint

Carnegie Mellon University
Human-Computer Interaction Institute
National Aeronautics and Space Administration
AMES Research Center