Evaluation
To reproduce the user experience of a time-pressured execution environment to effectively test the usability of the interface, we evaluated our prototype across 12 user tests comprised of a planetary surface mission simulation. Specifically, nine unique participants used LuMo to complete mock geological science tasks, specified by a time-constrained plan, at several locations around the NASA Ames Research Center campus. The simulation sought to reproduce the physical constraints of the space suit, remote communication with ground support, and the need to dynamically re-plan to return to base camp on time.
In addition to user evaluation, we utilized several Human-Computer Interaction methods, including speed dating, needs validation, bodystorming, heuristic evaluation, and think-aloud protocol.
Across our 12 user tests, participants consistently improved in executing the plan as measured by our key metrics: execution errors and total time to completion. These results suggest that our iterative design process produced a well-vetted and usable tool. In addition, expert user feedback from an astronaut specializing in planetary surface missions confirmed that we achieved our core design goal of creating an optimized view of the plan for astronauts that sup- ports the flexible and autonomous execution of planned activities.
