Process: User Testing

Our first round of user-testing was performed with contacts gathered at the Ames Arcjet complex.

Our first observation was that the users were inherently novices and unsure of various features of the device that would become familiar to trained experts. As NASA will make training mandatory, the value of the device in the hands of experts is the primary determinant for the value of the device as a whole. For example, users were unfamiliar and confused by quirks of the handheld interface such as tabbing between fields, selecting auto-complete options, and the shift modalities of the keypad, which effectively has three shift keys with multiple settings for uppercase & lowercase, blue shift, and yellow shift; as well as temporary and persistent mode shifts of each, without feedback. Users could not type numbers without use of the yellow shift.

Users were also unsure about the interaction of our interface design. The taxonomy of Constellation was new and confusing, and the interface vast and complex. Each entry of a problem report took upwards to 30 minutes. We ultimately determined that this was because we were forcing all three NASA roles into one interface, resulting in a task vastly more complex than necessary. We split the roles for further iterations.

Although we began compiling a tutorial to encourage familiarity with the taxonomy and handheld use of further iterations, and thereby move closer to true expert use of the design, some things could not be tested. Our novice users tend to grab the stylus first, and ignore walking over to synchronize with the base station, even though effective use of the keypad and base station synchronization could save time. Without a way to validly determine the most effective expert input balance of stylus & keypad, and handheld & base station, we ceased to focus on these factors in subsequent user tests.

One significant observation inspired a new direction in our design process. Users all invariably reacted positively to instances of auto-complete, which in this interface were limited to the native Windows spell-checking utility. We determined that designing an advanced form of auto-completion, specifically intended for our user base and created for the difficulties of mobile text entry, must be the future of our design.

A final outcome of the user-testing was an agreement to commit to a "flat" interaction, in which no depth of interface was permitted. In other words, every location in the interface must be both static and mutually exclusive to every other location, without exception, due to the minimal context awareness available through the tiny mobile screen. No two windows can be open simultaneously, and no file exploration trees can be permitted.