Explore, Refine, Persuade: HCII Faculty Take Playtesting to the Next Level
News
The last time you sat down to play a game — either the tabletop or video variety — how much thought did you give to how the game was developed? Did you think about the creators' motivation? How they designed the world you're briefly inhabiting? What kind of testing and revising it took to transform the game from a spark of inspiration to the experience you're about to enjoy?
Chances are you didn't think about any of those things. Fortunately, an interdisciplinary team of Carnegie Mellon researchers thinks about them constantly, and they're training the next generation of game designers to do the same.
Future game designers have plenty of opportunities to study their craft. They can take courses on making better user interfaces and user experiences, for example. But in conversations with her Entertainment Technology Center (ETC) colleagues, HCII and ETC Assistant Professor Jessica Hammer noticed that student game-design teams struggled with playtesting their work. When she and ETC Teaching Professor Mike Christel explored the problem further, they realized that students weren't actually being taught how to playtest their ideas because there's no set curriculum for teaching it.
The two teamed up with HCII Professor Jodi Forlizzi to change all that.
"We see playtesting as a process of making better games through engagement with players from ideation all the way through the final product," Hammer said. "We're really interested in that end-to-end trajectory."
To help the next generation of game designers understand playtesting and how it's used in game design, the CMU team developed a series of first-of-their-kind workshops, each of which focuses on a specific set of skills students need at a specific point in the game-design process. While students can access workshop materials online, the series is designed to be used in physical classrooms. "These workshops are designed to be in-person workshops with game design teams and a trained facilitator using the materials we're distributing online," Hammer said.
Hammer and her colleagues designed the first workshop, "Explore," to help game creators work with players at early stages, even when they're not sure what form their game will take. Using a composition box technique developed by HCII Ph.D. student Judeth Oden Choi, the design team first tries to articulate the inspirations and experiences they bring to the project and connect them to the way they want the users to feel when they play the game — basically, creating a vision of their ideal player experience. "Explore" also teaches the designers how to ask good questions, create a research plan using minimal prototyping, and observe players and listen to what they're really trying to say. The workshop is entirely hands-on, with students participating in actual exercises to learn early-stage playtesting techniques. At the end, the teams should be able to use the skills they developed and the data they gathered from playtesting to retool their initial game idea in a way that's meaningful to both the designers and the players.
While playtesting so early might seem a bit odd — after all, there's not much to actually play at the early stages — Hammer says that it often leads to more creative games.
"Integrating players really early can help you get away from your own misconceptions and assumptions about what's fun," she said. "That's one benefit of engaging with users early: you surprise yourself."
The second workshop, "Refine," teaches students how to iterate their games — to create prototypes, test them and then create new prototypes based on the results of that testing. "The changes you make may have wildly unpredictable results in terms of the way the players interact with the whole system," Hammer said. To help with the process, "Refine" covers important playtesting tools like collecting data, writing survey questions and conducting interviews. Most importantly, though, the workshop focuses on how to construct a hypothesis or question. "When you iterate your games, you're essentially making a hypothesis about how the new game will affect players," Hammer said. "How do you do that in a meaningful way?" "Refine" gives students the tools they need to actually help them do it.
In the third and final workshop, "Persuade," students learn how to use the data they collect from players to communicate with their stakeholders. A member of the team may need to justify a design decision to a fellow teammate, for example, or an entire team may need to convince an investor that their design is worth funding. In "Persuade," students learn how to understand the expectations of the audience they're talking to, and how to break down claims about their game. With that information in hand, the workshop helps students decide what evidence will best support those claims and then teaches them the testing techniques that can provide the data they need — things like expert panels, A/B testing, and pre- and post-testing. "Persuade" also focuses on helping designers use the data they collect to articulate how their decisions align with the goals the game is trying to accomplish, something that Forlizzi calls "designer's judgment."
"A designer uses their judgment to advance the research and development of a product, system or game — which will improve a given situation," Forlizzi said. "This can be informed by data, by sketching and making, or by relying on the designer’s expertise and prior experience."
The HCII/ETC team's playtesting efforts don't stop at just designing and implementing the workshops, though. They've also been using the workshops as a way to study how game designers apply the concepts they're taught and to identify areas where the design teams struggle. While the research is in its early stages, the team hopes to use their findings to recommend best practices for game designers, and to develop the tools and methods they need to playtest successfully.
Why all this effort? It's simple. If it weren't for playtesting, you wouldn't have nearly as much fun playing that game.
This playtesting work is just one aspect of game design research efforts in the HCII. Other faculty members active in this field include Vincent Aleven, Geoff Kaufman, Bruce McLaren and Amy Ogan.
Story by Susie Cribbs