Executive Summary
This website is an artifact of the MHCI capstone project for the Pittsburgh Science of Learning Center (PSLC). The MHCI project is an 8-month project that consists mainly of two phases:
User Needs Identification Phase
In the Spring Semester, or first phase, we performed research to determine the appropriate
requirements for the data shop. This document is a starting point intended to guide the
immediate (~2-3 year) development of the Data Shop. However, it is also expected to evolve and
change as the LearnLab studies come online and user needs adapt.
We began by soliciting users across the different LearnLab curricula in order to ensure that the
requirements would address the needs of all of the seven subjects. Finally, as a reality-check, we
developed a survey, which we conducted in-person with representatives from each of the courses,
to ensure that most of the general concepts required in the data shop (problems, errors, knowledge
components, hint requests, etc.) were present in each of the current and planned courses.
Appendix A: CI Models
Prototyping and User Testing Phase
The second phase of the capstone project focuses on iteratively designing and user testing a high-fidelity proof-of-concept for the design of the data shop. In the interest of time, the team was tasked with selecting a subset of the requirements that could be reasonably implemented over 18 months. We developed:
Learning Curves
Advantage: Difficult to produce by hand and valuable in their ability to measure learning during a study, long before pre- and post-test results can be compared.
Problem Profiles
Advantage: Provide researchers with a snapshot of the problem, including the problem text, which is essential to begin to understand and interpret the data.
Error Reports
Advantage: The need for Error Analysis was shared amongst all of the LearnLab courses that responded to our survey, including those whose researchers do not measure knowledge component-based learning.
Data Export
Advantage: The primary path to allowing users to export their data, which every user is likely to need to conduct personalized and in-depth statistical analysis.
Sample Selector
Advantage: This need emerged during our work on the above, and is a sub-set of the Data Export tool. It is intended to satisfy the needs of the audience to compare different groups of students at the same time.
Prototype
We hope that much of the future value of this report will be obtained via the design sections,
which provide an feature by feature breakdown of our design.
Design Section
These final designs were influenced by weekly user testing as well as client and advisor feedback.
We have also included some of the more interesting past design ideas with information about their
advantages and observed or anticipated problems.
User Testing