Executive Summary

This website is an artifact of the MHCI capstone project for the Pittsburgh Science of Learning Center (PSLC). The MHCI project is an 8-month project that consists mainly of two phases:

User Needs Identification Phase

In the Spring Semester, or first phase, we performed research to determine the appropriate requirements for the data shop. This document is a starting point intended to guide the immediate (~2-3 year) development of the Data Shop. However, it is also expected to evolve and change as the LearnLab studies come online and user needs adapt.

We began by soliciting users across the different LearnLab curricula in order to ensure that the requirements would address the needs of all of the seven subjects. Finally, as a reality-check, we developed a survey, which we conducted in-person with representatives from each of the courses, to ensure that most of the general concepts required in the data shop (problems, errors, knowledge components, hint requests, etc.) were present in each of the current and planned courses.

Appendix A: CI Models

Prototyping and User Testing Phase

The second phase of the capstone project focuses on iteratively designing and user testing a high-fidelity proof-of-concept for the design of the data shop. In the interest of time, the team was tasked with selecting a subset of the requirements that could be reasonably implemented over 18 months. We developed:

Learning Curves

Advantage: Difficult to produce by hand and valuable in their ability to measure learning during a study, long before pre- and post-test results can be compared.

Problem Profiles

Advantage: Provide researchers with a snapshot of the problem, including the problem text, which is essential to begin to understand and interpret the data.

Error Reports

Advantage: The need for Error Analysis was shared amongst all of the LearnLab courses that responded to our survey, including those whose researchers do not measure knowledge component-based learning.

Data Export

Advantage: The primary path to allowing users to export their data, which every user is likely to need to conduct personalized and in-depth statistical analysis.

Sample Selector

Advantage: This need emerged during our work on the above, and is a sub-set of the Data Export tool. It is intended to satisfy the needs of the audience to compare different groups of students at the same time.

Prototype

We hope that much of the future value of this report will be obtained via the design sections, which provide an feature by feature breakdown of our design.

Design Section

These final designs were influenced by weekly user testing as well as client and advisor feedback. We have also included some of the more interesting past design ideas with information about their advantages and observed or anticipated problems.

User Testing