Virtual Coach for Stroke Rehabilitation Therapy


This Virtual Coach evaluates and offers corrections and feedback for rehabilitation of stroke survivors. The Virtual Coach is composed of a tablet for clinician programming, a Kinect for monitoring motion, and a machine learning model to evaluate the quality of the exercise. Based on a set of rehabilitation exercises established by clinicians employed by our partner, Myomo, Inc., (e.g., bring a cup up to the mouth, lift an object from the floor and chair, one/two arm stand up and sit back, walking, etc.) we established the correct and most typical erroneous  postures and movements for those exercises. A normalized Hidden Markov Model (HMM) was trained to recognize correct and erroneous postures and exercise movements. Parameters for the HMM were selected to be normalized to height and distance between joints as measured in a calibration exercise. Feedback for typical problems with each exercise (e.g. go faster, lift higher, keep your posture straight, repetitions are too slow/fast) was solicited from therapists and built into the audio feedback to the patient. Graphical representation of the patient progress, performance  and scoring function outputs are presented on a dashboard, and can be easily interpreted by clinicians and patients.  Encouragement and corrections to techniques are provided by audio and graphical feedback. This Virtual Coach was demonstrated to clinicians at the Vincentian Rehabilitation Center in Pittsburgh, at in-service sessions for clinicians and therapists at the UPMC Mercy Rehabilitation Institute and HealthSouth Rehabilitation Hospital in Sewickley and Harmarville, Pennsylvania, as well as a pilot user study at CMU. We have also developed a game suite related to the stroke rehabilitation exercises, including self-calibration. MS Kinect is used for detecting user's motions. The combination of several games and option to chose one is shown on a menu looking like an island. Each game relates to a prescribed exercise. The set of available games is expending and should adjust its difficulty to the user current state of rehabilitation and improved capabilities.