2019 Summer Research Program Faculty Projects | Human-Computer Interaction Institute

Faculty Member	Research Projects
Vincent Aleven	Developing smart glasses for middle-school teachers We are creating smart glasses for middle-school teachers so they (the teachers) can help their students more effectively while the students learning with intelligent tutoring software. These glasses thus help bring about effective human/AI partnerships. Currently, the glasses display analytics so the teacher is more aware of how their students are doing. We are interested in extending the glasses so they provide decision support - help the teacher decide whom to help and how, while respecting the teacher's autonomy. We are interested as well in designing the glasses so they can help teachers team up students as the need arises, so students can work collaboratively on challenging aspects of the materials. The work involves a mix of data mining, design, creating prototypes with mixed-reality technology, and trying them out with teachers. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Data-driven authoring of intelligent tutoring software Past research shows that intelligent tutoring systems - software that helps students practice solving complex problems, can help students learn very effectively. The effectiveness of tutoring software has been shown in many different subject areas, including mathematics. Inspired by these results, we have developed authoring tools that make the creation of tutoring software much easier. In some cases, it can even be done without programming. We now want to extend the authoring tools so that they support a data-driven approach to authoring. That is, by using data collected from actual students, it may be possible to build tutors faster, and to build tutors that are better attuned to students' actual learning process. The work involves data mining, design, building tool prototypes, and trying them out with tutor authors. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Making a tutoring system for middle-school students more engaging In research studies, we found out that Lynnette, a basic tutoring system for middle-school equation solving, helps students learn but also that it is not as engaging for students as we would like it to be. We made worked to make Lynnette more engaging, by adding a space theme, a badge system, and a drag-and-drop interaction format for equation solving. We would like to make Lynnette even more engaging. We have a variety of ideas, including personalized story problems, story problems about space, adding a new theme (cats?), personalized dashboards, and social features. We are open to other ideas as well. The work involves design brainstorming, co-design with middle-school students, prototyping, and trying out prototypes with students. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Helping middle-school students gain skill and conceptual understanding in algebra From our past studies we know that Lynnette, a basic tutoring system for middle-school equation solving that we created, is very effective in helping students learn to solve equations. It is clear as well, unfortunately, that students' conceptual understanding of algebra does not improve much, when working with Lynnette. In this project, we are extending Lynnette, so it helps students come away with a much better mix of conceptual knowledge and equation-solving skill. We will design worked examples, self-explanation prompts, and interactive diagrams. The work involves design, developing prototypes, and trying them out with middle school students.
Jeff Bigham	Tools to Assist Online Workers Underlying all Machine Learning systems is data, and a lot of that data is produced by underpaid "crowd" workers. In this project, students will create machine learning systems to help improve worker efficiency and efficacy, in order to ultimately improve their wages and other outcomes. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Making Memes Accessible People who are blind do not have access to visual content on the web, including online memes. In this project, students will create computer vision models able to identify meme content, find its associated meme foundation, and use optical character recognition on the results to read the text. This will then be embedded into a web browser extension to add the text to images as blind users browse to make these images accessible. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Creating Accessible PDF Documents All of our research papers are produced in PDF format, yet many PDF documents are not created in an accessible way. In this project, students will put together a chain of tools in order to pseudo-automatically create accessible PDF documents. Students will encapsulate this in a supporting web application framework in order to make this toolchain accessible to web users.
Laura Dabbish	With Jason Hong: Design and evaluate better user interfaces for changing people's cybersecurity behaviors
Jodi Forlizzi	With John Zimmerman and Carolyn Rose: Investigate how clinicians in emergency rooms might better access clinical pathways. Want to make it easier for them to find the pathway they want and get the information from it they need. Want to improve visualization of pathway. Want to improve logging between pathway and EHR.
Jessica Hammer	Design and study in-class, real-time systems for peer feedback - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Design and develop interventions to support more productive conversations about race and racism - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Prototype in-home interactions with sticker-based RFID interfaces - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Design and develop novel interactions for a hybrid virtual-physical home
Jason Hong	Build Android smartphone privacy tools and user interfaces - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Build IoT privacy tools and user interfaces - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - With Laura Dabbish: Design and evaluate better user interfaces for changing people's cybersecurity behaviors
Aniket Kittur	Design and build new ways for people to make sense of information (e.g., searching the web, scientific papers) - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Accelerate innovation and discovery (e.g., finding and using analogies from other products or fields)
Kenneth Koedinger	LearnSphere: A community data infrastructure to support learning improvement online. The LearnSphere project integrates existing and new educational data and analysis repositories to offer the world’s largest learning analytics infrastructure with methods, linked data and portal access to relevant resources. - One student researcher will help analyze large amounts of educational data from students enrolled in online courses to help identify best learning practices. - Another student researcher will contribute to designs that enhance the general usability of the Tigris workflow tool. The intern will need to draw on their Adobe, HTML and CSS skills and prior experience to create mock-ups and implement team-approved designs. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - With Paulo Carvalho: Using interaction designs and Machine Learning to understand human learning The project explores learning processes by contrasting learning from retrieval practice and learning from studying examples. The goal of this project is to resolve and clarify how these processes compete for cognitive resources, including attention and working memory, in ways that depend on the knowledge content to be learned using a combination of experimental and computational approaches. - One student will contribute to the design and testing of Cognitive Tutors to be used as part of this project. - Another student will contribute to the development of computational models of learning, such as the Apprentice Learner architecture.
Chinmay Kulkarni	Integrating voice interfaces with direct manipulation: Google Assistant and similar systems have promise a more "natural" way to interact with computers. However, point-and-click (or touch) interfaces are still better at some tasks (such as choosing among a list, or seeing the differences between two options). In this project, students will create systems and design patterns for touch-and-talk interfaces that combine the strengths of both interaction modes. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Voting algorithms that people can work with: People are voting all the time, with Likes, upvotes and even project preferences for their summer internships! How should the "user-interface" of these voting algorithms be designed? How could they compensate for human biases (such as preferring a "sure thing" to an uncertain but larger reward)? In this project, students will test a wide variety of voting interfaces for their effects on human decision making. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Helping teachers teach better with a community: Teachers frequently turn to other teachers for help. How can we leverage this inherently social process to spread better teaching practices? In this project, students will create several small prototypes and pilot them with a small set of teachers - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Micro-scale peer feedback: Peer review and feedback are commonplace in classrooms. We're interested in "micro-scale" peer feedback systems that allow, for instance, two strangers on the Internet teach other about related concepts while they read the same news article, or two online creators (on Youtube or Etsy) giving each other feedback on how to succeed as a freelancer. In this project, students will create a Google Chrome extension to allow users to connect to each other and engage in micro peer feedback.
Marti Louw	Design, prototype and evaluate learning support tools (mobile app based games and tools) for citizen science water quality monitoring and environmental education - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Conduct design research, develop concepts, prototypes and test smart documentation (IoT) tools to support creative practice and learning Maker and project-based teaching and learning environments
Bruce McLaren	Porting a Chemistry Intelligent Tutor and Tests from Flash to HTML5/CSS3/JavaScript For this task, the intern will port online instructional and assessment materials for chemistry from Flash ActionScript to HTML5/CSS3/JavaScript. This task is for an educational technology research project in which intelligent tutors are being used to help high school students learn stoichiometry, a topic within an intro to chemistry course. By adopting a responsive design approach, the student will test the tutor on multiple devices to ensure responsiveness. The intern should have an interest in education and learning from technology, a computer science and programming background, and experience using web programming frameworks, HTML and JavaScript. The intern should also be willing to learn new technologies. The intern will learn research on learning with educational technology, how to develop web-based applications using industry standards and best practices - responsive web design, JS templating and design patterns - and version control, unit testing and debugging web applications. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Working with an Digital Learning Game for Learning Decimals For this task, the intern will implement error messages and hints within a digital learning game, and potentially undertake more substantial modifications to investigate questions about learning within digital learning games, all while maintaining a gameful design consistent with the context in which they are embedded. This task is for a learning games research project in which games are being used to help middle school students learn decimals. The intern should have an interest in education and learning from technology, a computer science and programming background, and experience using web programming frameworks, HTML and JavaScript. The intern should also be willing to learn new technologies. The intern will learn research on learning with educational technology, how to develop web-based applications using industry standards and best practices - responsive web design, JS templating and design patterns - and version control, unit testing and debugging web applications. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Working with an Intelligent Tutoring System for Learning Decimals For this task, the intern will implement error messages and hints within an intelligent tutor, and identify and implement other opportunities to improve the software. This task is for a educational technology research project in which technology is being used to help middle school students learn decimals. The intern should have an interest in education and learning from technology, a computer science and programming background, and experience using web programming frameworks, HTML and JavaScript. The intern should also be willing to learn new technologies. The intern will learn research on learning with educational technology, how to develop web-based applications using industry standards and best practices - responsive web design, JS templating and design patterns - and version control, unit testing and debugging web applications.
Brad Myers	Obsidian is a new programming language for writing software for blockchain platforms. We are designing Obsidian using a user-centered design process in order to make Obsidian as usable as possible by programmers, while integrating strong features to detect bugs at compile time. We are looking for students with an interest in usability of programming languages or programming environments to work on the Obsidian project. This might include building tools for programmers or designing and executing user studies, depending on your skills and interests. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Intelligent virtual assistants on smartphones like Siri and Google Assistant can perform tasks on the user's behalf, but their capabilities are limited to the apps and services they support, without a way for users to teach them new tasks. Prof. Brad Myers and PhD student Toby Li in the HCII (along with collaborators across SCS) have a research project on designing and implementing a multi-modal smartphone intelligent agent that enables the users to teach it new procedures, concepts and user preferences by demonstration and natural language instructions. We have finished the development of a first version of the system and are looking for students to help design and develop new features. More details about this project is available at http://www.cs.cmu.edu/~NatProg/sugilite.html and http://toby.li/research/. Possible projects may focus on a wide range of topics from interface design and user research to semantic parsing, dialog systems and machine learning depending on student interests. Required skills: Strong Java programming skills Preferred skills: Android development, natural language processing (especially semantic parsing), machine learning, user-centered research and design
Adam Perer	Designing data visualization for Interpretable Machine Learning - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Designing visual interfaces for Data Science in Healthcare
Dan Siewiorek	Deep neural networks are now in widespread usage for classifying images of objects. There is increasing interest in classifying activities, such as pitching a baseball. Often, such problems can be solved to some extent by recognizing the objects (holding a bat means the actor is not a pitcher). Our problem is not solvable by looking at objects, but rather by looking at movements. We would like to break up actions into meaningful units, based on Kinect (3D camera) data. We have had some success in using the actions when a person does a back exercise to recognize which of a set of exercises they are doing. In this project, students will: - Use the Kinect to extract information about a person’s sequence of movements - Use the data to determine useful movement components for detecting sub-actions in the sequence - Characterize success in terms of recognizing sub-actions and determining their timing - This is an open-ended project question and may require strong problem solving and engineering skills. Experience with Computer Vision and DNNs is preferred. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Using an Automatic External Defibrillator (AED) requires many pre-usage checks before users can proceed to mounting the AED. One of these checks is to confirm that the patient does not have a pacemaker implanted on the chest as the implant can interfere with placement of the AED pads. Currently, there is no elegant solution for the cognitive assistant to detect the pacemaker other than asking the users for confirmation. But due to lack of knowledge of pacemakers, previous user studies showed that the users were confused by the task. In this project, students will: - Develop possible solutions to detect implanted pacemakers - Integrate it with existing wearable cognitive assistant on using an AED machine - This is an open-ended project question and may require strong problem solving and engineering skills. Experience with Computer Vision is preferred.
John Zimmerman	With Jodi Forlizzi and Carolyn Rose: Investigate how clinicians in emergency rooms might better access clinical pathways. Want to make it easier for them to find the pathway they want and get the information from it they need. Want to improve visualization of pathway. Want to improve logging between pathway and EHR. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - With Amy Ogan: Prototype how a smart classroom that detects student and instructor behaviors might provide rich feedback that helps instructors improve their teaching. Design interfaces for teachers to review and reflect upon the feedback. Match feedback to theory on what constitutes effective instruction. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - With Jodi Forlizzi and Aaron Steinfeld: Prototype new behaviors for robots. We are particularly interested in re-embodiment and co-embodiment. Re-embodiment is where a robot personality jumps from body to body to follow a person as they complete a task. Co-embodiment is when two robot personalities reside within the same body, such as when an intelligent agent jumps into a driverless car so the traveller can compete tasks while in transit.

Faculty Member

Research Projects