Research Projects

Project #1
A New Bridge to the Digital Economy: Integrated AI-Augmented Learning and Collaboration

Mentor: Carolyn Rose, faculty

Description: This three-year NSF Future of Work (FOW) project, which started in October of 2022, seeks to address shortages of IT workers, while creating cost-effective, accessible pathways to living wage digital economy jobs for workers who previously lacked those opportunities. The tools and knowledge created by the project could eventually be applied to other STEM-focused community college degree programs across the nation, potentially impacting the lives of millions. This interdisciplinary project offers numerous opportunities to embed undergraduate research experiences related to advances in AI to enable learning interventions, design of interventions motivated by learning sciences principles, or development of extensions to an AI-augmented learning platform. The project will tackle five research questions: First, how can the Knowledge-Learning-Instruction (KLI) framework developed by learning scientists be used to align knowledge components in community college IT courses with the most effective AI-driven educational technologies to enhance and accelerate learning of those components? Second, to what extent do intelligent tutoring systems (ITS) and computer-supported collaborative learning (CSCL) experiences increase mastery and decrease the time needed to achieve it? Third, to what extent and in what ways do forms of example-based learning, used together with ITS and CSCL, further support learning and enable a wider range of learners to succeed? Fourth, to what extent can CSCL technology foster effective collaboration between community college students in 2-year information technology degree programs and the professional staff of partner firms on real-world (cloud computing) problems in the context of capstone projects and internships? Fifth, how successfully do students in the AI-augmented curricular pathway created by this project move into IT jobs relative to students in the standard course pathways?

Nature of Student Involvement: The two REU interns will work closely with McLaren, Rosé, and Teffera to achieve the project goals described above. The REU interns will be involved in weekly project meetings with McLaren’s or Rosé’s in which the research goals of the project and progress will be discussed. The REUs will also have the opportunity to work closely and exchange experiences with many other undergraduate summer interns who annually work on human-computer interaction (HCI) projects within Carnegie Mellon’s Human Computer Interaction Institute. A variety of learning opportunities will arise within the summer program at CMU, including a poster session and research talks by CMU faculty that the REU interns are encouraged to attend.

Skills we are interested in (not all required):

• Experience with curriculum design
• Running user studies, programming in Python
• Artificial intelligence/NLP/Machine Learning

Project #2
Active learning in STEM education

Mentor: Paulo Carvalho, systems scientist 

Description: Mastering material in STEM classes requires students to learn and memorize large amounts of new knowledge in a short period of time. One way that has long been argued to improve such learning is by having students practice new knowledge by spacing questions over time (spaced retrieval practice). However, the evidence for the benefits of spaced retrieval practice in STEM contexts is limited. How can learn by doing be optimized in STEM classes and what computational algorithms best capture this learning?

Nature of Student Involvement: Students will be involved in all steps of experimental research, data analytics, and student modeling.

Skills we are interested in (not all required):

• Quantitative data analyses
• Experience with experimental design and data collection
• Data science/learning analytics/student modeling experience preferred but not required

Project #3
Advancing Metamaterials by exploring novel structures, developing design tools and fabrication methods

Mentor: Alexandra Ion, faculty 

Description: We are looking to push the boundaries of mechanical metamaterials by unifying material and device. Metamaterials are advanced materials that can be designed to exhibit unusual properties and complex behavior. Their function is defined by their cell structure, i.e., their geometry. Such materials can incorporate entire mechanisms, computation, or re-configurable properties within their compliant cell structure, and have applications in product design, shape-changing interfaces, prosthetics, aerospace and many more.

In this project, we will develop design tools that allow novice users and makers to design their own complex materials and fabricate them using 3D printing or laser cutting. This may involve playfully exploring new cell designs, creating novel application examples by physical prototyping and developing open source software.

Nature of Student Involvement: Students will be part of all stages of research and will be fully embedded within the lab. We aim to give students a good insight into the nature of research and make it a fun summer!

Skills we are interested in (not all required):

• CS skills: software development, background in geometry, optimization, and/or simulation
• 3D modeling basics (CAD tools, e.g., Autodesk Fusion 360 or similar)
• Basic knowledge of classical mechanics or material science

Project #4
AI in the Accessible Kitchen: Supporting Blind and Visually Impaired People in Performing Activities of Daily Living

Mentor: Patrick Carrington, faculty 

Description: Poor nutrition is prevalent among people with vision impairments. Studies have shown that this poor nutritional status is due to a number of factors, including social and structural issues, financial barriers, as well as independent meal preparation challenges. Previously reported aversions to cooking have led to dietary choices that include eating at restaurants over 40% of the time. The significant financial burden of these choices, combined with the aversion to cooking “from-scratch meals” leads to the alternative option of buying and cooking frozen or prepared foods which are costly, unhealthy, and calorie-rich. Challenges associated with preparing meals include sensory, procedural, and physical challenges. Our research has aimed to address this gap by developing systems that bridge the digital and physical challenges faced by vision impaired people to enable more independent meal preparation. This project would involve hardware and software prototyping as well as user studies.

Nature of Student Involvement: The student would be involved in hardware and software prototyping as well as early user tests.

Skills we are interested in (not all required): The ideal student has some experience with 

• Sensors 
• UI design/development
• Qualitative methods/data analysis

Project #5
AI Privacy

Mentor: Sauvik Das, faculty

Description: This project focuses on advancing our vision of "Privacy through Design" in the development of AI products and services. REUs will collaborate with PIs Das and Forlizzi, as well their Ph.D. students. The project entails co-designing materials aimed at helping AI practitioners prioritize privacy in consumer-facing AI products. It builds upon two key research efforts: interviews with 35 industry AI professionals to understand their privacy practices, and the development of a taxonomy of consumer AI privacy harms based on AI incidents and failures. The participating undergraduates will have the opportunity to brainstorm, create, and evaluate tools and methods designed to mitigate privacy risks in AI product design, contributing directly to the evolving landscape of AI and privacy.

Nature of Student Involvement: Students will be involved in ideating, creating and/or evaluating tools and resources to help AI practitioners mitigate privacy risk

Skills we are interested in (not all required):

• Programming experience to build prototype systems
• Experience with qualitative methods (e.g., interviews)
• Prototyping systems with LLMs

Project #6
AI-CARING: Agents, Care Coordination, Trust and Affiliation

Mentor: Mai Lee Chang, postdoctoral fellow

Description: AI-CARING is a NSF AI Institute that is committed to both doing foundational AI research and developing technology that is useful and beneficial for society. The overall project focuses on developing AI systems to aid older adults, including those who experience cognitive decline, to continue living in their homes longer.

Our thrust focuses on how AI systems can learn the structures and forms of people's interpersonal relationships and how agents can provide support for tasks of daily living that support people's performance of self and reinforce their close familial bonds, friendships, and relationships with professionals like their doctors and other care providers.

Over the summer, we want to explore what an agent will need to learn about older adults’ and their informal caregivers’ (e.g., spouses, adult children, neighbors) day-to-day activities coordination (e.g., getting to doctor's appointments, getting food, arranging services, picking up meds) in order to provide support that is robust to uncertainty and changes in goals, care network structure, and environment. This work will capture the practices, priorities, values, triggers, and breakdowns in coordination. We are also interested in identifying critical moments or triggers that lead to changes in trust and affiliation (i.e., who the agent works for) when an agent or robot interacts with an older adult and their surroundings to design more trustworthy agents/robots for successful long-term support.

Nature of Student Involvement: Student research assistants will aid researchers in designing and executing studies to understand the needs and challenges that older adults and their informal caregivers face when making care coordination plans and how they adapt to changes. This will take a retrospective approach, interviewing stakeholders and reviewing their communication logs, calendars, and other coordination artifacts to reconstruct how they accomplished various care tasks. Student researchers will also conduct field study to discover triggers of trust and affiliation changes to understand what information an AI agent and robot need in order to learn about the changes in trust and affiliation.

Skills we are interested in (not all required):

• Fieldwork including observations, interviews, and directed storytelling
• Brainstorming
• Design of conversational interfaces
• Understanding of social psychology: social interaction, trust, affiliation
• Design and execution of user studies

Project #7
Chemistry Tutor Machine Learning Programmer

Mentor: Bruce McLaren, faculty

Description: Learn about intelligent tutoring systems and how to apply machine learning to them! You will be responsible for the development of machine learned detectors designed to identify specific student behaviors for the Stoich Tutor (https://stoichtutor.andrew.cmu.edu/). You will first complete work on the development of a detector for “gaming the system” and then move on to additional detectors. You will work with the research team that is investigating the link between behavioral, cognitive, and affective aspects of students and their engagement with the Stoich Tutor. You should have a computer science background with skills in HTML5/CSS3/JavaScript, Python, and an optional background in chemistry and familiarity or interest in machine learning. You will work with Prof. Bruce McLaren and Research Programmers Hayden Stec and Leah Teffera, with Stec and Teffera as the primary mentors.

Nature of Student Involvement: Programming and using machine learning to develop detectors of student behavior

Skills we are interested in (not all required):

• Computer Science background 
• Skills in HTML5/CSS3/JavaScript 
• Python is required 
• Background in chemistry and familiarity or interest in machine learning are optional

Project #8
Chemistry Tutor Programmer and Data Scientist

Mentor: Bruce McLaren, faculty

Description: Learn about intelligent tutoring systems and data science! You will be responsible for extending the Stoich Tutor (https://stoichtutor.andrew.cmu.edu/) and its associated grading script. The specific way in which the tutor and grading script will be extended will be determined by results from a study conducted during 2023-2024. Additionally, you will analyze data from prior studies to guide extensions to the tutor as well as assist in the overall project, in which the research team is investigating the link between behavioral, cognitive, and affective aspects of students and their engagement with the tutor. You should have a computer science background with skills in HTML5/CSS3/JavaScript, Python, and an optional background in chemistry. You will work with Prof. Bruce McLaren and Research Programmers Hayden Stec and Leah Teffera, with Stec and Teffera as the primary mentors.

Nature of Student Involvement: The student will program a tutor for chemistry as well as maintaining and extending a grading script that assesses log data from the program

Skills we are interested in (not all required):

• A Computer Science background 
• Skills in HTML5/CSS3/JavaScript
• Python is required
• A background in chemistry is optional but highly desirable.

Project #9
Cloud Administrator Intelligent Tutor Programmer

Mentor: Bruce McLaren, faculty

Description: In this project you will bring your knowledge of computer science and cloud computing to the task of building intelligent tutoring systems (ITSs) to help community college students learn about cloud computing. You will design and write code to develop intelligent tutoring systems that will be embedded in the SAIL Cloud Administrator course. The tutors will support local community college students in better understanding programming and computational thinking. You will learn about code repositories and good software engineering methodologies and practices. You should have a computer science background with skills in HTML5/CSS3/JavaScript, and an optional background in education (e.g. TA-ing a course) and familiarity or interest in cloud administration/computing. You will work with Prof. Bruce McLaren and Leah Teffera, with Teffera as the primary mentor.

Nature of Student Involvement: The student will develop intelligent tutors for an online Cloud Administrator course.

Skills we are interested in (not all required):

• Must have a computer science background
• Skills in HTML5/CSS3/JavaScript
• Optional background in education (e.g. TA-ing a course)
• Familiarity or interest in cloud administration/computing

Project #10
Codespec: a computer programming environment

Mentor: Carl Haynes-Magyar, Presidential Postdoctoral Fellow

Description: Despite the potential of Parsons problems, few environments offer a seamless transition between different problem types. Codespec supports learners in practicing how to solve a programming problem as a Pseudocode Parsons problem, a Parsons problem, a Faded Parsons problem, a fix-code problem, or a write-code problem. The goal of the project is to develop, evaluate, and implement algorithms for knowledge tracing, adaptive problem-sequencing, and ready-to-publish results that include learning curve analysis.

Nature of Student Involvement: Write code to develop back-end features for Codespec.

Skills we are interested in (not all required): 

• Experience or interest in computing education research, tools and environments.
• Experience with data science, learning analytics, student modeling, large language models.
• Preferred: CS (or other technical) major, HTML/CSS/JavaScript, Python, Django, VueJS, Figma skills.

Project #11
Computational Understanding of User Interfaces

Mentor: Jeffrey Bigham, faculty 

Description: The goal of our UI Understanding project is to build machine learning technologies that can learn to computationally understand and interact with user interfaces designed to be used by people. We have built technologies that understand graphical user interfaces from pixels, generate custom user interface code automatically, graphically reflow existing user interface to personalize them for specific abilities, and automate actions across different devices. Many of these projects target applications for people with disabilities who use user interfaces in ways other than those assumed by developers.

Nature of Student Involvement: Students will be involved in all aspects of research under the mentorship of senior graduate students and faculty -- including, defining project goals and scope, training or adapting large computer vision and natural language models, writing and presenting about results. Many of our past REU students have submitted their work to peer-reviewed venues and many have gone on to PhD programs in the area.

Skills we are interested in (not all required): Prefer students with familiarity with the following broad technical areas (will also get a chance to learn more during the REU!)

• Experience training/fine-tuning modern machine learning pipelines (computer vision, language, multimodal)
• Experience with one or more UI and interaction toolkit (e.g.., SwiftUI, React, others)
• Familiarity with LLM APIs and best practices (e.g., GPT4, Claude, etc)
• Knowledge of human-AI interaction, designing for ML systems, human-centered AI, etc.

Project #12
Designing algorithms for adaptive Extended Reality (XR)

Mentor: David Lindlbauer, faculty

Description: Extended Reality (XR) interfaces allow users to interact with the digital world anywhere and anytime. By embedding interfaces directly into users' environments, XR interfaces can both enhance productivity and be less distracting than traditional computing devices such as smartphones.

In this research, we aim to design algorithms that create context-aware XR interfaces, i.e., interfaces that automatically adjust when, where, and how to display XR interfaces. These algorithms can be optimization-based or learning-based, and form the backbone of XR systems that continuously adapt to users' needs and requirements when they interact with XR systems in a wide range of applications, from productivity, entertainment, manufacturing, or healthcare.

The research involves developing a novel algorithm for adaptive XR, and evaluating the approach in a comparative user study.

Nature of Student Involvement: Students will collaborate with other undergraduate and graduate students in creating the concept and planning of the research, and lead the implementation of the algorithm and the creation of the user study platform.

Skills we are interested in (not all required):

• Strong programming skill (c# or similar)
• Experience with Unity or Unreal
• Interest in XR

Project #13
Designing for workers’ experiences of health & wellbeing

Mentor: Franchesca Spektor, graduate student advised by faculty Sarah Fox and Jodi Forlizzi

Description: Low wage workers are at increased risk for injury and disablement while on the job. However, traditional ways of understanding, tracking, and reporting occupational injury may be insufficient. While formal reporting requirements from OSHA may address a torn ACL, regulatory bodies provide few avenues for reporting on the repetitive chronic strain which results in injuries over time. This project aims to conduct participatory design research with local service workers in the Pittsburgh region – from custodial staff to home care workers – to learn if new tools and technologies may help bridge the gap between health & safety policies and workers’ needs on the ground. We will explore training needs, threat of retaliation, administrative barriers to reporting, and health data governance.

Nature of Student Involvement: The REU students will be closely involved in the project, focusing specifically on understanding the health & safety needs of a local service context. The students will be responsible for conducting a literature review and may have the chance to:

• Conduct interviews, diary studies, and workshops with local workers
• Contribute to data analysis using thematic analysis
• Develop design prototypes to support health & safety reporting, and worker wellbeing

Skills we are interested in (not all required):

• The students should hold care and curiosity about worker wellbeing, workplace technologies, and labor issues
• Programming and technical prototyping skills for web and mobile applications
• Experience with user research, design thinking, and UX backgrounds
• Prior experience reading academic literature and conducting literature reviews

Project #14
Designing Inclusive Collaboration Environments

Mentor: Laura Dabbish, faculty 

Description: Open source software is important to sustaining the world’s infrastructure, and millions of volunteers help maintain it. However, growing evidence shows that people of different genders, particularly women, face particular barriers when contributing to open source software. Our research interviews people of diverse genders who have made significant open source contributions to understand how they became highly involved in open source, the barriers they face, and how they overcome them. We will also perform statistical analysis using data science on GitHub trace data to understand the extent to which our findings generalize, and the wider effects of barriers we uncover. Finally we are exploring interventions for enhancing inclusion in open collaboration environments.

Nature of Student Involvement: Students on this project will help us develop web based interventions for encouraging inclusive open source project collaboration environments, carry out interviews with open source contributors, and perform statistical analysis using data science on GitHub trace data.

Skills we are interested in (not all required):

• Front end web programming and UX design skills would be helpful for this project
• Strong organizational and interpersonal skills are important, other skills can be learned
• Any of the following skills helpful: experience conducting interviews, experience with data science pipelines (eg, using python, SQL or R)

Project #15
Developing Novel Interfaces for Live Streaming

Mentor: Noor Hammad, graduate student advised by Jessica Hammer

Description: This project explores how to build live streaming interfaces that afford new capabilities to viewers, streamers, and game developers. We will use a system architecture built by the Centre for Transformational Play that enables the creation of “game-aware” overlays for any Unity game streamed on the Twitch platform. You will be working closely with an interdisciplinary team to improve the system, add new features, and provide technical support for research studies on Twitch.

Nature of Student Involvement: The student will be collaborating with the research team on activities such as weekly meetings, low-fidelity prototyping, study design, and pilot tests. They will be expected to complete software development tasks independently.

Skills we are interested in: 

• Requirements:
  • Web development, particularly advanced JavaScript
  • Interest in live streaming and/or games
  • Prepared to use good collaborative software development practices (e.g. documentation, Git)
• Preferred skills:
  • Some experience with Unity game development
  • Experience working with AWS or other cloud services.

Project #16
Digital Learning Game Programmer

Mentor: Bruce McLaren, faculty 

Description: Decimal Point and Ocean Adventure are learning games developed in the McLearn Lab at CMU to help late elementary and middle school students learn about decimals and decimal operations. For this project, you will write and revise code to alter and extend the two games to prepare them for new classroom studies. You will learn about code repositories, state-of-the-art software engineering methodology, and good software practices. You should have a computer science background with skills in HTML5/CSS3/JavaScript, and an optional background in mathematics or education (e.g. TA-ing a class). You will work with Prof. Bruce McLaren and Research Programmer Hayden Stec, with Stec as the primary mentor.

Nature of Student Involvement: The student will be involved in design, development, testing, and revising of code for digital learning games.

Skills we are interested in (not all required):

• HTML5/CSS3/JavaScript
• optional background in mathematics or education (e.g. TA-ing a class)

Project #17
Digital Learning Games Quality Assurance Programmer

Mentor: Bruce McLaren, faculty 

Description: Decimal Point and Ocean Adventure are learning games developed in the McLearn Lab at CMU to help late elementary and middle school students learn about decimals and decimal operations, while Angle Jungle is a learning game to help middle school students learn about angles. For this position you will perform software quality assurance engineering to improve all of the McLearn Lab game materials — various decimal/angle tests, surveys, and three learning games (Decimal Point, Ocean Adventure, Angle Jungle) — testing the materials on all of the devices used in schools — an Apple laptop, an iPad and a ChromeBook -- and address any bugs that you found or have been previously reported. You will learn about code repositories, state-of-the-art software engineering and quality assurance methodology, and good software practices. You should have a computer science background with skills in HTML5/CSS3/JavaScript. You will work with Prof. Bruce McLaren and Research Programmer Hayden Stec, with Stec as the primary mentor.

Nature of Student Involvement: The student will be involved in design, development, testing, and revising of code for digital learning games

Skills we are interested in (not all required):

• Computer science background 
• Skills in HTML5/CSS3/JavaScript
• An interest in mathematics education is also valuable.

Project #18
Evaluate Impact of Transparency in Ride-Sharing Algorithm

Mentor: Seyun Kim, graduate student advised by faculty Motahhare Eslami and Haiyi Zhu

Description: Gig economy platforms such as Uber or Lyft use algorithmic decision-making that are blackbox and lack transparency in its decision making to users of the system including drivers. A way to evaluate and test the algorithm’s impact and output is to conduct algorithmic auditing. For this project, we develop an intervention to assess the impact of whether transparency of an algorithm’s decision-making process influences perceptions of equity. We will also assess what type of information to the users would be considered as more impactful.

Nature of Student Involvement: The students will be responsible for building an intervention in the form of a software interface that engages with gig economy workers. The software interface will be an additional add-on feature to an existing platform (project) with a research group at another institute. The student will be responsible for data collection and quantitative, qualitative data analysis to understand the impact of the intervention. The data collection will involve engaging with participants in the wild.

Skills we are interested in (not all required):

• Experience in software development (front end and back end) 
• Running experiments in the real-world (data collection in the wild) 
• Statistical analysis (quantitative analysis). The student does not necessarily need to have qualitative analysis experience.

Project #19
Human-Centered Data Science and Visualization

Mentor: Adam Perer, faculty 

Description: The Data Interaction Group (DIG) has a mission to empower everyone to analyze and communicate data with interactive systems. Our group conducts research in computer science at the intersection of human-computer interaction, machine learning, data science, programming languages, and data management.

This summer, we plan to build new tools for data scientists to help them better understand their data, which will hopefully result in better downstream machine-learning models derived from the data.

Nature of Student Involvement: Research, coding, user studies

Skills we are interested in (not all required):

• Programming experience
• Interest in data science and machine learning
• Web development skills

Project #20
Mixed-reality AI STEM Learning

Mentor: Nesra Yannier, senior systems scientist 

Description: This project focuses on developing a mixed-reality educational system and Intelligent Science Stations bridging physical and virtual worlds to improve children's STEM learning and enjoyment in a collaborative way. It uses depth camera sensing and computer vision to detect physical objects and provide personalized immediate feedback to children as they experiment and make discoveries in their physical environment. NoRILLA Intelligent Science Stations (www.norilla.org) are being used in many school districts, after school programs, children's museums and science centers (e.g., Carnegie Science Center, Children's Museum of Atlanta, Please Touch Museum, CaixaForum AI Museum in Spain). Research with hundreds of children has shown that it improves children's learning by 5 times compared to equivalent tablet or computer games. This REU project will focus on extending Intelligent Science Stations to different content areas, creating new modules and curriculum, designing new games and interfaces as well as collecting and analyzing data in schools and museums of children interacting with Intelligent Science Stations and Exhibits.

Nature of Student Involvement: The student will work closely with the project lead and will be involved in different aspects of the project including design, development and research. The student will help take the project further by developing new modules/games, computer vision algorithms/tools and AI enhancements on the platform and deployment of upcoming installations.

Skills we are interested in (not all required): 

• Familiarity with software and hardware components
• Familiarity with computer vision, interface development, Java/Processing, robotics and/or game design is a plus

Project #21
Scrolling Technique Library

Mentor: Brad Myers, faculty and HCII Director

Description: We have developed a new way to test how well a scrolling technique works, and we need to re-implement some older techniques to see how they compare. For example, the original Macintosh scrollbars from 1984 had arrows at the top and bottom, and a draggable indicator in the middle. Even earlier scrollbars worked entirely differently. I am hoping to recruit one good programmer to help recreate some old scrolling techniques, and possibly try out some brand new ones, like for Virtual Reality applications, to test how well they do compared to regular scrolling techniques like two-fingers on a touchpad or smartphone screen. If there is time, the project will include running user tests on the implemented techniques, and writing a CHI paper based in part on the results.

Nature of Student Involvement: The student on this project will be implementing all of the techniques as web applications.

Skills we are interested in (not all required):

• The student on this project must be an excellent programmer JavaScript or TypeScript
• Preferably with expertise in React or other web framework. 
• Experience with running user studies would be a plus.

Project #22
Studying Novel Live Streaming Interfaces

Mentor: Noor Hammad, graduate student advised by Jessica Hammer

Description: This project explores how people use live streaming interfaces to support shared attention in streamed experiences, including entertainment games and educational content. As part of this work, we will use novel live streaming “game-aware” interfaces developed by the Center for Transformational Play that allow viewers to customize their live streaming experience. You will be working closely with an interdisciplinary team to develop and execute studies into how viewers make use of these interfaces to support their streaming experiences.

Nature of Student Involvement: This student will collaborate with the research team on activities such as weekly meetings, participant recruitment, data collection, and data analysis. They will be expected to complete problem solving and logistical tasks independently.

Skills we are interested in:

• Requirements:
  • Interest in live streaming and/or games
  • Comfortable with independent work
  • Interest in mixed methods research
  • Good organizational skills
• Preferred Skills:
  • Experience with mixed research methods
  • Web development, particularly advanced JavaScript

Project #23
Supporting Designers in Learning to Co-create with AI for Complex Computational Design Tasks

Mentor: Nikolas Martelaro, faculty 

Description: Advancements in generative AI (GenAI) are rapidly disrupting creative professionals' work across a range of domains. To ensure that GenAI benefits creative professionals, rather than devaluing their labor, it is critical that we prepare the workforce to work with these technologies to effectively leverage their comparative advantages as humans. However, recent studies indicate that creative professionals face significant challenges in adopting GenAI successfully into their workflows. In this project, we will explore novel interactive interfaces and interaction patterns that allow professional designers to work more effectively with GenAI tools across different domains. First, we will iteratively build interactive prototypes and then evaluate their effectiveness through user studies. The results of this work will contribute to advancing future AI-augmented creative work.

Nature of Student Involvement: Attending weekly research meetings, supporting prototyping, and supporting preparation and facilitation of user studies

Skills we are interested in (not all required):

• Programming
• User research
• UX design
• Teamwork

Project #24
Supporting End-users in Auditing Harmful Algorithmic Behaviors in Generative AI

Mentor: Wesley Deng, graduate student advised by faculty Motahhare Eslami and Ken Holstein

Description: Despite impressive capabilities, text-to-image (TTI) generative AI also carries risks of biases and discrimination. Traditional algorithm audits done by small groups of AI experts often miss harmful biases due to the AI team's cultural blind spots and the difficulty of predicting the range of ways TTI systems will be used once deployed widely. Recent works from our research group have highlighted the effectiveness of end users in detecting biases overlooked by experts, demonstrating the value of engaging end users in algorithmic audits. Despite this potential, there is a lack of structured support and tools for public participation in auditing generative AI. In this project, you will build upon an existing interface (https://taiga.weaudit.org/) we developed to further design, develop, and evaluate new tools and mechanisms to better support end users in auditing and red teaming Stable Diffusion, an open source TTI model. Overall, this project aims to empower end users in the auditing process and enhance public involvement in creating a responsible and ethical generative AI landscape.

Nature of Student Involvement: Research assistants (RAs) will work closely with the research team and will be involved in the design, development, and evaluation of the system. This means the RA will get exposure to project ideation, rapid prototyping, front end web development (using Javascript/HTML/CSS/other web technologies), and conducting user evaluations.

Skills we are interested in (not all required):

• Experience in software development, and in particular the ability to learn new technologies
• Experience with web technologies such as JavaScript is preferred
• Experience in designing and conducting user studies evaluating interactive systems is preferred
• Familiarity with text-to-image generative AI (DALL-E, Stable Diffusion) is encouraged
• Interests in exploring topics such as Responsible AI, Human-AI Interaction, Algorithmic Fairness and Transparency

Project #25
Supporting middle school math homework with novel parent support tools

Mentor: Conrad Borchers, graduate student advised by Vincent Aleven and Ken Koedinger

Description: This project aims to design, implement, and test AI-based support tools that provide tailored recommendations to parents for how they might support their children's homework. The student will engage in high-fidelity prototyping and design research, studying parental engagement and student learning during interactions with the tool. The research output will contribute new scientific understanding of how cognitive and socio-emotional support can be merged productively. Prior work has identified that differences in parent styles during homework support relate to achievement gaps. Therefore, one potential research question is how to design interactive systems powered by AI that help parents and students adopt more favorable attitudes and approaches to homework.

The research extends prior project activities in the context of a grant on smart middle school mathematics homework support. We have conducted prototyping sessions with several students and their parents, identifying key needs for more effective and equitable homework support. We are also planning to pilot an initial prototype of the tool in late November. Therefore, the REU intern will contribute to design research activities at a stage of high fidelity.

Nature of Student Involvement: The REU intern will be tasked with conducting in-depth interview sessions and interactive usability studies with parents and children to refine the tool's design, including potential involvement in programming.

Skills we are interested in:

• Required: Demonstrated track record of conducting high-fidelity prototyping, usability testing, and user experience research, ideally in close collaboration with software engineers
• Desirable but not required: Experience in deploying machine learning applications in a web application, including experience with frontend engineering; experience with learning technologies

Project #26
Supporting Upper Extremity Health Monitoring and Management for Wheelchair Users

Mentor: Patrick Carrington, faculty

Description: Upper extremity (UE) health issues are a common concern among wheelchair users and have a large impact on their independence, social participation, and quality of life. However, despite the well-documented prevalence and negative impacts, these issues remain unresolved. Existing solutions (e.g. surgical repair, conservative treatments) often fail to promote sustained UE health improvement in wheelchair users’ day-to-day lives. In this project, we explore how health tracking technologies could support wheelchair users’ UE health self-care, including movement sensing, modeling body mechanics, and developing appropriate user interfaces for feedback and data analysis.

Nature of Student Involvement: The student will be involved in development, prototyping, and/or user studies.

Skills we are interested in (not all required): Candidates should ideally have experience programming, some experience with machine learning is helpful, experience with hardware and specifically IMUs is also positive.

Project #27
Tangible Privacy & Security

Mentor: Sauvik Das, faculty

Description: This project aims to address the persistent challenges of human error and negligence in cybersecurity and privacy by leveraging tangible computing. Building upon our lab's previous research - Spidey Sense, Bit Whisperer, and Smart Webcam Cover, we aim to overcome barriers related to awareness, ability, and motivation among end-users. Through the strategic introduction of tangible computing, our goal is to empower users with greater control and an enhanced understanding of privacy-invasive sensors in the physical world, thereby fostering proactive engagement in security and privacy practices. In pursuit of these goals, we are exploring two key ideas: 1) the development of a tangible control for privacy preferences in shared spaces, especially in large environments such as buildings or city-wide spaces, and 2) the creation of privacy-invasive sensors that provide clear indications of the data being captured and the range of data being collected—such as a webcam that visually communicates its field of view in the real world.

Nature of Student Involvement: Student will be involved in Ideation, Prototyping and Conducting Interviews.

Skills we are interested in (not all required):

• Hardware prototyping skills
• Conducting interviews and analyzing response data
• Programming skills

Project #28
Technologies for Training Everyday Mindfulness and Emotional Regulation

Mentor: Anna Fang, graduate student advised by Haiyi Zhu

Description: Practices like mindfulness and meditation that help people feel present without judgment and for reducing stress have become emerging topics of interest in HCI research. Technologies for improving people’s ability to be self-aware, emotionally self-regulate, or self-transcend often ask users to imagine themselves in environments like a quiet forest or watching the calm ocean waves, meant to feel separate and isolated from their everyday lives. However, the primary goal of practices is actually for people to generalize these skills to their day-to-day, in which most people do not usually live in these natural environments, silent atmospheres, or sitting in meditation postures.

As a result, in this project, we will be exploring a novel technological space for supporting practice of self-care skills in daily environments. We will not only explore and design for people’s needs, but also build and evaluate HCI technology (e.g. immersive technologies like VR or mixed reality, wearable devices, social computing systems) for practicing things like mindfulness, calm breathing, lowering anxiety and stress, or other self-regulation techniques. Our goal is to help people be better prepared for mental and emotional regulation when challenges arise.

Nature of Student Involvement: The student will work closely with the project leads. Students may participate in coding/development, organizing and carrying out interviews or other evaluation techniques for the project, organizing and analyzing findings, etc. Students will get a breadth of experience, such as learning things from need-finding to system building to paper writing. We welcome students to contribute their own ideas and feel ownership over guiding this project along with the faculty and student advisors!

Skills we are interested in (not all required):

• Experience in programming or software development (e.g. Python, Java)
• Experience or interest in interviewing or conducting user studies
• Interest in developing for or applying artificial intelligence, VR/AR technologies, wearables, or other technical HCI
• Passion and excitement about novel technologies for mental health!