HCII Ph.D. Thesis Proposal: Kexin Bella Yang
When
-
Description
Towards more customized, adaptive and reflective learning environments through human-AI collaboration: Design and evaluation of analytic-driven teacher support tools
Kexin Bella Yang
HCII PhD Thesis Proposal
Time & Location
Monday, Jul 22nd, 2024, 9:30 am EST
Virtual only
Zoom Meeting ID: 998 820 7001
Committee:
Vincent Aleven (Co-chair), HCII, Carnegie Mellon University
Nikol Rummel (Co-chair), CAIS Research Professor/ Ruhr University Bochum
Kenneth Holstein, HCII, Carnegie Mellon University
Xu Wang, Computer Science and Engineering, University of Michigan
Abstract
Classroom teaching is a complex job that requires different tasks in different phases. 1) Teachers need to do adequate lesson planning and preparation before class. It is important to ensure that the instructional materials and technology to leverage are customized to the classroom context and aligned with the curriculum. 2) During class, teachers frequently improvise and constantly match their actions to serve the differentiated needs of their students. 3) After class ends, post-class reflection can help teachers to think critically about what happened, to improve and flexibly adapt their instructions. It is important to provide teacher support in all phases. The advance of educational technology and multi-modal learning analytics (MMLA) (Cukurova et al., 2020; Escueta et al., 2017; Holstein et al., 2019b; Holstein & Aleven, 2022) holds promise to support teachers and students to achieve more effective teaching and learning, by empowering teachers to focus on instructional tasks that they do the best and that they enjoy doing. My PhD research aims to leverage the synergy of humans and AI to design, develop, and evaluate teacher-support tools (many are data-driven and analytics-based) and a crowd pipeline, with the goal of fostering more customized, differentiated, reflective learning environments.
Customization of Instructional Materials and Technology. While there are many ways to customize learning environments, my work focuses on instructional materials and technology (specifically, pairing algorithms). Prior literature has noted the need and importance of aligning instructional materials and technology with teachers’ practice and preferences to achieve sustained use and adoption.
Customization of instructional materials. For instructional materials, I studied how to achieve customized
Dynamic Differentiation between Individual and Collaborative Learning. There are many ways to make learning environments allow for differentiated learning, and my work focuses on dynamically combining different learning modes for students. Prior research found that combining individual and collaborative learning can be more beneficial than either way alone. However, prior work has yet to build a technology ecosystem to achieve dynamically differentiated learning between individual and collaborative learning and systematically evaluate the value of the dynamic combination of individual and collaborative learning. Grounded in the context of real-time analytic-based intelligent tutoring systems, my colleagues and I created a technology ecosystem to achieve the dynamic combination of individual and collaborative learning. I then conducted a series of studies to understand teachers’ preferences and boundaries in co-orchestrating with AI systems in such dynamic pairing, evaluated the feasibility of such dynamic combinations in classrooms, and how dynamic transitions are compared with traditional transitions.
I discovered teachers’ preferences about the respective roles of teachers, students, and AI in a co-orchestration system of dynamic pairing. I found that dynamic transitions between different activities have pedagogical value in the eyes of teachers and, to a lesser degree, students. Supporting dynamic transitions in actual classrooms is worthwhile from a pedagogical perspective, yet not without challenges, such as finding the optimal timing, content, and partner during such transitions. I found teachers and students demonstrated a subjective preference for the dynamic condition, which stemmed from its flexibility and individualization. However, for some of the test items, the standard condition had more learning gains than the dynamic condition.
Multi-modal Based Teacher Reflection Tool. Many ways can foster a more reflective learning environment, and my work focuses on supporting teacher reflection through multimodal learning analytics (MMLA) as teachers teach in classrooms where students use ITS. Teacher reflection can be an important component in advancing the quality of instruction. Prior literature has found that the practice of reflection is said to scaffold critical thinking (Korthagen, 2004), provide a source of knowledge construction in teaching (Conway, 2001), and promote self-regulation in teachers (Boud & Falchikov, 2007; Singh, 2008). However, more work is needed to understand how a reflection tool can be designed to fit into teachers’ routines and keep in mind their preferences for data sharing and privacy considerations. With the help of colleagues, I studied teachers’ preferences about multi-modal data analytics collection and analytics use in reflection tools. I found teachers showed the strongest interest in data about motivating students and interactions with students (teacher analytics) and students’ learning and progress (student analytics). I also learned about specific teachers’ data collection and sharing preferences in MMLA.
In my proposed work, I will leverage multi-modal data analytics (including location sensing and transaction log data) to support teacher reflection through design-based research and evaluation. I will address the following research questions: 1) How can we design a teacher reflection tool based on multi-modal learning analytics that reflects teachers’ practices and preferences? 2) How does leveraging a multi-modal learning analytics reflection tool affect teachers’ reflection practice? Specifically, my proposed work includes 1) refining and pilot testing the teacher reflection tool and 2) evaluating the use of the reflection tool with actual teachers and students.
My PhD research leverages various methodologies, including pipeline and tool design, data simulation and mining, user-centered research, as well as both prototyping field study and controlled classroom experiments. My research focuses on aiding teachers in customizing instructional materials and technology (i.e., pairing algorithms) and facilitating dynamically differentiated classrooms. I will further work on reflective instructional practices using Multimodal Learning Analytics (MMLA). This work contributes to further understanding the intersection of AIED and HCI research areas, specifically leveraging human-AI collaboration to make learning environments more customized, differentiated, and reflective.
Proposal Document
https://docs.google.com/