HCII PhD Thesis Proposal: Nesra Yannier
When
-
Where
GHC 4405
Description
THESIS PROPOSAL
Bridging Physical and Virtual Learning: A Mixed-Reality System for Early Science
Nesra Yannier
Committee:
Kenneth R. Koedinger (Co-Chair) (HCI & Psychology, CMU)
Scott E. Hudson (Co-Chair) (HCI, CMU)
Jessica Hammer (HCI, Entertainment Technology Center, CMU)
Kevin Crowley (Learning Research and Development Center, University of Pittsburgh)
Document:
http://www.nesrayannier.com/ThesisProposal_NesraYannier.pdf
Abstract:
Tangible interfaces and mixed-reality environments have potential to bring together the advantages of physical and virtual environments to improve children’s learning and enjoyment. However, there are too few controlled experiments that investigate whether experimenting with physical objects in the real world accompanied with interactive feedback may actually improve student learning compared to flat-screen interaction. Furthermore, we do not have a sufficient empirical basis for how a mixed-reality environment should be designed to maximize learning and enjoyment for children.
I created EarthShake, a mixed-reality game bridging physical and virtual worlds via Kinect depth-camera sensing to help children learn physics. I have conducted two controlled experiments with EarthShake that have identified features that are more and less important to student learning and enjoyment. The first experiment examined the effect of observing physical phenomena and collaboration (pairs versus solo), while the second experiment replicated the effect of observing physical phenomena while also testing whether adding simple physical control, such as shaking a tablet, improves learning and enjoyment. The experiments revealed that observing physical phenomena in the context of a mixed-reality game leads to significantly more learning (5 times more) and enjoyment compared to equivalent screen-only versions, while adding simple physical control or changing group size (solo or pairs) do not have significant effects. Furthermore, gesture analysis provides insight as to why experiencing physical phenomena may enhance learning.
My proposed work will further investigate what features of a mixed-reality system yield better learning and enjoyment, especially in the context of limited experimental results from other mixed-reality learning research. Most mixed-reality environments, including tangible interfaces, currently emphasize open-ended exploration and problem solving, and are claimed to be most effective when used in a discovery-learning mode with minimal guidance. I plan to investigate how critical to learning and enjoyment interactive guidance and feedback is (e.g. predict/observe/explain prompting structure with interactive feedback), in the context of EarthShake. I propose to conduct an experiment that compares the learning and enjoyment outcomes of children interacting with a version of EarthShake that supports guided-discovery, another version that supports exploration in discovery-learning mode, and a version that is a combination of both guided-discovery and exploration. This thesis will also explore how this approach can be extended to other content areas, with the goal of creating a new mixed-reality system that can be used in museum and school settings to improve young children’s science learning and enjoyment in a collaborative way, fostering productive dialogue and scientific curiosity.