Exploring the Integration of Reconfigurable Sensorimotor Interfaces into Educational STEM Simulations

In the move to digital learning platforms, education has largely left behind the ability for learners to touch and manipulate objects, despite the long tradition of manipulatives being used by educators like Friedrich Froebel and Maria Montessori, and more modern theories of embodied cognition suggesting the range of roles that touch and movement can play in supporting the acquisition of abstract STEM concepts. This shift to digital platforms may especially disadvantage learners with blindness and low vision (BLV), who often rely on audio alone to comprehend ever-more-complicated digital educational content like interactive simulations. This project will use co-design processes to iteratively create haptic input–output devices, TeleTangibles, that can be used to embody STEM concepts in support of the learning of both sighted students and those with BLV. The devices will provide two-way communication with educational PhET simulations, offering a tangible link between the simulation environment and the physical world. The devices can change shape based on input from educational simulations and from remote learners. The devices can also be reshaped manually, providing communication back to the simulation for remote pairs of learners to co-manipulate a shared tangible artifact. They are intended to be used flexibly – on their own, or integrated with a suite of STEM simulations customized to integrate with the devices. Such devices have the potential of expanding STEM participation and access to more learners, both learners with BLV and learners for whom tactile and kinesthetic engagement may offer a novel “way in” to abstract STEM content that may otherwise be forbidding or challenging. Researchers will partner with teachers, BLV students, and sighted students in the 5th to 8th grade to codesign mechatronic technology to explore three research questions: (1) What sensorimotor interactions with mechatronic technologies hold promise for enriching STEM learning for both sighted and BLV learners?; (2) What impact does integrating mechatronic technologies into educational simulations have on sensorially diverse learners’ STEM understandings?; and (3) What impact does the use of networked mechatronic devices have on remote and on co-located collaboration between learners who may or may not have similar sensory profiles? These questions will be addressed via five iterative rounds of Design-Based Research, using mixed methods approaches including semi-structured interviews, interaction analysis, log data analytics, grounded theory video analysis, pre-post STEM content learning assessments, ethnographic observation, micro-genetic analysis of learner conceptual change, and peer conversation transcripts. The technological innovation of the work lies in creating a low-cost, reconfigurable, tangible, haptic interface that can be deployed within a range of learning activities, adding tactile and kinesthetic means of engaging with STEM content; and providing best practices for the design of flexible, multi-purpose haptic systems. The innovation in learning research lies in how the project will evaluate and extend existing theories of embodied cognition by employing them with sensorially diverse learners. This will shed light on an under-studied population and also holds potential for re-evaluating and refining constructs that were developed with sighted populations. This project is funded by the Research on Innovative Technologies for Enhanced Learning (RITEL) program that supports early-stage exploratory research in emerging technologies for teaching and learning. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.