Understanding Learner Engagement in Informal STEM Learning Environments

Informal STEM learning environments such as science centers provide unique opportunities for K–8 learners to explore scientific concepts through hands-on inquiry, experimentation, collaboration, and play. Existing research shows that these environments can support curiosity, persistence, social interaction, problem solving, and early STEM identity by allowing children to engage with materials, peers, and exhibits in ways that complement formal classroom instruction. Research has also shown that learner engagement in these settings is multidimensional, including behavioral, emotional, cognitive, social, and disciplinary forms of engagement. However, educators currently lack scalable methods for understanding learner engagement in real time, making it difficult to provide timely support or personalize learning experiences. Existing approaches often rely on single-modality observations, controlled laboratory settings, or post hoc analyses. This project addresses this challenge by developing a privacy-preserving artificial intelligence framework that interprets learner engagement during exhibit interactions and translates those insights into personalized STEM learning experiences. Understanding learning at museums exhibits and providing personalized learning recommendations based on interactions with exhibits has been a core challenge to museum experience teams. This project takes an innovative technology-forward approach, which, in conjunction previous qualitative research from the field, seeks to strengthen trajectories of inquiry-based learning within museums. It would also likely broaden participation in STEM and support learners with diverse interaction and engagement profiles. The project investigates two fundamental research questions: (1) how multimodal artificial intelligence can accurately infer multidimensional learner engagement in authentic informal STEM environments, and (2) how these engagement representations can be used to generate personalized learning pathways that promote deeper inquiry and sustained participation. To address these questions, the research will collect, de-identify, and/or analyze multimodal data—including privacy-protected video, audio, student artifacts, and interaction logs—from K–8 learners participating in science-center exhibits. Transformer-based vision models, multimodal learning algorithms, structured exhibit knowledge representations, and large vision-language models will be integrated to infer engagement and support adaptive recommendations tailored to learners’ evolving needs during exhibit interactions. Expected outcomes include a privacy-preserving multimodal engagement dataset, validated engagement analytics, interpretable AI models for personalized STEM learning, educator-facing recommendation tools, and design principles for human-centered AI that can inform future informal STEM learning environments. This project was funded by the STEM K-12 program, which supports fundamental, applied, and translational research on STEM learning in both formal K-12 education and informal learning environments. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.