Collaborative Research: CISE-MSI: DP: HCC: Buenas - Giving All a Seat at the Table Using Mixed Reality

Online learning has become a critical aspect of modern post-secondary education, with many colleges and universities incorporating both synchronous and asynchronous online courses into their curricula. These kinds of courses can increase access to education for students from minority backgrounds, who are frequently of non-traditional student ages and may often need to travel farther and have fewer resources for attending in-class learning activities. However, online learning has not yet reached its full potential, with students reporting feeling disconnected from their on-campus classmates and instructor. Study groups are essential to undergraduate education, yet they have been especially problematic to replicate online. Conventional videoconferencing systems are inadequate for supporting the dynamic dialogue and good eye contact that are prerequisite to successful study groups. The Buenas research project will use mixed reality to develop a format for remote study group interaction that will allow local and remote students to interact naturally and effectively. Buenas will allow all students to see each other at their correct relative location and size, while providing advantages such as intuitiveness, situation awareness, depth perception, and increased sense of presence. The project has the potential to advance the effectiveness of small group meetings with hybrid local and remote attendance, especially for under-represented members of society, and will involve students from Minority-Serving Institutions (MSI) in research endeavors. Buenas will use mixed reality headsets to immerse remote students into local study groups. The goal is to allow local and remote students to interact naturally, as if all members of the group were co-located. At the local site, cardboard screens will be placed in unoccupied seats around the study group table to project video sprites (i.e., life-sized, background-subtracted videos) of the remote students onto them. At each remote site, a remote student will visualize the other students’ video sprites in mixed reality, integrated into their office spaces via virtual floating panels. The Buenas system will be validated via three user studies. The first study will compare study group sessions hosted with Buenas against sessions hosted via videoconferencing. The second study will compare various ways of representing the remote student, i.e., with a video sprite showing the person wearing a headset, with a computer animation avatar, and with a video sprite where the headset is removed through inpainting. The third study will evaluate Buenas with remote students connecting from their homes. The project will involve over two hundred students from the three collaborating institutions, two of which are MSIs This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.