Collaborative Research: VINES: Track 2: SCOPE-XR: Seeing the Channel: Optimizing Perception-aware Environments for XR

Extended reality (XR), including augmented, virtual, and mixed reality, will require wireless networks that can respond reliably to user motion, changing environments, and strict delay constraints. Existing Wi-Fi and 5G systems remain largely reactive and are not designed to support immersive, multi-user experiences that depend on stable spatial alignment and low motion-to-photon latency. This project will advance a new class of perception-aware wireless systems that will help future networks better support education, healthcare, manufacturing, and public-safety training. By enabling more robust and adaptive XR connectivity, the research will strengthen U.S. leadership in next-generation (NextG) wireless and immersive technologies, broaden access to advanced digital experiences, and train students at the intersection of wireless networking, machine learning, and XR systems. The project will also produce open software and evaluation assets that will support research, workforce development, and future technology translation. The project will develop an Artificial Intelligence (AI)-native, integrated sensing and communication (ISAC) framework for next-generation XR networking. The central technical idea is a Radio Radiance Field, a vision-aligned channel abstraction that will combine radio measurements with on-device perception to infer predictive channel priors in dynamic environments. These priors will drive software-defined network control functions for beamforming, scheduling, handover, and edge-assisted adaptation through Open Radio Access Network (O-RAN)-compliant xApps and rApps. The project will also couple XR pose, gaze, and rendering priorities with wireless control to enable closed-loop coordination between the sensing, communication, and application layers. The research team will validate the system in representative collaborative learning and industrial XR scenarios and will quantify communication gains together with latency and compute overhead. Expected outcomes will include deployable O-RAN software modules, benchmarked multimodal datasets and evaluation methods, and design principles for applying AI-native and ISAC-enabled control to XR and other perception-driven NextG wireless systems. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.