Collaborative Research: Ideas Lab: Breaking Low: End-to-End Delivery Technology for Interactive Multi-person XR Rehabilitation Activities

Extended reality (XR) technologies have shown significant promise in increasing user engagement and skill acquisition in a variety of domains. The goal of this project is to accelerate adoption of innovative XR applications for rehabilitation, for example, to enhance user experiences in treatments to improve motor function for diseases with motor disabilities. This project develops immersive XR exercise environments that enable users to move and interact in 3D space with each other and with virtual elements. A key goal is to make the experience of rehabilitation more enjoyable and effective by incorporating social interactions between remote users that can boost engagement and skill acquisition. The technology also allows clinicians to guide and interact with their patients remotely. To create a virtual environment that users experience as fast and seamless, this project develops novel approaches to the underlying networking infrastructure needed to run the application. Collaboration with industry partners will support technology adoption for XR-enabled rehabilitation technology and for the XR industry. Realizing multi-user, geo-distributed XR technology is challenging due to stringent motion-to-photon latency requirements for good user experience. Current wide-area Internet routing and cellular wireless management are one-size-fits-all across applications, hurting latency. A key insight of this project is that not all types of XR traffic require uniformly low latency; instead, the project enables prioritization of delivery of important XR traffic while intelligently managing lower-priority XR traffic. The project’s core technical contributions are an adaptive XR application with new delivery mechanisms, leveraging Internet path selection on the testbed, and programmable wireless resources using Open Radio Access Networks (Open RAN) and xApps. Performance of the XR environment will be assessed via user experience and key network performance indicators. The expected outcome is achieving key performance targets currently deemed impossible on today’s Internet by demonstrations of network-supported, multi-user XR technology for rehabilitation at different sites. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.