Collaborative Research: VINES: Track 1: Towards AI-Native Holographic Communication for Accessible Telepresence

Today, sending a moving, life size person across the internet as a hologram requires an impractical number of cameras, excessive bandwidth, and massive computing power. To address these limitations, this project introduces a highly efficient framework to capture a subject using only one or a few standard cameras, transmit a compact set of digital cues, and reconstruct a high-fidelity 3D hologram on smartphones, smart glasses, and headsets. Technically, the project advances a novel Generative Holographic Coder-Decoder (Codec) that integrates 3D Gaussian splatting, vector quantization, masked encoder-decoder learning, and analysis-by-rendering. The first task designs generalizable codecs for sparse multi-view and single-view inputs, with built-in recovery mechanisms for lost vector indices. The second task develops sparse computing methods for resource-constrained edge devices using heterogeneity-aware global structured pruning, mixed-precision quantization, pose- and point-aware attention mechanisms, and few-step generative sampling. The third task creates adaptive control via layered refinement and reinforcement learning to dynamically balance quality, bandwidth, latency, memory, and device hardware constraints. The framework will be comprehensively evaluated across wireless testbeds, long-distance network links, and large-scale simulations. By drastically reducing the data, hardware, and computing resources needed for high-quality holograms, this project broadens access to next-generation immersive communication. The resulting technology will support critical applications in remote healthcare, interactive classrooms, emergency response training, workforce development, collaborative design, entertainment, and the digital arts, including communities with limited network capacity or limited access to specialized equipment. Furthermore, the project integrates research with education through student thesis work, capstone projects, internships, course modules in networking and machine learning, and outreach demonstrations for kindergarten through grade twelve learners. These activities help prepare a diverse workforce for next-generation networks, artificial intelligence systems, and immersive media. This collaborative project brings together investigators from the University of North Carolina at Charlotte and the University of Central Florida to make live three dimensional (3D) communication significantly more accessible and practical. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.