Collaborative Research: Analog and Digital Semantic Communications with Pretrained Large Models

Modern wireless applications, including autonomous systems, augmented reality, and collaborative sensing, generate vast amounts of data that place increasing demands on communication networks. Semantic communication addresses this challenge by transmitting only the information needed to preserve meaning or accomplish a task, rather than reproducing every detail of the original data. However, most existing systems are designed for specific tasks and operating conditions, which limits their ability to generalize and adapt. This project overcomes these limitations by using large pretrained artificial intelligence models as a shared knowledge base between the transmitter and receiver, enabling the system to identify and represent task-relevant information efficiently across diverse applications. It establishes an integrated framework for analog, digital, and hybrid digital-analog communication that draws on their complementary strengths to preserve meaning, support reliable transmission, and adapt wireless resource use to changing application needs and network conditions. The intellectual merit of the project lies in creating a general and adaptive foundation for extracting, compressing, transmitting, and protecting critical information according to both its importance to the application and the available communication resources. The broader impacts include more efficient and resilient wireless services in bandwidth- and resource-constrained environments, together with advances in education and workforce development through student research and mentoring, curriculum development, high school outreach, and the open dissemination of software, experimental platforms, research findings, and educational materials. The project examines the fundamental problem of reliably communicating high-dimensional semantic representations generated by large pretrained artificial intelligence models over dynamic wireless channels. It develops three complementary communication architectures. The analog architecture transmits continuous semantic features and integrates adaptive feature selection, rate control, receiver feedback, and prompt-based adaptation to allocate communication resources according to semantic importance and channel quality. The digital architecture maps continuous features to discrete symbols through differentiable quantization and adaptive codebooks, then applies semantically prioritized channel coding, modulation, and rate control to provide stronger protection for more important information. The hybrid digital-analog architecture partitions semantic features between digital and analog transmission paths according to their relevance and sensitivity to distortion, while jointly optimizing communication rate and semantic reconstruction quality. Across all three architectures, lightweight contextual prompts, which serve as learned control inputs, coordinate encoding, quantization, and transmission in response to changing tasks and operating conditions without retraining the underlying pretrained models. Together, these advances provide an integrated foundation for semantic communication systems that operate efficiently, adaptively, and robustly under bandwidth limitations and time-varying wireless conditions. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.