Collaborative Research: CCSS: Semantic-Aware Collaborative Perception and Networking for Autonomous Urban Mobility

Autonomous urban mobility systems integrate distributed sensing, wireless communication, and automated decision-making to move people and goods safely and efficiently through complex, rapidly changing city environments. By enabling delivery robots, micro-vehicles, and other autonomous agents to perceive their surroundings and coordinate in real time, this emerging mobility paradigm can strengthen urban transportation services, improve last-mile logistics, and contribute to more responsive next-generation transportation ecosystems. Realizing these benefits, however, requires overcoming fundamental sensing and communication limitations in heterogeneous and dynamic environments. Buildings, pedestrians, and other obstacles can restrict sensing coverage, leaving individual agents with incomplete views and limited understanding of their surroundings, while wireless networks may be unable to transmit all available observations without delaying time-critical decisions. This project establishes a new paradigm for collaborative perception and networking that enhances collective situational awareness while reducing unnecessary network traffic. By enabling selective and predictive information exchange, the research advances communication and sensing technologies for resource-efficient autonomous mobility. These capabilities can help transportation agencies, network operators, and municipalities deploy more reliable and scalable urban mobility services. The project also produces an open-source simulation and evaluation platform that supports reproducible research and enables broader study of networked autonomous systems. Graduate and undergraduate students receive interdisciplinary training in wireless networking, intelligent mobility, and artificial intelligence. Outreach and dissemination activities broaden participation in science, technology, engineering, and mathematics and make project outcomes accessible to wider research and public communities. The project develops an innovative semantic-aware collaborative perception and networking framework that enables reliable perception and proactive coordination among heterogeneous autonomous mobility agents operating with incomplete observations, uncertain sensing conditions, and limited communication resources. The central scientific problem is to determine when collaboration is needed, which observations provide the greatest value, and how sensing, communication, and control resources can be coordinated across multiple timescales to support timely and resource-efficient sensing data exchange. The research comprises three integrated components. The first component, multi-modal collaborative perception formation, characterizes an agent’s perception confidence and uncertainty, identifies emerging perception needs, and selects suitable collaborators while accounting for variations in sensing data availability through a physics-informed learning method. The second component, value-of-information-aware semantic communication, quantifies the contribution and timeliness of individual observations and uses these measures to prioritize and adapt sensing data transmission under bandwidth and delay constraints. The third component, a multi-timescale management platform, models the coupled evolution of mobility, perception, and communication and combines simulation-driven prediction with closed-loop control to support proactive system planning. The framework is evaluated in realistic and synthetic urban mobility environments using measures of perception reliability, communication efficiency, responsiveness, and scalability. By integrating semantic reasoning, information valuation, predictive modeling, and adaptive network control, the project establishes a foundation for proactive and scalable coordination across autonomous mobility agents and the networks that connect them in dynamic urban environments. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.