SOLANA: Foundations of Sonar-Language Navigation and Action for Marine Robotics

The ocean remains one of the least explored environments on Earth because underwater robots cannot reliably operate where ambient light is limited, water is turbid, and satellite navigation and communication services are unavailable. While recent advances in artificial intelligence have enabled robots on land to understand human language and make intelligent decisions from camera images, these technologies are ineffective underwater, where acoustic sonars are the primary sensing modality. This project will develop the theoretical foundations and computational tools for a new generation of underwater robots that can understand language instructions, interpret sonar observations, and autonomously execute complex actions. These capabilities will enhance ocean exploration, environmental monitoring, offshore infrastructure inspection, and maritime security while reducing operational cost and risk. The project will also develop open educational resources and interdisciplinary training opportunities that prepare future workforces at the intersection of robotics, artificial intelligence, and ocean engineering. This project will establish the scientific foundations of Sonar-Language Autonomy (SOLANA) by developing the first computational framework that couples sonar perception and natural language understanding for autonomous navigation and action by underwater robots. The integrated research will develop three sets of algorithms and learning pipelines: (1) physics-aware sonar-language representations to learn semantic concepts directly from sonar measurements through geometry-aware tokenization, relational ping-graph formation, and scene-query embedding; (2) language-conditioned concept retrieval and spatial reasoning by grounding mission waypoint planning and goal-oriented policy adaptation in latent sonar-language space; and (3) explainable autonomy through hierarchical skill decomposition, sonar-language action execution, and post-mission semantic summarization. These outcomes will be validated through simulation and field experiments on commercial underwater robots operating with side-scan and forward-looking sonars in representative marine environments. Overall, this project will establish new algorithmic benchmarks, evaluation metrics, and open-source packages that define a research frontier in sonar-language autonomy for marine robotics. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.