CSR: Medium: Attention-Driven Software Architecture for Integrated Perception and Planning in Autonomous Agents

While significant progress has been made in developing autonomous systems, effectively processing and making sense of vast amounts of sensor data in real time remains a challenge, especially for small robotic agents with limited computing capabilities. To address this problem, this project will create an attention-driven software architecture that can identify and prioritize critical information from sensors, enabling timely decision-making while considering resource constraints and uncertainties in the environment. This architecture will holistically optimize computation scheduling, perception, and planning by adapting to the context and anticipate future actions. Three important advances to be made in this project are: (1) context adaptive scheduling of autonomous computation pipelines, (2) learning-based perception to anticipate future actions in dynamic environments, and (3) motion planning and decision making based on anticipated actions in the presence of uncertainty. By interrelating these components through the attention-driven architecture, this project will tackle fundamental challenges associated with time-sensitive scenarios in resource-constrained autonomous systems. This project will have broader impacts across various domains, including smart agriculture, manufacturing, surveillance, and robotic delivery. The proposed system-theoretic research will empower resource-constrained robotic agents with safe, reliable, and real-time autonomous capabilities in rapidly changing environments. The project plans to establish educational programs and engage with industry partners to provide undergraduate research opportunities. This research will cultivate a talented and diverse workforce for the future of autonomous systems. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.