I-Corps: Translation Potential of Resilient, Flexible, and Intuitive Situational Awareness via Next-Generation Artificial Intelligence (AI) and Flexible Middleware

This I-Corps project is based on the development of a software platform to provide a wide range of computer vision tasks with minimal computational effort for mobile applications, including autonomous vehicles, robotics, and traffic monitoring. Currently, traditional models using artificial intelligence (AI) solutions are static, monolithic, and optimized for worst-case scenarios, leading to significant computational demands that mobile devices often cannot meet. This technology provides operators with real-time, actionable insights, even in low-bandwidth environments. The solution is based on a new generation of AI algorithms, empowering systems with new analysis capabilities. In addition, it uses a modular approach that is designed to integrate seamlessly with existing systems, which may ensure greater scalability and adaptability for rapidly evolving missions. The ability of the platform to deploy AI algorithms in a flexible, resilient, and distributed fashion may make it possible to gather information from multiple sources from the environment, understand what the information means, and use the information to predict what may happen next, referred to as dynamic situational awareness. This technology may improve the decision making and safety of first responders and workers across fields and commercial areas and improve U.S. defense capabilities. This I-Corps project utilizes experiential learning coupled with first-hand investigation of the industry ecosystem to assess the translation potential of a software platform to provide a wide range of computer vision tasks by leveraging adaptive sensor fusion and edge processing. Computer vision is the cornerstone of a broad range of current and future mobile applications, such as vehicular autonomy, robotics, and traffic monitoring. In this domain, deep neural networks (DNN) achieve the best performance, where many generations of models have been designed to accomplish various tasks. Currently, these algorithms are designed for execution on a single machine and are static and monolithic. One downside of this approach is that the algorithms are tuned to process the “worst” case input, and have computational demands often failing to meet application requirements when considering the limitations of mobile devices. Moreover, offloading the execution to compute-capable devices (edge computing) requires the transfer of information-rich signals (e.g., camera, light detection and ranging (LiDAR) and radar imaging) over capacity constrained and volatile wireless channels. The goal is to provide operators with real-time, actionable insights, even in low-bandwidth or contested environments. The ability of the platform to deploy AI algorithms in a flexible, resilient and distributed fashion may make it possible to provide dynamic situational awareness in a broad range of applications. This may bring a new generation of AI algorithms to the market, empowering vehicles with new analysis capabilities that may improve decision making and safety. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.