CPS: Medium: Collaborative Research: Accelerating Societal-Scale Control Design via Offline Reinforcement Learning

This Cyber-Physical Systems (CPS) project will accelerate the design of controllers for large-scale engineering systems, focusing on the application of artificial intelligence in transportation. Traditional methods often rely on simulations that do not accurately represent real-world complexities, leading to an inefficient and costly process of collecting data, calibrating models, and testing controllers. This project aims to bridge the gap between simulated cyber environments and real-world physical operations by utilizing extensive offline datasets and offline reinforcement learning. Specifically, the research team will harness data derived from millions of vehicle miles collected on the I-24 MOTION open road testbed in Nashville, Tennessee. By developing efficient and adaptive control systems, such as improved cruise control for vehicles, the project seeks to enhance safety, reduce traffic congestion, and improve overall driving comfort. The anticipated result is a tenfold reduction in societal-scale transportation systems’ design cycles, leading to significant societal benefits in emissions reduction, air quality improvement, and transportation costs. Moreover, the project will contribute to education by offering courses that equip students with the skills needed to deploy these innovative systems, thereby preparing them to tackle future societal challenges. The collaborative project will explore critical questions surrounding the deployment of offline reinforcement learning in societal-scale cyber-physical systems in transportation. It addresses three key challenges: first, ensuring that controller designs align user preferences with system objectives; second, effectively processing and extracting useful information from vast time series datasets; and third, significantly reducing the number of iterations required in the design process. To achieve these aims, the multidisciplinary research team will develop novel reward functions informed by inverse reinforcement learning principles to encourage user participation. Additionally, advanced methods will be employed to explore the rich data generated by the open-road testbeds. The implementation of hybrid reinforcement learning strategies will facilitate real-time interactions of deployed controllers, enhancing design efficiency. Validation of the controllers will occur through extensive testing with vehicles on the open road, using the I-24 MOTION framework to ensure practical reliability and safety. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.