CPS: Medium: Collaborative Research: Developing Data-driven Robustness and Safety from Single Agent Settings to Stochastic Dynamic Teams: Theory and Applications

This Cyber-Physical Systems (CPS) project will make foundational methodological advances that enable safe and robust reinforcement learning (RL)-based control algorithmic solutions that are driven by problems in smart traffic signal control systems. Recent advances in computation, communication, storage, and sensing have led to a demand for data-driven learning-based decision-making and control in modern cyber-physical systems (CPSs), such as smart transportation systems. In such systems, decision-making agents need to operate safely and in a robust manner while working in complex environments with constraints that need to be respected. This project will develop foundational advances in robust RL solutions, and safe and constrained RL with provable guarantees by taking traffic signal control systems within smart transportation systems as our motivating CPS application and evaluation platform. This work will additionally focus on advancing curriculum development, recruitment of students from under-represented groups, involvement of undergraduate students in research, K-12 outreach, and also research community outreach via workshops, conference sessions, and seminars. The researchers will interface with companies and other stakeholders to communicate the results of the research as well as provide them with educational material on methodology. The technical approaches include: 1. Robust RL solutions incorporating model class knowledge, use of future predictions and robustness characterizations, and off-policy methods to address distributional shifts and data paucity arising from the use of a simulator/emulator or offline data; and 2. Efficient, safe, and constrained RL algorithms using model-free approaches and function-approximated methods, and also methods for partially-observed systems. To close the loop with the motivating CPS application, the RL algorithms will be evaluated in the context of traffic signal control via a comprehensive simulation-based evaluation using models of two instrumented sites. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.