CRII: Learning to simulate with small data

Reinforcement learning (RL) has shown great success in a series of artificial intelligence (AI) domains such as Go games. Since RL can provide optimized policies to achieve certain goals, people are eager to use advanced RL techniques to solve real-world decision-making problems. Despite its huge success in AI domains, RL has not yet shown the same degree of success for real-world applications because RL largely relies on simulation, and there is rarely a good simulator for real-world systems. Unlike simulated environments such as games where data could be unlimited, real-world physical systems like traffic systems are quite the opposite: data is small (i.e., sparse and hard to obtain). This raises important research questions about building realistic simulations from small data that can mimic complex and stochastic real-world dynamics. The solution from this project can help policymakers choose a better policy before implementing it in the real world, greatly facilitate the adoption of reinforcement learning techniques in the real world, and benefit many applications in which one would like to use real data to learn or understand real-world physical systems better. This project aims to build a realistic traffic simulator by investigating data mining algorithms and provides solutions toward mimicking real-world simulations with small data with applications to traffic simulations. This project will learn to simulate without making unrealistic assumptions on the real-world models and further learn with the real-world setting of small data. First, the project will try to learn data-driven models from incomplete and indirect observations from the real world. Second, this project will seek to innovate the data-driven model to meet with human knowledge, as human knowledge could guide us to learn a model that is not only relied on small and possibly biased data. Third, this project aims to leverage the influences from multiple parties in the real world for the data-driven model. New machine learning techniques will be developed in the data mining process. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.