Collaborative Research: Stochastic Nonlinear Control and Learning via Spectral Dynamics Embedding

This proposal aims to address the challenges of achieving optimal nonlinear control for dynamical systems in stochastic environments considering applications such as robots, aircraft, and automated manufacturing processes. Traditional methods to control these systems either provide sub-optimal solutions, lack rigorous analysis, or require a large amount of computation that could result in intractable solutions. Our research introduces a novel approach called spectral dynamic embedding, which aims to create efficient and reliable control algorithms suitable for a wide range of nonlinear systems. These methods will be tested in both virtual simulation environments and real-world robotic labs. The practical algorithms developed through this research can be applied to various applications, enhancing technologies in robotics, aerospace, manufacturing, energy, and beyond. The team will collaborate with industry partners to broaden the impact on society. Additionally, the project will involve students at various levels in cutting-edge research and experimentation, and also develop K-12 educational materials to inspire the next generation of scientists and engineers. The key innovation of this research lies in the unified spectral dynamic embedding approach, which reformulates the system dynamics in stochastic nonlinear control linearly to a nonlinear spectral feature space, rather than linearizing the dynamic model. This novel perspective allows for tractable dynamic programming or linear programming to solve the optimal policy and enables rigorous analysis of control optimality for general stochastic nonlinear dynamics. It also facilitates a simple learning procedure and computationally tractable exploration to accelerate data collection, both grounded in solid theoretical foundations. The research will develop computationally efficient methods for stochastic nonlinear control with either known or unknown models and will ensure the robustness and safety of the system. This interdisciplinary effort combines expertise in online control, reinforcement learning, optimization, statistical learning, and reproducing kernel Hilbert space to tackle this longstanding problem, aiming for transformative impacts on both control theory and machine learning. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.