FRG: Collaborative Research: Variationally Stable Neural Networks for Simulation, Learning, and Experimental Design of Complex Physical Systems

A ubiquitous and often critical task in science and technology is to synthesize information from governing physical laws and noisy observational data, such as those provided by sensor systems, in order to optimize important quantities of interest. Examples touched upon in this project include subsurface flow through porous media, fiber optics, waveguide design, and material science applications. The overarching goal of this project is to develop a mathematically rigorous framework for “learning” the underlying complex models from the given sources of information. The recent stunning successes of modern machine learning, especially deep learning, in error-tolerant applications, does not automatically imply its success in error-sensitive scientific tasks. Targeting the latter, this project aims to significantly advance prediction capabilities through rigorous accuracy quantification and certification, arguably an indispensable feature of next generation simulation tools in science and technology. This requires integrating conceptual tools from diverse areas such as numerical and functional analysis, machine learning, statistics, optimization, and information geometry. The project gathers a diverse team for this purpose, and as a byproduct, creates a unique educational framework for students and young researchers. Governing physical laws are formulated in terms of (systems of) parameter dependent partial differential equations (PDEs) of various types depending on the application. Partially observed states of interest are then among (or close to) all those solutions that are obtained when traversing the parameter space. Learning or optimizing such states boils down to ill-posed inverse problems involving functions of many (parametric) variables. To cope with these obstructions, this project formulates a “learning” framework as a nonlinear regression problem over hypothesis classes comprised of deep neural networks. Residual type loss functions are employed to avoid expensive computation of a large number of high-fidelity training samples. Accuracy quantification and a posteriori certification is then warranted by so-called variationally-correct residual risks. This means that the size of the loss at any stage of the optimization is uniformly proportional to the error incurred by the resulting estimation in a physically relevant metric. The variational correctness is achieved through stable variational formulations of the underlying PDEs. They are typically based on currently evolving (discontinuous) Petrov-Galerkin methodologies. Due to the inherent appearance of dual norms, this requires new strategies for efficiently evaluating resulting loss functions in the high dimensional parametric context. Moreover, specially adapted gradient flows will serve as an important constituent in developing robust integrated optimization/adaptation/regularization strategies. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.