Comparative Study of Finite Element and Neural Network Discretizations for Partial Differential Equations

This research connects two different fields, machine learning from data science and numerical partial differential equations from scientific and engineering computing, through the comparative study of the finite element method and finite neuron method. Finite element methods have undergone decades of study by mathematicians, scientists and engineers in many fields and there is a rich mathematical theory concerning them. They are widely used in scientific computing and modelling to generate accurate simulations of a wide variety of physical processes, most notably the deformation of materials and fluid mechanics. By contrast, deep neural networks are relatively new and have only been widely used in the last decade. In this short time, they have demonstrated remarkable empirical performance on a wide variety of machine learning tasks, most notably in computer vision and natural language processing. Despite this great empirical success, there is still a very limited mathematical understanding of why and how deep neural networks work so well. We hope to leverage the success of deep learning to improve numerical methods for partial differential equations and to leverage the theoretical understanding of the finite element method to better understand deep learning. The interdisciplinary nature of the research will also provide a good training experience for junior researchers. This project will support 1 graduate student each year of the three year project. Piecewise polynomials represent one of the most important functional classes in approximation theory. In classical approximation theory and numerical methods for partial differential equations, these functional classes are often represented by linear functional spaces associated with a priori given grids, for example, by splines and finite element spaces. In deep learning, function classes are typically represented by a composition of a sequence of linear functions and coordinate-wise non-linearities. One important non-linearity is the rectified linear unit (ReLU) function and its powers (ReLUk). The resulting functional class, ReLUk-DNN, does not form a linear vector space but is rather parameterized non-linearly by a high-dimensional set of parameters. This function class can be used to solve partial differential equations and we call the resulting numerical algorithms the finite neuron method (FNM). Proposed research topics include: error estimates for the finite neuron method, universal construction of conforming finite elements for arbitrarily high order partial differential equations, an investigation into how and why the finite neuron method gives a much better asymptotic error estimate than the corresponding finite element method, and the development and analysis of efficient algorithms for using the finite neuron method. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.