Collaborative Research: CIF: Small: New Theory and Applications of Non-smooth and Non-Lipschitz Riemannian Optimization

Non-convex optimization problems are ubiquitous in fields as diverse as data science, machine learning, and information science and engineering, thereby creating a need for algorithms to efficiently solve such problems. This project will study an important but less developed area in non-convex optimization, namely non-smooth and non-Lipschitz Riemannian optimization. The outcomes of this project will provide insights into important classes of non-convex optimization problems, and will lead to the development of new tools for solving them. New teaching material on non-convex optimization problems will be produced for educating the next generation students in this important class of applications. The societal impact of this exploration will be to benefit new applications in areas such as gene expression, autonomous driving and cancer studies. While existing theory and algorithms for Riemannian optimization usually require the objective function to be differentiable, in contrast this project focuses on non-smooth and non-Lipschitz Riemannian optimization. In particular, the project will study several algorithms for non-smooth optimization that are less developed in the Riemannian setting, including the manifold alternating direction method of multipliers, the inertial manifold proximal gradient method, the stochastic manifold proximal point algorithm, and the manifold prox-linear algorithm. For Riemannian optimization with a non-Lipschitz objective, the investigators will derive the corresponding optimality conditions and then design two algorithms that are based on a smoothing technique, namely the Riemannian smoothing gradient descent method and the Riemannian smoothing trust region method. The proposed algorithms will be implemented to solve real-world applications such as the clustering of single-cell RNA sequencing data, and 3D object detection and 3D tracking in autonomous driving. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.