III: Small: Towards Efficient Adaptive Federated Learning: Algorithms, Theories, and Applications

In federated learning, machine learning models are trained using the data of individual users without sharing the data outside of the original secure system. The paradigm is popular, in part because of the promise it holds for protecting privacy of data while still using data from multiple sources to create a powerful central model. Algorithms for federated learning require expensive computations for deriving large sets of hyperparameters (the variables that control learning) that control their operation. Recently, Adaptive Federated Learning (AFL) is being used, which offers improved training performance and reduced effort for identifying the hyperparameters. Despite these improvements, AFL is still in its early stages and suffers from issues in both communication of large number of hyperparameters and training efficiency. Moreover, the worst-case efficiency of adaptive federated learning also remain largely understudied from a theoretical perspective. The broad goal of this project is to fill this research gap by: (1) providing a provably efficient approach to adaptive federated learning, and (2) building practical adaptive federated learning systems for real-world machine learning settings such as the Internet of Things (IoT) and healthcare. This project will offer a novel perspective for developing practical and theoretically-sound efficient adaptive federated learning. The core idea of this project is the integration of adaptive federated system design with the perspective of nonconvex optimization, which provides a systematic way to deliver intuitive solutions with theoretical insights/guarantees. Specifically, this project consist of three aims: (i) improving the communication efficiency of adaptive federated learning with new algorithm designs and rigorous convergence guarantees; (ii) improving the training efficiency of adaptive federated learning under heterogeneous data distributions with rigorous convergence guarantees; (iii) building and evaluating the practical adaptive federated learning systems in real-world applications such as IoT and healthcare. Taken together, this project provides new approaches and insights into efficient adaptive federated learning systems design and will promote advances in the fields of machine learning, data mining, and related application field. It will also help develop and culture graduate and undergraduate students with machine learning and federated optimization techniques, recruit undergraduates from diverse backgrounds into research activities, interest K-12 school students for a broader education in computer science and make the proposed research accessible to a broader audience. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.