Collaborative Research: SHF: Medium: Heterogeneous Architecture for Collaborative Machine Learning

The recent breakthrough of on-device machine learning with specialized artificial-intelligence hardware brings machine intelligence closer to individual devices. To leverage the power of the crowd, collaborative machine learning makes it possible to build up machine-learning models based on datasets that are distributed across multiple devices while preventing data leakage. However, most existing efforts are focused on homogeneous devices; given the widespread yet heterogeneous participants in practice, it is urgently important but challenging to manage immense heterogeneity. The research team develops heterogeneous architectures for collaborative machine learning to achieve three objectives under heterogeneity: efficiency, adaptivity, and privacy. The proposed heterogeneous architecture for collaborative machine learning is bringing tangible benefits for a wide range of disciplines that employ artificial intelligence technologies, such as healthcare, precision medicine, cyber physical systems, and education. The research findings of this project are intended to be integrated with the existing courses and K-12 programs. Furthermore, the research team is actively engaged in activities that encourage students from underrepresented groups to participate in computer science and engineering research. This project provides the theoretical underpinning and empirical evidence for an efficient, adaptive and privacy-preserving design under heterogeneity, which fills a critical void - the existing collaborative machine-learning approach fails to manage the immense heterogeneity in practice. This project centers on three aspects: (1) design of specialized neural architectures for heterogeneous hardware platforms to cope with the limited efficiency of collaborative training due to heterogeneity; (2) design of an efficient and adaptive knowledge-transfer framework to bridge heterogeneous participants based on their underlying proximity benefits; (3) privacy strategies for heterogeneous collaboration by identifying new vulnerabilities and developing privacy-preserving mechanisms. A general-purpose testbed is built to rigorously validate the proposed research and expand the impact of this project. It is expected that this project opens a new research paradigm to unleash the utmost potential of heterogeneous and collaborative machine intelligence. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.