Collaborative Research: SaTC: CORE: Medium: Private Model Personalization

Modern machine learning's success has brought with it a serious challenge for privacy: it is now widely documented that the models currently in use encode individual inputs in surprising ways. Understanding how to detect such memorization, and training methods that avoid it, is a major topic of current research. However, prior investigations have focused mostly on the batch model of machine learning, in which training data are all drawn from a single underlying population. This project seeks to understand the privacy risks that arise when the training data from many populations are pooled in order to take advantage of structure that is shared across populations. For example, many individuals’ photos could be pooled to train better face recognition algorithms (even though each person is interested in a different set of faces). Such settings—called “model personalization”, “multitask learning” or “meta-learning”—provide a powerful framework for combining insights from far-flung, disparate data sources. However, their power raises fundamental questions about the extent to which the results of joint analysis violate the privacy of individual users' data. The project looks both at attacks on privacy—methods for extracting individual-level or dataset-level information from the resulting predictions or models—as well as mitigation strategies based on the now-standard, state-of-the-art framework, differential privacy. The project involves both theoretical analysis and real-world experimentation. It will inform the development of training algorithms for these complex settings and provide tools for use by companies and other research groups. This impact will be facilitated by the project team's existing collaborations with industry researchers. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.