Non-parametric estimation under covariate shift: From fundamental bounds to efficient algorithms

Machine learning (ML) methods for large-scale prediction have had dramatic impacts on various branches of science and engineering over the past decade. However, these methods --- when they are used on data sets that differ from the training data --- are often reported to be unstable, or to fail in mysterious ways. Failures of this type --- and our lack of understanding of their root causes --- pose a major roadblock to the adoption of modern ML methods in high-stake settings, where the cost of failure might be significant (e.g., self-driving cars, financial risk assessments, medical diagnoses). The goal of this research project is to characterize the fundamental causes of such failures, and to develop new algorithms that mitigate these issues. The project will also integrate research and education through: (a) the involvement of both undergraduate and graduate students in the research and in the dissemination of research results; (b) the inclusion of the research results in courses at MIT and in the web-based course materials, which are accessed from other universities; and (c) short courses at summer schools and workshops. The project will also support mentoring graduate students and postdocs that are under-represented in the STEM fields, with continued professional support in their careers. In more detail, the research focuses on the challenge of covariate shift, in which the distribution of the feature vectors used to train a large-scale prediction model differ from those on which it is evaluated. An initial goal is to characterize fundamental limits and develop efficient algorithms for estimation under covariate shift, in the finite-sample non-asymptotic setting. Armed with such an understanding, a follow-up goal is to develop computationally efficient procedures that achieve the fundamental limits along with theoretical understanding of their behavior. The work in this project will leverage and build upon techniques and tools from non-parametric analysis, empirical process theory, concentration of measure, reproducing kernel Hilbert spaces, and information theory. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.