Robust Classification and uncertainty quantification for non-iid samples

Although advancements in machine learning have significantly improved classification accuracy in various applications, recent research has expanded the focus beyond solely prediction accuracy. There is now an increased emphasis on the necessity for robust quantification of prediction uncertainty, self-awareness in handling abnormal samples, and the ability of classification models to generalize to minor or novel populations. Addressing challenges in these settings, where the traditional independent and identically distributed (IID) data generating assumption no longer holds, is crucial for effectively applying machine learning techniques in safety-critical and fairness-critical systems, such as medical diagnosis or policy making. The project will also contribute to the training of students through their involvement in the research. This project aims to advance robust methods for inferring class labels and quantifying uncertainty under complex non-IID settings by employing techniques from distributionally robust optimization, fairness learning, conformal prediction, and semi-supervised learning. The project will propose innovative classification strategies that exhibit improved worst-group performance across latent sub-populations and enhanced fairness with respect to potentially latent sensitive attributes. These strategies will be integrated with state-of-the-art machine learning techniques, including neural networks and gradient boosting, to develop robust, generalizable, and flexible machine learning algorithms. Additionally, the project will develop novel adaptive classification strategies and investigate their theoretical guarantees. These strategies will leverage both labeled training data and unlabeled test samples. Finally, the project aims to facilitate the in-depth application of the developed methods in safety-critical and fairness-critical systems, particularly in the domains of medical and immunological studies. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.