SLES: Foundations of Safety-Aware Learning in the Wild

Machine learning (ML) models today must operate amid increasingly dynamic and unpredictable environments. A crucial challenge for models deployed in the wild is that they will encounter unknown out-of-distribution (OOD) data, in addition to the known in-distribution (ID) data. Current approaches for training ML models, particularly in a supervised setting, are known to be brittle and lack necessary safety awareness, e.g., OOD data may be blindly classified as a known class with high confidence. The project's novelties are developing safety-aware learning methodologies and theoretical guarantees that can provably detect OOD data as models are deployed in the wild. The project's impacts are to enhance safety for a broad range of downstream applications that depend on artificial intelligence (AI) classification, including transportation, healthcare, commerce, and scientific discovery, so that they can properly handle unexpected input. This project will make AI understand better what it knows and doesn't know, so that it abstains from unexpected input instead of wrongly classifying with supreme confidence. Technically, the team of researchers will design new algorithms that can leverage a large amount of real-world unlabeled data that arises ubiquitously in the model's deployment environment. Learning from such data can be challenging due to its heterogeneity (mixed with ID and OOD data) and non-stationarity (changes over time). To address the challenges, the project designs new machine learning algorithms that provably use unlabeled wild data for OOD safety-aware learning, online OOD detection to adapt to changing environments, and OOD detection with foundation models. The learning framework will be evaluated by a balance of empirical experimentation and theoretical understanding. This research is supported by a partnership between the National Science Foundation and Open Philanthropy. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.