CAREER: Enabling Reliable Uncertainty-Aware Decision Making with Unreliable Data

With the availability of abundant data, deep learning models have advanced significantly and achieved unprecedented prediction capabilities. However, when these models are used in critical applications like medical diagnostics, it becomes crucial to measure how confident the models are in their predictions. Accurate uncertainty estimation can help decision makers understand the reliability of predictions and ensure transparency, safety, and trust in high-stakes scenarios. Recently, significant attention has been given to conformal inference, a statistical technique that offers rigorous measures to quantify the uncertainty of the predictions. This allows users to know how much potential uncertainty is the case for individual predictions. Its impact spans a broad spectrum of real-world application domains. However, existing conformal inference frameworks are mostly data-driven and face significant challenges when handling unreliable, real-world data (e.g., noisy and incomplete data), which are common in many applications. This project aims to tackle these challenges, improving conformal inference to better handle unreliable data and enhance its effectiveness in real-world uncertainty studies. This project develops a suite of novel conformal inference models and algorithms to enable reliable uncertainty-aware decision making with unreliable data. Specifically, it focuses on four research objectives. The first objective addresses challenges posed by data noise by relaxing overly optimistic data assumptions across the essential stages of the conformal inference framework. The second objective explores innovative solutions for imputing missing data within the framework, while addressing the optimistic assumptions that may be violated due to noise introduced during the imputation process. The third objective enhances the usability of the obtained uncertainty information to improve model performance. The fourth objective systematically validates the proposed research across various application domains and incorporates expert feedback to refine the approach. The outcomes of this project will empower researchers and practitioners to integrate predictive uncertainties in data mining and machine learning across diverse domains, enabling more informed and reliable decision making to advance scientific discovery. Results will be integrated into existing curricula and new developed courses. This project will also provide research opportunities for undergraduate and graduate students. Customized research and teaching initiatives will be developed to attract K-12 students to STEM fields and introduce them to conformal inference and data science research. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.