CAREER: Information-Theoretic Measures for Fairness and Explainability in High-Stakes Applications

Machine learning is becoming increasingly prevalent in various aspects of our lives, including several high-stakes applications, such as finance, education, and employment. These machine learning models have shown remarkable success at learning patterns present in the historical data. However, indiscriminate learning of all patterns can sometimes lead to predictor inaccuracies that have unintended consequences affecting impartiality of downstream decisions with respect to subgroups, e.g., people or organizations. This project seeks to advance the foundations of machine learning by developing a rigorous mathematical framework for explainable artificial intelligence (AI). Rethinking the traditional paradigm of separately addressing fairness and explainability, this research project will jointly examine fairness and explainability through a unified information-theoretic lens. Furthermore, through extensive outreach and student engagements, this project aims to instill interest in mathematically-principled approaches and STEM education among undergraduate and high-school students to spearhead the next generation of AI technology. The research project will provide a novel information-theoretic view of explainable machine learning, by leveraging a body of work in information theory called Partial Information Decomposition (PID). PID is closely tethered to the principles of Blackwell sufficiency in statistical decision theory and provides a formal way of quantifying when a random variable is “more informative” than another with respect to a target variable. Combined with estimation and optimization techniques, this project will enable us to disentangle the joint information content that several random variables share about another target variable. Four research thrusts will be investigated: (i) Providing an information-theoretic framework for explainable AI; (ii) Performing systematic feature selection and representation learning with impartiality constraints; (iii) Investigating fundamental limits with a focus on distributed and federated settings; and (iv) Validating these findings on real-world datasets in finance and education. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.