SHF: Medium: Scallop: A Neurosymbolic Programming Framework for Combining Logic with Deep Learning

Neurosymbolic programming is an emerging paradigm that aims to address fundamental challenges for deep learning by combining it with classical logical reasoning. This project aims to realize the promise of neurosymbolic programming by developing a methodology, algorithms, software implementation, benchmarks, and case studies. To execute the proposed research, the team brings together researchers with expertise spanning machine learning, logical reasoning, formal methods, and programming systems, and will collaborate with a radiology researcher on a real-world application in healthcare. The primary outcome of the project will be an open-source framework for neurosymbolic programming that will be of technical interest to researchers in both machine learning and logical reasoning. In a broader sense, the project contributes to the vision of Trustworthy Artificial Intelligence (AI), and enables its deployment in critical applications such as healthcare. Through education and knowledge transfer activities, the project will build a community of researchers at the intersection of machine learning and logic, to facilitate tight integration of research results with education. The project is centered around a neurosymbolic programming framework, called Scallop, that integrates neural architectures with a declarative rule-based logic programming language. Such a design allows convenient specification of challenging tasks such as visual question answering by a suitable decomposition of the desired computation into neural and symbolic components. To address the core challenge of developing end-to-end gradient-descent-based learning algorithms for such neurosymbolic programs, the research is organized along three foundational themes: 1) symbolic reasoning constructs that allow specifying rich domain knowledge yet enable efficient inference and learning; 2) scalable learning for neurosymbolic programs based on ideas rooted in abductive inference and data provenance; and 3) theory and techniques for semantic robustness of neurosymbolic programs. Complementary research tasks include development of the Scallop compiler and toolchain, collection of benchmarks for neurosymbolic learning from a wide range of computational tasks, empirical evaluation, and exploring an application to improve breast cancer risk assessment by integrating neural-network-based image processing of mammograms with rule-based expert knowledge. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.