Collaborative Research: III: Medium: Interpretable Scientific Discovery with Foundation Model Programs

Generating nuanced yet interpretable hypotheses from noisy and high-dimensional observations is a core activity across numerous scientific disciplines. For example, across neuroscience, genomics, biomechanics, and ecology, researchers analyze images and videos to study phenomena ranging from the genetic expression of lab mice to global species distributions of birds. Useful scientific hypotheses are typically instantiated on symbolically interpretable concepts and attributes (such as stride periods and center of mass oscillations when studying kinematics of walking). A key challenge is to extract such symbolically interpretable hypotheses from raw high-dimensional observational data. To address this challenge, this project aims to develop a novel neurosymbolic programming framework, called foundation model programming, for generating symbolically interpretable scientific hypotheses from high dimensional observational data. The main idea is to represent interpretable hypotheses as neurosymbolic programs that use symbolic primitives as well as neural modules, including foundation models. The use of neural and foundation models allows the hypotheses to be full-stack, modeling both the extraction of relevant patterns and motifs from high-dimensional raw data and reasoning over those patterns. The core technical benefit of this approach is that it inherits both the contextual flexibility of modern foundation models given high-dimensional inputs, and the rigorous reasoning abilities offered by neurosymbolic approaches. The proposal is backed by a substantial amount of prior work on both neurosymbolic learning and applying foundational models in scientific domains, including foundation model-enabled hypothesis generation for low-dimensional data, self-supervised symbolic feature extraction from high-dimensional data, data-efficient expert-in-the-loop training approaches, and deep deployment into real scientific workflows. Importantly, the use of foundation models enables building methods that are more autonomous, and require fewer manual annotations by the expert. This project will develop algorithms that can jointly reason over what are the most useful symbolically interpretable concepts or motifs that can be extracted from the raw data, as well as how to compose those concepts into coherent hypotheses. This paradigm mirrors how humans develop hypotheses, by jointly establishing a discrete vocabulary of concepts (from continuous high dimensional descriptions) and reasoning over those concepts, thus enabling interpretability. This project will benchmark the developed methods on several scientific tasks and domains and collaborations. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.