Collaborative Research: CAIG: Assemblage-based machine learning to quantify oceanic geochemical fluxes from marine particle images

Sinking particles in the ocean are known as “marine snow.” These particles provide food for deep sea life and transfer nutrients as well as pollutants from the surface into deeper water. Scientists want to know the number and composition of particles to understand the impact on the chemistry and biology of the ocean. Underwater cameras reveal particle numbers but not composition. This project will develop artificial intelligence (AI)-powered tools to predict marine snow composition from photographs. Human expertise and knowledge will inform each development stage to build an interpretable model. The research will support training of graduate students in AI and oceanography. Microplastic particles will be quantified as part of the project to better understand how this pollutant moves through marine food webs. The blueprint for understandable computer vision tools built by this project has applications in other image-rich sciences like ecology and medicine. This project aims to predict the chemical composition of sinking marine snow from imaging data, testing the fundamental assumption that visual elements like shape and transparency are conservative indicators of composition. To address the inaccuracies of current allometric models, this project will develop an interpretable AI framework that utilizes a prototype-driven instance segmentation model, multi-output regression, and a fusion model that integrates environmental metadata. The developed AI framework will move beyond black-box processing to link predicted masks with specific morphological prototypes. Key activities will include analyzing data from six major oceanographic field campaigns to compare assemblage-based predictions against single-particle metrics; carrying out a human-in-the-loop interpretability audit, enabling the interactive refinement of model logic to align with physical reality; and creating a reproducible pipeline between AI and geosciences to better understand nutrient cycling, biomineral ballasting, and the vertical transport of pollutants. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.