CRII: III: Harnessing Deep-Learning to Simplify Biological Inference from Complex Imaging Data

Imaging data at high resolutions is being produced at unprecedented scales. Advances in computer vision are required to meet the challenge of understanding and extracting meaningful information in these images to inform scientists and physicians accurately and precisely. To detect what is in each image, discover where the informative patterns occur, and determine what differences in these patterns may mean requires the deployment of deep learning methods. These methods require training data sets that will inform the network of meaningful patterns in the image that will enable proper classification and quantification. An unmet challenge is developing the proper training data that represents variation observed in the real world and implementing the appropriate computational methods to cope with this variation. This award has two goals: (1) the exploration of variation in model performance on highly variable data sets; and, (2) the unification of the processes of biological data collection with the development of deep learning models to analyze this data. The first two objectives develop a deep-learning framework that is trained on highly variable data (from non-model organisms collected from the wild) and imaged under controlled levels of variation at varying magnifications, staining procedures, and anatomical planes. The third objective also addresses the concept of data variation in an educational context, which develops an active learning seminar where two different data types (images versus sound files) that represent the same data source (bat species), teaching students how different features can be extracted to represent the same classification scheme. All three concepts thread together data collection; that is, how samples are collected and imaged by the investigator and students and followed by the deployment of deep-learning architectures to analyze this complex data. An outstanding need in deep learning with biomedical and histological imaging is accurate model prediction despite notable variation in data driven by both biological and methodological sources. The project explores the performance of the combination of convolutional neural networks and pre-trained networks (e.g., U-Net) in the prediction of categorical outcomes of bat histological data of the intestine. It also produces novel models of deep-learning image segmentation to quantify histological data that is often performed manually and currently not scalable. The networks will make use of numerous training data sets obtained throughout the career of the investigator and combine this training data with pre-trained models to improve accuracy. The products of the award will be the implementation of an application programming interface (API) in which users can upload their own images, which will further contribute to training data sets and allow for expedited high-throughput image processing. This project bridges multiple disparate disciplines in biology and computer science, leveraging the properties of recent advances in neural network algorithms for histological data in medicine to solve complex analytical issues that arise in comparative morphology, anatomical sciences, and evolution. A final objective of the project is to also develop a seminar course module for graduate students to build deep-learning neural networks to identify bat species from acoustic call libraries, uniting ecological data collection with computational biology and machine learning. Specifically, students will generate neural networks to classify different bat species based on acoustic calls that they record. The course is unique, as it exposes students to the entire pipeline of data informatics, including data acquisition via collecting bat acoustic data in the field, to processing the data in different formats, to generating the networks, and analyze the accuracies. Understanding sources of variation and noise that is introduced and how to overcome this variation is a key learning objective. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.