DL-based CT image formation with characterization and control of resolution and noise

Deep learning (DL) based CT image formation methods have proliferated over the past few years. The existing approaches mostly follow the paradigm established in computer vision, and build a deep neural network (DNN) with standard modules that capture salient image features useful for computer vision tasks. These standard modules also work very well for CT images, placing DL-based CT image formation methods at the forefront of research and innovation. However, current DNNs are oblivious to the fact that CT images, unlike natural images, must be interpretable by a radiologist to make a diagnosis. CT image interpretation is affected by image features such as image resolution and noise variance-covariance, which are under exploited by the standard modules from computer vision. Consequently, current DL-based CT image formation has no direct characterization, let alone prospective control, of image resolution and noise variance-covariance. These properties can only be assessed after an image is generated, but resolution/noise has no direct influence during the image formation process. In this proposal, we challenge this established paradigm and propose an innovative DL framework named GradDNN to (1) characterize the resolution and noise properties of a DNN’s output during network training or parameter fine-tuning, and to (2) guide the image formation process so that the output has the desired resolution/noise properties. GradDNN (which stands for gradient + DNN) applies network linearization, i.e., gradient computation, to a mother DNN to extract local resolution and noise properties of the images generated by the mother, and make these properties available during network training and parameter fine tuning. The linearization method for analyzing nonlinear systems such as a DNN has never been attempted before. Conceptually, GradDNN associates a daughter module to any mother DNN for noise/resolution characterization, thereby making the resulting network CT-specific. We will develop GradDNN and demonstrate its capability in the context of two important clinical tasks: (1) mitigation of calcium blooming in coronary CT angiography, and (2) low contrast lesion detection in abdominal CT, both of which have high requirements on resolution and noise. Data for DL network training will be generated using digitally augmented patient data prospectively collected at two sites. Image quality comparison between the mother DNN alone and the mother+daughter duo, using a number of effective mother DNN architectures, will be carried out to demonstrate the additional gain of DL with joint resolution/noise learning. The comparison will use both digitally augmented patient data and real patient data to further establish robustness and generalizability. In this exploratory proposal, we focus on DL networks that perform image-to-image transformation. However, the learning framework using GradDNN is general and can be applied to DL networks that perform direct projection-to-image transformation. Successful completion of this proposal will generate strong preliminary data for a follow-up R01 that extends our results to such DL networks.