Deep Learning for Individualized Treatment Effect Inference with Multimodal Depiction

PROJECT SUMMARY Predicting the outcome of a treatment given the pre-operative patient status, i.e., individualized treatment effect (ITE) inference, is of great clinical importance for precise treatment planning. For example, the ITE w.r.t. survival time estimation of glioblastoma (GBM) patients undergoing different treatments enables assessing these possible multi-treatments by answering the question: ”would this patient have lived longer (and by how much), had an alternative treatment been applied?” Improving beyond subjective experience-driven therapy, the widespread accumulation of big medical data offers unprecedented opportunities for the data-driven deep learning (DL) algorithms to learn the underlying causal relations between multimodal depict patient imaging and clinical data, multi-treatments, and corresponding ITE. The practical ITE prediction requires a DL framework, which is largely unavailable at present. The current methods have limitations, including only utilizing the partial and incomplete status depiction, not applicable to multi-treatment on the outcome, and neglecting the ordinal ITE labels. In addition, the lack of reliability information and interpretability in the conventional DL model also hinders its large-scale clinical implementation. We propose to use our previous successful DL model to take both multimodal status and multi-treatments for accurate ITE inference for both factual and counterfactual cases with either continuous or ordinal labels. We will further establish a deep self-training scheme for reliable and interpretable ITE inference with quantified uncertainty and visualized DL-focused pathology region. The overall goal of this project is to develop an accurate, reliable, and interpretable pipeline for ITE inference by leveraging our advanced DL technique, which can be widely generalizable. This concept could significantly advance individualized treatment planning. The overall hypothesis is that the proposed solution can offer a unique opportunity to characterize causal relations among multimodal status depictions, multi-treatments, and corresponding ITEs with the novel DL model, which is not provided by current direct models. In addition, enabling the reliability quantification and interpretation of the underlying patterns of DL decisions could open a new window for ITE outcome utilization and treatment- specific pathology patterns investigation, thus leading to a multitude of new applications. The specific aims of this exploratory proposal are (1) to develop a multimodal multi-treatment DL framework for accurate ITE inference, (2) to establish a deep self-training scheme for reliable and interpretable ITE inference with calibrated uncertainty and visualized 4D (3D+modal) gradient activation. We will apply the proposed DL-based ITE inference framework to the clinical GBM survival dataset with different resections and test it based on various figure-of-merits. Successful completion of the project will provide a clinically applicable DL technique for better assessment and prediction of outcomes with individualized treatment. Using this overall strategy, which is only just now beginning to be explored, DL will play a major role in advising the best interventions.