Integrating Clinical Knowledge: A Multimodal AI System for Reliable Head CT Understanding and Interpretation

SUMMARY The field of radiology is undergoing a significant transformation with the integration of artificial intelligence (AI), particularly in head computed tomography (CT) scan interpretation. While AI systems are evolving from narrow-task solutions to comprehensive systems for detecting multiple findings and generating preliminary reports, we lack robust frameworks for evaluating their accuracy across both radiological observations and diverse clinical scenarios. This project advances head CT interpretation through three interconnected approaches focused on knowledge representation, image-based report verification, and longitudinal analysis of sequential scans. We propose developing a comprehensive knowledge graph and similarity metric for standardizing radiological concepts, leveraging large language models to create more granular taxonomies than existing medical ontologies. Additionally, we will implement an AI-enabled verification system for real-time assessment of radiology reports, employing uncertainty estimation and visual entailment models to verify factual accuracy of statements against imaging findings. In addition, we will develop novel methods for automated longitudinal comparison of sequential head CT scans, addressing the challenge of tracking multiple findings across complex neuroanatomical structures. This will enable both quantitative measurements and qualitative assessments of temporal changes. The project leverages datasets from multiple health systems and brings together expertise in biomedical AI, radiology, neurology, and neurosurgery. Our work will establish essential foundations for reliable AI-assisted head CT interpretation, enhancing radiologist trust and improving patient care through more accurate and verifiable reporting systems.