Excellence in Research: Trustworthy AI for Healthcare: Evidence-Grounded, Compliant, Human-in-the-Loop

Artificial intelligence (AI) could help healthcare professionals summarize records, coordinate care, and make faster decisions, but today’s large language models can invent facts, expose sensitive patient information, and provide answers confidently without evidence for the projected confidence. This project asks how healthcare AI can be made accurate, private, and accountable by design. The central idea is to tie important claims to trusted medical evidence, limit each AI component to the minimum patient information needed for its task, and involve clinicians whenever uncertainty or risk is high. Rather than replacing clinical judgment, the system is intended to help experts work more efficiently while preserving their authority over final decisions. The research could enable safer clinical summarization and decision support while producing methods that transfer to other regulated fields such as law and finance. It will also strengthen AI research and education at Tuskegee University, expand student opportunities, broaden participation in the national AI workforce, and share resources with partner institutions and other Historically Black Colleges and Universities. The project integrates three research thrusts. First, a Clinical Knowledge-grounded Retrieval-Augmented Generation framework will organize validated medical sources as an evidence graph of atomic claims and typed relationships, optimize query-specific support subgraphs, constrain generation to high-confidence evidence paths, and return provenance linking claims to sources. Second, a compliance-focused large language model agent will semantically audit and remediate data through redaction, generalization, or synthesis, while a compliance-aware orchestrator builds least-privilege task graphs for specialized agents. Third, risk-aware uncertainty monitoring will route selected outputs to clinicians; structured diagnostic traces will localize failures in evidence, agents, or data flows; and a remediation planner will propose reversible fixes with targeted tests for expert approval. Evaluation will use medical question-answering and de-identification benchmarks, a mock electronic health record, simulated clinical workflows, penetration testing, and clinician studies with clinical partners to measure accuracy, privacy, efficiency, diagnostic value, and human-AI team performance. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.