Research Initiation Award: Enhancing Robustness in Large Language Models for Natural Language Understanding through Comprehensive Bias Analysis

The Historically Black Colleges and Universities Undergraduate Program (HBCU-UP) through Research Initiation Awards (RIA) provide support for junior and mid-career faculty at Historically Black Colleges and Universities who are building new research programs or redirecting and rebuilding existing research programs. It is expected that the award will help to further the faculty member's research capability and effectiveness and improve research and teaching at the home institution. Artificial intelligence (AI) systems that generate and interpret human language are increasingly used to support decisions in education, health care, finance, and public services. Although these systems can improve access to information, they may also produce incorrect information with unwarranted confidence, provide inconsistent answers to the same question, or perform less effectively in different social contexts. They can also be overly sensitive to small changes in wording or input, reducing their reliability in real-world applications. This research investigates why these problems occur and develops practical methods to improve the reliability, fairness, and robustness of modern AI systems. The research will also engage undergraduate students in research, provide hands-on training in trustworthy AI, and develop educational materials that strengthen research capacity at HBCUs. The research develops and evaluates new methods for improving the trustworthiness of large language models (LLMs) and related multimodal AI systems. First, the research investigates optimization of continuous prompt representations to systematically identify failure cases that cannot be discovered through conventional text-based prompt searches, enabling the development of more reliable mitigation strategies. Second, the project studies knowledge and bias in retrieval-augmented language models by grounding model responses in verified knowledge sources. Carefully designed synthetic datasets are used to evaluate and mitigate bias while ensuring realism and ethical use. Third, the project investigates multimodal robustness by analyzing how biases introduced through visual and textual inputs affect the behavior of modern language-and-image models under adversarial attacks, distribution shifts, and challenging real-world conditions. Efficient mitigation strategies will be developed to improve robust performance across multimodal tasks. The proposed methods are evaluated using openly available LLMs, established benchmark suites for reliability, fairness, and robustness, and predefined quantitative measures of success. All software, benchmark datasets, and evaluation tools developed through the project will be released as open-source resources to support future research and broader adoption of trustworthy AI. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.