Collaborative Research: RI: Medium: Multilingual Long-form QA with Retrieval-Augmented Language Models

This project aims to enable automatic question answering systems to produce paragraph-level answers. Prior work on question answering has focused on simpler questions that can be answered with short phrases. Building systems to produce paragraph-level answers opens up exciting opportunities to answer complicated questions, and to offer more nuanced and comprehensive answers to simpler questions. This project will create comprehensive and reliable evaluation protocols for long form question answering (LFQA), pioneer multilingual studies to broaden information access to a wider population, and develop new algorithms that integrate web search with LFQA systems to provide verifiable long form answers paired with human-written evidence documents. This project focuses on three core dimensions of LFQA – datasets, evaluation, and modeling. Expanding the scope of prior English-centric LFQA, this research will investigate multilingual capabilities of large language models by constructing multilingual LFQA datasets and studying knowledge transfer across languages. In terms of modeling, it will propose a new framework that iteratively weaves together – in a transparent manner—knowledge retrieved from documents and memorized knowledge from a language model. Finally for evaluation, the project will engage domain experts who are familiar with the question topic to provide rationales for their evaluation of model generated answers. Such feedback will be used to derive a fine-grained annotation framework which localizes errors and unpack the weaknesses of generated answers. Together, the proposed work will bring significant progress to LFQA, an emerging topic for natural language processing and artificial intelligence research. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.