Collaborative Research: An Intelligent Natural Language Interface to Improve the Accessibility of Complex Data Tasks for Blind or Visually Impaired STEM Employees

Researchers at Northeastern University, the University of Maine, and Colby College are conducting research focused on developing and testing a technology-based solution that improves the accessibility of spreadsheet data tasks for blind and low vision (BLV) STEM employees. Data tasks include data exploration, data manipulation, and data analysis, which are often inaccessible to BLV employees due to the visual attributes of commonly used data tools and the limitations of current keyboard-based functions. Through this research, keyboard functions will be enhanced with the addition of natural language-based functions, resulting in an intuitive and accessible approach to performing complex data tasks. Proficiency in the use of data is foundational for many STEM jobs. This project addresses a longstanding accessibility issue faced by BLV employees in STEM fields Employing a user-centered approach, this project has three phases: 1) establish design considerations and guidelines for developing BLV-focused natural language (NL) interfaces that can facilitate data analytics tasks, 2) develop a prototype NL interface to support BLV users to perform data analytics tasks on STEM-specific datasets, and 3) empirically measure the practical utility of the prototype system for its ability to support BLV users in performing these data tasks and assess its acceptability and usability among target end users. With BLV STEM professionals and researchers involved across all aspects of the research, the methods include interviews, co-design workshops, Wizard of Oz prototyping, and experimental evaluation. The results of this research will be a prototype of an intelligent natural language-based system, called Auxel, that will support BLV STEM professionals with spreadsheet data tasks. This award has been made in response to the NSF solicitation “Workplace Equity for Persons with Disabilities in STEM and STEM Education” (NSF 23-593). The Directorate funds this project for Social, Behavioral and Economic Sciences’ Office of Multidisciplinary Activities and the Division of Graduate Education’s EDU CORE Research program. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.