Collaborative Research: HCC: Trustworthy Web Interaction for People with Visual Impairments through Intelligent Detection and Management of Deceptive Web Designs

The internet plays a central role in most areas of society, including, but not limited to communication, education, employment, healthcare, government services, and commerce. However, some websites use deceptive design practices, often called dark patterns, that manipulate users into making unintended decisions, sharing personal information, or spending money. These practices can pose unique challenges for people with visual impairments who rely on screen readers and other nonvisual technologies, because important visual and spatial cues may be inaccessible or misleading. This project seeks to make web interaction safer and more trustworthy by developing intelligent technologies that help people with visual impairments recognize, understand, and manage deceptive web designs. The project will benefit people with visual impairments by reducing risks, such as unintended purchases or privacy violations, and decrease cognitive burden during everyday online activities. The resulting knowledge and technologies will also contribute to more effective and ethical digital design practices, support the training of students in human-computer interaction and trustworthy artificial intelligence, and inform the development of digital tools and design practices that improve transparency, user protection, and trustworthiness across the broader Web. This project investigates how artificial intelligence can be used to detect and mitigate deceptive web design patterns that disproportionately affect screen reader users. The research combines advances in large language model–based agentic artificial intelligence, accessibility research, and human-centered design. First, the project will develop intelligent detection algorithms that automatically analyze webpage layouts, interaction workflows, user actions, and accessible navigation paths to identify deceptive patterns, including those that arise from mismatches between visual webpage structure and screen reader navigation. Second, the project will design and develop customizable nonvisual assistive interfaces that help users understand and manage detected deceptive patterns, with interface requirements and refinements informed through iterative studies involving people with visual impairments. Third, the project will integrate these technologies into a browser extension and evaluate their real world effectiveness, usability, privacy implications, cost, and latency through controlled experiments and user studies. More broadly, the project will investigate how intelligent agents can model screen reader navigation behavior to identify deceptive patterns, explain associated risks, and support informed user decision making under real-world constraints. The project will generate new knowledge on trustworthy artificial intelligence, accessible computing, and human-centered web technologies, while advancing methods for protecting users from online deception in real-world digital environments. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.