HCC: Rethinking Visualization Evaluation through Image-Based Models of Human Vision

Data visualizations are used by both everyday citizens and experts to make consequential decisions, such as selecting a stock to invest in or making a diagnosis from a medical scan. Before settling on an effective visualization, a designer often must test several designs with users in order to identify the one that leads to the best decisions. However, this testing procedure is costly and time-consuming, requiring many rounds of evaluation. Moreover, results from user studies can be imprecise, as even small changes to a visualization or its underlying data can render a previously effective design unreliable. This project addresses this problem by developing new computational tools for evaluating visualizations that reduce reliance on costly human studies. To do this, the research team will simulate how the human visual system responds to a visualization image. The team will then develop algorithms that judge visualization effectiveness based on whether that simulated response faithfully preserves the data properties encoded in the image. These judgments will be calibrated against human study results. The resulting algorithms will be released as open-source software for designers and researchers and will be applied to biomedical visualizations for evaluating potential cancer treatments. More specifically, this project will develop a framework for computational visualization evaluation grounded in image-computable models of human vision. The research team will extend existing models of texture perception to operate on visualization images. These models will be used to predict which data properties are retained by the early-to-mid visual cortex, and hence are available to support perception and judgment by the viewer. The resulting framework will be validated against empirical data from controlled human studies spanning a range of chart and map types, data properties, and visual inference tasks. The developed models will be embedded in accessible, open-source software that enables designers to assess visualizations without having to run user studies. Finally, the team will translate these methods into a visualization tool for biomedical scientists evaluating how well various potential cancer treatments might work for patients. If successful, the project will shift visualization evaluation from a labor-intensive empirical practice toward a more principled, scalable, model-based analysis. This stands to broadly improve how visualizations are designed, tested, and deployed across scientific and public-facing domains. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.