CAREER: Resolving Uncertainty Visualization Reasoning Errors with Mental Model Design and Training

People worldwide use data visualizations that show forecasts of future events to decide how to respond to impending hazards. For example, television news meteorologists often use visualizations of a hurricane's forecasted path to inform the public about an approaching storm. Unfortunately, a large body of research demonstrates that people misinterpret the most common methods for visualizing uncertainty in forecasts such as these. This project’s goal is to learn more about why people have difficulty using forecast visualizations and how to create more effective ones. One key outcome of the project will be a theory of uncertainty visualization literacy that will identify the skills needed to effectively use visualized uncertainty and provide cognitively informed rules for new visualization designs. This work will develop more effective methods to convey forecast uncertainty, along with uncertainty literacy training that will support the public in making informed decisions in response to natural disasters and public health crises. The research plan includes a series of empirical studies to test competing hypotheses, including a novel theory centered around integrating users' mental models into visualization design and training. The new hypothesis predicts that uncertainty visualization reasoning errors result from discrepancies between how people conceptualize a forecast (a priori schemas) and how the visualization presents the data (visualization-driven schemas). The first phase of this work will create tools to reveal a priori schemas and visualization-driven schemas using previously established cognitive methods for evaluating schemas. These methods include analysis of participants' drawings, eye tracking, and memory tests. In the second phase, the team will develop a measurement tool to determine the relative distance between the two schemas, using a mapping agreement unit based on methods developed in human factors, then use that tool to empirically test the schema-based theory compared to alternative explanations for reasoning errors when using uncertainty visualizations. In the third phase, the team will use the winning approaches to develop visualization training and new visualization designs of hurricane path forecasts and COVID-19 morbidity projections as testbeds. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.