CAREER: Advancing Experimental Frameworks and Empirically-Based Guidelines for Designing and Assessing Statistical Data Graphics

Charts, graphs, and other visual representations of statistical data are often used to help people communicate and make decisions. However, when creating these graphics, many designers use rules of thumb, guidelines, and individual opinions that are often not well-grounded in experimental evidence. Further, studies that do assess the design of graphics tend to focus on a relatively narrow set of uses and quality measures. These risks can lead designers to design graphics that harm rather than help communication and decision-making. This project's goal is to develop a more evidence-based and holistic approach to studying the quality of graphics and creating guidelines for designing effective graphics. This will include developing a wider range of measures and use cases to consider when studying how people use graphics, then conducting experiments using these measures and developing guidelines for effective design based on the results. These guidelines and experimental methods will then be made available to scientists who create graphics when talking about their work with the public. The research team will also integrate the work with math and statistics education courses, as well as outreach activities with middle and high school math teachers, in order to provide additional data for the research itself while enhancing those educational activities. Together, the project will improve visual communication of data by developing design guidelines supported by empirical studies that evaluate graphics across a range of real-world tasks. The research focus of the project is to develop and validate an integrated approach to the study of data visualizations that examines multiple levels of user engagement with each chart, using simultaneous talk-aloud, direct annotation, and interactive tools that support graphical decision-making and record participant strategies for engaging with the graphics. Once an optimal combination of measurement methods has been selected that balances cognitive load with holistic assessment, the research team will develop a statistical framework and research infrastructure to collect multi-modal data for the analysis of visualization experiments. Combining multiple data streams will enable both the PI and other researchers to assess design decisions with respect to their impact on different types of user engagement. Leveraging these measurement and analysis methods, the project will empirically assess common chart design guidelines to determine how design decisions impact a range of different interactions with data visualizations, including estimation, inference, prediction, and application. The empirical assessment of chart design decisions will provide experiential learning opportunities for undergraduate students in introductory statistics and numerical literacy courses, reinforcing the material taught in class and increasing student awareness of the importance of data visualization for effective communication. The research team will also improve quantitative literacy by developing continuing education modules to equip secondary school science and mathematics educators with hands-on activities that integrate statistics and data collection with civics, health, and art to connect with and excite students about math and science. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.