CAREER: Promoting Metacognition in Visual Analytics

Data-driven decision-making requires the people engaged in it to make a number of choices about the data to collect, and the methods for collecting it and analyzing it, as well as the interpretation of the results. Cognitive, cultural, and data biases can interfere with these processes at every stage, requiring data analysts to be thoughtful and reflective as they do their work. This project’s goal is to help analysts reduce their biases through tools that help them critically assess their thought processes using metacognition, or thinking about thinking. Metacognition will be a guiding idea for developing tool features that help analysts be aware of possible biases. Studies of metacognition have shown that it can be helpful in other educational and analysis settings; this project will use ideas from those studies to develop methods that identify potential biases and present activities to help analysts avoid them. The project team will also create educational materials and work with non-profit partner organizations to help the general public think more deeply about their own analytic strategies and how they might be improved. This project will apply theories of metacognition to address human biases and improve decision-making processes on an individual level. The project is structured around four research thrusts. The researchers will first organize theories of metacognition and translate them into an actionable design space of metacognitive interventions in visual analytics (Thrust I). Next, the researchers will work alongside non-profit partner organizations to co-design and develop a suite of metacognitive interventions (Thrust II) and evaluate those interventions in a series of laboratory experiments (Thrust III). Finally, the researchers will assess the extent to which empirical findings translate to real-world efficacy in a case study deployment of metacognitive interventions (Thrust IV). This work will test whether meta-cognitive interventions, when successfully applied in data-driven decision-making, can both boost awareness of the technical accuracy of analytic results and foster more thoughtful, socially accountable, and diligent practices in data analysis. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.