Improving Science Communication via Results-Blind Selection: Lessons from Registered Reports

We are often told that science is trustworthy because scientists focus on how to answer a question without getting to decide what that answer will be. For instance, a psychological scientist might carefully design a study to compare strict and lenient teaching styles, opening up the possibility that the answer could be unexpected, or even disappointing. Unlike scientists, however, the public learns the answers that have been selected by science journalists and editors, a system that opens the door to distrust. This distrust is apparent in a recent PEW report, which found that only 52% of Americans say that the media do a good job of covering science, a rate that puts the US second-to-last among the 20 countries surveyed. If journalists are more likely to report on sensationalistic findings over those supported by strong evidence, this could damage the dependability of those findings as source of guidance. Further, if journalists are more likely to report findings that reinforce a particular ideology, this could foster polarization and public distrust. The present proposal will test one strategy for improving the trustworthiness of psychological science reporting: journalists could select research based on methods rather than results (results-blind selection). This approach thus shifts the focus to prioritizing sound, rather than sensational, findings. By pursuing concrete strategies for improving the trustworthiness of science journalism, this proposal aligns with the National Science Foundation’s goal to “increase the public value of scientific activity.” The idea that results-blind evaluation could reduce bias and improve the quality of featured research is the foundation for the Registered Reports academic article format, first introduced at Cortex in 2013. Previous research shows that Registered Reports are rated higher than the comparison papers on rigorousness and quality, suggesting that results-blind selection may be an effective way to promote quality and replicability. In the present project, we propose three studies. In Study 1, we will recruit 400 journalists and test whether results-blind selection of scientific studies can improve the trustworthiness of reported research. In Study 2, we will recruit 250 participants and test whether results-blind selection of scientific studies can increase public trust in science journalism. Finally, in Study 3 we will re-contact 32 journalists from Study 1 and use group interviews to explore how results-blind selection could be effectively integrated with existing journalistic norms and routines. These findings will inform our understanding of existing mechanisms that rely on results-blind selection – like Registered Reports – as well as our understanding of the potential value of this approach to improving the quality the psychological science that gets communicated to the public. The project is co-funded by the Science of Science: Discovery, Communications, and Impact and the Advancing Informal STEM Learning Programs. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.