Conference: Workshop on Establishing Standards and Metrics for Probability Survey Panels

This project creates clear and easy-to-follow rules to help researchers, leaders, and businesses understand how survey data are collected and how trustworthy they are. Surveys are used to learn what people think, do, and experience. The information from surveys helps leaders make better decisions that affect everyone. A new, faster, and less expensive way to collect survey data is through probability survey panels, which include people from all over the country who are selected using scientific methods and asked to take surveys regularly. These panels are used in thousands of studies each year to help make decisions about health, education, and public policy. But it is not always clear how these panels are created, how data are collected, or how reliable the results are. If it is not clear how surveys are done or how good the data are, it is hard to trust the results. This workshop brings together experts from different fields to create simple and clear rules for how probability survey panels should share information about their quality and methods. By making survey methods more open and easier to understand, this project helps make sure the data behind important decisions can be trusted and are worth the support of taxpayers. In particular, the project helps ensure that probability survey panel data are of sufficiently high quality and trustworthiness to be valuable in training AI models. This project also facilitates the translation of research on survey practices to businesses that conduct surveys. The project addresses a growing dilemma in survey research: the lack of clear and consistent transparency standards for probability survey panels, which are widely used to collect high-quality, representative data. As more researchers use these panels, it is critical to know how the panels were created, managed, and used to gather data, so the quality of the findings can be evaluated. This workshop gathers leaders from government, academia, journalism, and industry to develop shared guidelines for what should be reported about these panels. The workshop focuses on three main areas of a panel's lifecycle: how participants are recruited, how the panel is maintained, and how individual surveys are carried out. The team produces a widely disseminated public report with easy-to-follow guidance and definitions to help everyone, from creators and users of probability survey panels, including users of AI models and businesses, to the public and decision makers, better understand, compare, trust, and use the results of surveys that these panels produce. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.