Understanding Equivalent Equations: Using Clinical Interviews and Nominal Response Cognitive Diagnostic Models to Measure and Describe Students’ Sensemaking

This project aims to serve the national interest by developing a tool that researchers and instructors can use to assess what students understand about core ideas needed to work with equations. The U.S. currently faces a shortage of STEM graduates, and algebra is often a barrier to earning STEM degrees. This project aims to provide instructors and researchers with assessments to measure student understanding of equations, using innovative statistical techniques to measure a wider range of student thinking than existing tests. Equations are used extensively across STEM fields to model real-world problems, and therefore any tools that strengthen students’ understanding of equations have the potential to substantially expand and strengthen the STEM workforce. Instructors could use assessments produced by this project to improve their instruction, and researchers could use them to assess the effectiveness of different courses and teaching approaches. Once developed, assessments and learning models from this project will also have the potential to be validated or adapted to other contexts, such as high school. In addition, the innovative methods used by this project to develop assessments could then be used as a model for developing more effective assessments on other topics. This mixed methods project will combine nominal response cognitive diagnostic models (NRCDMs) and in-depth cognitive interviews to describe new conceptions of equivalent equations not documented in existing literature, better define the boundaries of conceptions mentioned in existing literature, and explore the relationships between conceptions (e.g., analyzing which conceptions commonly co-occur and whether some conceptions have a hierarchical relationship with one another). NRCDMs are an innovative approach that draws on the full range of student responses. The most commonly used psychometric models discard information about the specific answer option students chose by scoring each item only as correct/incorrect. In contrast, NRCDMs model the specific answer options chosen, allowing for analysis of larger student response patterns that correspond to specific conceptions that each answer option was designed to measure. The goal of this project will be twofold: 1) to generate a robust framework of conceptions that describes the varied ways that college students think about equivalent equations; and 2) to produce a diagnostic instrument that instructors and researchers can use to determine which conceptions of equivalent equations students may hold, which can subsequently be used in future to better adapt and differentiate curriculum and instruction and to inform research. This project thus has the potential to contribute to a richer understanding of how students think about equivalent equations and provide a useful diagnostic instrument for practitioners, as well as to provide an assessment development model that can subsequently be used to develop other diagnostic instruments to measure conceptions in other mathematical domains. This project plans to share assessments, learning frameworks, and methods of assessment development resulting from this research with practitioners, researchers, policymakers, and the general public through a wide variety of dissemination efforts. This project is supported by NSF’s EDU Core Research (ECR) program and NSF's IUSE: EDU Program. The IUSE: EDU Program supports research and development projects to improve the effectiveness of STEM education for all students. The ECR program emphasizes fundamental STEM education research that generates foundational knowledge in the field. Investments are made in critical areas that are essential, broad and enduring. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.