Development of an Adaptive Assessment System of Mature Number Sense

Mature number sense (MNS) is the ability to understand numbers and operations then use that understanding to flexibly solve problems. It is a foundation for success in mathematics and everyday numerical decision making. However, teachers currently have few practical tools to quickly and accurately understand if their students are developing these skills. Existing assessments are often too time-consuming, difficult to administer, or not flexible enough to meet the needs of all learners across multiple grade levels. This project will create a new adaptive assessment system for students in grades 3–12 that can efficiently measure students’ number sense. In addition, a result from the project is a score report that provides teachers with clear, actionable information to guide their instruction. The assessment will adjust to students’ skill levels, reduce unnecessary testing time, discourage teaching to the test, allow multiple attempts to better capture student thinking and reduce anxiety and frustration, and provide results that educators can use to inform future instruction. This project has the potential to improve mathematics instruction for all learners in grades 3-12, support student success across a wider range of learners, and advance research on how mathematical understanding grows over time. This project will develop and validate the results and interpretations of an adaptive assessment system of Mature Number Sense (MNS) for students in Grades 3–12. This MNS test will be psychometrically rigorous, instructionally informative, and scalable for classroom implementation. The proposed work advances both assessment scholarship and practices, and deepening understanding of mathematical proficiency. Specifically, the research team will expand and calibrate a large item bank through expert review, teacher partnerships, student think alouds, and a multi-agent large language model (LLM) framework designed to generate and vet candidate items efficiently while minimizing calibration burden. On the basis of the item bank, the project will construct an adaptive MNS assessment system capable of personalizing MNS assessment, measuring MNS across a broad range of proficiency levels, and maintaining brief administration times. To address limitations of traditional scoring approaches that rely solely on final responses, a novel sequential item response theory (SIRT) framework that incorporates students’ sequences of responses in multiple attempts is used. This in turn leads to more accurate characterizations of students' MNS. Because classroom utility is central to the project, the research team will work closely with educators to evaluate the interpretability and instructional usefulness of score reports. Leveraging an interdisciplinary expertise in psychometrics, mathematics education, cognitive development, information technology, and AI, as well as longstanding partnerships with schools and educators, this project will produce an adaptive assessment that supports foundational research on the development of MNS, applied research linking MNS to broader outcomes in mathematics education, improved instructional decision-making in grade 3–12 classrooms, and wide applications of innovative assessment methodologies across STEM education. This project is supported by NSF's STEM K12 program. The STEM K12 program supports fundamental, applied, and translational research that advances STEM teaching and learning and improves understanding of education across the human lifespan and a range of formal and informal settings. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.