Validation of a digital tool for diagnosing mathematical proficiency

Abstract

This study was aimed to validate a digital tool for diagnosing mathematical proficiency in the Number and Algebra strand of 1,504 Thai seventh-grade students. Researchers employed a multidimensional approach, an extension of the Rasch model to measure its quality. A design-based research method was adopted to create the diagnostic tool which consists of four components, namely register system, input data, process system, and diagnostic feedback report. The diagnostic framework consists of 18 tasks encompassing 11 and 7 tasks in the Mathematical Procedures Dimension and Structure of the Observed Learning Outcome dimension, respectively. The results revealed that there is internal structure evidence of validity based on the comparison of model fit and the Wright map. The results also indicated that the reliability evidence and item fit are compliant with the quality of the digital tool as shown in the analysis of standard error of measurement and infit and outfit of the items. In conclusion, the developed digital tool can diagnose seventh-grade students’ multiple mathematical proficiencies in terms of accuracy, consistency, and stability. This implies that the digital tool can provide fruitful information, particularly to those intermediate and high mathematical proficiency levels because the error for estimating proficiency in each dimension was at the lowest value for those students.