Effectiveness of anchoring vignettes in re-evaluating self-rated social and emotional skills in mathematics

Abstract

In current assessment practice, self-ratings and questionnaires are a dominant tool used to measure the skills called social and emotional skills or non-cognitive skills, although the tools are affected by various biases. In this regard, the anchoring vignette approach was introduced against the biases, correcting individuals’ self-rated responses based on their rating of hypothetical individuals in the scenario. Drawing from students’ self-rated social and emotional skills in mathematics, this paper presented the study which examined the effect of anchoring vignette approach on reliability and correlation by comparing self-rated and vignette-corrected scales. Research participants were Mongolian students in ninth grade (N=308). The participants were administered in two scales: self-ratings for math perseverance and cooperative learning in math, followed by a vignette set. The vignette-corrected scale showed higher reliability than the self-rating scale for both math perseverance and cooperative learning in math. Besides, the vignette-corrected scales for gender and region showed a stronger and more significant correlation than the self-rating scales, suggesting that there might be gender- and region-related differences in the way the students act in math class. In summary, these findings suggest that the anchoring vignette approach has the potential to measure social and emotional or non-cognitive aspects of mathematics in a more reliable way. Future studies could further investigate the reliability and validity of the anchoring vignette approach by including more cultural groups and designing the vignettes while considering various math contents and vignette gender.