An investigation of item bias in the four-tier diagnostic test using Rasch model

Abstract

The existence of item bias in a set of measuring instruments can threaten the instrument’s validity. Based on the Rasch model, this study evaluated item bias in the four-tier heat and temperature diagnostic test (4T-HTDT). This study used a cross-sectional quantitative survey method. There were 241 students selected using a stratified random sampling technique. The 4T-HTDT instrument consisted of 20 items grouped into five concept groups. Students’ conceptual understanding was grouped into five categories, namely scientific knowledge (Rating=5), false positive (Rating=4), false negative (Rating=3), misconceptions (Rating=2), and lack of knowledge (Rating=1). The differential item functioning (DIF) score was used to evaluate item bias in the 4T-HTDT. Bias was reviewed based on the respondent’s gender, class, and school. The item has DIF if the probability value is <5%. The results showed that 35% (7 out of 20 items) spread over five groups of heat and temperature concepts were biased. However, excluding seven DIF items from the measurement set would not significantly affect the composition and distribution of items. Thus, the 13 items in the 4T-HTDT instrument are free from bias and can be used to evaluate the conceptual understanding of high school students.