Simulated computer adaptive testing method choices for ability estimation with empirical evidence

Abstract

Computer adaptive testing (CAT) is a technological advancement for educational assessments that requires thorough feasibility studies through computer simulations to ensure strong testing foundations. This advancement is especially germane in Africa being adopters of technology, and this should not be done blindly without empirical evidence. A quasi-experimental design was adopted for this study to establish methodological choices for CAT ability estimation. Five thousand candidates were simulated with 100 items simulate through the three-parameter logistic model. The simulation design stipulated a fixed-length test of 30 items, while examinee characteristics were drawn from a normal distribution with a mean of 0 and a standard deviation of 1. Also, controls for the simulation were set not to control item exposure or to use the progressive restricted method. Data gathered were analyzed using descriptive statistics (mean and standard deviation) and inferential statistics (Two-way multivariate analysis of variance: MANOVA) for testing the generated hypotheses. This study provided empirical evidence for choosing ability estimation methods for CAT as part of the efforts geared towards designing accurate testing programs for use in higher education.