Design and Implementation of Performance Metrics for Evaluation of Assessments Data

Abstract

Evocative evaluation of assessment data is essential to quantify the achievements at course and program levels. The objective of this paper is to design performance metrics and respective formulas to quantitatively evaluate the achievement of set objectives and expected outcomes at the course levels for program accreditation. Even though assessment processes for accreditation are well documented but existence of an evaluation process is assumed. This work provides performance metrics such as attainment, student achievement, and x-th percentile for the evaluation of assessment data at course and program level. Then, a sample course data and uniformly distributed synthetic data are used to analyze the results from designed metrics. The primary findings of this work are twofold: (i) analysis with sample course assessment data reveals that qualitative mapping between marks obtained in assessments to the defined outcomes is essential for meaningful results, (ii) analysis with synthetic data shows that higher values of one metric does not imply higher values of the other metrics and they depend upon the obtained marks distribution. In particular, for uniformly distributed marks, achievement < attainment for meanOfUniformDistr. < averageMarks < passingT hreshold(α). Authors hope that the articulated description of evaluation formulas will help convergence to high quality standard in evaluation process.