Academic engagement and artificial intelligence platform behaviors in grammar achievement

Abstract

This study is among the first to use archival institutional records to test the incremental validity of artificial intelligence platform behaviors (AI_index) in predicting grammar achievement (GA). Using data from 405 non–English-major freshmen enrolled in a compulsory grammar course at a private Chinese university, we examined whether AI_index predicts end-of-semester grammar exam performance beyond course-embedded behavioral academic engagement (AE_index). AE_index was derived from grade-book quizzes and class interactions, whereas AI_index was constructed from institutional platform logs capturing coursework completion and assigned video viewing. Indices were scaled to a 0–100 range, and GA was measured by a unified final exam. Descriptive statistics, correlations, and hierarchical regression analyses showed that AE_index was a small but significant predictor of exam performance, whereas AI_index was weak and non-significant and added no incremental predictive value beyond AE_index. Together, the two indices explained a modest proportion of variance in GA. These findings suggest that completion-based platform metrics are unlikely to reflect effortful learning unless platform tasks align with summative assessment demands (e.g., translation and proofreading). The findings caution against using completion-based AI metrics as high-stakes indicators without demonstrated task–assessment alignment.