ChatGPT as scaffold: quiz performance across session complexity

Abstract

Many studies examine the use of ChatGPT in education, but most measure student perceptions, not performance, and few track performance across multiple sessions of different complexity. In particular, no study has tracked whether this association varies across sessions of different cognitive complexity. Using a quasi-experimental design, first-year undergraduates (N=193) at the Higher International Institute of Tourism (ISITT) in Tangier, Morocco were followed across seven introductory statistics sessions. One group had access to ChatGPT during learning activities, while the other followed the same instruction without artificial intelligence (AI) access. Performance was measured through end-of-session quizzes (1,136 observations) and analyzed using a linear mixed-effects model. No consistent overall advantage was associated with either condition. However, a significant interaction between condition and session was identified (χ²(6)=61.50, p<0.001). The most pronounced divergence occurred in session 6, which involved multi-step computation (harmonic mean and grouped-data mode interpolation), where the AI-permitted group scored higher (9.15 vs. 7.18, d=1.24). Across the remaining six sessions, effect sizes ranged from d=−0.20 to d=+0.19. Our findings suggest that AI integration should be selective, used during complex procedural sessions rather than uniformly across all course content.