Effectiveness evaluation and application of large language model in data-driven teaching decision-making

Abstract

This study aims to examine teachers’ perceptions of the effectiveness of large language models (LLM) in supporting data-driven decision-making in educational contexts. Specifically, the study explores the influence of technological pedagogical knowledge, technological content knowledge, and technological pedagogical content knowledge on teachers’ utilization of LLMs for informed decision-making. Additionally, it investigates the moderating role of ethical considerations in the use of LLMs. A survey-based methodology was employed to collect data from university teachers in Chengdu, Sichuan, China, resulting in a sample of 319 respondents, which was analyzed using Smart PLS 4. The findings indicate that technological pedagogical knowledge, technological content knowledge, and technological pedagogical content knowledge for LLM use significantly impact data-driven decision-making in teaching. Moreover, ethical considerations were found to significantly moderate the relationship between these knowledge domains and data-driven decision-making. This study contributes novel insights by addressing the evaluation and application of LLM effectiveness from teachers’ perspectives, ultimately fostering the advancement of quality education.