Development and validation of a scale measuring students’ use of generative artificial intelligence tools

Abstract

The emergence of artificial intelligence (AI) in education, such as students’ use of generative artificial intelligence (GAI) tools in academic work, has profoundly transformed the learning ecosystem, offering both promising opportunities and potential challenges. Considering that such tools are still a developing area of study in education, this paper aimed to develop a scale that can assess and describe students’ practices and perspectives towards using GAI tools. Through an exploratory-sequential mixed methods design, an interview of 20 higher education students and a scoping literature review were used to generate scale items in the first phase of the study. In the second phase of the study, two pilot tests of the scale participated by 793 students were implemented. Exploratory and confirmatory factor analysis (CFA), internal consistency, and convergent validity tests were also carried out. The developed scale consists of 26 statements covering the following factors: i) research tool; ii) communication tool; iii) reliance; and iv) ethical use. The CFA model confirmed these factors, and all fit indices show that the overall structure of the scale has acceptable to good fit. The scale’s psychometric properties reveal that it is valid and reliable. This scale development study implicates schools to use a structured way of assessing how students engage with GAI tools in academic settings and rethink ways to support students’ learning through the responsible use of AI tools.