A psychometric validation of a creativity assessment instrument for pre-service teachers in using generative AI

Anggraeni Dian Permatasari, Janu Arlinwibowo, Virgiawan Listanto, Khofifa Najma Iftitah, Cahya Kusuma Ratih, Ilham Penta Priyadi

Abstract


The integration of generative artificial intelligence (GenAI) in education is accelerating rapidly, creating new demands for teachers to engage creatively with AI-supported pedagogical practices. This study aimed to develop and validate a measurement instrument for assessing the creativity of pre-service teachers in using GenAI. Content validity was established through expert judgment involving specialists in computer science, educational technology and assessment. Empirical data were collected using a performance-based assessment administered to pre-service teachers from ten universities across five provinces in Indonesia. Data analysis employed Aiken’s V for content validity, confirmatory factor analysis (CFA) for construct validity, and generalizability theory (G theory) for instrument reliability. The results indicated satisfactory content validity, with an Aiken’s V index of 0.78. CFA demonstrated the measurement model exhibited a good fit with the empirical data and all indicators contributed significantly to their respective construct dimensions (p<0.05). Reliability analysis showed score variance was primarily attributable to differences in students’ abilities (0.406), rather than item variance (0.064) or person–rater interaction (0.032). The relative (0.852) and absolute (0.843) G-coefficients exceeded recommended thresholds. Overall, 54.18% of participants demonstrated high to very high levels of creativity, confirming the instrument meets psychometric standards for measuring pre-service teachers’ creativity in using GenAI.

Keywords


Creativity; GenAI; Instrument quality; Pre-service teacher; Psychometric analysis

Full Text:

PDF


DOI: http://doi.org/10.11591/ijere.v15i6.39901

Refbacks

  • There are currently no refbacks.


Copyright (c) 2026 Janu Arlinwibowo, Virgiawan Listanto, Khofifa Najma Iftitah, Cahya Kusuma Ratih, Ilham Penta Priyadi

International Journal of Evaluation and Research in Education (IJERE)
p-ISSN: 2252-8822, e-ISSN: 2620-5440
The journal is published by Institute of Advanced Engineering and Science (IAES).

View IJERE Stats

Creative Commons License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.