Exploring Pre-Service Teachers’ AI Pedagogical Judgement in Assessment Design: A Document Analysis of Technology-Integrated Tasks
Keywords:
Artificial Intelligence (AI); Pre-Service Teachers (PSTs’); assessment design; ethics; contextual understanding; AI Pedagogical JudgementAbstract
The increasing integration of Artificial Intelligence (AI) in education has significant implications for assessment design and teacher preparation. This empirical study explores how Pre-Service Teachers (PSTs) engage with AI tools in the design of curriculum-aligned assessment tasks within a structured digital literacy module. Using qualitative document analysis of 63 assessment artefacts produced at a South African private higher education institution, the study examines how PSTs utilised AI-generated outputs, the extent of their critical and ethical reflection, and how their designs aligned with curriculum requirements. Thematic analysis, informed by the Technological Pedagogical Content Knowledge (TPACK) framework and the Substitution Augmentation Modification and Redefinition model, revealed three interrelated patterns: efficiency-oriented and creative use of AI, critical evaluation and adaptation of AI-generated content, and emerging ethical awareness regarding overreliance, bias, and contextual misalignment. While most PSTs used AI at substitution and augmentation levels, several PSTs demonstrated movement toward modification through interactive and digitally mediated assessments. Building on the findings, the study proposes the concept of AI Pedagogical Judgement to explain how PSTs negotiate generative efficiency, curriculum alignment practices, ethical reflexivity and professional agency when deciding whether to accept, adapt, or reject AI outputs. The study contributes to emerging scholarship on AI literacy in teacher education by showing that meaningful AI integration relies not only on access to technology, but on the quality of pedagogical judgement exercised in assessment design.
https://doi.org/10.26803/ijlter.25.6.26
References
Antonenko, P., & Abramowitz, B. (2023). In-service teachers’ (mis)conceptions of artificial intelligence in K-12 science education. Journal of Research on Technology in Education, 55(1), 64-78. https://doi.org/10.1080/15391523.2022.2119450
Ayyoub, A. M., Khlaif, Z. N., Shamali, M., Abu Eideh, B., Assali, A., Hattab, M. K., Barham, K. A., & Bsharat, T. R. K. (2025). Advancing higher education with GenAI: factors influencing educator AI literacy. Frontiers in Education, 10. https://doi.org/10.3389/feduc.2025.1530721
Bezuidenhout, R., Davis, C., & du Plooy-Cilliers, F. (2021). Research matters (2nd ed.). Juta.
Boateng, S., & Kalonde, G. (2024). Exploring the synergy of the SAMR (substitution, augmentation, modification, and redefinition) model and technology integration in education: The key to unlocking student engagement and motivation. Proceedings of The International Conference on Advanced Research in Education Teaching and Learning, 1(1), 37-46. https://doi.org/10.33422/aretl.v1i1.185
Bowen, G. A. (2009). Document analysis as a qualitative research method. Qualitative Research Journal, 9(2), 27-40. https://doi.org/10.3316/QRJ0902027
Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative research in psychology, 3(2), 77-101. https://doi.org/10.1191/1478088706qp063oa
Casal-Otero, L., Catala, A., Fernandez-Morante, C., Taboada, M., Cebreiro, B., & Barro, S. (2023). AI literacy in K-12: a systematic literature review. International Journal of STEM Education, 10(1). https://doi.org/10.1186/s40594-023-00418-7
Cope, B., Kalantzis, M., & Searsmith, D. (2020). Artificial intelligence for education: Knowledge and its assessment in AI-enabled learning ecologies. Educational Philosophy and Theory, 53(12), 1229-1245. https://doi.org/10.1080/00131857.2020.1728732
Guan, L., Zhang, Y., & Gu, M. M. (2025). Pre-service teachers preparedness for AI-integrated education: An investigation from perceptions, capabilities and teachers' identity changes. Computers and Education: Artificial Intelligence, 8(2). https://doi.org/10.1016/j.caeai.2024.100341
Harkin, B. (2024). Moving from Digital Tansformation to Digital Acceleration 1. In evolving from digital transformation to digital acceleration using the galapagos framework (pp. 1-306). Auerbach Publications. https://doi.org/10.1201/9781003404217
Holmes, W., & Littlejohn, A. (2024). Artificial intelligence for professional learning. In Handbook of artificial intelligence at work (pp. 191-211). https://doi.org/10.4337/9781800889972.00018
Holmes, W., Iniesto, F., Anastropoulou, S., & Boticario, J. G. (2023). Stakeholders perspectives on the ethics of AI in distance-based higher education. International Review of Research in Open and Distributed Learning, 24(2), 96-117. https://doi.org/10.19173/irrodl.v24i2.6089
Joseph, G. V., Athira, P., Thomas, M. A., Jose, D., Roy, T. V., & Prasad, M. (2024). Impact of digital literacy, use of AI tools and peer collaboration on AI assisted learning: Perceptions of the University Students. Digital Education Review, 45, 43-49. https://doi.org/10.1344/der.2024.45.43-49
Kaka, S., & Kessner, T. (2025). Bots and beginners: A comparative study of AI- and PST-generated social studies lesson plans. The Journal of Social Studies Research, 50(2), 98-114. https://doi.org/10.1177/0885985X251385910
Kelly, M. M., Martin-Peters, T., & Farber, J. S. (2024). Secondary data analysis: Using existing data to answer new questions. Journal of Pediatric Health CARE, 38(4), 615-618. https://doi.org/10.1016/j.pedhc.2024.03.005
Khlaif, Z. N., Alkouk, W. A., Salama, N., & Abu Eideh, B. (2025). Redesigning assessments for AI-enhanced learning: A framework for educators in the generative AI era. Education Sciences, 15(2), 174. https://doi.org/10.3390/educsci15020174
Khoehler, M., Mishra, P., Akcaoglu, M., & Rosenburg, J. (2006). Technological pedagogical content Knowledge: A framework for teacher knowledge. Teachers College Record: The Voice of Scholarship in Education, 108(6), 1017-1054. https://journals.sagepub.com/doi/10.1111/j.1467-9620.2006.00684.x
Kohnke, L., & Zou, D. (2026). Artificial intelligence integration in TESOL teacher education: Promoting a critical lens guided by TPACK and SAMR. TESOL Quarterly, 59, 267-278. https://doi.org/10.1002/tesq.3396
Lim, W. (2025). What is qualitative research? An overview and guidelines. Australasian Marketing Journal, 33(2), 199-229. http://doi.org/10.1177/14413582241264619
Lucas, M., Bem-haja, P., Zhang, Y., Llorente-Cejudo, C., & Palacios-Rodriguez, A. (2025). A comparative analysis of pre-service teachers' readiness for AI integration. Computers and Education: Artificial Intelligence, 8. https://doi.org/10.1016/j.caeai.2025.100396
Luckin, R., Cukurova, M., & du Boulay, B. (2022). Empowering educators to be AI-ready. Computers and Education: Artificial Intelligence, 3, 1-11. https://doi.org/10.1016/j.caeai.2022.100076
Ou, A. W., Ströhr, C., & Malmström, H. (2024). Academic communication with AI- powered language tools in higher education: From a post-humanist perspective. Systems, 121, 103225. https://doi.org/10.1016/j.system.2024.103225
Owan, V. J., Abang, B. K., Idika, O. D., Etta, O. E., & Bassey, A. B. (2023). Exploring the potential of artificial intelligence tools in educational measurement and assessment. EURASIA Journal of Mathematics, Science and Technology Education, 19(8), em2307. https://doi.org/10.29333/ejmste/13428
Rashid, A., & Kausik, M. K. (2024). AI revolutionizing industries worldwide: A comprehensive overview of its diverse applications. Hybrid Advances, 7, 1-34. https://doi.org/10.1016/j.hybadv.2024.100277
Rougeaux, R., & Sharp, B. (2023). Using artificial intelligence (AI) for lesson plan (LP) design in teacher education. ICERI2023 Proceedings, 967-974. https://doi.org/10.21125/iceri.2023.0324
Sa??n, F. G., Özkaya, A. B., Tengiz, F., Geyik, Ö. G., & Geyik, C. (2023). Current evaluation and recommendations for the use of artificial intelligence tools in education. Turkish Journal of Biochemistry, 48(6), 620-625. https://doi.org/10.1515/tjb-2023-0254
Sayed, B.T., Bani Younes, Z.B., Alkhayyat, A. (2024). To be with artificial intelligence in oral test or not to be a probe into the traces of success in speaking skill, psychological wellbeing, autonomy, and academic buoyancy. Language Testing in Asia, 14 (49). https://doi.org/10.1186/s40468-024-00321-0
Sun, J., Ma, H., Zeng, Y., Han, D., & Jin , Y. (2023). Promoting the AI teaching competency of K-12 computer science teachers: A TPACK-based professional development approach. Education and Information Technologies, 28(2), 1509-1533. https://doi.org/10.1007/s10639-022-11256-5
Tunjera, N., & Chigona, A. (2025). AI literacy in preservice teachers preparation programs: A global meta-analysis. European Conference of E-Learning. 24, pp. 363-369. Academic Conferences & Publishing International Ltd. https://doi.org/10.34190/ecel.24.1.4174
Wang, K., Ruan, Q., Zhang, X., Fu, C., & Duan, B. (2024). Pre-Service Teachers’ GenAI Anxiety, Technology Self-Efficacy, and TPACK: Their Structural Relations with Behavioral Intention to Design GenAI-Assisted Teaching. Behavioral Sciences, 14(5), 373. https://doi.org/10.3390/bs14050373
Williamson, B., & Eynon, R. (2020). Historical threads, missing links, and future directions in AI in education. Learning Media and Technology, 45(3), 223-235. https://doi.org/10.1080/17439884.2020.1798995
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Warda Abrahams
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
All articles published by IJLTER are licensed under a Creative Commons Attribution Non-Commercial No-Derivatives 4.0 International License (CCBY-NC-ND4.0).
