Clearer, not necessarily more accurate: generative artificial intelligence, exemplars, and revision of assessment rubrics in initial teacher education
Parole chiave:
Rubriche di valutazione, Intelligenza Artificiale Generativa, Exemplars, Giudizio valutativo, Formazione degli insegnantiAbstract
Constructing and revising an assessment rubric requires pre-service teachers to articulate a conception of quality, and revision may act either on the formulation of the descriptors or on the design of the underlying assessment framework. This study examines how pre-service primary teachers revised an analytic rubric with the support of exemplars and a generative artificial intelligence (GenAI) tool, within an authentic task. In a single-group pre–post design, two raters assessed 183 pairs of initial and final rubrics on an overall score and on nine criteria related to formulation (descriptor language) and design (rubric structure and assessment content); students indicated the resource they found most useful and rated the experience (n = 177). Overall quality improved in a widespread but moderate manner (Hodges–Lehmann estimate = 0.53 on a 1–10 scale; rrb = .78; dz = 0.57). Improvement varied across criteria (Time × Criterion interaction, η2p = .13): it was greatest for language clarity and moderate for structure and content, with structural precision and content validity remaining the weakest dimensions even after revision. Overall, 96.0% of students attributed a role to GenAI in the revision, whereas the perceived development of assessment competence was more modest and independent of the preferred resource. The findings suggest that revision supported by exemplars and GenAI acts more readily on the formulation of quality than on its design, and point to the need for task instructions that direct comparison towards the precision and validity of criteria.
Riferimenti bibliografici
Bearman, M., Tai, J., Dawson, P., Boud, D., & Ajjawi, R. (2024). Developing evaluative judgement for a time of generative artificial intelligence. Assessment & Evaluation in Higher Education. Advance online publication. https://doi.org/10.1080/02602938.2024.2335321
Boud, D., & Soler, R. (2016). Sustainable assessment revisited. Assessment & Evaluation in Higher Educa-tion, 41(3), 400–413. https://doi.org/10.1080/02602938.2015.1018133
Brookhart, S. M. (2013). How to create and use rubrics for formative assessment and grading. ASCD.
Carless, D., & Chan, K. K. H. (2017). Managing dialogic use of exemplars. Assessment & Evaluation in Higher Education, 42(6), 930–941. https://doi.org/10.1080/02602938.2016.1211246
Cooper, G., Ilich, K., & Sturrock, K. (2026). Using generative artificial intelligence to reimagine feedback in higher education: A collaborative autoethnography. Assessment & Evaluation in Higher Education, 1–15.
Dai, W., Tsai, Y.-S., Lin, J., Aldino, A., Jin, H., Li, T., . . . Chen, G. (2024). Assessing the proficiency of large language models in automatic feedback generation: An evaluation study. Computers and Educa-tion: Artificial Intelligence, 7, Article 100299.
Doria, B., Slaviero, G., Zaggia, C., & Grion, V. (2026). Generative artificial intelligence as evaluative scaffold-ing: Effects on students' perceptions and dispositions in an authentic learning task. The Turkish Online Journal of Educational Technology, 25(3), 11–29.
Fraile, J., Panadero, E., & Pardo, R. (2017). Co-creating rubrics: The effects on self-regulated learning, self-efficacy and performance of establishing assessment criteria with students. Studies in Educational Evaluation, 53, 69–76. https://doi.org/10.1016/j.stueduc.2017.03.003
Grion, V., & Serbati, A. (2019). Valutazione sostenibile e feedback nei contesti universitari [Sustainable assessment and feedback in university contexts]. Pensa MultiMedia.
Heinrich, T., Baily, S., Chen, K. W., DeOliveira, J., Park, S., & Wang, N. C. H. (2025). AI-assisted grading and personalized feedback in large political science classes: Results from randomized controlled trials. PLOS ONE, 20(8), Article e0328041.
Holm, S. (1979). A simple sequentially rejective multiple test procedure. Scandinavian Journal of Statistics, 6(2), 65–70.
The jamovi project. (2024). jamovi (Version 2.6) [Computer software]. https://www.jamovi.org
Kerby, D. S. (2014). The simple difference formula: An approach to teaching nonparametric correlation. Com-prehensive Psychology, 3. https://doi.org/10.2466/11.IT.3.1
Koo, T. K., & Li, M. Y. (2016). A guideline of selecting and reporting intraclass correlation coefficients for reliability research. Journal of Chiropractic Medicine, 15(2), 155–163. https://doi.org/10.1016/j.jcm.2016.02.012
Lakens, D. (2013). Calculating and reporting effect sizes to facilitate cumulative science: A practical primer for t-tests and ANOVAs. Frontiers in Psychology, 4, Article 863. https://doi.org/10.3389/fpsyg.2013.00863
Lenth, R. (2023). emmeans: Estimated marginal means, aka least-squares means [R package]. https://cran.r-project.org/package=emmeans
Lim, J. M., & Go, C. (2025). Integrating ChatGPT into teacher feedback: Practical insights for L2 writing in-struction. TESL-EJ, 29(3).
McGraw, K. O., & Wong, S. P. (1996). Forming inferences about some intraclass correlation coefficients. Psychological Methods, 1(1), 30–46. https://doi.org/10.1037/1082-989X.1.1.30
Mills, E., Mizouri, A., & Peach, A. (2025). Prompting better feedback: A study of custom GPT for formative assessment in undergraduate physics. Education Sciences, 15(8), Article 1058.
Nicol, D. (2021). The power of internal feedback: Exploiting natural comparison processes. Assessment & Evaluation in Higher Education, 46(5), 756–778.
Nicol, D., & McCallum, S. (2022). Making internal feedback explicit: Exploiting the multiple comparisons that occur during peer review. Assessment & Evaluation in Higher Education, 47(3), 424–443. https://doi.org/10.1080/02602938.2021.1924620
Pan, Y. (2025). Leveraging generative AI powered rubric-indexed feedback as a formative assessment strategy for enhancing medical English education. Discover Computing, 28(1), Article 284.
Panadero, E., & Jonsson, A. (2013). The use of scoring rubrics for formative assessment purposes revisited: A review. Educational Research Review, 9, 129–144.
Panadero, E., Tapia, J. A., & Huertas, J. A. (2012). Rubrics and self-assessment scripts effects on self-regulation, learning and self-efficacy in secondary education. Learning and Individual Differences, 22(6), 806–813.
Picasso, F., Agostini, D., & Serbati, A. (2025). Exploring the potential of AI for authentic assessment in edu-cation: Towards a new model of interaction. In Vilalta-Perdomo, Scroccaro, Salinas-Navarro, & Michel-Villarreal (Eds.), The Emerald handbook of active learning for authentic assessment. Emerald. https://doi.org/10.1108/9781837978571
R Core Team. (2024). R: A language and environment for statistical computing (Version 4.4) [Computer software]. https://cran.r-project.org
Sadler, D. R. (1989). Formative assessment and the design of instructional systems. Instructional Science, 18(2), 119–144.
Sadler, D. R. (2010). Beyond feedback: Developing student capability in complex appraisal. Assessment & Evaluation in Higher Education, 35(5), 535–550. https://doi.org/10.1080/02602930903541015
Singmann, H. (2023). afex: Analysis of factorial experiments [R package]. https://cran.r-project.org/package=afex
Stephenson, E., Morrigan, K., Irfan, A., Bascombe, K., & Okorie, M. (2026). Artificial intelligence-assisted feedback in pharmacology education: A pilot evaluation of a custom generative model. BMC Medical Education, 26(1), Article 1238.
Tai, J., Ajjawi, R., Boud, D., Dawson, P., & Panadero, E. (2018). Developing evaluative judgement: Enabling students to make decisions about the quality of work. Higher Education, 76(3), 467–481. https://doi.org/10.1007/s10734-017-0220-3
Taylor, B., Kisby, F., & Reedy, A. (2024). Rubrics in higher education: An exploration of undergraduate stu-dents' understanding and perspectives. Assessment & Evaluation in Higher Education, 49(6), 799–809. https://doi.org/10.1080/02602938.2023.2299330
Tomazin, L., Bentenuto, A., Perzolli, S., Venuti, P., Serbati, A., Picasso, F., & Lipnevich, A. A. (2026). Teacher feedback, rubrics, and annotated exemplars: Examining the effects on Italian psychology uni-versity students' performance on writing assignments. Assessment & Evaluation in Higher Education, 1–14.
Villarroel, V., Bloxham, S., Bruna, D., Bruna, C., & Herrera-Seda, C. (2018). Authentic assessment: Creating a blueprint for course design. Assessment & Evaluation in Higher Education, 43(5), 840–854.
Yang, J., & Niu, T. (2026). Exploring of the impact of AI feedback on college students' critical thinking: A in-tervention experiment based on design discipline. Frontiers in Psychology, 17, Article 1806913.
##submission.downloads##
Pubblicato
Come citare
Fascicolo
Sezione
Licenza
Copyright (c) 2026 Beatrice Doria, Valentina Grion, Giorgia Slaviero, Federica Picasso, Anna Dipace

Questo lavoro è fornito con la licenza Creative Commons Attribuzione - Non commerciale - Non opere derivate 4.0 Internazionale.