Clearer, not necessarily more accurate: generative artificial intelligence, exemplars, and revision of assessment rubrics in initial teacher education

Autori

  • Beatrice Doria Università Pegaso
  • Valentina Grion Pegaso Telematic University
  • Giorgia Slaviero https://orcid.org/0009-0000-9312-4683
  • Federica Picasso federica.picasso@unipegaso.it;
  • Anna Dipace

Parole chiave:

Rubriche di valutazione, Intelligenza Artificiale Generativa, Exemplars, Giudizio valutativo, Formazione degli insegnanti

Abstract

Constructing and revising an assessment rubric requires pre-service teachers to articulate a conception of quality, and revision may act either on the formulation of the descriptors or on the design of the underlying assessment framework. This study examines how pre-service primary teachers revised an analytic rubric with the support of exemplars and a generative artificial intelligence (GenAI) tool, within an authentic task. In a single-group pre–post design, two raters assessed 183 pairs of initial and final rubrics on an overall score and on nine criteria related to formulation (descriptor language) and design (rubric structure and assessment content); students indicated the resource they found most useful and rated the experience (n = 177). Overall quality improved in a widespread but moderate manner (Hodges–Lehmann estimate = 0.53 on a 1–10 scale; rrb = .78; dz = 0.57). Improvement varied across criteria (Time × Criterion interaction, η2p = .13): it was greatest for language clarity and moderate for structure and content, with structural precision and content validity remaining the weakest dimensions even after revision. Overall, 96.0% of students attributed a role to GenAI in the revision, whereas the perceived development of assessment competence was more modest and independent of the preferred resource. The findings suggest that revision supported by exemplars and GenAI acts more readily on the formulation of quality than on its design, and point to the need for task instructions that direct comparison towards the precision and validity of criteria.

Riferimenti bibliografici

Bearman, M., Tai, J., Dawson, P., Boud, D., & Ajjawi, R. (2024). Developing evaluative judgement for a time of generative artificial intelligence. Assessment & Evaluation in Higher Education. Advance online publication. https://doi.org/10.1080/02602938.2024.2335321

Boud, D., & Soler, R. (2016). Sustainable assessment revisited. Assessment & Evaluation in Higher Educa-tion, 41(3), 400–413. https://doi.org/10.1080/02602938.2015.1018133

Brookhart, S. M. (2013). How to create and use rubrics for formative assessment and grading. ASCD.

Carless, D., & Chan, K. K. H. (2017). Managing dialogic use of exemplars. Assessment & Evaluation in Higher Education, 42(6), 930–941. https://doi.org/10.1080/02602938.2016.1211246

Cooper, G., Ilich, K., & Sturrock, K. (2026). Using generative artificial intelligence to reimagine feedback in higher education: A collaborative autoethnography. Assessment & Evaluation in Higher Education, 1–15.

Dai, W., Tsai, Y.-S., Lin, J., Aldino, A., Jin, H., Li, T., . . . Chen, G. (2024). Assessing the proficiency of large language models in automatic feedback generation: An evaluation study. Computers and Educa-tion: Artificial Intelligence, 7, Article 100299.

Doria, B., Slaviero, G., Zaggia, C., & Grion, V. (2026). Generative artificial intelligence as evaluative scaffold-ing: Effects on students' perceptions and dispositions in an authentic learning task. The Turkish Online Journal of Educational Technology, 25(3), 11–29.

Fraile, J., Panadero, E., & Pardo, R. (2017). Co-creating rubrics: The effects on self-regulated learning, self-efficacy and performance of establishing assessment criteria with students. Studies in Educational Evaluation, 53, 69–76. https://doi.org/10.1016/j.stueduc.2017.03.003

Grion, V., & Serbati, A. (2019). Valutazione sostenibile e feedback nei contesti universitari [Sustainable assessment and feedback in university contexts]. Pensa MultiMedia.

Heinrich, T., Baily, S., Chen, K. W., DeOliveira, J., Park, S., & Wang, N. C. H. (2025). AI-assisted grading and personalized feedback in large political science classes: Results from randomized controlled trials. PLOS ONE, 20(8), Article e0328041.

Holm, S. (1979). A simple sequentially rejective multiple test procedure. Scandinavian Journal of Statistics, 6(2), 65–70.

The jamovi project. (2024). jamovi (Version 2.6) [Computer software]. https://www.jamovi.org

Kerby, D. S. (2014). The simple difference formula: An approach to teaching nonparametric correlation. Com-prehensive Psychology, 3. https://doi.org/10.2466/11.IT.3.1

Koo, T. K., & Li, M. Y. (2016). A guideline of selecting and reporting intraclass correlation coefficients for reliability research. Journal of Chiropractic Medicine, 15(2), 155–163. https://doi.org/10.1016/j.jcm.2016.02.012

Lakens, D. (2013). Calculating and reporting effect sizes to facilitate cumulative science: A practical primer for t-tests and ANOVAs. Frontiers in Psychology, 4, Article 863. https://doi.org/10.3389/fpsyg.2013.00863

Lenth, R. (2023). emmeans: Estimated marginal means, aka least-squares means [R package]. https://cran.r-project.org/package=emmeans

Lim, J. M., & Go, C. (2025). Integrating ChatGPT into teacher feedback: Practical insights for L2 writing in-struction. TESL-EJ, 29(3).

McGraw, K. O., & Wong, S. P. (1996). Forming inferences about some intraclass correlation coefficients. Psychological Methods, 1(1), 30–46. https://doi.org/10.1037/1082-989X.1.1.30

Mills, E., Mizouri, A., & Peach, A. (2025). Prompting better feedback: A study of custom GPT for formative assessment in undergraduate physics. Education Sciences, 15(8), Article 1058.

Nicol, D. (2021). The power of internal feedback: Exploiting natural comparison processes. Assessment & Evaluation in Higher Education, 46(5), 756–778.

Nicol, D., & McCallum, S. (2022). Making internal feedback explicit: Exploiting the multiple comparisons that occur during peer review. Assessment & Evaluation in Higher Education, 47(3), 424–443. https://doi.org/10.1080/02602938.2021.1924620

Pan, Y. (2025). Leveraging generative AI powered rubric-indexed feedback as a formative assessment strategy for enhancing medical English education. Discover Computing, 28(1), Article 284.

Panadero, E., & Jonsson, A. (2013). The use of scoring rubrics for formative assessment purposes revisited: A review. Educational Research Review, 9, 129–144.

Panadero, E., Tapia, J. A., & Huertas, J. A. (2012). Rubrics and self-assessment scripts effects on self-regulation, learning and self-efficacy in secondary education. Learning and Individual Differences, 22(6), 806–813.

Picasso, F., Agostini, D., & Serbati, A. (2025). Exploring the potential of AI for authentic assessment in edu-cation: Towards a new model of interaction. In Vilalta-Perdomo, Scroccaro, Salinas-Navarro, & Michel-Villarreal (Eds.), The Emerald handbook of active learning for authentic assessment. Emerald. https://doi.org/10.1108/9781837978571

R Core Team. (2024). R: A language and environment for statistical computing (Version 4.4) [Computer software]. https://cran.r-project.org

Sadler, D. R. (1989). Formative assessment and the design of instructional systems. Instructional Science, 18(2), 119–144.

Sadler, D. R. (2010). Beyond feedback: Developing student capability in complex appraisal. Assessment & Evaluation in Higher Education, 35(5), 535–550. https://doi.org/10.1080/02602930903541015

Singmann, H. (2023). afex: Analysis of factorial experiments [R package]. https://cran.r-project.org/package=afex

Stephenson, E., Morrigan, K., Irfan, A., Bascombe, K., & Okorie, M. (2026). Artificial intelligence-assisted feedback in pharmacology education: A pilot evaluation of a custom generative model. BMC Medical Education, 26(1), Article 1238.

Tai, J., Ajjawi, R., Boud, D., Dawson, P., & Panadero, E. (2018). Developing evaluative judgement: Enabling students to make decisions about the quality of work. Higher Education, 76(3), 467–481. https://doi.org/10.1007/s10734-017-0220-3

Taylor, B., Kisby, F., & Reedy, A. (2024). Rubrics in higher education: An exploration of undergraduate stu-dents' understanding and perspectives. Assessment & Evaluation in Higher Education, 49(6), 799–809. https://doi.org/10.1080/02602938.2023.2299330

Tomazin, L., Bentenuto, A., Perzolli, S., Venuti, P., Serbati, A., Picasso, F., & Lipnevich, A. A. (2026). Teacher feedback, rubrics, and annotated exemplars: Examining the effects on Italian psychology uni-versity students' performance on writing assignments. Assessment & Evaluation in Higher Education, 1–14.

Villarroel, V., Bloxham, S., Bruna, D., Bruna, C., & Herrera-Seda, C. (2018). Authentic assessment: Creating a blueprint for course design. Assessment & Evaluation in Higher Education, 43(5), 840–854.

Yang, J., & Niu, T. (2026). Exploring of the impact of AI feedback on college students' critical thinking: A in-tervention experiment based on design discipline. Frontiers in Psychology, 17, Article 1806913.

##submission.downloads##

Pubblicato

2026-11-19

Come citare

Doria, B., Grion, V., Slaviero, G., Picasso, F., & Dipace, A. (2026). Clearer, not necessarily more accurate: generative artificial intelligence, exemplars, and revision of assessment rubrics in initial teacher education. Journal of Inclusive Methodology and Technology in Learning and Teaching, 6(3). Recuperato da https://www.inclusiveteaching.it/index.php/inclusiveteaching/article/view/815

Puoi leggere altri articoli dello stesso autore/i

<< < 1 2