Voice, Text, and Large Language Models for Socio-Affective Analysis: Educational Implications and Ethical Considerations

Autori

  • Gianni Fenu University of Cagliari
  • Salvatore Carta
  • Alessandro Giuliani University of Cagliari
  • Alessandro Sebastian Podda
  • Marco Manolo Manca University of Cagliari
  • Livio Pompianu University of Cagliari

Parole chiave:

learning analytics, socio-affective analytics, speech analysis, large language models, affective computing

Abstract

The increasing use of artificial intelligence in learning analytics is expanding the range of behavioral, linguistic, and socio-affective signals that can be computationally analyzed. This paper investigates the methodological potential and limitations of voice-, text-, and Large Language Model (LLM)-based analysis by reframing an experimental framework originally developed for depression detection as a case study for socio-affective analytics in education. We conducted experiments on the DAIC-WOZ corpus using three analytical approaches: raw-speech modeling with an ECAPA-TDNN architecture and Non-Uniform Speaker Disentanglement; text-based prediction using Word2Vec and BERT representations with supervised regressors; and full-conversation assessment with DeepSeek under zero-shot and few-shot prompting. The best LLM configuration achieved an accuracy of 0.851 and an F1-score of 0.788, while the audio approach showed comparatively balanced performance, and conventional text representations were less consistent. These findings illustrate the different information captured by acoustic, lexical, contextual, and generative approaches. Rather than advocating automated psychological diagnosis in educational settings, we use the results to discuss construct validity, domain transfer, interpretability, bias, privacy, and human oversight, arguing that socio-affective learning analytics should treat computational outputs as uncertain, contextual indicators for human interpretation.

Riferimenti bibliografici

Alfredo, R., Echeverria, V., Jin, Y., Yan, L., Swiecki, Z., Gašević, D., & Martinez-Maldonado, R. (2024). Human-centred learning analytics and AI in education: A systematic literature review. Computers and Education: Artificial Intelligence, 6, 100215. DOI: 10.1016/j.caeai.2024.100215.

Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021). On the Dangers of Stochastic Par-rots: Can Language Models Be Too Big? FAccT ’21, 610–623. DOI: 10.1145/3442188.3445922.

Bergdahl, N., Bond, M., Sjöberg, J., Dougherty, M., & Oxley, E. (2024). Unpacking student engagement in higher education learning analytics: A systematic review. International Journal of Educational Tech-nology in Higher Education, 21, 63. DOI: 10.1186/s41239-024-00493-y.

Chancellor, S., & De Choudhury, M. (2020). Methods in predictive techniques for mental health status on so-cial media: A critical review. npj Digital Medicine, 3, 43. DOI: 10.1038/s41746-020-0233-7.

Dai, A., Shi, W., Gu, X., & Huang, L. (2026). Large Language Models for Depression Detection: A Review with Prospects of Incomplete Multimodality. Brain Sciences, 16(6), 593. DOI: 10.3390/brainsci16060593.

Donaghy, P., Ennis, E., Mulvenna, M., et al. (2024). A Review of Studies Using Machine Learning to Detect Voice Biomarkers for Depression. Journal of Technology in Behavioral Science. DOI: 10.1007/s41347-024-00454-2.

Fareed, M., Fatima, M., Uddin, J., Ahmed, A., & Sattar, M. A. (2025). A systematic review of ethical consid-erations of large language models in healthcare and medicine. Frontiers in Digital Health, 7, 1653631. DOI: 10.3389/fdgth.2025.1653631.

Fisher, H., Jaffe, N. M., Pidvirny, K., et al. (2026). Language-based detection of depression with machine learning: systematic review and meta-analysis. npj Digital Medicine, 9, 273. DOI: 10.1038/s41746-026-02448-1.

Gratch J, Artstein R, Lucas GM, Stratou G, Scherer S, Nazarian A, Wood R, Boberg J, DeVault D, Marsella S, Traum DR. The Distress Analysis Interview Corpus of Human and Computer Interviews. In Pro-ceedings of LREC 2014 May (pp. 3123-3128).

Ji, J., Dong, W., Li, J., et al. (2024). Depressive and mania mood state detection through voice as a bi-omarker using machine learning. Frontiers in Neurology, 15, 1394210. DOI: 10.3389/fneur.2024.1394210.

Khosravi, H., Buckingham Shum, S., Chen, G., et al. (2022). Explainable Artificial Intelligence in education. Computers and Education: Artificial Intelligence, 3, 100074. DOI: 10.1016/j.caeai.2022.100074.

Liu, L., Liu, L., Wafa, H. A., Tydeman, F., Xie, W., & Wang, Y. (2024). Diagnostic accuracy of deep learn-ing using speech samples in depression: A systematic review and meta-analysis. JAMIA, 31(10), 2394–2404. DOI: 10.1093/jamia/ocae189.

Liu, Q., & Khalil, M. (2023). Understanding privacy and data protection issues in learning analytics using a systematic review. British Journal of Educational Technology, 54, 1715–1747. DOI: 10.1111/bjet.13388.

Menne, F., Dörr, F., Schräder, J., et al. (2024). The voice of depression: speech features as biomarkers for major depressive disorder. BMC Psychiatry, 24, 794. DOI: 10.1186/s12888-024-06253-6.

Omar, M., & Levkovich, I. (2025). Exploring the efficacy and potential of large language models for depres-sion: A systematic review. Journal of Affective Disorders, 371, 234–244. DOI: 10.1016/j.jad.2024.11.052.

Ouhaichi, H., Spikol, D., & Vogel, B. (2023). Research trends in multimodal learning analytics: A systematic mapping study. Computers and Education: Artificial Intelligence, 4, 100136. DOI: 10.1016/j.caeai.2023.100136.

Prinsloo, P., Slade, S., & Khalil, M. (2023). Multimodal learning analytics - In-between student privacy and encroachment: A systematic review. British Journal of Educational Technology, 54(6). DOI: 10.1111/bjet.13373.

Ravi, V., Wang, J., Flint, J., & Alwan, A. (2024). Enhancing accuracy and privacy in speech-based depres-sion detection through speaker disentanglement. Computer Speech & Language, 86, 101605. DOI: 10.1016/j.csl.2023.101605.

Rets, I., Herodotou, C., & Gillespie, A. (2023). Six Practical Recommendations Enabling Ethical Use of Predictive Learning Analytics in Distance Education. Journal of Learning Analytics, 10(1), 149–167. DOI: 10.18608/jla.2023.7743.

Sailer, M., Ninaus, M., Huber, S. E., Bauer, E., & Greiff, S. (2024). The End is the Beginning is the End: The closed-loop learning analytics framework. Computers in Human Behavior, 158, 108305. DOI: 10.1016/j.chb.2024.108305.

Shin, D., Kim, H., Lee, S., Cho, Y., & Jung, W. (2024). Using Large Language Models to Detect Depression From User-Generated Diary Text Data as a Novel Approach in Digital Mental Health Screening. Journal of Medical Internet Research, 26, e54617. DOI: 10.2196/54617.

Wang, J., Ravi, V., & Alwan, A. (2023). Non-uniform Speaker Disentanglement For Depression Detection From Raw Speech Signals. Interspeech 2023, 2343–2347. DOI: 10.21437/Interspeech.2023-2101.

##submission.downloads##

Pubblicato

2026-11-19

Come citare

Fenu, G., Carta, S., Giuliani, A., Podda, A. S., Manca, M. M., & Pompianu, L. (2026). Voice, Text, and Large Language Models for Socio-Affective Analysis: Educational Implications and Ethical Considerations. Journal of Inclusive Methodology and Technology in Learning and Teaching, 6(3). Recuperato da https://www.inclusiveteaching.it/index.php/inclusiveteaching/article/view/804

Puoi leggere altri articoli dello stesso autore/i