Voice, Text, and Large Language Models for Socio-Affective Analysis: Educational Implications and Ethical Considerations
Parole chiave:
learning analytics, socio-affective analytics, speech analysis, large language models, affective computingAbstract
The increasing use of artificial intelligence in learning analytics is expanding the range of behavioral, linguistic, and socio-affective signals that can be computationally analyzed. This paper investigates the methodological potential and limitations of voice-, text-, and Large Language Model (LLM)-based analysis by reframing an experimental framework originally developed for depression detection as a case study for socio-affective analytics in education. We conducted experiments on the DAIC-WOZ corpus using three analytical approaches: raw-speech modeling with an ECAPA-TDNN architecture and Non-Uniform Speaker Disentanglement; text-based prediction using Word2Vec and BERT representations with supervised regressors; and full-conversation assessment with DeepSeek under zero-shot and few-shot prompting. The best LLM configuration achieved an accuracy of 0.851 and an F1-score of 0.788, while the audio approach showed comparatively balanced performance, and conventional text representations were less consistent. These findings illustrate the different information captured by acoustic, lexical, contextual, and generative approaches. Rather than advocating automated psychological diagnosis in educational settings, we use the results to discuss construct validity, domain transfer, interpretability, bias, privacy, and human oversight, arguing that socio-affective learning analytics should treat computational outputs as uncertain, contextual indicators for human interpretation.
Riferimenti bibliografici
Alfredo, R., Echeverria, V., Jin, Y., Yan, L., Swiecki, Z., Gašević, D., & Martinez-Maldonado, R. (2024). Human-centred learning analytics and AI in education: A systematic literature review. Computers and Education: Artificial Intelligence, 6, 100215. DOI: 10.1016/j.caeai.2024.100215.
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021). On the Dangers of Stochastic Par-rots: Can Language Models Be Too Big? FAccT ’21, 610–623. DOI: 10.1145/3442188.3445922.
Bergdahl, N., Bond, M., Sjöberg, J., Dougherty, M., & Oxley, E. (2024). Unpacking student engagement in higher education learning analytics: A systematic review. International Journal of Educational Tech-nology in Higher Education, 21, 63. DOI: 10.1186/s41239-024-00493-y.
Chancellor, S., & De Choudhury, M. (2020). Methods in predictive techniques for mental health status on so-cial media: A critical review. npj Digital Medicine, 3, 43. DOI: 10.1038/s41746-020-0233-7.
Dai, A., Shi, W., Gu, X., & Huang, L. (2026). Large Language Models for Depression Detection: A Review with Prospects of Incomplete Multimodality. Brain Sciences, 16(6), 593. DOI: 10.3390/brainsci16060593.
Donaghy, P., Ennis, E., Mulvenna, M., et al. (2024). A Review of Studies Using Machine Learning to Detect Voice Biomarkers for Depression. Journal of Technology in Behavioral Science. DOI: 10.1007/s41347-024-00454-2.
Fareed, M., Fatima, M., Uddin, J., Ahmed, A., & Sattar, M. A. (2025). A systematic review of ethical consid-erations of large language models in healthcare and medicine. Frontiers in Digital Health, 7, 1653631. DOI: 10.3389/fdgth.2025.1653631.
Fisher, H., Jaffe, N. M., Pidvirny, K., et al. (2026). Language-based detection of depression with machine learning: systematic review and meta-analysis. npj Digital Medicine, 9, 273. DOI: 10.1038/s41746-026-02448-1.
Gratch J, Artstein R, Lucas GM, Stratou G, Scherer S, Nazarian A, Wood R, Boberg J, DeVault D, Marsella S, Traum DR. The Distress Analysis Interview Corpus of Human and Computer Interviews. In Pro-ceedings of LREC 2014 May (pp. 3123-3128).
Ji, J., Dong, W., Li, J., et al. (2024). Depressive and mania mood state detection through voice as a bi-omarker using machine learning. Frontiers in Neurology, 15, 1394210. DOI: 10.3389/fneur.2024.1394210.
Khosravi, H., Buckingham Shum, S., Chen, G., et al. (2022). Explainable Artificial Intelligence in education. Computers and Education: Artificial Intelligence, 3, 100074. DOI: 10.1016/j.caeai.2022.100074.
Liu, L., Liu, L., Wafa, H. A., Tydeman, F., Xie, W., & Wang, Y. (2024). Diagnostic accuracy of deep learn-ing using speech samples in depression: A systematic review and meta-analysis. JAMIA, 31(10), 2394–2404. DOI: 10.1093/jamia/ocae189.
Liu, Q., & Khalil, M. (2023). Understanding privacy and data protection issues in learning analytics using a systematic review. British Journal of Educational Technology, 54, 1715–1747. DOI: 10.1111/bjet.13388.
Menne, F., Dörr, F., Schräder, J., et al. (2024). The voice of depression: speech features as biomarkers for major depressive disorder. BMC Psychiatry, 24, 794. DOI: 10.1186/s12888-024-06253-6.
Omar, M., & Levkovich, I. (2025). Exploring the efficacy and potential of large language models for depres-sion: A systematic review. Journal of Affective Disorders, 371, 234–244. DOI: 10.1016/j.jad.2024.11.052.
Ouhaichi, H., Spikol, D., & Vogel, B. (2023). Research trends in multimodal learning analytics: A systematic mapping study. Computers and Education: Artificial Intelligence, 4, 100136. DOI: 10.1016/j.caeai.2023.100136.
Prinsloo, P., Slade, S., & Khalil, M. (2023). Multimodal learning analytics - In-between student privacy and encroachment: A systematic review. British Journal of Educational Technology, 54(6). DOI: 10.1111/bjet.13373.
Ravi, V., Wang, J., Flint, J., & Alwan, A. (2024). Enhancing accuracy and privacy in speech-based depres-sion detection through speaker disentanglement. Computer Speech & Language, 86, 101605. DOI: 10.1016/j.csl.2023.101605.
Rets, I., Herodotou, C., & Gillespie, A. (2023). Six Practical Recommendations Enabling Ethical Use of Predictive Learning Analytics in Distance Education. Journal of Learning Analytics, 10(1), 149–167. DOI: 10.18608/jla.2023.7743.
Sailer, M., Ninaus, M., Huber, S. E., Bauer, E., & Greiff, S. (2024). The End is the Beginning is the End: The closed-loop learning analytics framework. Computers in Human Behavior, 158, 108305. DOI: 10.1016/j.chb.2024.108305.
Shin, D., Kim, H., Lee, S., Cho, Y., & Jung, W. (2024). Using Large Language Models to Detect Depression From User-Generated Diary Text Data as a Novel Approach in Digital Mental Health Screening. Journal of Medical Internet Research, 26, e54617. DOI: 10.2196/54617.
Wang, J., Ravi, V., & Alwan, A. (2023). Non-uniform Speaker Disentanglement For Depression Detection From Raw Speech Signals. Interspeech 2023, 2343–2347. DOI: 10.21437/Interspeech.2023-2101.
##submission.downloads##
Pubblicato
Come citare
Fascicolo
Sezione
Licenza
Copyright (c) 2026 Gianni Fenu, Salvatore Carta, Alessandro Giuliani, Alessandro Sebastian Podda, Marco Manolo Manca, Livio Pompianu

Questo lavoro è fornito con la licenza Creative Commons Attribuzione - Non commerciale - Non opere derivate 4.0 Internazionale.