Video traffic analysis for real-time emotion recognition and visualization in online learning

Sassi, Ayoub, Jaafar, Wael, Cherif, Safa, Abderrazak, Jihene Ben et Yanikomeroglu, Halim. 2023. « Video traffic analysis for real-time emotion recognition and visualization in online learning ». IEEE Access, vol. 11. pp. 99376-99386.

Prévisualisation

PDF Jaafar-W-2023-27890.pdf - Version publiée Licence d'utilisation : Creative Commons CC BY. Télécharger (2MB) | Prévisualisation

Résumé

Since the outbreak of the COVID-19 crisis, the transition to remote education presented several challenges to educational institutions. Unlike face-to-face classes where educators can modify and keep track of the lessons and content according to the students’ observed emotions and participation, such activities are difficult to complete in online learning environments. To address this issue, we propose here a novel and comprehensive framework that leverages advanced computer vision and analysis techniques to detect students’ emotions during online learning and assess their state of mind regarding the taught content. Our framework is composed of three modules. The first module uses a novel lightweight machine learning method, called convolutional neural network-random forest (CNN-RF), to efficiently detect the students’ basic emotions, e.g., sad, happy, etc., during the online course. Our approach surpasses existing benchmarks in terms of accuracy (over 71%) on the FER-2013 dataset, while being less complex (i.e., using a smaller number of parameters). The second module consists of mapping the basic emotions to an education-aware state of mind, e.g., interest, boredom, distraction, etc. Unlike the few works that proposed simplistic mapping, we propose here a Plutchik wheel’s inspired mapping system, which is more precise and reflects better the relationship between combinations of basic emotions and the resulting education-aware state of mind. Thus, our understanding of the students’ cognitive and affective experiences during online learning can be enhanced. The third module is a visualization dashboard that offers clear and intuitive real-time representations of basic emotions and states of mind. This tool provides educators with invaluable insights into students’ emotional dynamics, enabling them to identify learning difficulties with high precision and make informed recommendations for improvements in course content and online teaching methods. In summary, the proposed framework presents a novel and powerful tool that addresses the challenges related to online learning. By accurately detecting the students’ emotions, assessing their states of mind, and providing real-time visualization, our approach represents a significant advancement toward the optimization of online education, which is critically needed in rural and remote areas of the globe.

Type de document:	Article publié dans une revue, révisé par les pairs
Professeur:	Professeur Jaafar, Waël
Affiliation:	Génie logiciel et des technologies de l'information
Date de dépôt:	03 oct. 2023 14:46
Dernière modification:	31 janv. 2024 20:19
URI:	https://espace2.etsmtl.ca/id/eprint/27890

Actions (Authentification requise)

Dernière vérification avant le dépôt