Continuous Facial Emotion Analysis Based Stress Detection and Personalized Multimedia Recommendation System For Stress Management

Abstract

Problem: Mental stress significantly affects physical and mental health, leading to decreased productivity and long-term health issues. Traditional detection methods, such as physiological monitoring and self-reporting, are invasive, inconsistent, and impractical for continuous real-time monitoring. Most current machine learning systems for stress detection and content recommendation rely on single-point emotion assessments, lacking real-time adaptability and often overlooking gender biases, which can reduce accuracy and inclusivity. Solution: This project introduces a non-invasive, continuous stress detection system using facial emotion recognition through Convolutional Neural Networks (CNNs). The system detects stress-related emotions in real-time and recommends personalized multimedia content based on detected stress levels. Gender-specific and mixed datasets were used to enhance model accuracy, with MobileNetV2 and transfer learning incorporated for efficient emotion recognition. The system operates with minimal user input, providing a seamless experience. Results: The prototype achieved satisfactory results in detecting stress across the CK+ female dataset, achieving 100% accuracy, and 92.3% accuracy on the CK+ male dataset. The precision and recall for the female model were both 1, resulting in an F1 score of 1.0, indicating balance between precision and recall. The male model performed slightly lower, with an F1 score of 0.9333, reflecting performance drops. FER2013 common model showed lower performance compared to CK+ models, achieving 74.29% accuracy and 0.7429 precision and recall, with an F1 score of 0.7200. Despite these challenges, the real-time multimedia recommendation system successfully offered user preferred content based on detected stress state highlighting the system’s potential in personalized stress management. Further work is required to enhance generalizability across diverse environments and populations.