Analysis of Mental Health Assessment Methods Based on Multimodal Signals
DOI:
https://doi.org/10.54097/e3cptf06Keywords:
Multimodal Architecture; Multimodal Signaling; Mental Health; Emotion Recognition.Abstract
According to the World Health Organization, there are more than 300 million people with depression in the world. And if this trend is not controlled, it is expected that depression will become the most serious and common disease in the world in 2030, and the mental health problems of college students are particularly serious. In order to effectively detect mental health, this review analyzes and summarizes each of the three commonly used multimodal architectures at this stage, compares their typical studies and experiments to explore the demand relationship between multimodal signals and architectures, explores the advantages and disadvantages of architecture algorithms, gives new algorithm optimization schemes, and gives reasonable suggestions for new application environments. The results show that the three multimodal architectures explored have higher accuracy than the single-modality. However, there are deficiencies in anti-interference, dataset collection, and scenario interaction, which can be optimized by dataset update and lotus effect algorithm.
Downloads
References
[1] Flores R, Tlachac M, Shrestha A and Rundensteiner E A, WavFace: A Multimodal Transformer-Based Model for Depression Screening. in IEEE Journal of Biomedical and Health Informatics, 2025, vol. 29, no. 5, pp. 3632-3641. DOI: https://doi.org/10.1109/JBHI.2025.3529348
[2] Dong Z, Ji X, Lai C S, Qi D, Zhou G and Lai L L, Memristor-Based Hierarchical Attention Network for Multimodal Affective Computing in Mental Health Monitoring, in IEEE Consumer Electronics Magazine, 2023, vol. 12, no. 4, pp. 94-106 DOI: https://doi.org/10.1109/MCE.2022.3159350
[3] Ishikawa, Y et al. Learning Cross-Modal Factors from Multimodal Physiological Signals for Emotion Recognition. PRICAI : Trends in Artificial Intelligence. 2023, vol. 14325. Singapore: Springer, 2023. 438–450. DOI: https://doi.org/10.1007/978-981-99-7019-3_40
[4] Jiang Z et al., HealthPrism: A Visual Analytics System for Exploring Children's Physical and Mental Health Profiles with Multimodal Data, in IEEE Transactions on Visualization and Computer Graphics, 2024, vol. 30, no. 1, pp. 1205-1215. DOI: https://doi.org/10.1109/TVCG.2023.3326943
[5] Mehmood, I, Li, H, Umer, W, Arsalan, A, Anwer, S, Mirza, MA, Ma, J, Antwi-Afari, M F, et al. Multimodal integration for data-driven classification of mental fatigue during construction equipment operations: Incorporating electroencephalography, electrodermal activity, Developments in The Built Environment 2023 vol.15 pp. 2666-1659. DOI: https://doi.org/10.1016/j.dibe.2023.100198
[6] Guo W. Zhao M. Alrashoud A. Tolba S. F and Xia F, Multimodal Educational Data Fusion for Students’ Mental Health Detection, in IEEE Access,2022 vol. 10, pp. 70370-70382 DOI: https://doi.org/10.1109/ACCESS.2022.3187502
[7] Joshi S, Jain K, Joshi U, Jain Y, Balpande S and Kulkarni R, MindMatrics: A Framework for Advanced Mental Health Diagnosis Using Multimodal Data Analysis, 2025 International Conference on Advanced Computing Technologies (ICoACT), Sivalasi, India, 2025, pp. 01-08 DOI: https://doi.org/10.1109/ICoACT63339.2025.11004973
[8] Titung R, Interactive Machine Learning for Multimodal Affective Computing, 2022 10th International Conference on Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW), Nara, Japan, 2022, pp. 1-4 DOI: https://doi.org/10.1109/ACIIW57231.2022.10085996
[9] Fusion K S et al, Net: A Multi-Modal Deep Learning Framework for Emotion Recognition and Mental Health Assessment, 2024 International Conference on Emerging Research in Computational Science (ICERCS), Coimbatore, India, 2024, pp. 1-5. DOI: https://doi.org/10.1109/ICERCS63125.2024.10895596
[10] Dalirinia E, Yaghoobi M, Tabatabaee H, et al. Multimodal Lotus Effect Algorithm for Engineering Optimization Problems. Engineering Reports, 2025, 7(4): e70137-e70137. DOI: https://doi.org/10.1002/eng2.70137
Downloads
Published
Issue
Section
License
Copyright (c) 2025 Highlights in Science, Engineering and Technology
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
