从压力到情感:跨可穿戴传感器模态的生理情绪识别多模态深度学习
英文标题:From Stress to Affect: Multimodal Deep Learning for Physiological Emotion Recognition Across Wearable Sensor Modalities
该研究比较了 LSTM、TCN 和 Transformer 三类时序深度学习架构在可穿戴设备生理情绪识别任务中的表现,使用 WESAD 和 EmoWear 两个多模态数据集,并采用受试者独立的留一受试者交叉验证。Transformer 在 WESAD 多模态配置下取得最高准确率 99.02% ± 0.51%,而 LSTM 在 EmoWear 上的唤醒度和效价识别中表现最佳,准确率分别为 91.80% ± 1.06% 和 89.96% ± 0.36%。研究发现,模型架构的相对优劣取决于数据集特性,并不存在始终占优的单一架构;多模态传感在两个数据集上均优于仅腕部或仅胸部传感。采样频率分析表明,4 Hz 能以显著更低的训练成本实现接近更高频率的性能,为开发者选择模型、传感器组合与采样频率提供了参考。
640 B
612 B
380 B
9.1 KB
32.7 KB
22.8 KB
24.2 KB
34.7 KB
中文
[提交于 2026 年 9 月 17 日]
标题:从压力到情感:跨可穿戴传感器模态的生理情绪识别多模态深度学习
查看 Desta Haileselassie Hagos 及另外 2 位作者撰写的论文《从压力到情感:跨可穿戴传感器模态的生理情绪识别多模态深度学习》PDF。
摘要:利用可穿戴传感器进行生理情绪识别,在心理健康监测、情感计算和人机交互领域具有重要应用价值。然而,现有研究通常仅评估单一模型、传感配置或数据集,这限制了我们对这些因素如何影响识别性能的理解。本文利用两个多模态可穿戴数据集 WESAD 和 EmoWear,对用于生理情绪识别的时序深度学习架构进行了比较研究。研究在仅腕部、仅胸部和多模态传感配置下,采用受试者独立的留一受试者交叉验证(LOSO-CV),评估双向长短期记忆网络(LSTM)、时序卷积网络(TCN)和 Transformer 模型。我们还研究了软投票集成、传感器消融、采样频率和基于梯度的显著性分析。Transformer 在 WESAD 的多模态配置下取得最高准确率(99.02% +/- 0.51%);而 LSTM 在 EmoWear 的多模态配置下,对唤醒度和效价均取得最佳准确率,分别为 91.80% +/- 1.06% 和 89.96% +/- 0.36%。这些结果表明,不同架构之间的相对性能取决于数据集特性,而非某一种架构始终更优。在两个数据集上,多模态传感均持续优于仅腕部和仅胸部配置。采样频率分析显示,4 Hz 是一个实用的运行点:其性能可与更高频率相当,同时显著降低训练成本。这些发现为可穿戴生理情绪识别中的架构、传感模态和采样频率选择提供了指导。
提交历史
来自:Desta Haileselassie Hagos [
]
[v1]
2026 年 9 月 17 日,周四,18:45:17 UTC(133 KB)
English / 原文
[Submitted on 17 Sep 2026]
Title:From Stress to Affect: Multimodal Deep Learning for Physiological Emotion Recognition Across Wearable Sensor Modalities
View a PDF of the paper titled From Stress to Affect: Multimodal Deep Learning for Physiological Emotion Recognition Across Wearable Sensor Modalities, by Desta Haileselassie Hagos and 2 other authors
Abstract:Physiological emotion recognition using wearable sensors has important applications in mental health monitoring, affective computing, and human-computer interaction. However, existing studies typically evaluate a single model, sensing configuration, or dataset, limiting our understanding of how these factors influence recognition performance. We present a comparative study of temporal deep learning architectures for physiological emotion recognition using two multimodal wearable datasets: WESAD and EmoWear. Bidirectional long short-term memory (LSTM), temporal convolutional network (TCN), and Transformer models are evaluated under wrist-only, chest-only, and multimodal sensing configurations using participant-independent leave-one-subject-out cross-validation (LOSO-CV). We also investigate soft-voting ensembles, sensor ablation, sampling frequency, and gradient-based saliency. The Transformer achieved the highest multimodal accuracy on WESAD (99.02% +/- 0.51%), whereas the LSTM achieved the best multimodal accuracy on EmoWear for both arousal (91.80% +/- 1.06%) and valence (89.96% +/- 0.36%). These results show that relative architecture performance depends on dataset characteristics rather than one architecture being uniformly superior. Multimodal sensing consistently outperformed wrist-only and chest-only configurations across both datasets. Sampling-frequency analysis showed that 4 Hz provides a practical operating point, with performance comparable to higher frequencies at substantially lower training cost. These findings provide guidance for selecting architectures, sensing modalities, and sampling frequencies for wearable physiological emotion recognition.
Submission history
From: Desta Haileselassie Hagos [
]
[v1]
Thu, 17 Sep 2026 18:45:17 UTC (133 KB)