Decoupled Multimodal Distilling for Emotion Recognition
Nanjing University of Information Science and Technology · Nanjing University of Science and Technology
Abstract
Human multimodal emotion recognition (MER) aims to perceive human emotions via language, visual and acoustic modalities. Despite the impressive performance of previous MER approaches, the inherent multimodal heterogeneities still haunt and the contribution of different modalities varies significantly. In this work, we mitigate this issue by proposing a decoupled multimodal distillation (DMD) approach that facilitates flexible and adaptive crossmodal knowledge distillation, aiming to enhance the discriminative features of each modality. Specially, the representation of each modality is decoupled into two parts, i.e., modality-irrelevant/-exclusive spaces, in a self-regression manner. DMD utilizes a graph…
Citation impact
- FWCI
- 41.97
- Percentile
- 100%
- References
- 47
Authors
3- YLYong LiCorresponding
Nanjing University of Information Science and Technology, Nanjing University of Science and Technology
- YWYuanzhi Wang
Nanjing University of Information Science and Technology, Nanjing University of Science and Technology
- ZCZhen Cui
Nanjing University of Science and Technology, Nanjing University of Information Science and Technology
Topics & keywords
- Computer science
- Modality (human–computer interaction)
- Crossmodal
- Modalities
- Visualization
- Artificial intelligence
- Graph
- Human–computer interaction
- Reduced inequalities