MISA
National University of Singapore · Singapore University of Technology and Design
Abstract
Multimodal Sentiment Analysis is an active area of research that leverages multimodal signals for affective understanding of user-generated videos. The predominant approach, addressing this task, has been to develop sophisticated fusion techniques. However, the heterogeneous nature of the signals creates distributional modality gaps that pose significant challenges. In this paper, we aim to learn effective modality representations to aid the process of fusion. We propose a novel framework, MISA, which projects each modality to two distinct subspaces. The first subspace is modality-invariant, where the representations across modalities learn their commonalities and reduce the modality gap. The second subspace…
Citation impact
- FWCI
- 54.85
- Percentile
- 100%
- References
- 46
Authors
3Topics & keywords
- Modality (human–computer interaction)
- Computer science
- Modalities
- Linear subspace
- Task (project management)
- Artificial intelligence
- Subspace topology
- Process (computing)