RAFT: Robust Adversarial Fusion Transformer for multimodal sentiment analysis

Multimodal sentiment analysis (MSA) has emerged as a key technology for understanding human emotions by jointly processing text, audio, and visual cues. Despite significant progress, existing fusion models remain vulnerable to real-world challenges such as modality noise, missing channels, and weak inter-modal coupling. This paper addresses these limitations by introducing RAFT (Robust Adversarial Fusion Transformer), which integrates cross-modal and self-attention mechanisms with noise-imitation adversarial training to strengthen feature interactions and resilience under imperfect inputs. We first formalize the problem of noisy and incomplete data in MSA and demonstrate how adversarial noise simulation can…

• NN
  National Natural Science Foundation of China

  Award: 61771299