Audio-visual speech recognition using deep learning

Audio-visual speech recognition (AVSR) system is thought to be one of the most promising solutions for reliable speech recognition, particularly when the audio is corrupted by noise. However, cautious selection of sensory features is crucial for attaining high recognition performance. In the machine-learning community, deep learning approaches have recently attracted increasing attention because deep neural networks can effectively extract robust latent features that enable various recognition algorithms to demonstrate revolutionary generalization capabilities under diverse application conditions. This study introduces a connectionist-hidden Markov model (HMM) system for noise-robust AVSR. First, a deep…

• MO
  Ministry of Education, Culture, Sports, Science and Technology

  Award: 24119003
• JS
  Japan Society for the Promotion of Science

  Awards: 265114, 24220006
• PR
  Precursory Research for Embryonic Science and Technology