Decoding lip language using triboelectric sensors with deep learning
Tsinghua University · Chinese Academy of Sciences · +4 more institutions
Abstract
Lip language is an effective method of voice-off communication in daily life for people with vocal cord lesions and laryngeal and lingual injuries without occupying the hands. Collection and interpretation of lip language is challenging. Here, we propose the concept of a novel lip-language decoding system with self-powered, low-cost, contact and flexible triboelectric sensors and a well-trained dilated recurrent neural network model based on prototype learning. The structural principle and electrical properties of the flexible sensors are measured and analysed. Lip motions for selected vowels, words, phrases, silent speech and voice speech are collected and compared. The prototype learning model reaches a test…
Citation impact
- FWCI
- 32.98
- Percentile
- 100%
- References
- 48
Authors
8Topics & keywords
- Computer science
- Triboelectric effect
- Decoding methods
- Speech recognition
- Artificial intelligence
- Telecommunications