A Deep Learning Based Approach to Synthesize Intelligible Speech with Limited Temporal Envelope Information

Ching-Ju Hsiao1; Fei Chen2; Ji-Yan Han1; Wei-Zhong Zheng1; Ying-Hui Lai3

10.1109/EMBC48229.2022.9871247

2022

2375-7477

978-1-7281-2783-5

2022 44th Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC)

1972-1976

11-15 July 2022

Glasgow, Scotland, United Kingdom

Envelope waveforms can be extracted from multiple frequency bands of a speech signal, and envelope waveforms carry important intelligibility information for human speech communication. This study aimed to investigate whether a deep learning-based model with features of temporal envelope information could synthesize an intelligible speech, and to study the effect of reducing the number (from 8 to 2 in this work) of temporal envelope information on the intelligibility of the synthesized speech. The objective evaluation metric of short-time objective intelligibility (STOI) showed that, on average, the synthesized speech of the proposed approach provided higher STOI (i.e., 0.8) scores in each test condition; and the human listening test showed that the average word correct rate of eight listeners was higher than 97.5%. These findings indicated that the proposed deep learning-based system can be a potential approach to synthesize a highly intelligible speech with limited envelope information in the future.

Measurement Deep learning Medical devices Biological system modeling Oral communication Logic gates Hearing aids

其他

IEEE

1.Department of Biomedical Engineering, National Yang Ming Chiao Tung University, Taipei, Taiwan
2.Department of Electrical and Electronic Engineering, Southern University of Science and Technology, Shenzhen, China
3.Department of Biomedical Engineering and Medical Device Innovation & Translation Center, National Yang Ming Chiao Tung University, Taipei, Taiwan

Ching-Ju Hsiao,Fei Chen,Ji-Yan Han,et al. A Deep Learning Based Approach to Synthesize Intelligible Speech with Limited Temporal Envelope Information[C],2022:1972-1976.