A Method To Map EEG Signals To Spoken Speech Using Gaussian Process Modeling

Wu，Hongde; Pan，Changjie; Li，Mingtao; Chen，Fei

10.23919/EUSIPCO54536.2021.9615959

2021

2219-5491

978-1-6654-0900-1

European Signal Processing Conference

2021-August

1140-1144

23-27 Aug. 2021

Dublin, Ireland

This work aimed to map electroencephalography (EEG) signals recorded during speech production to an intelligible speech. Experiments were designed to record EEG and spoken speech signals from normal participants. EEG features were processed with a Gaussian process regression method, and used to estimate multiple temporal amplitude envelopes of a spoken speech signal. The estimated envelopes were further applied to synthesize an intelligible speech signal by using a temporal envelope-based vocoder model. The performance of reconstructing the spoken speech signal was evaluated by the short-term objective intelligibility (STOI) index and the root mean square error (RMSE) between the reconstructed vocoded speech and the original spoken speech. Results showed a small RMSE between two sets of mel-frequency cepstral coefficients, and a STOI measurement up to 0.71. Both measures outperformed results from existing studies with similar tasks, indicating the potential in synthesizing an intelligible spoken speech with EEG signals in brain-computer interface based speech communication.

Brain computer interface Electroencephalography Speech synthesis Stochastic analysis

第一

英语

EI

20220411505686

Audio signal processing ; Biomedical signal processing ; Brain computer interface ; Electrophysiology ; Gaussian distribution ; Gaussian noise (electronic) ; Mean square error ; Regression analysis ; Speech communication ; Speech intelligibility ; Speech recognition ; Speech synthesis ; Stochastic systems

Biomedical Engineering:461.1 ; Medicine and Pharmacology:461.6 ; Information Theory and Signal Processing:716.1 ; Computer Peripheral Equipment:722.2 ; Control Systems:731.1 ; Speech:751.5 ; Probability Theory:922.1 ; Mathematical Statistics:922.2 ; Systems Science:961

2-s2.0-85123189451

Scopus

University Key Laboratory of Advanced Wireless Communications of Guangdong Province,Department of Electrical and Electronic Engineering,Southern University of Science and Technology,Shenzhen,518055,China

Wu，Hongde,Pan，Changjie,Li，Mingtao,et al. A Method To Map EEG Signals To Spoken Speech Using Gaussian Process Modeling[C],2021:1140-1144.