3S-TSE: Efficient Three-Stage Target Speaker Extraction for Real-Time and Low-Resource Applications

Shulin He1; Jinjiang Liu1; Hao Li2,3; Yang Yang1; Fei Chen2,3; Xueliang Zhang1

10.1109/ICASSP48485.2024.10448000

2024-04-19

1520-6149

979-8-3503-4486-8

ICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

14-19 April 2024

Seoul, Korea, Republic of

Target speaker extraction (TSE) aims to isolate a specific voice from multiple mixed speakers relying on a registerd sample. Since voiceprint features usually vary greatly, current end-to-end neural networks require large model parameters which are computational intensive and impractical for real-time applications, espetially on resource-constrained platforms. In this paper, we address the TSE task using microphone array and introduce a novel three-stage solution that systematically decouples the process: First, a neural network is trained to estimate the direction of the target speaker. Second, with the direction determined, the Generalized Sidelobe Canceller (GSC) is used to extract the target speech. Third, an Inplace Convolutional Recurrent Neural Network (ICRN) acts as a denoising post-processor, refining the GSC output to yield the final separated speech. Our approach delivers superior performance while drastically reducing computational load, setting a new standard for efficient real-time target speaker extraction.

其他

EI

1.College of Computer Science, Inner Mongolia University, China
2.Department of Electrical and Electronic Engineering, Southern University of Science and Technology, Shenzhen, China
3.Key Laboratory of Urban Land Resources Monitoring and Simulation, Ministry of Natural Resources, China

Shulin He,Jinjiang Liu,Hao Li,et al. 3S-TSE: Efficient Three-Stage Target Speaker Extraction for Real-Time and Low-Resource Applications[C],2024.