Mixture to Mixture: Leveraging Close-Talk Mixtures as Weak-Supervision for Speech Separation

Wang, Zhong-Qiu

2024

10.1109/LSP.2024.3417284

IEEE SIGNAL PROCESSING LETTERS   影响因子和分区

1070-9908

1558-2361

We propose mixture to mixture (M2M) training, a weakly-supervised neural speech separation algorithm that leverages close-talk mixtures as a weak supervision for training discriminative models to separate far-field mixtures. Our idea is that, for a target speaker, its close-talk mixture has a much higher signal-to-noise ratio (SNR) of the target speaker than any far-field mixtures, and hence could be utilized to design a weak supervision for separation. To realize this, at each training step we feed a far-field mixture to a deep neural network (DNN) to produce an intermediate estimate for each speaker, and, for each of considered close-talk and far-field microphones, we linearly filter the DNN estimates and optimize a loss so that the filtered estimates of all the speakers can sum up to the mixture captured by each of the considered microphones. Evaluation results on a 2-speaker separation task in simulated reverberant conditions show that M2M can effectively leverage close-talk mixtures as a weak supervision for separating far-field mixtures.

Microphones Machine-to-machine communications Training Signal processing algorithms Microwave integrated circuits Signal to noise ratio Indexes Weakly-supervised neural speech separation

SCI ; EI

英语

第一 ; 通讯

Engineering

Engineering, Electrical & Electronic

WOS:001262728700003

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC

ENGINEERING

Web of Science

Southern Univ Sci & Technol, Dept Comp Sci & Engn, Shenzhen 518055, Peoples R China

Wang, Zhong-Qiu. Mixture to Mixture: Leveraging Close-Talk Mixtures as Weak-Supervision for Speech Separation[J]. IEEE SIGNAL PROCESSING LETTERS,2024,31.

Wang, Zhong-Qiu.(2024).Mixture to Mixture: Leveraging Close-Talk Mixtures as Weak-Supervision for Speech Separation.IEEE SIGNAL PROCESSING LETTERS,31.

Wang, Zhong-Qiu."Mixture to Mixture: Leveraging Close-Talk Mixtures as Weak-Supervision for Speech Separation".IEEE SIGNAL PROCESSING LETTERS 31(2024).