Cross-Attention-Guided Wavenet for Mel Spectrogram Reconstruction in The ICASSP 2024 Auditory EEG Challenge

Yuan Fang1; Hao Li2,3; Xueliang Zhang1; Fei Chen2,3; Guanglai Gao1

10.1109/ICASSPW62465.2024.10627006

2024-04-19

979-8-3503-7452-0

2024 IEEE International Conference on Acoustics, Speech, and Signal Processing Workshops (ICASSPW)

14-19 April 2024

Seoul, Korea, Republic of

This paper provides an overview of our submission to Task 2 of the Auditory EEG Challenge at ICASSP 2024 Signal Processing Grand Challenge (SPGC). We introduce a novel approach, employing a cross-attention-guided WaveNet with a coarse-to-fine generation strategy, aimed at enhancing the detailed reconstruction of Mel spectrograms from time-domain EEG. Specifically, the model utilizes WaveNet to sequentially reconstruct the envelope, 10-band Mel, 80-band Mel, and magnitude from coarse to fine granular levels. To bridge the gap between different modalities, we introduce a cross-attention mechanism, exploring correlations across modalities. A combined loss function is employed to refine the reconstruction performance. Notably, we achieved Pearson correlation values of 0.0651 ± 0.0153 for the validation set and 0.0413 ± 0.0169 for the heldout-subjects test set, securing the second position in the competition. We release the training code for our model online1.

其他

EI

1.College of Computer Science, Inner Mongolia University, China
2.Department of Electrical and Electronic Engineering, Southern University of Science and Technology, Shenzhen, China
3.Key Laboratory of Urban Land Resources Monitoring and Simulation, Ministry of Natural Resources, China

Yuan Fang,Hao Li,Xueliang Zhang,et al. Cross-Attention-Guided Wavenet for Mel Spectrogram Reconstruction in The ICASSP 2024 Auditory EEG Challenge[C],2024.