A Spiking LSTM Accelerator for Automatic Speech Recognition Application Based on FPGA

Yin, Tingting1,2; Dong, Feihong1,2; Chen, Chao2; Ouyang, Chenghao3; Wang, Zheng2; Yang, Yongkui2

2024-03-01

10.3390/electronics13050827

ELECTRONICS   影响因子和分区

2079-9292

Long Short-Term Memory (LSTM) finds extensive application in sequential learning tasks, notably in speech recognition. However, existing accelerators tailored for traditional LSTM networks grapple with high power consumption, primarily due to the intensive matrix-vector multiplication operations inherent to LSTM networks. In contrast, the spiking LSTM network has been designed to avoid these multiplication operations by replacing multiplication and nonlinear functions with addition and comparison. In this paper, we present an FPGA-based accelerator specifically designed for spiking LSTM networks. Firstly, we employ a low-cost circuit in the LSTM gate to significantly reduce power consumption and hardware cost. Secondly, we propose a serial-parallel processing architecture along with hardware implementation to reduce inference latency. Thirdly, we quantize and efficiently deploy the synapses of the spiking LSTM network. The power consumption of the accelerator implemented on Artix-7 and Zynq-7000 is only about 1.1 W and 0.84 W, respectively, when performing the inference for speech recognition with the Free Spoken Digit Dataset (FSDD). Additionally, the energy consumed per inference is remarkably efficient, with values of 87 mu J and 66 mu J, respectively. In comparison with dedicated accelerators designed for traditional LSTM networks, our spiking LSTM accelerator achieves a remarkable reduction in power consumption, amounting to orders of magnitude.

spiking LSTM spiking neural networks hardware acceleration FPGA automatic speech recognition (ASR)

SCI

英语

第一

Computer Science ; Engineering ; Physics

Computer Science, Information Systems ; Engineering, Electrical & Electronic ; Physics, Applied

WOS:001182667100001

MDPI

Web of Science

1.Southern Univ Sci & Technol, Coll Engn, Shenzhen 518055, Peoples R China
2.Chinese Acad Sci, Shenzhen Inst Adv Technol, Shenzhen 518055, Peoples R China
3.Huazhong Univ Sci & Technol, Sch Comp Sci & Technol, Wuhan 430074, Peoples R China

Yin, Tingting,Dong, Feihong,Chen, Chao,et al. A Spiking LSTM Accelerator for Automatic Speech Recognition Application Based on FPGA[J]. ELECTRONICS,2024,13(5).

Yin, Tingting,Dong, Feihong,Chen, Chao,Ouyang, Chenghao,Wang, Zheng,&Yang, Yongkui.(2024).A Spiking LSTM Accelerator for Automatic Speech Recognition Application Based on FPGA.ELECTRONICS,13(5).

Yin, Tingting,et al."A Spiking LSTM Accelerator for Automatic Speech Recognition Application Based on FPGA".ELECTRONICS 13.5(2024).