VSPIM: SRAM Processing-in-Memory DNN Acceleration via Vector-Scalar Operations

Chen Nie1; Chenyu Tang1; Jie Lin2; Huan Hu3; Chenyang Lv1; Ting Cao4; Weifeng Zhang5; Li Jiang1; Xiaoyao Liang1; Weikang Qian6; Yanan Sun7; Zhezhi He1

2024-10-01

10.1109/TC.2023.3285095

IEEE Transactions on Computers   影响因子和分区

2326-3814

Processing-in-Memory (PIM) has been widely explored for accelerating data-intensive machine learning computation that mainly consists of general-matrix-multiplication (GEMM), by mitigating the burden of data movements and exploiting the ultra-high memory parallelism. The two mainstreams of PIM, the analog- and digital-type, have both been exploited in accelerating machine learning workloads by numerous outstanding prior works. Currently, the digital-PIM is increasingly favored due to the broader computing support and the avoidance of errors caused by intrinsic non-idealities, e.g., process variation. Nevertheless, it still lacks further optimization considering the characteristics of the GEMM computation, including better efficient data layout and scheduling, and the ability to handle the sparsity of activations at the bit-level. To boost the performance and efficiency of digital SRAM PIM, we propose the architecture called VSPIM that performs the computation in a bit-serial fashion, with unique support of vector-scalar computing pattern. The novelties of the VSPIM can be concluded as follows: 1) support bit-serial based scalar-vector computing via ingenious parallel bit-broadcasting; 2) refine the GEMM mapping strategy and computing pattern to enhance performance and efficiency; 3) powered by the introduced scalar-vector operation, the bit-sparsity of activation is leveraged to halt unnecessary computation to maximize efficiency and throughput. Our comprehensive evaluation shows that, compared to the state-of-the-art SRAM-based digital-PIM design (Neural Cache), VSPIM can significantly boost the performance and energy efficiency by up to $8.87\times$8.87× and $4.81\times$4.81× respectively, with negligible area overhead, upon multiple representative neural networks.

其他

1.Department of Computer Science and Engineering, Shanghai Jiao Tong University, Shanghai, China
2.University of Central Florida, Orlando, FL, USA
3.School of Microelectronics, Southern University of Science and Technology, Shenzhen, China
4.Microsoft Research Asia, Beijing, China
5.Alibaba Group US Inc, San Diego, CA, USA
6.University of Michigan-Shanghai Jiao Tong University Joint Institute, Ann Arbor, MI, USA
7.Department of Micro-Nano Electronics, Shanghai Jiao Tong University, Shanghai, China

Chen Nie,Chenyu Tang,Jie Lin,et al. VSPIM: SRAM Processing-in-Memory DNN Acceleration via Vector-Scalar Operations[J]. IEEE Transactions on Computers,2024,73(10).

Chen Nie.,Chenyu Tang.,Jie Lin.,Huan Hu.,Chenyang Lv.,...&Zhezhi He.(2024).VSPIM: SRAM Processing-in-Memory DNN Acceleration via Vector-Scalar Operations.IEEE Transactions on Computers,73(10).

Chen Nie,et al."VSPIM: SRAM Processing-in-Memory DNN Acceleration via Vector-Scalar Operations".IEEE Transactions on Computers 73.10(2024).