Tolerating Device-to-Device Variation for Memristive Crossbar-Based Neuromorphic Computing Systems: A New Bayesian Perspective

Yang Xiao1; Qi Xu2; Bo Yuan3

10.1109/IJCNN54540.2023.10191448

2023

International Joint Conference on Neural Networks (IJCNN)

2161-4393

978-1-6654-8868-6

2023 International Joint Conference on Neural Networks (IJCNN)

1-7

18-23 June 2023

Gold Coast, Australia

345 E 47TH ST, NEW YORK, NY 10017 USA

IEEE

Memristive crossbar-based architecture provides an energy-efficient platform to accelerate neural networks (NNs) thanks to its Processing-in-Memory (PIM) nature. However, the device-to-device variation (DDV), which is typically modeled as Lognormal distribution, deviates the programmed weights from their target values, resulting in significant performance degradation. This paper proposes a new Bayesian Neural Network (BNN) approach to enhance the robustness of weights against DDV. Instead of using the widely-used Gaussian variational posterior in conventional BNNs, our approach adopts a DDV-specific variational posterior distribution, i.e., Lognormal distribution. Accordingly, in the new BNN approach, the prior distribution is modified to keep consistent with the posterior distribution to avoid expensive Monte Carlo simulations. Furthermore, the mean of the prior distribution is dynamically adjusted in accordance with the mean of the Lognormal variational posterior distribution for better convergence and accuracy. Compared with the state-of-the-art approaches, experimental results show that the proposed new BNN approach can significantly boost the inference accuracy with the consideration of DDV on several well-known datasets and modern NN architectures. For example, the inference accuracy can be improved from 18% to 74% in the scenario of ResNet-18 on CIFAR-10 even under large variations.

device-to-device variation neuromorphic computing bayesian inference

第一

英语

CPCI-S

National Natural Science Foundation of China["61976111","62250710682","62141415"] ; Guangdong Provincial Key Laboratory[2020B121201001]

Computer Science ; Engineering

Computer Science, Artificial Intelligence ; Computer Science, Hardware & Architecture ; Engineering, Electrical & Electronic

WOS:001046198703030

IEEE

1.Research Institute of Trustworthy Autonomous System (RITAS), Southern University of Science and Technology (SUSTech), Shenzhen, China
2.School of Microelectronics, University of Science and Technology of China (USTC)
3.Department of Computer Science and Engineering, Guangdong Provincial Key Laboratory of Brain-inspired Intelligent Computation, Southern University of Science and Technology (SUSTech), Shenzhen, China

Yang Xiao,Qi Xu,Bo Yuan. Tolerating Device-to-Device Variation for Memristive Crossbar-Based Neuromorphic Computing Systems: A New Bayesian Perspective[C]. 345 E 47TH ST, NEW YORK, NY 10017 USA:IEEE,2023:1-7.