Efficient DNN neuron pruning by minimizing layer-wise nonlinear reconstruction error

Jiang，Chunhui1; Li，Guiying1; Qian，Chao1; Tang，Ke2

2018

1045-0823

IJCAI International Joint Conference on Artificial Intelligence

2018-July

2298-2304

Deep neural networks (DNNs) have achieved great success, but the applications to mobile devices are limited due to their huge model size and low inference speed. Much effort thus has been devoted to pruning DNNs. Layer-wise neuron pruning methods have shown their effectiveness, which minimize the reconstruction error of linear response with a limited number of neurons in each single layer pruning. In this paper, we propose a new layer-wise neuron pruning approach by minimizing the reconstruction error of nonlinear units, which might be more reasonable since the error before and after activation can change significantly. An iterative optimization procedure combining greedy selection with gradient decent is proposed for single layer pruning. Experimental results on benchmark DNN models show the superiority of the proposed approach. Particularly, for VGGNet, the proposed approach can compress its disk space by 13.6× and bring a speedup of 3.7×; for AlexNet, it can achieve a compression rate of 4.1× and a speedup of 2.2×, respectively.

其他

英语

2-s2.0-85055715831

Scopus

1.Anhui Province Key Lab of Big Data Analysis and Application, University of Science and Technology of China, ,Hefei,230027,China
2.Shenzhen Key Lab of Computational Intelligence, Department of Computer Science and Engineering, Southern University of Science and Technology, ,Shenzhen,518055,China

Jiang，Chunhui,Li，Guiying,Qian，Chao,et al. Efficient DNN neuron pruning by minimizing layer-wise nonlinear reconstruction error[C],2018:2298-2304.