Soft filter pruning for accelerating deep convolutional neural networks

He, Yang1,2; Kang, Guoliang2; Dong, Xuanyi2; Fu, Yanwei3; Yang, Yi1,2

2018

1045-0823

27th International Joint Conference on Artificial Intelligence, IJCAI 2018

2018-July

2234-2240

Stockholm, Sweden

International Joint Conferences on Artificial Intelligence

This paper proposed a Soft Filter Pruning (SFP) method to accelerate the inference procedure of deep Convolutional Neural Networks (CNNs). Specifically, the proposed SFP enables the pruned filters to be updated when training the model after pruning. SFP has two advantages over previous works: (1) Larger model capacity. Updating previously pruned filters provides our approach with larger optimization space than fixing the filters to zero. Therefore, the network trained by our method has a larger model capacity to learn from the training data. (2) Less dependence on the pre-trained model. Large capacity enables SFP to train from scratch and prune the model simultaneously. In contrast, previous filter pruning methods should be conducted on the basis of the pre-trained model to guarantee their performance. Empirically, SFP from scratch outperforms the previous filter pruning methods. Moreover, our approach has been demonstrated effective for many advanced CNN architectures. Notably, on ILSCRC-2012, SFP reduces more than 42% FLOPs on ResNet-101 with even 0.2% top-5 accuracy improvement, which has advanced the state-of-the-art.
© 2018 International Joint Conferences on Artificial Intelligence. All right reserved.

第一

EI

Division of Arctic Sciences[DE170101415] ; Google[CRC] ; Data to Decisions Cooperative Research Centres[]

20184406016514

Convolution ; Neural networks

Information Theory and Signal Processing:716.1

EV Compendex

1.SUSTech-UTS Joint Centre of CIS, Southern University of Science and Technology, United Kingdom
2.CAI, University of Technology Sydney, Australia
3.School of Data Science, Fudan University, China

He, Yang,Kang, Guoliang,Dong, Xuanyi,et al. Soft filter pruning for accelerating deep convolutional neural networks[C]:International Joint Conferences on Artificial Intelligence,2018:2234-2240.