HOTCAKE: Higher Order Tucker Articulated Kernels for Deeper CNN Compression

Lin，Rui1; Ko，Ching Yun2; He，Zhuolun3; Chen，Cong4; Cheng，Yuan5; Yu，Hao1; Chesi，Graziano1; Wong，Ngai1

10.1109/ICSICT49897.2020.9278257

2020-11-03

978-1-7281-6236-2

2020 IEEE 15th International Conference on Solid-State and Integrated Circuit Technology, ICSICT 2020 - Proceedings

1-4

3-6 Nov. 2020

Kunming, China

The emerging edge computing has produced immense interests in compacting a neural network without sacrificing much accuracy. In this regard, low-rank tensor decomposition constitutes a powerful tool to compress convolutional neural networks (CNNs) by decomposing the 4-way kernel tensor into multi-stage smaller ones. Building on top of Tucker-2 decomposition, we propose a generalized Higher Order Tucker Articulated Kernels (HOT-CAKE) scheme comprising four steps: input channel decomposition, guided Tucker rank selection, higher order Tucker decomposition and fine-tuning. By subjecting each CONV layer to HOTCAKE, a highly compressed CNN model with graceful accuracy trade-off is obtained. Experiments show HOTCAKE can compress even pre-compressed models and produce state-of-the-art lightweight networks.

Tensors Kernel Neural networks Matrix decomposition Faces Convolution Time complexity

其他

英语

EI

20210309772771

Economic and social effects ; Integrated circuits ; Tensors

Semiconductor Devices and Integrated Circuits:714.2 ; Algebra:921.1 ; Social Sciences:971

2-s2.0-85099173487

Scopus

1.University of Hong Kong,Department of Electrical and Electronic Engineering,Hong Kong
2.Massachusetts Institute of Technology,Department of Electrical Engineering and Computer Science,United States
3.Chinese University of Hong Kong,Department of Computer Science and Engineering,Hong Kong
4.Shanghai Jiao Tong University,Department of MicrolNanoe1ectronics,Shanghai,China
5.Southern University of Science and Technology,Department of Electrical and Electronic Engineering,China

Lin，Rui,Ko，Ching Yun,He，Zhuolun,et al. HOTCAKE: Higher Order Tucker Articulated Kernels for Deeper CNN Compression[C],2020:1-4.