Strip-MLP: Efficient Token Interaction for Vision MLP

Guiping Cao1,2; Shengda Luo1; Wenjian Huang1; Xiangyuan Lan2; Dongmei Jiang2; Yaowei Wang2; Jianguo Zhang1,2

arxiv-2307.11458

2023-10-01

IEEE International Conference on Computer Vision 2023

1550-5499

979-8-3503-0719-1

2023 IEEE/CVF International Conference on Computer Vision (ICCV)

1494-1504

2023.10

Paris

10662 LOS VAQUEROS CIRCLE, PO BOX 3014, LOS ALAMITOS, CA 90720-1264 USA

IEEE COMPUTER SOC

Token interaction operation is one of the core modules in MLP-based models to exchange and aggregate information between different spatial locations. However, the power of token interaction on the spatial dimension is highly depen- (a) Down-sampled image. (b) Token mixing of MLP layer. dent on the spatial resolution of the feature maps, which limits the model's expressive ability, especially in deep layers where the feature are down-sampled to a small spatial size. To address this issue, we present a novel method called Strip-MLP to enrich the token interaction power in three ways. Firstly, we introduce a new MLP paradigm called Strip MLP layer that allows the token to interact with other tokens in a cross-strip manner, enabling the tokens in a row (or column) to contribute to the information aggregations in adjacent but different strips of rows (or columns). Secondly, a Cascade Group Strip Mixing Module (CGSMM) is proposed to overcome the performance degradation caused by small spatial feature size. The module allows tokens to interact more effectively in the manners of within-patch and cross-patch, which is independent to the feature spatial size. Finally, based on the Strip MLP layer, we propose a novel Local Strip Mixing Module (LSMM) to boost the token interaction power in the local region. Extensive experiments demonstrate that Strip-MLP significantly improves the performance of MLP-based models on small datasets and obtains comparable or even better results on ImageNet. In particular, Strip-MLP models achieve higher average Top-1 accuracy than existing MLP-based models by +2.44% on Caltech-101 and +2.16% on CIFAR-100. The source codes will be available at https://github.com/MedProcess/Strip MLP.

Degradation Strips Computer vision Analytical models Source coding Computational modeling Task analysis

第一 ; 通讯

英语

CPCI-S

National Key Research and Development Program of China[2021YFF1200800] ; Peng Cheng Laboratory Research Project[PCL2023AS6-1] ; Guangdong Basic and Applied Basic Research Foundation[2022A1515110573]

Computer Science ; Imaging Science & Photographic Technology

Computer Science, Artificial Intelligence ; Computer Science, Theory & Methods ; Imaging Science & Photographic Technology

WOS:001159644301070

人工提交

正式出版

1.Southern University of Science and Technology, Shenzhen, China
2.Peng Cheng Laboratory, Shenzhen, China

Guiping Cao,Shengda Luo,Wenjian Huang,et al. Strip-MLP: Efficient Token Interaction for Vision MLP[C]. 10662 LOS VAQUEROS CIRCLE, PO BOX 3014, LOS ALAMITOS, CA 90720-1264 USA:IEEE COMPUTER SOC,2023:1494-1504.