LoGo Transformer: Hierarchy Lightweight Full Self-Attention Network for Corneal Endothelial Cell Segmentation

Yinglin Zhang1; Zichao Cai2; Risa Higashita3; Jiang Liu2

10.1109/IJCNN54540.2023.10191116

2023

International Joint Conference on Neural Networks (IJCNN)

2161-4393

978-1-6654-8868-6

2023 International Joint Conference on Neural Networks (IJCNN)

2023-June

1-7

18-23 June 2023

Gold Coast, Australia

345 E 47TH ST, NEW YORK, NY 10017 USA

IEEE

Corneal endothelial cell segmentation plays an important role in quantifying clinical indicators for the cornea health state evaluation. Although Convolution Neural Networks (CNNs) are widely used for medical image segmentation, their receptive fields are limited. Recently, Transformer outperforms convolution in modeling long-range dependencies but lacks local inductive bias so the pure transformer network is difficult to train on small medical image datasets. Moreover, Transformer networks cannot be effectively adopted for secular microscopes as they are parameter-heavy and computationally complex. To this end, we find that appropriately limiting attention spans and modeling information at different granularity can introduce local constraints and enhance attention representations. This paper explores a hierarchy full self-attention lightweight network for medical image segmentation, using Local and Global (LoGo) transformers to separately model attention representation at lowlevel and high-level layers. Specifically, the local efficient transformer (LoTr) layer is employed to decompose features into finergrained elements to model local attention representation, while the global axial transformer (GoTr) is utilized to build long-range dependencies across the entire feature space. With this hierarchy structure, we gradually aggregate the semantic features from different levels efficiently. Experiment results on segmentation tasks of the corneal endothelial cell, the ciliary body, and the liver prove the accuracy, effectiveness, and robustness of our method. Compared with the convolution neural networks (CNNs) and the hybrid CNN-Transformer state-of-the-art (SOTA) methods, the LoGo transformer obtains the best result.

Corneal endothelial cell segmentation Transformer Lightweight Robustness

其他

英语

CPCI-S ; EI

Computer Science ; Engineering

Computer Science, Artificial Intelligence ; Computer Science, Hardware & Architecture ; Engineering, Electrical & Electronic

WOS:001046198700079

20233614678334

Convolution ; Cytology ; Medical imaging ; Semantic Segmentation ; Semantics

Biomedical Engineering:461.1 ; Biological Materials and Tissue Engineering:461.2 ; Biology:461.9 ; Information Theory and Signal Processing:716.1 ; Artificial Intelligence:723.4 ; Imaging Techniques:746

IEEE

1.School of Computer Science, University of Nottingham Ningbo China, Ningbo, China
2.Research Institute of Trustworthy Autonomous Systems, Southern University of Science and Technology, Shenzhen, China
3.Tomey Corporation, Nagoya, Japan

Yinglin Zhang,Zichao Cai,Risa Higashita,et al. LoGo Transformer: Hierarchy Lightweight Full Self-Attention Network for Corneal Endothelial Cell Segmentation[C]. 345 E 47TH ST, NEW YORK, NY 10017 USA:IEEE,2023:1-7.