SSiT: Saliency-guided Self-supervised Image Transformer for Diabetic Retinopathy Grading

Huang，Yijin1; Lyu，Junyan1; Cheng，Pujin1; Tam，Roger2; Tang，Xiaoying1

2024

10.1109/JBHI.2024.3362878

IEEE Journal of Biomedical and Health Informatics   影响因子和分区

2168-2194

2168-2208

Self-supervised Learning (SSL) has been widely applied to learn image representations through exploiting unlabeled images. However, it has not been fully explored in the medical image analysis field. In this work, Saliency-guided Self-Supervised image Transformer (SSiT) is proposed for Diabetic Retinopathy (DR) grading from fundus images. We novelly introduce saliency maps into SSL, with a goal of guiding self-supervised pre-training with domain-specific prior knowledge. Specifically, two saliency-guided learning tasks are employed in SSiT: (1) Saliency-guided contrastive learning is conducted based on the momentum contrast, wherein fundus images' saliency maps are utilized to remove trivial patches from the input sequences of the momentum-updated key encoder. Thus, the key encoder is constrained to provide target representations focusing on salient regions, guiding the query encoder to capture salient features. (2) The query encoder is trained to predict the saliency segmentation, encouraging the preservation of fine-grained information in the learned representations. To assess our proposed method, four publicly-accessible fundus image datasets are adopted. One dataset is employed for pre-training, while the three others are used to evaluate the pre-trained models' performance on downstream DR grading. The proposed SSiT significantly outperforms other representative state-of-the-art SSL methods on all downstream datasets and under various evaluation settings. For example, SSiT achieves a Kappa score of 81.88% on the DDR dataset under fine-tuning evaluation, outperforming all other ViT-based SSL methods by at least 9.48%.

Biomedical imaging Diabetic retinopathy Fundus image Lesions Medical diagnostic imaging Saliency map Self-supervised learning Self-supervised learning Task analysis Training Transformers Vision transformer

SCI ; EI

英语

第一

2-s2.0-85184826507

Scopus

1.Department of Electronic and Electrical Engineering, Southern University of Science and Technology, Shenzhen, China
2.School of Biomedical Engineering, The University of British Columbia, Vancouver, BC, Canada

Huang，Yijin,Lyu，Junyan,Cheng，Pujin,et al. SSiT: Saliency-guided Self-supervised Image Transformer for Diabetic Retinopathy Grading[J]. IEEE Journal of Biomedical and Health Informatics,2024,PP(99):1-13.

Huang，Yijin,Lyu，Junyan,Cheng，Pujin,Tam，Roger,&Tang，Xiaoying.(2024).SSiT: Saliency-guided Self-supervised Image Transformer for Diabetic Retinopathy Grading.IEEE Journal of Biomedical and Health Informatics,PP(99),1-13.

Huang，Yijin,et al."SSiT: Saliency-guided Self-supervised Image Transformer for Diabetic Retinopathy Grading".IEEE Journal of Biomedical and Health Informatics PP.99(2024):1-13.