JOINEDTrans: Prior guided multi-task transformer for joint optic disc/cup segmentation and fovea detection

He，Huaqing1,2; Qiu，Jiaming1; Lin，Li1,2,3; Cai，Zhiyuan1,4; Cheng，Pujin1,3; Tang，Xiaoying1,2

2024-07-01

10.1016/j.compbiomed.2024.108613

Computers in Biology and Medicine   影响因子和分区

0010-4825

1879-0534

Deep learning-based image segmentation and detection models have largely improved the efficiency of analyzing retinal landmarks such as optic disc (OD), optic cup (OC), and fovea. However, factors including ophthalmic disease-related lesions and low image quality issues may severely complicate automatic OD/OC segmentation and fovea detection. Most existing works treat the identification of each landmark as a single task, and take into account no prior information. To address these issues, we propose a prior guided multi-task transformer framework for joint OD/OC segmentation and fovea detection, named JOINEDTrans. JOINEDTrans effectively combines various spatial features of the fundus images, relieving the structural distortions induced by lesions and other imaging issues. It contains a segmentation branch and a detection branch. To be noted, we employ an encoder with prior-learning in a vessel segmentation task to effectively exploit the positional relationship among vessel, OD/OC, and fovea, successfully incorporating spatial prior into the proposed JOINEDTrans framework. There are a coarse stage and a fine stage in JOINEDTrans. In the coarse stage, OD/OC coarse segmentation and fovea heatmap localization are obtained through a joint segmentation and detection module. In the fine stage, we crop regions of interest for subsequent refinement and use predictions obtained in the coarse stage to provide additional information for better performance and faster convergence. Experimental results demonstrate that JOINEDTrans outperforms existing state-of-the-art methods on the publicly available GAMMA, REFUGE, and PALM fundus image datasets. We make our code available at https://github.com/HuaqingHe/JOINEDTrans.

Coarse-to-fine Fovea detection Multi-task learning Optic disc and cup segmentation Prior

EI

英语

第一 ; 通讯

20242116130982

Deep learning ; Image segmentation ; Learning systems

Ergonomics and Human Factors Engineering:461.4

COMPUTER SCIENCE

2-s2.0-85193623474

Scopus

1.Department of Electronic and Electrical Engineering,Southern University of Science and Technology,Shenzhen,Guangdong,China
2.Jiaxing Research Institute,Southern University of Science and Technology,Jiaxing,Zhejiang,China
3.Department of Electrical and Electronic Engineering,The University of Hong Kong,Hong Kong
4.Department of Computer Science and Engineering,The Hong Kong University of Science and Technology,Hong Kong

He，Huaqing,Qiu，Jiaming,Lin，Li,et al. JOINEDTrans: Prior guided multi-task transformer for joint optic disc/cup segmentation and fovea detection[J]. Computers in Biology and Medicine,2024,177.

He，Huaqing,Qiu，Jiaming,Lin，Li,Cai，Zhiyuan,Cheng，Pujin,&Tang，Xiaoying.(2024).JOINEDTrans: Prior guided multi-task transformer for joint optic disc/cup segmentation and fovea detection.Computers in Biology and Medicine,177.

He，Huaqing,et al."JOINEDTrans: Prior guided multi-task transformer for joint optic disc/cup segmentation and fovea detection".Computers in Biology and Medicine 177(2024).