基于深度学习与医学先验的眼底关键结构检测算法

眼底摄影已被常规用于记录各种视网膜退行性疾病的存在和严重程度，如年龄相关性黄斑变性、青光眼和糖尿病视网膜病变，其中中心凹、视盘和视杯是重要的解剖标志。识别这些解剖标志具有重要的临床意义。然而，视网膜变性过程中病变和其他异常的存在使自动标志检测和分割变得严重复杂。大多数现有工作将每个地标的识别视为单个任务，并且通常不使用任何临床先验信息。本研究本文详细介绍了两个主要的基于深度学习的方法的研究方向：首先分析了视网膜关键结构之间的空间位置关系，并提出基于医学先验与多任务学习的视杯视盘分割与黄斑中心凹定位方法，提升了视杯视盘分割和黄斑中心定位任务的准确性和鲁棒性；其次针对在临床普查中光学相干断层扫描（Optical Coherence Tomography, OCT）模态数量少、质量参差不齐的现象，提出了；结合眼底彩照与OCT的多模态黄斑中心定位算法，保证即使引入少量的高质量OCT图像也能很好的提升中心凹定位任务的准确性。

具体的，在视杯视盘分割与黄斑中心凹定位联合检测方面，本文提出了一种引入血管位置的医学先验，增强了网络对眼底图像中视杯视盘区域的识别和分割能力。在黄斑中心定位方面，我们开发了一种多模态融合网络，该网络能够有效地利用眼底彩照与OCT图像中的互补信息，提高中心凹定位的准确性和鲁棒性。

本文的实验部分通过大量实验验证了所提方法的有效性，与现有技术相比，我们的方法在视杯视盘分割和黄斑中心定位的准确率上都有显著提升。特别是在公开的眼底图像数据集上，本文提出的方法能够有效地处理各种复杂情况，如视网膜病变、图像质量参差不齐等，显示了良好的泛化能力和实用性。

[1] 视网膜血管改变用于心血管疾病评估及预测的研究进展 - 中国医师杂志[EB/OL]. /2024-02-27. https://rs.yiigle.com/cmaid/1325259.
[2] Liu J, Wang X, Tai X-C. Deep Convolutional Neural Networks with Spatial Regularization, Volume and Star-Shape Priors for Image Segmentation[J]. Journal of Mathematical Imaging and Vision, 2022, 64(6): 625–645.
[3] Jing Z, Xiao-yan D, Jia-qing L I. Choroidal thickness of Chinese population and its relevant factors[J]. Chinese Journal of Ocular Fundus Diseases, Chinese Medical Journals Publishing House Co., Ltd., 2011, 27(05): 450–453.
[4] Song S, Dang K, Yu Q, 等. Bilateral-ViT For Robust Fovea Localization[A]. 2022 IEEE 19th International Symposium on Biomedical Imaging (ISBI)[C]. 2022: 1–5.
[5] 秦运输. 基于深度学习的青光眼智能诊断研究[D]. 中国科学技术大学, 2020.
[6] Cai Z, Lin L, He H, 等. Corolla: An Efficient Multi-Modality Fusion Framework with Supervised Contrastive Learning for Glaucoma Grading[A]. 2022 IEEE 19th International Symposium on Biomedical Imaging (ISBI)[C]. 2022: 1–4.
[7] Fu H. REFUGE: Retinal Fundus Glaucoma Challenge[J]. IEEE, 2019.
[8] Huang Y, Lin L, Cheng P, 等. Lesion-Based Contrastive Learning for Diabetic Retinopathy Grading from Fundus Images[A]. M. de Bruijne, P.C. Cattin, S. Cotin, 等. Medical Image Computing and Computer Assisted Intervention – MICCAI 2021[C]. Cham: Springer International Publishing, 2021: 113–123.
[9] 乔良, 袁烨, 张万虎, 等. 间接眼底镜下外路手术治疗孔源性视网膜脱离对患者视力与黄斑水肿的影响[J]. 现代生物医学进展, 2022, 22(15): 2870–2874.
[10] Wu J, Fang H, Li F, 等. GAMMA challenge: Glaucoma grAding from Multi-Modality imAges[J]. Medical Image Analysis, 2023, 90: 102938.
[11] Phene S, Dunn R C, Hammel N, 等. Deep Learning and Glaucoma Specialists: The Relative Importance of Optic Disc Features to Predict Glaucoma Referral in Fundus Photographs[J]. Ophthalmology, 2019, 126(12): 1627–1639.
[12] Fu H, Cheng J, Xu Y, 等. Joint Optic Disc and Cup Segmentation Based on Multi-Label Deep Network and Polar Transformation[J]. IEEE Transactions on Medical Imaging, 2018, 37(7): 1597–1605.
[13] 赵仕成, 莫娟. 一种基于递归深度特征融合的视杯视盘联合分割方法[J]. 信息技术与信息化, 2022(06): 36–42.
[14] 方玲玲, 张丽榕. 视盘和视杯分割在计算机辅助青光眼诊断中的应用综述[J]. 中国图象图形学报, 2022, 27(10): 2952–2971.
[15] Fu H, Cheng J, Xu Y, 等. Disc-Aware Ensemble Network for Glaucoma Screening From Fundus Image[J]. IEEE Transactions on Medical Imaging, 2018, 37(11): 2493–2501.
[16] 滕娟, 陈小红, 陈梅珠, 等. 非增生型糖尿病视网膜病变患者黄斑水肿与中心凹下脉络膜厚度的关系[J]. 眼科新进展, 2017, 37(3): 244–247.
[17] Huang Y, Zhong Z, Yuan J, 等. Efficient and robust optic disc detection and fovea localization using region proposal network and cascaded network[J]. Biomedical Signal Processing and Control, 2020, 60: 101939.
[18] Peng L, Lin L, Cheng P, 等. Unsupervised Domain Adaptation for Cross-Modality Retinal Vessel Segmentation via Disentangling Representation Style Transfer and Collaborative Consistency Learning[A]. 2022 IEEE 19th International Symposium on Biomedical Imaging (ISBI)[C]. Kolkata, India: IEEE, 2022: 1–5.
[19] Fraga A, Barreira N, Ortega M, 等. Precise Segmentation of the Optic Disc in Retinal Fundus Images[A]. R. Moreno-Díaz, F. Pichler, A. Quesada-Arencibia. Computer Aided Systems Theory – EUROCAST 2011[C]. Berlin, Heidelberg: Springer, 2012: 584–591.
[20] Tjandrasa H, Wijayanti A, Suciati N. Optic Nerve Head Segmentation Using Hough Transform and Active Contours[J]. TELKOMNIKA (Telecommunication Computing Electronics and Control), 2012, 10(3): 531.
[21] Edupuganti V G, Chawla A, Kale A. Automatic Optic Disk and Cup Segmentation of Fundus Images Using Deep Learning[A]. 2018 25th IEEE International Conference on Image Processing (ICIP)[C]. 2018: 2227–2231.
[22] Zheng Y, Zhang X, Xu X, 等. Deep level set method for optic disc and cup segmentation on fundus images[J]. Biomedical Optics Express, Optica Publishing Group, 2021, 12(11): 6969–6983.
[23] Pachade S, Porwal P, Kokare M, 等. NENet: Nested EfficientNet and adversarial learning for joint optic disc and cup segmentation[J]. Medical Image Analysis, 2021, 74: 102253.
[24] Meng Y, Zhang H, Zhao Y, 等. Dual Consistency Enabled Weakly and Semi-Supervised Optic Disc and Cup Segmentation with Dual Adaptive Graph Convolutional Networks[J]. IEEE Transactions on Medical Imaging, 2022: 1–1.
[25] El Jurdi R, Petitjean C, Honeine P, 等. High-level prior-based loss functions for medical image segmentation: A survey[J]. Computer Vision and Image Understanding, 2021, 210: 103248.
[26] Sun G, Zhang Z, Zhang J, 等. Joint optic disc and cup segmentation based on multi-scale feature analysis and attention pyramid architecture for glaucoma screening[J]. Neural Computing and Applications, 2021.
[27] Xie R, Liu J, Cao R, 等. End-to-End Fovea Localisation in Colour Fundus Images With a Hierarchical Deep Regression Network[J]. IEEE Transactions on Medical Imaging, 2021, 40(1): 116–128.
[28] Hu X, Li F, Samaras D, 等. Topology-Preserving Deep Image Segmentation[A]. Advances in Neural Information Processing Systems[C]. Curran Associates, Inc., 2019, 32.
[29] He H, Lin L, Cai Z, 等. JOINED: Prior Guided Multitask Learning for Joint Optic Disc/Cup Segmentation and Fovea Detection[A]. 2022.
[30] Camarasa R, Kervadec H, Bos D, 等. Differentiable Boundary Point Extraction for Weakly Supervised Star-shaped Object Segmentation[A]. Proceedings of The 5th International Conference on Medical Imaging with Deep Learning[C]. PMLR, 2022: 188–198.
[31] Medhi J P, Dandapat S. Automatic detection of fovea using property of vessel free region[A]. 2015 Twenty First National Conference on Communications (NCC)[C]. 2015: 1–6.
[32] Deka D, Medhi J P, Nirmala S R. Detection of macula and fovea for disease analysis in color fundus images[A]. 2015 IEEE 2nd International Conference on Recent Trends in Information Systems (ReTIS)[C]. 2015: 231–236.
[33] Fu Y, Zhang G, Li J, 等. Fovea localization by blood vessel vector in abnormal fundus images[J]. Pattern Recognition, 2022, 129: 108711.
[34] Meyer M I, Galdran A, Mendonça A M, 等. A Pixel-Wise Distance Regression Approach for Joint Retinal Optical Disc and Fovea Detection[A]. A.F. Frangi, J.A. Schnabel, C. Davatzikos, 等. Medical Image Computing and Computer Assisted Intervention – MICCAI 2018[C]. Cham: Springer International Publishing, 2018: 39–47.
[35] Li Y, El Habib Daho M, Conze P-H, 等. Multimodal Information Fusion for Glaucoma and Diabetic Retinopathy Classification[A]. B. Antony, H. Fu, C.S. Lee, 等. Ophthalmic Medical Image Analysis[C]. Cham: Springer International Publishing, 2022: 53–62.
[36] Cai Z, Lin L, He H, 等. Uni4Eye: Unified 2D and 3D Self-supervised Pre-training via Masked Image Modeling Transformer for Ophthalmic Image Classification[A]. L. Wang, Q. Dou, P.T. Fletcher, 等. Medical Image Computing and Computer Assisted Intervention – MICCAI 2022[C]. Cham: Springer Nature Switzerland, 2022: 88–98.
[37] Anantrasirichai N, Zheng R, Selesnick I, 等. Image fusion via sparse regularization with non-convex penalties[J]. Pattern Recognition Letters, 2020, 131: 355–360.
[38] Pappas O, Anantrasirichai N, Nicholson L, 等. Curvelet domain image fusion of OCT and fundus imagery using convolution of Meridian distributions[A]. 2013 IEEE International Conference on Image Processing[C]. 2013: 1423–1427.
[39] Stein D M, Ishikawa H, Hariprasad R, 等. A new quality assessment parameter for optical coherence tomography[J]. British Journal of Ophthalmology, BMJ Publishing Group Ltd, 2006, 90(2): 186–190.
[40] Meng Y, Zhang H, Zhao Y, 等. Graph-Based Region and Boundary Aggregation for Biomedical Image Segmentation[J]. IEEE Transactions on Medical Imaging, 2022, 41(3): 690–701.
[41] Salazar-Gonzalez A, Kaba D, Li Y, 等. Segmentation of the Blood Vessels and Optic Disk in Retinal Images[J]. IEEE Journal of Biomedical and Health Informatics, 2014, 18(6): 1874–1886.
[42] Pascal L, Perdomo O J, Bost X, 等. Multi-task deep learning for glaucoma detection from color fundus images[J]. Scientific Reports, Nature Publishing Group, 2022, 12(1): 12361.
[43] Papastefanou V P, Al-Jamal R T, Ali Z C, 等. Ultra-wide-field imaging assessment of small choroidal pigmented lesions using red and green colour channels[J]. Eye, 2021, 35(1): 282–288.
[44] Chen L-C, Zhu Y, Papandreou G, 等. Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation[A]. Computer Vision – ECCV 2018[C]. Springer, Cham, 2018: 833–851.
[45] Li W, Liao H, Miao S, 等. Unsupervised Learning of Landmarks based on Inter-Intra Subject Consistencies[J]. arXiv, 2020.
[46] Ronneberger O, Fischer P, Brox T. U-Net: Convolutional Networks for Biomedical Image Segmentation[A]. N. Navab, J. Hornegger, W.M. Wells, 等. Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015[C]. Cham: Springer International Publishing, 2015: 234–241.
[47] Deng J, Dong W, Socher R, 等. ImageNet: A large-scale hierarchical image database[A]. 2009 IEEE Conference on Computer Vision and Pattern Recognition[C]. 2009: 248–255.
[48] Tan M, Le Q V. EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks[J]. arXiv, 2020.
[49] Kamble R, Samanta P, Singhal N. Optic Disc, Cup and Fovea Detection from Retinal Images Using U-Net++ with EfficientNet Encoder[A]. H. Fu, M.K. Garvin, T. MacGillivray, 等. Ophthalmic Medical Image Analysis[C]. Cham: Springer International Publishing, 2020: 93–103.
[50] Hagos M T, Kant S. Transfer Learning based Detection of Diabetic Retinopathy from Small Dataset[J]. arXiv, 2019.
[51] Cao H, Wang Y, Chen J, 等. Swin-Unet: Unet-Like Pure Transformer for Medical Image Segmentation[A]. L. Karlinsky, T. Michaeli, K. Nishino. Computer Vision – ECCV 2022 Workshops[C]. Cham: Springer Nature Switzerland, 2023: 205–218.
[52] Liu Z, Lin Y, Cao Y, 等. Swin Transformer: Hierarchical Vision Transformer using Shifted Windows[A]. 2021 IEEE/CVF International Conference on Computer Vision (ICCV)[C]. 2021: 9992–10002.
[53] Lyu J, Zhang Y, Huang Y, 等. AADG: Automatic Augmentation for Domain Generalization on Retinal Image Segmentation[J]. IEEE Transactions on Medical Imaging, 2022, 41(12): 3699–3711.
[54] Zhou Z, Siddiquee M M R, Tajbakhsh N, 等. UNet++: A Nested U-Net Architecture for Medical Image Segmentation[J]. arXiv, 2018.
[55] Chen L-C, Papandreou G, Kokkinos I, 等. DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs[J]. arXiv, 2017.
[56] He K, Zhang X, Ren S, 等. Deep Residual Learning for Image Recognition[J]. arXiv, 2015.
[57] Dolz J, Gopinath K, Yuan J, 等. HyperDense-Net: A Hyper-Densely Connected CNN for Multi-Modal Image Segmentation[J]. IEEE Transactions on Medical Imaging, 2019, 38(5): 1116–1126.
[58] Oktay O, Schlemper J, Folgoc L L, 等. Attention U-Net: Learning Where to Look for the Pancreas[J]. arXiv, 2018.
[59] Shen S, Xia Y, Eich A, 等. SegTrans: Semantic Segmentation With Transfer Learning for MLS Point Clouds[J]. IEEE Geoscience and Remote Sensing Letters, 2023, 20: 1–5.
[60] Chen J, Lu Y, Yu Q, 等. TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation[J]. arXiv, 2021.
[61] Zamir S W, Arora A, Khan S, 等. Restormer: Efficient Transformer for High-Resolution Image Restoration[A]. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)[C]. 2022: 5718–5729.
[62] Wu Z, Liu Z, Lin J, 等. Lite Transformer with Long-Short Range Attention[A]. 2020.
[63] Density estimation using Real NVP | OpenReview[EB/OL]. /2024-03-14. https://openreview.net/forum?id=HkpbnH9lx.
[64] Kingma D P, Ba J. Adam: A Method for Stochastic Optimization[J]. arXiv, 2017.
[65] Gegundez-Arias M E, Marin D, Bravo J M, 等. Locating the fovea center position in digital fundus images using thresholding and feature extraction techniques[J]. Computerized Medical Imaging and Graphics, 2013, 37(5): 386–393.