基于OCT图像的视网膜色素上皮断裂检测技术研究与应用

  视网膜疾病会引发眼底结构变化，导致视功能减退，严重影响患者的正常工作和生活。作为视网膜最外层的视网膜色素上皮（Retinal Pigment Epithelium，RPE）会在疾病发生或治疗过程中显现异常，出现断裂，导致患者在疾病严重程度、视力和预后恢复时间等方面表现出差异，影响医生制定和调整治疗方案。诊断监测过程中会产生大量的光学相干断层扫描（Optical Coherence Tomography， OCT）图像，在医疗资源有限的情况下，亟需智能辅助算法的帮助。目前现有相关方法大多依赖视网膜分层效果，并且缺乏对该问题的针对性。
    为此，本文聚焦于湿性年龄性黄斑变性（wet Age-related Macular Degeneration，wAMD）场景，通过与宁波眼科医院的深度合作，收集并建立了一个wAMD的OCT数据集，逐步展开了三个方面的研究：
  
  （1）提出了一种基于残差神经网络的RPE断裂识别网络。该方法探究无需分层条件下深度学习用于RPE断裂与否检测的可行性，取得了优于普通眼科医师的性能。
    （2）提出了一种面向结构的Transformer算法。该方法通过融入临床结构先验知识，使用新颖的结构指导模块增强视网膜结构特征，使用Vision Transformer建模病变与视网膜的关联性，通过更具泛化性的标签投票策略提升了RPE断裂与否的检测性能。
    （3）提出了一种新颖的面向交互的特征分解RPE断裂检测网络。受临床诊断流程启发，该方法将通过OCT图像进行RPE断裂与否的分类任务转化为通过定位辅助RPE断裂与否的检测任务。方法通过全局上下文嵌入模块保留全局上下文特征，将定位网络得到的RPE异常特征和全局上下文特征通过全局上下文交叉注意力模块进行融合建模，缓解了由于断裂与未断裂相似导致的误分类问题，进一步提升了检测性能和辅助效果。

  本文提出的RPE断裂检测算法在湿性AMD数据集上进行了实验验证，提高了检测准确率，有效检测RPE断裂与否情况，起到了一定的辅助作用。

Retinal diseases can cause changes in the retina, leading to decreased visual function and affecting the work and life of patients. The retinal pigment epithelium (RPE), the outermost layer of the retina, may interrupt during the disease occurrence or treatment, resulting in differences in the severity of the disease, visual acuity, and prognosis recovery time of patients, which will affect the ophthalmologist's treatment plan and adjustment. The diagnosis generates a large number of optical coherence tomography (OCT) images. In the case of limited medical resources, intelligent auxiliary algorithms are urgently needed. At present, most of the existing related methods rely on retinal layering, and lack of pertinence to this problem.
  
  This thesis focuses on wet Age-related Macular Degeneration (wAMD) scenarios. Through in-depth cooperation with Ningbo Eye Hospital, a wAMD OCT dataset was collected and established, and three aspects of the research were gradually carried out:
  
  (1) An RPE rupture recognition network based on a residual neural network is proposed. This method explores the feasibility of using deep learning for RPE rupture detection without layering and achieves better performance than ordinary ophthalmologists.
  
  (2) A Structure-Oriented Transformer algorithm is proposed. Incorporating the prior knowledge of clinical structure, the method uses a novel structure-guided module to enhance layer features and uses Vision Transformer to model the relationship between the lesion and the retina. The performance of RPE rupture detection is better through a more generalized Token Vote strategy. The algorithm improves the detection performance of RPE rupture or not.
  
  (3) A novel interaction-oriented feature decomposition RPE fracture detection network is proposed. Inspired by the clinical diagnosis, the method transforms the task of classifying RPE rupture through OCT images into the detection task of RPE fractures assisted by localization. The method uses the global context embedding module to retain the global context features and integrates the RPE abnormal features and global context features obtained by the localization network through the global context cross attention module, which alleviates the misclassification problem caused by the similarity between broken and unbroken, and further Improved detection performance and assistance effects.

  The RPE rupture detection algorithm proposed in this paper is experimentally verified on the wAMD dataset, which improves the accuracy and effectively detects whether the RPE is ruptured or not, which plays a certain auxiliary role.

[1] SCHMITT J M. Optical coherence tomography (OCT): a review[J]. IEEE Journal of SelectedTopics in Quantum Electronics, 1999, 5(4): 1205-1215.
[2] HUANG D, SWANSON E A, LIN C P, et al. Optical coherence tomography[J]. Science, 1991,254(5035): 1178-1181.
[3] MARMOR M F, WOLFENSBERGER T. The retinal pigment epithelium[J]. Function andDisease, 1998: 103-134.
[4] KLETTNER A K, DITHMAR S. Retinal pigment epithelium in health and disease[M].Springer, 2020.
[5] KHATRI M, SAXENA S, KAUR A, et al. Resistive index of ophthalmic artery correlateswith retinal pigment epithelial alterations on spectral domain optical coherence tomography indiabetic retinopathy[J]. International Journal of Retina and Vitreous, 2018, 4(1): 1-7.
[6] SCHLANITZ F, BAUMANN B, SACU S, et al. Impact of drusen and drusenoid retinal pigmentepithelium elevation size and structure on the integrity of the retinal pigment epithelium layer[J]. British Journal of Ophthalmology, 2019, 103(2): 227-232.
[7] MITCHELL P, RODRÍGUEZ F J, JOUSSEN A M, et al. Management of retinal pigment epithelium tear during anti-VEGF therapy[J]. Retina (Philadelphia, Pa.), 2021, 41(4): 671.
[8] HOSKIN A, BIRD A, SEHMI K. Tears of detached retinal pigment epithelium[J]. BritishJournal of Ophthalmology, 1981, 65(6): 417-422.
[9] BRESSLER N M, BRESSLER S B, FINE S L. Age-related macular degeneration[J]. Surveyof Ophthalmology, 1988, 32(6): 375-413.
[10] FERRIS F L, FINE S L, HYMAN L. Age-related macular degeneration and blindness due toneovascular maculopathy[J]. Archives of Ophthalmology, 1984, 102(11): 1640-1642.
[11] GROSSNIKLAUS H, GASS J D. Clinicopathologic correlations of surgically excised type 1 andtype 2 submacular choroidal neovascular membranes[J]. American Journal of Ophthalmology,1998, 126(1): 59-69.
[12] SUGMK J, KIATTISIN S, LEELASANTITHAM A. Automated classifcation between agerelated macular degeneration and diabetic macular edema in OCT image using image segmentation[C]//The 7th 2014 Biomedical Engineering International Conference. IEEE, 2014: 1-4.
[13] HASSAN B, RAJA G, HASSAN T, et al. Structure tensor based automated detection of macularedema and central serous retinopathy using optical coherence tomography images[J]. JOSA A,2016, 33(4): 455-463.
[14] KOPROWSKI R, TEPER S, WRÓBEL Z, et al. Automatic analysis of selected choroidal diseases in OCT images of the eye fundus[J]. Biomedical Engineering Online, 2013, 12(1): 1-18.
[15] LIU Y Y, CHEN M, ISHIKAWA H, et al. Automated macular pathology diagnosis in retinalOCT images using multi-scale spatial pyramid and local binary patterns in texture and shapeencoding[J]. Medical Image Analysis, 2011, 15(5): 748-759.61参考文献
[16] VENHUIZEN F G, VAN GINNEKEN B, BLOEMEN B, et al. Automated age-related maculardegeneration classifcation in OCT using unsupervised feature learning[C]//Medical Imaging2015: Computer-Aided Diagnosis: volume 9414. SPIE, 2015: 391-397.
[17] LEE C S, BAUGHMAN D M, LEE A Y. Deep learning is efective for classifying normal versusage-related macular degeneration OCT images[J]. Ophthalmology Retina, 2017, 1(4): 322-327.
[18] GRASSMANN F, MENGELKAMP J, BRANDL C, et al. A deep learning algorithm for prediction of age-related eye disease study severity scale for age-related macular degeneration fromcolor fundus photography[J]. Ophthalmology, 2018, 125(9): 1410-1420.
[19] TREDER M, LAUERMANN J L, ETER N. Automated detection of exudative age-relatedmacular degeneration in spectral domain optical coherence tomography using deep learning[J]. Graefe’s Archive for Clinical and Experimental Ophthalmology, 2018, 256(2): 259-265.
[20] DAS V, DANDAPAT S, BORA P K. Automated Classifcation of Retinal OCT Images using aDeep Multi-Scale Fusion CNN[J]. IEEE Sensors Journal, 2021, 21(20): 23256-23265.
[21] SUNIJA A, KAR S, GAYATHRI S, et al. Octnet: A lightweight cnn for retinal disease classifcation from optical coherence tomography images[J]. Computer Methods and Programs inBiomedicine, 2021, 200: 105877.
[22] RUNKLE A P, KAISER P K, SRIVASTAVA S K, et al. OCT angiography and ellipsoid zonemapping of macular telangiectasia type 2 from the AVATAR study[J]. Investigative Ophthalmology & Visual Science, 2017, 58(9): 3683-3689.
[23] ZHU W, CHEN H, ZHAO H, et al. Automatic three-dimensional detection of photoreceptorellipsoid zone disruption caused by trauma in the OCT[J]. Scientifc Reports, 2016, 6(1): 1-10.
[24] MUKHERJEE D, LAD E M, VANN R R, et al. Correlation between macular integrity assessment and optical coherence tomography imaging of ellipsoid zone in macular telangiectasiatype 2[J]. Investigative Ophthalmology & Visual Science, 2017, 58(6): BIO291-BIO299.
[25] WANG Z, CAMINO A, HAGAG A M, et al. Automated detection of preserved photoreceptoron optical coherence tomography in choroideremia based on machine learning[J]. Journal ofBiophotonics, 2018, 11(5): e201700313.
[26] DE SILVA T, JAYAKAR G, GRISSO P, et al. Deep Learning-Based Automatic Detection of Ellipsoid Zone Loss in Spectral-Domain OCT for Hydroxychloroquine Retinal Toxicity Screening[J]. Ophthalmology Science, 2021, 1(4): 100060.
[27] CHEN X, ZHANG L, SOHN E H, et al. Quantifcation of external limiting membrane disruptioncaused by diabetic macular edema from SD-OCT[J]. Investigative Ophthalmology & VisualScience, 2012, 53(13): 8042-8048.
[28] WANG L, ZHU W, LIAO J, et al. Support vector machine based IS/OS disruption detection fromSD-OCT images[C]//Medical Imaging 2014: Image Processing: volume 9034. InternationalSociety for Optics and Photonics, 2014: 90341U.
[29] LECUN Y, BOTTOU L, BENGIO Y, et al. Gradient-based learning applied to document recognition[J]. Proceedings of the IEEE, 1998, 86(11): 2278-2324.
[30] SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale imagerecognition[C]//International Conference on Learning Representations. 2015: 1-14.62参考文献
[31] HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition[C]//Proceedingsof the IEEE Conference on Computer Vision and Pattern Recognition. 2016: 770-778.
[32] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[J]. Advances inNeural Information Processing Systems, 2017, 30.
[33] DOSOVITSKIY A, BEYER L, KOLESNIKOV A, et al. An image is worth 16x16 words:Transformers for image recognition at scale[A/OL]. 2020. arXiv: 2010.11929. https://arxiv.org/abs/2010.11929.
[34] LIU Z, LIN Y, CAO Y, et al. Swin transformer: Hierarchical vision transformer using shiftedwindows[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.2021: 10012-10022.
[35] CHU X, TIAN Z, WANG Y, et al. Twins: Revisiting the design of spatial attention in visiontransformers[J]. Advances in Neural Information Processing Systems, 2021, 34: 9355-9366.
[36] CHEN C F R, FAN Q, PANDA R. Crossvit: Cross-attention multi-scale vision transformer forimage classifcation[C]//Proceedings of the IEEE/CVF International Conference on ComputerVision. 2021: 357-366.
[37] D’ASCOLI S, TOUVRON H, LEAVITT M L, et al. Convit: Improving vision transformers withsoft convolutional inductive biases[C]//International Conference on Machine Learning. PMLR,2021: 2286-2296.
[38] DAI Z, LIU H, LE Q V, et al. Coatnet: Marrying convolution and attention for all data sizes[J].Advances in Neural Information Processing Systems, 2021, 34: 3965-3977.
[39] HU J, SHEN L, SUN G. Squeeze-and-excitation networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018: 7132-7141.
[40] WOO S, PARK J, LEE J Y, et al. Cbam: Convolutional block attention module[C]//Proceedingsof the European Conference on Computer Vision (ECCV). 2018: 3-19.
[41] PARK J, WOO S, LEE J Y, et al. Bam: Bottleneck attention module[A/OL]. 2018. arXiv:1807.06514. https://arxiv.org/abs/1807.06514.
[42] GIRSHICK R, DONAHUE J, DARRELL T, et al. Rich feature hierarchies for accurate objectdetection and semantic segmentation[C]//Proceedings of the IEEE Conference on ComputerVision and Pattern Recognition. 2014: 580-587.
[43] UIJLINGS J R, VAN DE SANDE K E, GEVERS T, et al. Selective search for object recognition[J]. International Journal of Computer Vision, 2013, 104: 154-171.
[44] GIRSHICK R. Fast r-cnn[C]//Proceedings of the IEEE International Conference on ComputerVision. 2015: 1440-1448.
[45] REN S, HE K, GIRSHICK R, et al. Faster r-cnn: Towards real-time object detection with regionproposal networks[J]. Advances in Neural Information Processing Systems, 2015, 28.
[46] REDMON J, DIVVALA S, GIRSHICK R, et al. You only look once: Unifed, real-time objectdetection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016: 779-788.63参考文献
[47] ZHOU B, KHOSLA A, LAPEDRIZA A, et al. Learning deep features for discriminative localization[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2016: 2921-2929.
[48] SELVARAJU R R, COGSWELL M, DAS A, et al. Grad-cam: Visual explanations from deepnetworks via gradient-based localization[C]//Proceedings of the IEEE International Conferenceon Computer Vision. 2017: 618-626.
[49] LI X, WANG W, HU X, et al. Selective kernel networks[C]//Proceedings of the IEEE/CVFConference on Computer Vision and Pattern Recognition. 2019: 510-519.
[50] ZHANG X, HU Y, XIAO Z, et al. Machine learning for cataract classifcation/grading onophthalmic imaging modalities: a survey[J/OL]. Machine Intelligence Research, 2022. DOI:http://doi.org/10.1007/s11633-022-1329-0.
[51] ZHANG X, XIAO Z, HIGASHITA R, et al. Adaptive feature squeeze network for nuclearcataract classifcation in AS-OCT image[J]. Journal of Biomedical Informatics, 2022, 128:104037.
[52] XU L, WANG L, CHENG S, et al. MHANet: A hybrid attention mechanism for retinal diseasesclassifcation[J]. Plos One, 2021, 16(12): e0261285.
[53] ZHANG X, XIAO Z, LI X, et al. Mixed pyramid attention network for nuclear cataract classifcation based on anterior segment OCT images[J]. Health Information Science and Systems,2022, 10(1): 1-12.
[54] LUO W, LI Y, URTASUN R, et al. Understanding the efective receptive feld in deep convolutional neural networks[J]. Advances in Neural Information Processing Systems, 2016, 29.
[55] DING X, ZHANG X, HAN J, et al. Scaling up your kernels to 31x31: Revisiting large kerneldesign in cnns[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and PatternRecognition. 2022: 11963-11975.
[56] XIE X, NIU J, LIU X, et al. A survey on incorporating domain knowledge into deep learningfor medical image analysis[J/OL]. Medical Image Analysis, 2021, 69: 101985. https://www.sciencedirect.com/science/article/pii/S1361841521000311. DOI: https://doi.org/10.1016/j.media.2021.101985.
[57] SRINIVASAN P P, KIM L A, METTU P S, et al. Fully automated detection of diabetic macularedema and dry age-related macular degeneration from optical coherence tomography images[J]. Biomedical Optics Express, 2014, 5(10): 3568-3577.
[58] BUSLAEV A, IGLOVIKOV V I, KHVEDCHENYA E, et al. Albumentations: fast and fexibleimage augmentations[J]. Information, 2020, 11(2): 125.
[59] WIGHTMAN R. PyTorch image models[J/OL]. GitHub Repository, 2019. https://github.com/rwightman/pytorch-image-models. DOI: 10.5281/zenodo.4414861.
[60] RASTI R, RABBANI H, MEHRIDEHNAVI A, et al. Macular OCT classifcation using a multiscale convolutional neural network ensemble[J]. IEEE Transactions on Medical Imaging, 2017,37(4): 1024-1034.
[61] CAI Z, VASCONCELOS N. Cascade r-cnn: Delving into high quality object detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018: 6154-6162.64参考文献
[62] ZHANG H, CHANG H, MA B, et al. Dynamic R-CNN: Towards high quality object detectionvia dynamic training[C]//European Conference on Computer Vision. Springer, 2020: 260-275.
[63] WANG J, ZHANG W, CAO Y, et al. Side-aware boundary localization for more precise objectdetection[C]//European Conference on Computer Vision. Springer, 2020: 403-419.
[64] MCROBERT A P, CAUSER J, VASSILIADIS J, et al. Contextual information infuences diagnosis accuracy and decision making in simulated emergency medicine emergencies[J]. BMJQuality & Safety, 2013, 22(6): 478-484.
[65] LU X, LI B, YUE Y, et al. Grid r-cnn[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019: 7363-7372.
[66] ZHONG Q, LI C, ZHANG Y, et al. Cascade region proposal and global context for deep objectdetection[J]. Neurocomputing, 2020, 395: 170-177.
[67] CHEN Z M, JIN X, ZHAO B R, et al. HCE: Hierarchical Context Embedding for Region-BasedObject Detection[J]. IEEE Transactions on Image Processing, 2021, 30: 6917-6929.
[68] WU Y, CHEN Y, YUAN L, et al. Rethinking classifcation and localization for object detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2020: 10186-10195.
[69] LIN T Y, DOLLÁR P, GIRSHICK R, et al. Feature pyramid networks for object detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2017:2117-2125.
[70] ROSENFELD A, THURSTON M. Edge and curve detection for visual scene analysis[J]. IEEETransactions on Computers, 1971, 100(5): 562-569.
[71] HE K, GKIOXARI G, DOLLÁR P, et al. Mask r-cnn[C]//Proceedings of the IEEE InternationalConference on Computer Vision. 2017: 2961-2969.
[72] CHENG J, HUANG W, CAO S, et al. Enhanced performance of brain tumor classifcation viatumor region augmentation and partition[J]. PloS One, 2015, 10(10): e0140381.
[73] DENG J, DONG W, SOCHER R, et al. Imagenet: A large-scale hierarchical image database[C]//2009 IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 2009: 248-255.
[74] SHAO Q, GONG L, MA K, et al. Attentive CT lesion detection using deep pyramid inferencewith multi-scale booster[C]//Medical Image Computing and Computer Assisted Intervention–MICCAI 2019: 22nd International Conference, Shenzhen, China, October 13–17, 2019, Proceedings, Part VI 22. Springer, 2019: 301-309.
[75] FENG C, ZHONG Y, GAO Y, et al. Tood: Task-aligned one-stage object detection[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. 2021: 3510-3519.