基于眼科OCTA图像的脑疾病分析——痴呆症智能辅助诊断

痴呆症是一种以认知功能障碍和日常活动能力减退为主要表现的临床综合征，是全球重要的公共卫生问题。痴呆症的病程不可逆，及早发现并采取措施控制病情将有助于提升患者的生活质量。但受限于大脑结构和功能的复杂性，目前缺乏对脑部的直接观测手段，导致现有痴呆症辅助诊断方法存在局限性。 有研究表明，眼和脑在组织来源上具有同源性，在结构和功能机制上具有相似性。相比于脑部检查的复杂性，眼部检查具有无创、无放射性、采集方便等独特优势。近年来，已有研究通过眼科图像研究痴呆症，并发现痴呆症会影响视网膜不同区域血流情况。光学相干层析血流成像（Optical coherence tomography angiography，OCTA）是一种视网膜血流成像技术，可以获取多投影OCTA图像用于痴呆症辅助诊断。然而OCTA图像中血管分布复杂，临床参数获取难度大，并且仅用单投影的OCTA图像难以全面地捕获视网膜血流情况。针对上述问题，本文基于眼科OCTA图像对痴呆症的早期辅助诊断展开了深入的研究，主要的工作内容如下： 首先，为了更加精确地提取血管相关特征用于痴呆症的分析，本文提出了一种视网膜血管检测方法。该方法提出了血管连接模块和多尺度学习模块，用于缓解末端血管分割困难的问题，更好地检测不同尺度的血管。之后，通过特征提取与分析，发现痴呆症患者在多层投影的OCTA图像上均存在血管面积密度和长度密度降低等现象，验证了痴呆症可能与视网膜血管系统的异常有关。 其次，由于痴呆症患者在多投影图像上均存在异常，单投影OCTA图像难以全面反映疾病的特征。为了联合多投影OCTA图像用于痴呆症早期辅助诊断，本文提出了一种多投影一致性互补性学习网络。该方法通过一致性和互补性注意力模块和跨视图融合模块，结合不同投影的多尺度特征，有效利用多投影OCTA图像的信息用于痴呆症诊断。 最后，为了缓解多投影OCTA图像中一致性和互补性信息不平衡的挑战，同时尽可能去除含有无关细节的噪声输入数据，本文提出了基于信息瓶颈理论的多投影学习方法。该方法提出了多投影信息瓶颈平衡损失，以重构多投影OCTA图像的一致性互补性信息。此外，设计了用于多投影学习的三元组网络结构，用于建模OCTA图像的全局结构分布一致性和局部病灶互补性。

[1] HUGHES S, YANG H, CHAN-LING T. Vascularization of the human fetal retina: roles of vasculogenesis and angiogenesis[J]. Investigative ophthalmology & visual science, 2000, 41(5): 1217-1228.
[2] VAHIA V N. Diagnostic and statistical manual of mental disorders 5: A quick glance[J]. Indian journal of psychiatry, 2013, 55(3): 220.
[3] REN R, QI J, LIN S, et al. The China Alzheimer Report 2022[J]. General Psychiatry, 2022, 35(1).
[4] JIA L, DU Y, CHU L, et al. Prevalence, risk factors, and management of dementia and mild cognitive impairment in adults aged 60 years or older in China: a cross-sectional study[J]. The Lancet Public Health, 2020, 5(12): e661-e671.
[5] 贾建平, 王荫华, 李焰生, 等. 中国痴呆与认知障碍诊治指南 (二): 痴呆分型及诊断标准[J].中华医学杂志, 2011, 91(10): 651-655.
[6] MEIBURGER K M, SALVI M, ROTUNNO G, et al. Automatic segmentation and classification methods using optical coherence tomography angiography (OCTA): a review and handbook[J]. Applied Sciences, 2021, 11(20): 9734.
[7] BULUT M, KURTULUŞ F, GÖZKAYA O, et al. Evaluation of optical coherence tomography angiographic findings in Alzheimer’s type dementia[J]. British Journal of Ophthalmology, 2018, 102(2): 233-237.
[8] ZHANG Y S, ZHOU N, KNOLL B M, et al. Parafoveal vessel loss and correlation between peripapillary vessel density and cognitive performance in amnestic mild cognitive impairment and early Alzheimer’s disease on optical coherence tomography angiography[J]. PloS one, 2019, 14(4): e0214685.
[9] YOON S P, THOMPSON A C, POLASCIK B W, et al. Correlation of OCTA and volumetric MRI in mild cognitive impairment and Alzheimer’s disease[J]. Ophthalmic Surgery, Lasers and Imaging Retina, 2019, 50(11): 709-718.
[10] SALOBRAR-GARCIA E, MÉNDEZ-HERNÁNDEZ C, HOZ R D, et al. Ocular vascularchanges in mild Alzheimer’s disease patients: Foveal avascular zone, choroidal thickness, and ONH hemoglobin analysis[J]. Journal of personalized medicine, 2020, 10(4): 231.
[11] ROBBINS C B, THOMPSON A C, BHULLAR P K, et al. Characterization of retinal microvascular and choroidal structural changes in Parkinson disease[J]. JAMA ophthalmology, 2021, 139(2): 182-188.
[12] SHI C, CHEN Y, KWAPONG W R, et al. Characterization by fractal dimension analysis of the retinal capillary network in Parkinson disease[J]. Retina, 2020, 40(8): 1483-1491.
[13] MUTLU U, COLIJN J M, IKRAM M A, et al. Association of retinal neurodegeneration on optical coherence tomography with dementia: a population-based study[J]. JAMA neurology, 2018, 75(10): 1256-1263.
[14] ZHANG S, KWAPONG W R, YANG T, et al. Choriocapillaris Changes Are Correlated With Disease Duration and MoCA Score in Early-Onset Dementia[J]. Frontiers in Aging Neuroscience, 2021, 13: 192.
[15] MOONS L, DE GROEF L. Multimodal retinal imaging to detect and understand Alzheimer’s and Parkinson’s disease[J]. Current opinion in neurobiology, 2022, 72: 1-7.
[16] LAU A Y, MOK V, LEE J, et al. Retinal image analytics detects white matter hyperintensitiesin healthy adults[J]. Annals of clinical and translational neurology, 2019, 6(1): 98-105.
[17] JEENA R, SUKESH KUMAR A, MAHADEVAN K. Stroke diagnosis from retinal fundus images using multi texture analysis[J]. Journal of Intelligent & Fuzzy Systems, 2019, 36(3): 2025-2032.
[18] RAVEENDRAN SUSHA J. Computation of retinal fundus parameters for stroke prediction[J]. Computer Methods in Biomechanics and Biomedical Engineering: Imaging & Visualization, 2020, 8(4): 374-381.
[19] TIAN J, SMITH G, GUO H, et al. Modular machine learning for Alzheimer’s disease classification from retinal vasculature[J]. Scientific Reports, 2021, 11(1): 1-11.
[20] NUNES A, SILVA G, DUQUE C, et al. Retinal texture biomarkers may help to discriminate between Alzheimer’s, Parkinson’s, and healthy controls[J]. PloS one, 2019, 14(6): e0218826.
[21] JEENA R, SHINY G, SUKESH KUMAR A, et al. A Comparative analysis of stroke diagnosis from retinal images using hand-crafted features and CNN[J]. Journal of Intelligent & Fuzzy Systems, 2021(Preprint): 1-9.
[22] SANDEEP C, KUMAR A S, MAHADEVAN K, et al. Classification of OCT images for the early diagnosis of Alzheimer’s disease[C]//2017 International Conference on Intelligent Computing and Control (I2C2). IEEE, 2017: 1-5.
[23] MA Y, HAO H, XIE J, et al. ROSE: a retinal OCT-angiography vessel segmentation dataset and new model[J]. IEEE transactions on medical imaging, 2020, 40(3): 928-939.
[24] WISELY C E, WANG D, HENAO R, et al. Convolutional neural network to identify symptomatic Alzheimer’s disease using multimodal retinal imaging[J]. British Journal of Ophthalmology, 2022, 106(3): 388-395.
[25] DE FAUW J, LEDSAM J R, ROMERA-PAREDES B, et al. Clinically applicable deep learning for diagnosis and referral in retinal disease[J]. Nature medicine, 2018, 24(9): 1342-1350.
[26] RAJKOMAR A, DEAN J, KOHANE I. Machine learning in medicine[J]. New England Journal of Medicine, 2019, 380(14): 1347-1358.
[27] BADAR M, HARIS M, FATIMA A. Application of deep learning for retinal image analysis: A review[J]. Computer Science Review, 2020, 35: 100203.
[28] YANAGIHARA R T, LEE C S, TING D S W, et al. Methodological challenges of deep learning in optical coherence tomography for retinal diseases: a review[J]. Translational Vision Science & Technology, 2020, 9(2): 11-11.
[29] CHEN X, HOU P, JIN C, et al. Quantitative analysis of retinal layer optical intensities on three-dimensional optical coherence tomography[J]. Investigative ophthalmology & visual science, 2013, 54(10): 6846-6851.
[30] SUN M, ZHANG Z, MA C, et al. Quantitative analysis of retinal layers on three-dimensional spectral-domain optical coherence tomography for pituitary adenoma[J]. PloS one, 2017, 12(6): e0179532.
[31] SUN Z, CHEN H, SHI F, et al. An automated framework for 3D serous pigment epithelium detachment segmentation in SD-OCT images[J]. Scientific reports, 2016, 6(1): 1-10.
[32] GUO J, ZHU W, SHI F, et al. A framework for classification and segmentation of branch retinal artery occlusion in SD-OCT[J]. IEEE Transactions on Image Processing, 2017, 26(7): 3518-3527.
[33] CHENG J, LIU J, XU Y, et al. Superpixel classification based optic disc and optic cup segmentation for glaucoma screening[J]. IEEE transactions on medical imaging, 2013, 32(6): 1019-1032.
[34] TAN N M, XU Y, GOH W B, et al. Robust multi-scale superpixel classification for optic cup localization[J]. Computerized Medical Imaging and Graphics, 2015, 40: 182-193.
[35] LIU Z, WANG C, CAI X, et al. Discrimination of Diabetic Retinopathy From Optical Coherence Tomography Angiography Images Using Machine Learning Methods[J]. IEEE access, 2021, 9: 51689-51694.
[36] GULSHAN V, PENG L, CORAM M, et al. Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs[J]. Jama, 2016, 316(22): 2402-2410.
[37] FU H, CHENG J, XU Y, et al. Joint optic disc and cup segmentation based on multi-label deep network and polar transformation[J]. IEEE transactions on medical imaging, 2018, 37(7): 1597-1605.
[38] ORLANDO J I, PROKOFYEVA E, DEL FRESNO M, et al. An ensemble deep learning based approach for red lesion detection in fundus images[J]. Computer methods and programs in biomedicine, 2018, 153: 115-127.
[39] ZHAO M, HAMARNEH G. Retinal Image Classification via Vasculature-Guided Sequential Attention[C]//2019 IEEE/CVF International Conference on Computer Vision Workshop (IC CVW). IEEE Computer Society, 2019: 381-387.
[40] SALMA A, BUSTAMAM A, YUDANTHA A R, et al. Diabetic Retinopathy Detection and Classification Using GoogleNet and Attention Mechanism Through Fundus Images[J]. Turkish Journal of Computer and Mathematics Education (TURCOMAT), 2021, 12(14): 590-597.
[41] WANG W, XU Z, YU W, et al. Two-stream CNN with loose pair training for multimodal AMD categorization[C]//International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 2019: 156-164.
[42] HEISLER M, KARST S, LO J, et al. Ensemble deep learning for diabetic retinopathy detection using optical coherence tomography angiography[J]. Translational Vision Science & Technology, 2020, 9(2): 20-20.
[43] HUA R, XIONG J, LI G, et al. Development and validation of a deep learning algorithm based on fundus photographs for estimating the CAIDE dementia risk score[J]. Age and Ageing, 2022, 51(12): 282.
[44] THAKOOR K, BORDBAR D, YAO J, et al. Hybrid 3d-2d Deep Learning For Detection Of Neovascularage-Related Macular Degeneration Using Optical Coherence Tomography B-Scans And Angiography Volumes[C]//2021 IEEE 18th International Symposium on Biomedical Imaging (ISBI). IEEE, 2021: 1600-1604.
[45] HE X, DENG Y, FANG L, et al. Multi-Modal Retinal Image Classification With Modality-Specific Attention Network[J]. IEEE Transactions on Medical Imaging, 2021, 40(6): 1591-1602.
[46] LI M, CHEN Y, JI Z, et al. Image projection network: 3D to 2D image segmentation in OCTA images[J]. IEEE Transactions on Medical Imaging, 2020, 39(11): 3343-3354.
[47] LIN L, WANG Z, WU J, et al. BSDA-Net: A Boundary Shape and Distance Aware Joint Learning Framework for Segmenting and Classifying OCTA Images[C]//International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 2021: 65-75.
[48] HUANG D, SWANSON E A, LIN C P, et al. Optical coherence tomography[J]. science, 1991, 254(5035): 1178-1181.
[49] LE D, ALAM M N, LIM J I, et al. Deep learning for objective OCTA detection of diabetic retinopathy[C]//Ophthalmic Technologies: volume 11218. SPIE, 2020: 98-103.
[50] ZANG P, GAO L, HORMEL T T, et al. Dcardnet: Diabetic retinopathy classification at multiple levels based on structural and angiographic optical coherence tomography[J]. IEEE Transactions on Biomedical Engineering, 2020, 68(6): 1859-1870.
[51] DE CARLO T E, ROMANO A, WAHEED N K, et al. A review of optical coherence tomography angiography (OCTA)[J]. International journal of retina and vitreous, 2015, 1: 1-15.
[52] PELLEGRINI M, VAGGE A, FERRO DESIDERI L, et al. Optical coherence tomography angiography in neurodegenerative disorders[J]. Journal of clinical medicine, 2020, 9(6): 1706.
[53] ALAM M, LE D, SON T, et al. AV-Net: deep learning for fully automated artery-vein classification in optical coherence tomography angiography[J]. Biomedical optics express, 2020, 11(9): 5249.
[54] AOYAMA Y, MARUKO I, KAWANO T, et al. Diagnosis of central serous chorioretinopathy by deep learning analysis of en face images of choroidal vasculature: A pilot study[J]. PLoS One, 2021, 16(6): e0244469.
[55] PRENTAŠIĆ P, HEISLER M, MAMMO Z, et al. Segmentation of the foveal microvasculature using deep learning networks[J]. Journal of biomedical optics, 2016, 21(7): 075008-075008.
[56] MOU L, ZHAO Y, CHEN L, et al. CS-Net: channel and spatial attention network for curvilinear structure segmentation[C]//Medical Image Computing and Computer Assisted Intervention–MICCAI 2019: 22nd International Conference, Shenzhen, China, October 13–17, 2019, Proceedings, Part I 22. Springer, 2019: 721-730.
[57] PISSAS T, BLOCH E, CARDOSO M J, et al. Deep iterative vessel segmentation in OCT angiography[J]. Biomedical Optics Express, 2020, 11(5): 2490-2510.
[58] GIARRATANO Y, BIANCHI E, GRAY C, et al. Automated segmentation of optical coherence tomography angiography images: benchmark data and clinically relevant metrics[J]. Translational vision science & technology, 2020, 9(13): 5-5.
[59] LO J, HEISLER M, VANZAN V, et al. Microvasculature segmentation and intercapillary area quantification of the deep vascular complex using transfer learning[J]. Translational Vision Science & Technology, 2020, 9(2): 38-38.
[60] KIRZHEVSKY A, SUTSKEVER I, HINTON G E. Imagenet classification with deep convolutional neural networks[J]. Advances in neural information processing systems, 2012, 25: 1097-1105.
[61] DENG J, DONG W, SOCHER R, et al. Imagenet: A large-scale hierarchical image database[C]//2009 IEEE conference on computer vision and pattern recognition. Ieee, 2009: 248-255.
[62] AJIT A, ACHARYA K, SAMANTA A. A review of convolutional neural networks[C]//2020 international conference on emerging trends in information technology and engineering (ic ETITE). IEEE, 2020: 1-5.
[63] HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2016: 770-778.
[64] LONG J, SHELHAMER E, DARRELL T. Fully convolutional networks for semantic segmentation[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2015: 3431-3440.
[65] ROSENBERGER O, FISCHER P, BROX T. U-net: Convolutional networks for biomedical image segmentation[C]//International Conference on Medical image computing and computer-assisted intervention. Springer, 2015: 234-241.
[66] RAO A, PARK J, WOO S, et al. Studying the effects of self-attention for medical image analysis[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. 2021: 3416-3425.
[67] GUO M H, XU T X, LIU J J, et al. Attention mechanisms in computer vision: A survey[J]. Computational Visual Media, 2022, 8(3): 331-368.
[68] CHEN L, ZHANG H, XIAO J, et al. Sca-cnn: Spatial and channel-wise attention in convolutional networks for image captioning[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2017: 5659-5667.
[69] HU J, SHEN L, SUN G. Squeeze-and-excitation networks[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2018: 7132-7141.
[70] MNIH V, HEESS N, GRAVES A, et al. Recurrent models of visual attention[J]. Advances in neural information processing systems, 2014, 27: 2204-2212.
[71] JADERBERG M, SIMONYAN K, ZISSERMAN A, et al. Spatial transformer networks[J]. Advances in neural information processing systems, 2015, 28: 2017-2025.
[72] WANG X, GIRSHICK R, GUPTA A, et al. Non-local neural networks[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2018: 7794-7803.
[73] WOO S, PARK J, LEE J Y, et al. Cbam: Convolutional block attention module[C]//Proceedings of the European conference on computer vision (ECCV). 2018: 3-19.
[74] PARK J, WOO S, LEE J Y, et al. Bam: Bottleneck attention module[A/OL]. 2018. arXiv: 1807.06514. https://arxiv.org/abs/1807.06514.
[75] HAN K, WANG Y, CHEN H, et al. A survey on vision transformer[J]. IEEE transactions on pattern analysis and machine intelligence, 2022, 45(1): 87-110.
[76] MA Y, HAO H, XIE J, et al. ROSE: A Retinal OCT-Angiography Vessel Segmentation Dataset and New Model[J]. IEEE transactions on medical imaging, 2020, 40(3): 928-939.
[77] CHENG H X, HAN X F, XIAO G Q. CENet: Toward Concise and Efficient LiDAR Semantic Segmentation for Autonomous Driving[M]//2022 IEEE International Conference on Multimedia and Expo (ICME). IEEE, 2022: 01-06.
[78] PIAO S, LIU J. Accuracy Improvement of UNet Based on Dilated Convolution[M]//Journal of Physics Conference Series: volume 1345. 2019: 052066.
[79] CAO Y, LIU S, PENG Y, et al. DenseUNet: densely connected UNet for electron microscopy image segmentation[J]. IET Image Processing, 2020, 14(12): 2682-2689.
[80] ZHUANG J. LadderNet: Multi-path networks based on U-Net for medical image segmentation[A/OL]. 2018. arXiv: 1810.07810. http://arxiv.org/abs/1810.07810.
[81] OKTAY O, SCHLEMPER J, FOLGOC L L, et al. Attention U-Net: Learning Where to Look for the Pancreas[A/OL]. 2018. arXiv: 1804.03999. https://arxiv.org/abs/1804.03999.
[82] XING G, CHEN L, WANG H, et al. Multi-scale pathological fluid segmentation in OCT with a novel curvature loss in convolutional neural network.[J]. IEEE Transactions on Medical Imaging, 2022, 41(6): 1547-1559.
[83] RONNEBERGER O, FISCHER P, BROX T. U-Net: Convolutional Networks for Biomedical Image Segmentation[J]. medical image computing and computer assisted intervention, 2015: 234-241.
[84] ZHAO Y, ZHANG J, PEREIRA E, et al. Automated tortuosity analysis of nerve fibers in corneal confocal microscopy[J]. IEEE transactions on medical imaging, 2020, 39(9): 2725-2737.
[85] DE JESUS D A, BREA L S, BREDA J B, et al. OCTA multilayer and multisector peripapillary microvascular modeling for diagnosing and staging of glaucoma[J]. Translational Vision Science & Technology, 2020, 9(2): 58-58.
[86] FU H, GENG Y, ZHANG C, et al. Red-nets: Redistribution networks for multi-view classification[J]. Information Fusion, 2021, 65: 119-127.
[87] LI S Y, JIANG Y, ZHOU Z H. Partial multi-view clustering[C]//Proceedings of the AAAI conference on artificial intelligence: volume 28. 2014: 1968-1974.
[88] CHEN C F R, FAN Q, PANDA R. Crossvit: Cross-attention multi-scale vision transformer for image classification[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. 2021: 357-366.
[89] YAN S, XIONG X, ARNAB A, et al. Multiview Transformers for Video Recognition[M]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022: 3333-3343.
[90] HERMESSI H, MOURALI O, ZAGROUBA E. Multimodal medical image fusion review: Theoretical background and recent advances[J]. Signal Processing, 2021, 183: 108036.
[91] WANG X, SHU K, KUANG H, et al. The Role of Spatial Alignment in Multimodal Medical Image Fusion Using Deep Learning for Diagnostic Problems[C]//2021 the 3rd International Conference on Intelligent Medicine and Health. 2021: 40-46.
[92] ZHOU T, CANU S, VERA P, et al. 3D Medical Multi-modal Segmentation Network Guided by Multi-source Correlation Constraint[C]//2020 25th International Conference on Pattern Recognition (ICPR). IEEE, 2021: 10243-10250.
[93] ZHANG N, DING S, LIAO H, et al. Multimodal correlation deep belief networks for multi-view classification[J]. Applied Intelligence, 2019, 49(5): 1925-1936.
[94] SELVARAJU R R, COGSWELL M, DAS A, et al. Grad-cam: Visual explanations from deep networks via gradient-based localization[C]//Proceedings of the IEEE international conference on computer vision. 2017: 618-626.
[95] LYNCH C J, LISTON C. New machine-learning technologies for computer-aided diagnosis[J]. Nature medicine, 2018, 24(9): 1304-1305.
[96] LIU J, JIANG Y, LI Z, et al. Partially shared latent factor learning with multiview data[J]. IEEE transactions on neural networks and learning systems, 2014, 26(6): 1233-1246.
[97] HARDOON D R, SZEDMAK S, SHAWE-TAYLOR J. Canonical correlation analysis: An overview with application to learning methods[J]. Neural computation, 2004, 16(12): 2639-2664.
[98] TISHBY N, ZASLAVSKY N. Deep learning and the information bottleneck principle[C]//2015 ieee information theory workshop (itw). IEEE, 2015: 1-5.
[99] WAN Z, ZHANG C, ZHU P, et al. Multi-View Information-Bottleneck Representation Learning[C]//Proceedings of the AAAI Conference on Artificial Intelligence: volume 35. 2021: 10085-10092.
[100] SMITH S M, JENKINSON M, WOOLRICH M W, et al. Advances in functional and structural MR image analysis and implementation as FSL[J]. Neuroimage, 2004, 23: 208-219.