基于眼底图内容的视网膜病变检索研究

得益于数字图像处理技术和深度学习的发展，基于眼底图的计算机辅助视网膜病变筛查已广泛应用于临床，帮助眼科医生提高筛查的敏感性和特异性。然而，目前分类和分割这两个主流任务直接提供自动化决策结果，违背了信息驱动医学范式中医生作为决策主体的原则。因此，本文探索基于内容的图像检索（Content-based Image Retrieval，CBIR）的计算机辅助视网膜病变筛查任务，通过提供过往相似病例作为证据来支持医生循证决策，从而避免计算机自主决策。

为支持视网膜病变筛查递进的三层循证决策，定类、定级、定量，本文按照检索粒度从粗到细开展多病变类别语义检索、单病变级别语义检索、单病变病灶实例检索的表征学习和度量学习研究。多病变包括糖尿病性视网膜病变（Diabetic Retinopathy，DR）、年龄相关性黄斑病变等；单病变则以DR为研究对象，因其是全球劳动年龄人口致盲的主要原因且早期症状的视觉表征不易察觉。语义检索期望返回的图像和查询图像属于同一类别或级别，实例检索则期望二者在病灶大小、数量等病变程度上一致。

本文为CBIR辅助视网膜病变筛查的三个任务提出了四种创新方法，旨在将眼底图中的区分性表征有效编码到低维图像描述符中，并通过计算图像描述符的相似度为医生提供可靠的证据，如下：

（1）多病变语义检索从含有多种疾病的眼底图库中检索到同一类视网膜病变的眼底图，以为定类循证决策提供证据。本文研发了基于注意力的深度哈希网络，以捕捉眼底图中用于判别不同眼部疾病的显著性区域表征，从而增强紧凑哈希码的区分性。本文首先构造基于空间注意力的二元哈希网络，利用二元损失函数优化多疾病的相似度，取得了优异的类别相关性排序性能；进而改良空间注意力机制并构建三元哈希网络，结合三元和交叉熵损失函数同时实现类和哈希码的区分性，进一步提升了排序质量。

（2）如定类中筛出疾病类别为DR，单病变语义检索支持从多级DR眼底图库中检索出同一级DR，以为DR分级提供证据。非增生性DR分级筛查依赖于微血管瘤和视网膜内出血的识别，但二者在视觉表征上区分性低。为有效编码DR病灶表征，本文基于眼底图对称的空间结构，提出了一种基于特征加权的镜像注意力模块，可以捕捉病灶在水平和垂直两个方向上的空间依赖关系；并基于自监督学习框架，提出了一种精细三元损失函数，强化三元组中的三边距离约束；显著提升了眼底图DR语义检索的相关性排序性能，但在其他医学影像上表现不佳。

为此，本文从参数优化角度研发了一种集成谱范数权重衰减（Spectral Norm-based Weight Decay，SNWD）正则项的一致性约束三元（Concordance-constrained Triplet，CCT）损失函数。CCT损失函数通过建模类内和类间相似度的一致性增强数据对模型优化的驱动；SNWD正则项利用谱范数的秩一近似实现权值不同程度的衰减，和损失函数共同影响权值方差偏移；强化了低维图像描述符的判别性，有效提升了其他影像语义检索的相关性排序性能。

（3）DR分级定性评估病变程度，为制定最佳治疗方案，临床上需要进一步定量检查病变程度。单病变实例检索期望检索出病灶高相关性程度的DR眼底图，为定量循证决策提供证据。鉴于DR微血管瘤和视网膜内出血的小、多、散，本文基于像素级语义分割标签自动构建三种相关性程度标签，即病灶的最小外切直径、数量、分布象限数，以探索利用归一化折损累计增益评价相关性程度排序。为有效学习小、多、散病灶的视觉内容及其边缘上下文突变信息，本文在变分自编码器中引入平均差分卷积并利用像素语义分割引导隐向量生成。本文所提出的方法取得了和眼科医生相当的相关性程度排序性能，具有为临床定量循证决策提供证据的实用性。

[1] 李筱荣, 杨千惠. 美国眼科学会《糖尿病视网膜病变临床指南》解读[J]. 中华实验眼科杂志, 2020, 38(9): 795-798.
[2] CHO N H, SHAW J, KARURANGA S, et al. Idf diabetes atlas: Global estimatesof diabetes prevalence for 2017 and projections for 2045[J]. Diabetes research and clinical practice, 2018, 138: 271-281.
[3] SOLOMON S D, GOLDBERG M F. Etdrs grading of diabetic retinopathy: stillthe gold standard?[J]. Ophthalmic research, 2019, 62(4): 190- 195.
[4] YANASE J, TRIANTAPHYLLOU E. A systematic survey of computer-aided di-agnosis in medicine: Past and present developments[J]. Expert Systems with Ap-plications, 2019, 138: 112821.
[5] 郑光远, 刘峡壁, 韩光辉. 医学影像计算机辅助检测与诊断系统综述[J]. 软件学报, 2018, 29(5): 1471- 1514.
[6] AKRAM M U, KHALID S, TARIQ A, et al. Detection and classification of retinallesions for grading of diabetic retinopathy[J]. Computers in biology and medicine,2014, 45: 161- 171.
[7] ZHAO Z, ZHANG K, HAO X, et al. Bira-net: Bilinear attention net for diabeticretinopathy grading[C]//2019 IEEE International Conference on Image Processing(ICIP). IEEE, 2019: 1385- 1389.
[8] ARAÚJO T, ARESTA G, MENDONÇA L, et al. Dr| graduate: Uncertainty-awaredeep learning-based diabetic retinopathy grading in eye fundus images[J]. Medical Image Analysis, 2020, 63: 101715.
[9] AHSAN M A, QAYYUM A, RAZI A, et al. An active learning method for diabeticretinopathy classification with uncertainty quantification[J]. Medical & Biological Engineering & Computing, 2022, 60(10): 2797-2811.
[10] ABDELMAKSOUD E, BARAKAT S, ELMOGY M. Diabetic retinopathy gradingsystem based on transfer learning[J]. International Journal of Advanced ComputerResearch, 2021, 11: 52.
[11] GULSHAN V, RAJAN R P, WIDNER K, et al. Performance of a deep-learningalgorithm vs manual grading for detecting diabetic retinopathy in india[J]. JAMA ophthalmology, 2019, 137(9): 987-993.
[12] MÜLLER H, UNAY D. Retrieval from and understanding of large-scale multi-modal medical datasets: a review[J]. IEEE transactions on multimedia, 2017, 19 (9): 2093-2104.
[13] KUMAR A, KIM J, CAI W, et al. Content-based medical image retrieval: a surveyof applications to multidimensional and multimodality data[J]. Journal of digital imaging, 2013, 26(6): 1025- 1039.
[14] HAMEED I M, ABDULHUSSAIN S H, MAHMMOD B M. Content-based imageretrieval: A review of recent trends[J]. Cogent Engineering, 2021, 8(1): 1927469.
[15] 胡志军, 徐勇. 基于内容的视频检索综述[J]. 计算机科学, 2020, 47(1): 117-123.
[16] YE M, SHEN J, LIN G, et al. Deep learning for person re-identification: A surveyand outlook[J]. IEEE transactions on pattern analysis and machine intelligence, 2021, 44(6): 2872-2893.
[17] ZHANG Y, PAN P, ZHENG Y, et al. Visual search at alibaba[C]//Proceedings ofthe 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2018: 993- 1001.
[18] JING Y, LIU D, KISLYUK D, et al. Visual search at pinterest[C]//Proceedings ofthe 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 2015: 1889- 1898.
[19] KIESEBERG P, WEIPPL E, HOLZINGER A. Trust for the doctor-in-the-loop[J].ERCIM news, 2016, 104(1): 32-33.
[20] KIESEBERG P, SCHANTL J, FRÜHWIRT P, et al. Witnesses for the doctor inthe loop[C]//International Conference on Brain Informatics and Health. Springer, 2015: 369-378.
[21] LITJENS G, KOOI T, BEJNORDI B E, et al. A survey on deep learning in medicalimage analysis[J]. Medical image analysis, 2017, 42: 60-88.
[22] ABDULSAHIB A A, MAHMOUD M A, MOHAMMED M A, et al. Compre-hensive review of retinal blood vessel segmentation and classification techniques: intelligent solutions for green computing in medical images, current challenges,open issues, and knowledge gaps in fundus medical images[J]. Network Modeling Analysis in Health Informatics and Bioinformatics, 2021, 10(1): 1-32.
[23] ZHENG L, YANG Y, TIAN Q. Sift meets cnn: A decade survey of instance re-trieval[J]. IEEE transactions on pattern analysis and machine intelligence, 2017, 40(5): 1224- 1244.
[24] 蒋鹏飞, 李翔, 彭清华, 等. 眼底血管病理学改变特征研究[J]. 国际眼科杂志, 2019, 19(3): 393-397.
[25] 中华医学会糖尿病学分会. 中国 2 型糖尿病防治指南 (2010 年版)[J]. 中国医学前沿杂志, 2011, 3(6): 54- 109.
[26] CHEW E Y, DAVIS M D, DANIS R P, et al. The effects of medical management onthe progression of diabetic retinopathy in persons with type 2 diabetes: the action to control cardiovascular risk in diabetes (accord) eye study[J]. Ophthalmology, 2014, 121(12): 2443-2451.
[27] LOWE D G. Distinctive image features from scale-invariant keypoints[J]. Inter-national journal of computer vision, 2004, 60(2): 91- 110.
[28] SIVIC J, ZISSERMAN A. Video google: A text retrieval approach to object match-ing in videos[C]//Computer Vision, IEEE International Conference on: volume 3.IEEE Computer Society, 2003: 1470- 1470.
[29] BABENKO A, SLESAREV A, CHIGORIN A, et al. Neural codes for image re-trieval[C]//European conference on computer vision. Springer, 2014: 584-599.
[30] LI W J, WANG S, KANG W C. Feature learning based deep supervised hashingwith pairwise labels[C]//Proceedings of the Twenty-Fifth International Joint Con- ference on Artificial Intelligence. 2016: 1711- 1717.
[31] BABENKO A, LEMPITSKY V. Aggregating local deep features for image re-trieval[C]//Proceedings of the IEEE international conference on computer vision. 2015: 1269- 1277.
[32] RAZAVIAN A S, SULLIVAN J, CARLSSON S, et al. Visual instance retrievalwith deep convolutional networks[J]. ITE Transactions on Media Technology andApplications, 2016, 4(3): 251-258.
[33] WANG J, ZHANG T, SEBE N, et al. A survey on learning to hash[J]. IEEE trans-actions on pattern analysis and machine intelligence, 2017, 40(4): 769-790.
[34] SPYROMITROS-XIOUFIS E, PAPADOPOULOS S, KOMPATSIARIS I Y, et al.A comprehensive study over vlad and product quantization in large-scale image retrieval[J]. IEEE Transactions on Multimedia, 2014, 16(6): 1713- 1728.
[35] JIN S, YAO H, SUN X, et al. Deep saliency hashing for fine-grained retrieval[J].IEEE Transactions on Image Processing, 2020, 29: 5336-5351.
[36] DO T T, HOANG T, LE TAN D K, et al. Compact hash code learning with binarydeep neural network[J]. IEEE Transactions on Multimedia, 2019, 22(4): 992- 1004.
[37] WU D, DAI Q, LIU J, et al. Deep incremental hashing network for efficient im-age retrieval[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2019: 9069-9077.
[38] KLEIN B, WOLF L. End-to-end supervised product quantization for image searchand retrieval[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019: 5041-5050.
[39] YU T, YUAN J, FANG C, et al. Product quantization network for fast image re-trieval[C]//Proceedings of the European Conference on Computer Vision (ECCV).2018: 186-201.
[40] XIAO H C, ZHAO W L, LIN J, et al. Deeply activated salient region for instancesearch[J]. ACM Transactions on Multimedia Computing, Communications andApplications, 2022, 18(3s): 1- 19.
[41] NG T, BALNTAS V, TIAN Y, et al. Solar: second-order loss and attention for imageretrieval[C]//European conference on computer vision. Springer, 2020: 253-270.
[42] REVAUD J, ALMAZÁN J, REZENDE R S, et al. Learning with average precision:Training image retrieval with a listwise loss[C]//Proceedings of the IEEE/CVF In- ternational Conference on Computer Vision. 2019: 5107-5116.
[43] HUANG L K, CHEN J, PAN S J. Accelerate learning of deep hashing with gradientattention[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. 2019: 5271-5280.
[44] TOLIAS G, SICRE R, JÉGOU H. Particular object retrieval with integral max-pooling of cnn activations[C]//ICLR 2016-International Conference on LearningRepresentations. 2016: 1- 12.
[45] KALANTIDIS Y, MELLINA C, OSINDERO S. Cross-dimensional weighting foraggregated deep convolutional features[C]//European conference on computer vi- sion. Springer, 2016: 685-701.
[46] GORDO A, ALMAZAN J, REVAUD J, et al. End-to-end learning of deep visualrepresentations for image retrieval[J]. International Journal of Computer Vision,2017, 124(2): 237-254.
[47] RADENOVIĆ F, TOLIAS G, CHUM O. Fine-tuning cnn image retrieval with nohuman annotation[J]. IEEE transactions on pattern analysis and machine intelli-gence, 2018, 41(7): 1655- 1668.
[48] CONJETI S, PASCHALI M, KATOUZIAN A, et al. Deep multiple instance hash-ing for scalable medical image retrieval[C]//International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 2017: 550-558.
[49] CONJETI S, ROY A G, KATOUZIAN A, et al. Hashing with residual networksfor image retrieval[C]//International Conference on Medical Image Computing andComputer-Assisted Intervention. Springer, 2017: 541-549.
[50] CHEN Z, CAI R, LU J, et al. Order-sensitive deep hashing for multimorbiditymedical image retrieval[C]//International Conference on Medical Image Comput- ing and Computer-Assisted Intervention. Springer, 2018: 620-628.
[51] YANG E, YAO D, CAO B, et al. Deep disentangled hashing with momentumtriplets for neuroimage search[C]//International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 2020: 191-201.
[52] ACTON S T, SOLIZ P, RUSSELL S, et al. Content based image retrieval:The foundation for future case-based and evidence-based ophthalmology[C]//2008 IEEE International Conference on Multimedia and Expo. IEEE, 2008: 541-544.
[53] CHAUM E, KARNOWSKI T P, GOVINDASAMY V P, et al. Automated diagnosisof retinopathy by content-based image retrieval[J]. Retina, 2008, 28(10): 1463-1477.
[54] DEEPAK K S, JOSHI G D, SIVASWAMY J. Content-based retrieval of retinalimages for maculopathy[C]//Proceedings of the 1st ACM International Health In- formatics Symposium. 2010: 135- 143.
[55] 杨锋, 魏国辉, 曹慧, 等. 基于内容的医学图像检索研究进展[J]. 激光与光电子学进展, 2020, 57(6): 060003.
[56] 刘桂慧. 基于深度学习的医学图像特征提取方法研究[J]. 信息通信, 2020(7): 3.
[57] 梁洪. 基于内容的医学图像检索及语义建模关键技术研究[D]. 哈尔滨工程大学, 2010.
[58] 张宁. 基于 Hausdorff 距离的医学影像检索方法研究[D]. 哈尔滨理工大学,2019.
[59] 崔少国, 熊舒羽, 刘畅, 等. 基于深度哈希卷积神经网络的医学图像检索[J].重庆理工大学学报: 自然科学, 2020, 34(8): 134- 142.
[60] 李启. 基于深度学习的肺结节图像检索研究与实现[D]. 中北大学, 2019.
[61] LECUN Y, BENGIO Y, HINTON G. Deep learning[J]. nature, 2015, 521(7553):436-444.
[62] ABRÀMOFF M D, LOU Y, ERGINAY A, et al. Improved automated detectionof diabetic retinopathy on a publicly available dataset through integration of deeplearning[J]. Investigative ophthalmology & visual science, 2016, 57(13): 5200-5206.
[63] GULSHAN V, PENG L, CORAM M, et al. Development and validation of adeep learning algorithm for detection of diabetic retinopathy in retinal fundus pho- tographs[J]. Jama, 2016, 316(22): 2402-2410.
[64] GARGEYA R, LENG T. Automated identification of diabetic retinopathy usingdeep learning[J]. Ophthalmology, 2017, 124(7): 962-969.
[65] TING D S W, CHEUNG C Y L, LIM G, et al. Development and validation of adeep learning system for diabetic retinopathy and related eye diseases using retinalimages from multiethnic populations with diabetes[J]. Jama, 2017, 318(22): 2211-2223.
[66] NIRTHIKA R, MANIVANNAN S, RAMANAN A. Siamese network based finegrained classification for diabetic retinopathy grading[J]. Biomedical Signal Pro-cessing and Control, 2022, 78: 103874.
[67] EYEPACS. Diabetic retinopathy detection | kaggle[EB/OL]. 2016. https://www.kaggle.com/competitions/diabetic-retinopathy-detection/data.
[68] LI X, JIANG Y, ZHANG J, et al. Lesion-attention pyramid network for diabeticretinopathy grading[J]. Artificial Intelligence in Medicine, 2022, 126: 102259.
[69] DECENCIERE E, CAZUGUEL G, ZHANG X, et al. Teleophta: Machine learningand image processing methods for teleophthalmology[J]. Irbm, 2013, 34(2): 196-203.
[70] LIU Q, LIU H, ZHAO Y, et al. Dual-branch network with dual-sampling modulateddice loss for hard exudate segmentation in color fundus images[J]. IEEE Journal of Biomedical and Health Informatics, 2021, 26(3): 1091- 1102.
[71] LI T, GAO Y, WANG K, et al. Diagnostic assessment of deep learning algorithmsfor diabetic retinopathy screening[J]. Information Sciences, 2019, 501: 511-522.
[72] PORWAL P, PACHADE S, KOKARE M, et al. Idrid: Diabetic retinopathy–segmentation and grading challenge[J]. Medical image analysis, 2020, 59: 101561.
[73] LIU Q, LIU H, KE W, et al. Automated lesion segmentation in fundus images withmany-to-many reassembly of features[J]. Pattern Recognition, 2023, 136: 109191.
[74] WANG Z, YIN Y, SHI J, et al. Zoom-in-net: Deep mining lesions for diabeticretinopathy detection[C]//International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 2017: 267-275.
[75] GUO C, SZEMENYEI M, HU Y, et al. Channel attention residual u-net for reti-nal vessel segmentation[C]//ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2021: 1185- 1189.
[76] LI L, XU M, WANG X, et al. Attention based glaucoma detection: a large-scaledatabase and cnn model[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2019: 10571- 10580.
[77] HE A, LI T, LI N, et al. Cabnet: category attention block for imbalanced diabeticretinopathy grading[J]. IEEE Transactions on Medical Imaging, 2020, 40(1): 143- 153.
[78] LI X, HU X, YU L, et al. Canet: cross-disease attention network for joint diabeticretinopathy and diabetic macular edema grading[J]. IEEE transactions on medical imaging, 2019, 39(5): 1483- 1493.
[79] WANG Y L, YANG J Y, YANG J Y, et al. Progress of artificial intelligence in dia-betic retinopathy screening[J]. Diabetes/Metabolism Research and Reviews, 2021,37(5): e3414.
[80] ZHOU K, GU Z, LIU W, et al. Multi-cell multi-task convolutional neural networksfor diabetic retinopathy grading[C]//2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). IEEE, 2018: 2724-2727.
[81] LIU S, GONG L, MA K, et al. Green: a graph residual re-ranking network forgrading diabetic retinopathy[C]//International Conference on Medical Image Com- puting and Computer-Assisted Intervention. Springer, 2020: 585-594.
[82] WILKINSON C P, FERRIS III F L, KLEIN R E, et al. Proposed internationalclinical diabetic retinopathy and diabetic macular edema disease severity scales[J].Ophthalmology, 2003, 110(9): 1677- 1682.
[83] SRIVASTAVA R. Integrative computational genomics based approaches to un-cover the tissue-specific regulatory networks in development and disease[M]. In- diana University-Purdue University Indianapolis, 2020.
[84] 邵毅, 周琼. 糖尿病视网膜病变诊治规范——2018 年美国眼科学会临床指南解读[J]. 眼科新进展, 2019, 39(6).
[85] 胡延秋, 杨雪蓝, 葛畅. 糖尿病视网膜病变患者血糖管理的最佳证据总结[J]. 护理学杂志, 2019, 2.
[86] APTOS. Aptos 2019 blindness detection | kaggle[EB/OL]. 2019. https://www.kaggle.com/c/aptos2019-blindness-detection.
[87] QOMARIAH D, NOPEMBER I, TJANDRASA H, et al. Segmentation of microa-neurysms for early detection of diabetic retinopathy using mresunet[J]. Interna- tional Journal of Intelligent Engineering and Systems, 2021, 14(3): 359-373.
[88] ZHAO G, FU D, YANG T. A deep learning method for microaneurysms seg-mentation in fundus images[C]//Proceedings of 2021 Chinese Intelligent Systems Conference. Springer, 2022: 772-780.
[89] TAN J H, FUJITA H, SIVAPRASAD S, et al. Automated segmentation of exu-dates, haemorrhages, microaneurysms using single convolutional neural network[J]. Information sciences, 2017, 420: 66-76.
[90] HUANG G, LIU Z, VAN DER MAATEN L, et al. Densely connected convolutionalnetworks[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2017: 4700-4708.
[91] RONNEBERGER O, FISCHER P, BROX T. U-net: Convolutional networks forbiomedical image segmentation[C]//International Conference on Medical image computing and computer-assisted intervention. Springer, 2015: 234-241.
[92] YOON S, KANG W Y, JEON S, et al. Image-to-image retrieval by learning simi-larity between scene graphs[C]//Proceedings of the AAAI Conference on Artificial Intelligence: volume 35. 2021: 10718- 10726.
[93] ZAEEMZADEH A, GHADAR S, FAIETA B, et al. Face image retrieval with at-tribute manipulation[C]//Proceedings of the IEEE/CVF International Conferenceon Computer Vision. 2021: 12116- 12125.
[94] ZHUANG B, LIN G, SHEN C, et al. Fast training of triplet-based deep binaryembedding networks[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2016: 5955-5964.
[95] GONG Y, LAZEBNIK S, GORDO A, et al. Iterative quantization: A procrusteanapproach to learning binary codes for large-scale image retrieval[J]. IEEE transac- tions on pattern analysis and machine intelligence, 2012, 35(12): 2916-2929.
[96] LAI H, PAN Y, LIU Y, et al. Simultaneous feature learning and hash coding withdeep neural networks[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2015: 3270-3278.
[97] WOO S, PARK J, LEE J Y, et al. Cbam: Convolutional block attention module[C]//Proceedings of the European conference on computer vision (ECCV). 2018: 3- 19.
[98] CHEN L, ZHANG H, XIAO J, et al. Sca-cnn: Spatial and channel-wise attentionin convolutional networks for image captioning[C]//Proceedings of the IEEE con- ference on computer vision and pattern recognition. 2017: 5659-5667.
[99] SLANEY M, CASEY M. Locality-sensitive hashing for finding nearest neighbors[lecture notes][J]. IEEE Signal processing magazine, 2008, 25(2): 128- 131.
[100] TANG Y K, MAO X L, HAO Y J, et al. Locality-sensitive hashing for findingnearest neighbors in probability distributions[C]//Chinese National Conference on Social Media Processing. Springer, 2017: 3- 15.
[101] LI J, CHENG J, YANG F, et al. Losha: A general framework for scalable localitysensitive hashing[C]//Proceedings of the 40th International ACM SIGIR Confer- ence on Research and Development in Information Retrieval. 2017: 635-644.
[102] CONJETI S, KATOUZIAN A, KAZI A, et al. Metric hashing forests[J]. Medicalimage analysis, 2016, 34: 13-29.
[103] ZHU H, LONG M, WANG J, et al. Deep hashing network for efficient similarityretrieval[C]//Proceedings of the AAAI conference on Artificial Intelligence: vol- ume 30. 2016.
[104] LIU H, WANG R, SHAN S, et al. Deep supervised hashing for fast image retrieval[C]//Proceedings of the IEEE conference on computer vision and pattern recogni- tion. 2016: 2064-2072.
[105] NIE D, GAO Y, WANG L, et al. Asdnet: Attention based semi-supervised deepnetworks for medical image segmentation[C]//International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 2018: 370-378.
[106] ABRAHAM N, KHAN N M. A novel focal tversky loss function with improved at- tention u-net for lesion segmentation[C]//2019 IEEE 16th international symposium on biomedical imaging (ISBI 2019). IEEE, 2019: 683-687.
[107] SCHLEMPER J, OKTAY O, SCHAAP M, et al. Attention gated networks: Learn-ing to leverage salient regions in medical images[J]. Medical image analysis, 2019,53: 197-207.
[108] LI G, YU Y. Visual saliency based on multiscale deep features[C]//Proceedings ofthe IEEE conference on computer vision and pattern recognition. 2015: 5455-5463.
[109] ZHAO R, OUYANG W, LI H, et al. Saliency detection by multi-context deeplearning[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2015: 1265- 1274.
[110] LIU N, HAN J. Dhsnet: Deep hierarchical saliency network for salient object de-tection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016: 678-686.
[111] CHOPRA S, HADSELL R, LECUN Y. Learning a similarity metric discrimina-tively, with application to face verification[C]//2005 IEEE Computer Society Con- ference on Computer Vision and Pattern Recognition (CVPR’05): volume 1. IEEE, 2005: 539-546.
[112] WEI X, WANG H, SCOTNEY B, et al. Minimum margin loss for deep face recog-nition[J]. Pattern Recognition, 2020, 97: 107012.
[113] GENTILE C, WARMUTH M K. Linear hinge loss and average margin[J]. Ad-vances in neural information processing systems, 1998, 11.
[114] SCHROFF F, KALENICHENKO D, PHILBIN J. Facenet: A unified embeddingfor face recognition and clustering[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2015: 815-823.
[115] WANG X, SHI Y, KITANI K M. Deep supervised hashing with triplet labels[C]//Asian conference on computer vision. Springer, 2016: 70-84.
[116] SUN Y, CHENG C, ZHANG Y, et al. Circle loss: A unified perspective of pair sim-ilarity optimization[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020: 6398-6407.
[117] HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2016: 770-778.
[118] IOFFE S, SZEGEDY C. Batch normalization: Accelerating deep network train-ing by reducing internal covariate shift[C]//International conference on machine learning. PMLR, 2015: 448-456.
[119] NAIR V, HINTON G E. Rectified linear units improve restricted boltzmann ma-chines[C]//Proceedings of the 27th international conference on machine learning(ICML- 10). 2010: 807-814.
[120] ZHOU B, KHOSLA A, LAPEDRIZA A, et al. Learning deep features for discrimi-native localization[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2016: 2921-2929.
[121] KRIZHEVSKY A, SUTSKEVER I, HINTON G E. Imagenet classification withdeep convolutional neural networks[C]//Advances in neural information processing systems. 2012: 1097- 1105.
[122] LIN T Y, GOYAL P, GIRSHICK R, et al. Focal loss for dense object detection[C]//Proceedings of the IEEE international conference on computer vision. 2017:2980-2988.
[123] JOHNSON J, DOUZE M, JÉGOU H. Billion-scale similarity search with gpus[J].IEEE Transactions on Big Data, 2019, 7(3): 535-547.
[124] SCHMIDT-ERFURTH U, SADEGHIPOUR A, GERENDAS B S, et al. Artificialintelligence in retina[J]. Progress in retinal and eye research, 2018, 67: 1-29.
[125] LI T, BO W, HU C, et al. Applications of deep learning in fundus images: A review[J]. Medical Image Analysis, 2021, 69: 101971.
[126] SILVA W, POELLINGER A, CARDOSO J S, et al. Interpretability-guided content-based medical image retrieval[C]//International Conference on MedicalImage Computing and Computer-Assisted Intervention. Springer, 2020: 305-314.
[127] NOH H, ARAUJO A, SIM J, et al. Large-scale image retrieval with attentive deeplocal features[C]//Proceedings of the IEEE international conference on computer vision. 2017: 3456-3465.
[128] KOCH G, ZEMEL R, SALAKHUTDINOV R, et al. Siamese neural networks forone-shot image recognition[C]//ICML deep learning workshop: volume 2. Lille,2015: 0.
[129] HOFFER E, AILON N. Deep metric learning using triplet network[C]// International workshop on similarity-based pattern recognition. Springer, 2015:84-92.
[130] HU J, SHEN L, SUN G. Squeeze-and-excitation networks[C]//Proceedings of theIEEE conference on computer vision and pattern recognition. 2018: 7132-7141.
[131] QILONG W, BANGGU W, PENGFEI Z, et al. Eca-net: Efficient channel atten- tion for deep convolutional neural networks.[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2020.
[132] WANG X, GIRSHICK R, GUPTA A, et al. Non-local neural networks[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2018: 7794-7803.
[133] LIU Z, MAO H, WU C Y, et al. A convnet for the 2020s[C]//Proceedings ofthe IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022: 11976- 11986.
[134] BELLO I, ZOPH B, VASWANI A, et al. Attention augmented convolutional net-works[C]//Proceedings of the IEEE/CVF international conference on computer vi- sion. 2019: 3286-3295.
[135] CHEN J, HE T, ZHUO W, et al. Tvconv: Efficient translation variant convolution for layout-aware visual processing[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022: 12548- 12558.
[136] PHAISANGITTISAGUL E. An analysis of the regularization between l2 anddropout in single hidden layer neural network[C]//2016 7th International Confer- ence on Intelligent Systems, Modelling and Simulation (ISMS). IEEE, 2016: 174- 179.
[137] ZHANG C, BENGIO S, HARDT M, et al. Understanding deep learning (still)requires rethinking generalization[J]. Communications of the ACM, 2021, 64(3): 107- 115.
[138] ZHAO X, QI H, LUO R, et al. A weakly supervised adaptive triplet loss fordeep metric learning[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops. 2019: 0-0.
[139] KRIZHEVSKY A, HINTON G, et al. Learning multiple layers of features fromtiny images[J]. Handbook of Systemic Autoimmune Diseases, 2009, 1(4).
[140] DENG J, DONG W, SOCHER R, et al. Imagenet: A large-scale hierarchical imagedatabase[C]//2009 IEEE conference on computer vision and pattern recognition.Ieee, 2009: 248-255.
[141] KORNBLITH S, SHLENS J, LE Q V. Do better imagenet models transfer bet-ter?[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2019: 2661-2671.
[142] HUANG J, RATHOD V, SUN C, et al. Speed/accuracy trade-offs for modern con-volutional object detectors[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2017: 7310-7311.
[143] CHEN L C, COLLINS M, ZHU Y, et al. Searching for efficient multi-scale archi-tectures for dense image prediction[J]. Advances in neural information processing systems, 2018, 31.
[144] CHENG G, LAI P, GAO D, et al. Class attention network for image recognition [J/OL]. SCIENCE CHINA Information Sciences, 2022: 1. DOI: https://doi.org/ 10.1007/s11432-021-3493-7.
[145] LI Z, SUN Y, ZHANG L, et al. Ctnet: Context-based tandem network for semanticsegmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2021, 44(12): 9904-9917.
[146] CHEN C F R, FAN Q, PANDA R. Crossvit: Cross-attention multi-scale visiontransformer for image classification[C]//Proceedings of the IEEE/CVF Interna- tional Conference on Computer Vision. 2021: 357-366.
[147] GUO J, HAN K, WU H, et al. Cmt: Convolutional neural networks meet visiontransformers[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022: 12175- 12185.
[148] BABILONI F, MARRAS I, SLABAUGH G, et al. Tesa: Tensor element self-attention via matricization[C]//Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition. 2020: 13945- 13954.
[149] WANG H, WANG Y, ZHOU Z, et al. Cosface: Large margin cosine loss for deepface recognition[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2018: 5265-5274.
[150] WANG J, ZHOU F, WEN S, et al. Deep metric learning with angular loss[C]//Proceedings of the IEEE international conference on computer vision. 2017: 2593-2601.
[151] QIAN Q, SHANG L, SUN B, et al. Softtriple loss: Deep metric learning with-out triplet sampling[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. 2019: 6450-6458.
[152] GE W. Deep metric learning with hierarchical triplet loss[C]//Proceedings of theEuropean Conference on Computer Vision (ECCV). 2018: 269-285.
[153] EBRAHIMPOUR M K, QIAN G, BEACH A. Multi-head deep metric learningusing global and local representations[C]//Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. 2022: 3031-3040.
[154] KIM S, KIM D, CHO M, et al. Proxy anchor loss for deep metric learning[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recog- nition. 2020: 3238-3247.
[155] WU C Y, MANMATHA R, SMOLA A J, et al. Sampling matters in deep embed-ding learning[C]//Proceedings of the IEEE International Conference on Computer Vision. 2017: 2840-2848.
[156] OH SONG H, XIANG Y, JEGELKA S, et al. Deep metric learning via lifted struc-tured feature embedding[C]//Proceedings of the IEEE conference on computer vi- sion and pattern recognition. 2016: 4004-4012.
[157] SOHN K. Improved deep metric learning with multi-class n-pair loss objective[J].Advances in neural information processing systems, 2016, 29.
[158] DO T T, TRAN T, REID I, et al. A theoretically sound upper bound on the tripletloss for improving the efficiency of deep distance metric learning[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019: 10404- 10413.
[159] LI T, WU B, YANG Y, et al. Compressing convolutional neural networks viafactorized convolutional filters[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019: 3977-3986.
[160] SWAMINATHAN S, GARG D, KANNAN R, et al. Sparse low rank factorizationfor deep neural network compression[J]. Neurocomputing, 2020, 398: 185- 196.
[161] WANG M, LIU B, FOROOSH H. Factorized convolutional neural networks[C]//Proceedings of the IEEE International Conference on Computer Vision Workshops.2017: 545-553.
[162] KHODAK M, TENENHOLTZ N, MACKEY L, et al. Initialization and regular-ization of factorized neural layers[C]//9th International Conference on Learning Representations, ICLR. 2021.
[163] GRAY G, CROWLEY E J, STORKEY A. Separable layers enable structured effi-cient linear substitutions[J]. arXiv preprint arXiv:1906.00859, 2019.
[164] KOSSAIFI J, LIPTON Z C, KOLBEINSSON A, et al. Tensor regression networks[J]. Journal of Machine Learning Research, 2020, 21: 1-21.
[165] KESKAR N S, NOCEDAL J, TANG P T P, et al. On large-batch training for deeplearning: Generalization gap and sharp minima[C]//5th International Conference on Learning Representations, ICLR. 2017.
[166] YOSHIDA Y, MIYATO T. Spectral norm regularization for improving the gener-alizability of deep learning[J]. stat, 2017, 1050: 31.
[167] MIYATO T, KATAOKA T, KOYAMA M, et al. Spectral normalization for gen-erative adversarial networks[C]//6th International Conference on Learning Repre- sentations, ICLR. 2018.
[168] MOHRI M, ROSTAMIZADEH A, TALWALKAR A. Foundations of machinelearning[M]. MIT press, 2018.
[169] SHAMIR O. Fast stochastic algorithms for svd and pca: Convergence propertiesand convexity[C]//International Conference on Machine Learning. PMLR, 2016:248-256.
[170] SANDLER M, HOWARD A, ZHU M, et al. Mobilenetv2: Inverted residuals andlinear bottlenecks[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2018: 4510-4520.
[171] FOWLKES E B, MALLOWS C L. A method for comparing two hierarchical clus-terings[J]. Journal of the American statistical association, 1983, 78(383): 553-569.
[172] NGUYEN H Q, LAM K, LE L T, et al. Vindr-cxr: An open dataset of chest x-rays with radiologist ’s annotations[J]. Scientific Data, 2022, 9(1).
[173] LOYMAN M, GREENSPAN H. Lung nodule retrieval using semantic similarityestimates[C]//Medical Imaging 2019: Computer-Aided Diagnosis: volume 10950. International Society for Optics and Photonics, 2019: 109503P.
[174] SU Z, LIU W, YU Z, et al. Pixel difference networks for efficient edge detection[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.2021: 5117-5127.
[175] ÖZTÜRK Ş . Stacked auto-encoder based tagging with deep features for content-based medical image retrieval[J]. Expert Systems with Applications, 2020, 161:113693.
[176] ÖZTÜRK Ş . Two-stage sequential losses based automatic hash code generationusing siamese network[J]. Avrupa Bilim ve Teknoloji Dergisi, 2020: 39-46.
[177] JIMENEZ A, ALVAREZ J M, GIRO-I NIETO X. Class-weighted convolutionalfeatures for visual instance search[C]//Proceedings of the 28th British Machine Vi- sion Conference 2017. 2017: 1- 12.
[178] MOHEDANO E, MCGUINNESS K, O’CONNOR N E, et al. Bags of local con-volutional features for scalable instance search[C]//Proceedings of the 2016 ACM on International Conference on Multimedia Retrieval. 2016: 327-331.
[179] MOHEDANO E, MCGUINNESS K, GIRÓ-I NIETO X, et al. Saliency weightedconvolutional features for instance search[C]//2018 international conference oncontent-based multimedia indexing (CBMI). IEEE, 2018: 1-6.
[180] LASKAR Z, KANNALA J. Context aware query image representation for partic-ular object retrieval[C]//Scandinavian Conference on Image Analysis. Springer, 2017: 88-99.
[181] CAO J, LIU L, WANG P, et al. Where to focus: Query adaptive match-ing for instance retrieval using convolutional feature maps[J]. arXiv preprintarXiv:1606.06811, 2016.
[182] SALVADOR A, GIRÓ-I NIETO X, MARQUÉS F, et al. Faster r-cnn features forinstance search[C]//Proceedings of the IEEE conference on computer vision and pattern recognition workshops. 2016: 9- 16.
[183] KIM J, YOON S E. Regional attention based deep feature for image retrieval.[C]//BMVC. 2018: 209.
[184] TSCHANNEN M, BACHEM O, LUCIC M. Recent advances in autoencoder-basedrepresentation learning[J]. arXiv preprint arXiv:1812.05069, 2018.
[185] VAN DEN OORD A, VINYALS O, et al. Neural discrete representation learning[J]. Advances in neural information processing systems, 2017, 30.
[186] WU H, FLIERL M. Learning product codebooks using vector-quantized autoen-coders for image retrieval[C]//2019 IEEE Global Conference on Signal and Infor-mation Processing (GlobalSIP). IEEE, 2019: 1-5.
[187] JIN G, ZHANG Y, LU K. Deep hashing based on vae-gan for efficient similarityretrieval[J]. Chinese Journal of Electronics, 2019, 28(6): 1191- 1197.
[188] DOERSCH C. Tutorial on variational autoencoders[J]. stat, 2016, 1050: 13.
[189] ZHANG M, XIAO T Z, PAIGE B, et al. Improving vae-based representation learn-ing[J]. arXiv preprint arXiv:2205. 14539, 2022.
[190] NISHIMAKI K, IKUTA K, ONGA Y, et al. Loc-vae: Learning structurally local-ized representation from 3d brain mr images for content-based image retrieval[C]// 2022 IEEE International Conference on Systems, Man, and Cybernetics (SMC). IEEE, 2022: 2433-2438.
[191] LEE K, LEE Y, KO H H, et al. A study on the channel expansion vae for content-based image retrieval[J]. Applied Sciences, 2022, 12(18): 9160.
[192] WANG X, DU Y, YANG S, et al. Retccl: clustering-guided contrastive learningfor whole-slide image retrieval[J]. Medical Image Analysis, 2022: 102645.
[193] ALVES C, TRAINA A J. Variational autoencoders for medical image retrieval[C]//2022 International Conference on INnovations in Intelligent SysTems and Appli- cations (INISTA). IEEE, 2022: 1-6.
[194] AHONEN T, HADID A, PIETIKAINEN M. Face description with local binarypatterns: Application to face recognition[J]. IEEE transactions on pattern analysis and machine intelligence, 2006, 28(12): 2037-2041.
[195] JAIN A K, FARROKHNIA F. Unsupervised texture segmentation using gabor fil-ters[J]. Pattern recognition, 1991, 24(12): 1167- 1186.
[196] JUEFEI-XU F, NARESH BODDETI V, SAVVIDES M. Local binary convolu-tional neural networks[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2017: 19-28.
[197] LUAN S, CHEN C, ZHANG B, et al. Gabor convolutional networks[J]. IEEETransactions on Image Processing, 2018, 27(9): 4357-4366.
[198] YU F, KOLTUN V. Multi-scale context aggregation by dilated convolutions[J].arXiv preprint arXiv:1511.07122, 2015.
[199] DAI J, QI H, XIONG Y, et al. Deformable convolutional networks[C]//Proceedingsof the IEEE international conference on computer vision. 2017: 764-773.
[200] YU Z, ZHAO C, WANG Z, et al. Searching central difference convolutional networks for face anti-spoofing[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020: 5295-5305.
[201] YU Z, QIN Y, ZHAO H, et al. Dual-cross central difference network for face anti-spoofing[C]//ZHOU Z H. Proceedings of the Thirtieth International Joint Con- ference on Artificial Intelligence, IJCAI-21. International Joint Conferences on Artificial Intelligence Organization, 2021: 1281- 1287.
[202] AKBAR S, SHARIF M, AKRAM M U, et al. Automated techniques for bloodvessels segmentation through fundus retinal images: A review[J]. Microscopy re- search and technique, 2019, 82(2): 153- 170.
[203] JIN K, HUANG X, ZHOU J, et al. Fives: A fundus image dataset for artificialintelligence based vessel segmentation[J]. Scientific Data, 2022, 9(1): 1-8.
[204] FU Y, CHEN J, LI J, et al. Optic disc segmentation by u-net and probability bubblein abnormal fundus images[J]. Pattern Recognition, 2021, 117: 107971.
[205] HASAN M K, ALAM M A, ELAHI M T E, et al. Drnet: Segmentation and lo-calization of optic disc and fovea from diabetic retinopathy image[J]. Artificial Intelligence in Medicine, 2021, 111: 102001.
[206] GARIFULLIN A, LENSU L, UUSITALO H. Deep bayesian baseline for segment-ing diabetic retinopathy lesions: Advances and challenges[J]. Computers in Biol- ogy and Medicine, 2021, 136: 104725.
[207] HUANG S, LI J, XIAO Y, et al. Rtnet: relation transformer network for diabeticretinopathy multi-lesion segmentation[J]. IEEE Transactions on Medical Imaging,2022, 41(6): 1596- 1607.
[208] ROYCHOWDHURY S, KOOZEKANANI D D, PARHI K K. Iterative vessel seg-mentation of fundus images[J]. IEEE Transactions on Biomedical Engineering,2015, 62(7): 1738- 1749.
[209] HE A, WANG K, LI T, et al. Progressive multi-scale consistent network for multi-class fundus lesion segmentation[J]. IEEE Transactions on Medical Imaging, 2022.
[210] LI Y, ZHU M, SUN G, et al. Weakly supervised training for eye fundus lesionsegmentation in patients with diabetic retinopathy[J]. Mathematical Biosciences and Engineering, 2022, 19(5): 5293-5311.
[211] KINGMA D P, WELLING M. Auto-encoding variational bayes[J]. stat, 2014,1050: 1.