DCAMIL: Eye-tracking guided dual-cross-attention multi-instance learning for refining fundus disease detection

Jiang，Hongyang1; Gao，Mengdi2; Huang，Jingqi3; Tang，Chen4; Zhang，Xiaoqing3; Liu，Jiang1,3,4,5,6

2024-06-01

10.1016/j.eswa.2023.122889

Expert Systems with Applications   影响因子和分区

0957-4174

Deep neural networks (DNNs) have facilitated the development of computer-aided diagnosis (CAD) systems for fundus diseases, helping ophthalmologists to reduce missed diagnoses and misdiagnosis rates. However, the majority of CAD systems are data-driven, but lack the prior medical knowledge that can be performance-friendly. In this regard, we innovatively proposed a human-in-the-loop (HITL) CAD system by leveraging ophthalmologists’ eye-tracking information. Concretely, the HITL CAD system was implemented on the multi-instance learning (MIL), where clinicians’ gaze maps were beneficial to cherry-pick diagnosis-related instances. Furthermore, the dual-cross-attention MIL (DCAMIL) network was utilized to curb the adverse effects of noisy instances. Meanwhile, both the sequence augmentation (SA) module and the domain adversarial network (DAN) were introduced to enrich and standardize the instances in the training bag, respectively, thereby enhancing the robustness of our method. We conduct comparative experiments on our newly-constructed datasets (namely, AMD-Gaze and DR-Gaze) for the AMD and early DR detection, respectively. Rigorous experiments demonstrate the feasibility of our HITL CAD system and the superiority of the proposed DCAMIL, which fully exploits ophthalmologists’ eye-tracking information. These investigations indicate that clinicians’ gaze maps, as prior medical knowledge, is potential to contribute to the CAD systems of clinical diseases.

Computer-aided diagnosis Eye-tracking Fundus disease Human-in-the-loop Multi-instance learning

SCI

英语

第一 ; 通讯

ENGINEERING

2-s2.0-85180403846

Scopus

1.Department of Computer Science and Engineering,Southern University of Science and Technology,Shenzhen,518055,China
2.Department of Biomedical Engineering,Chinese University of Hong Kong,Hong Kong
3.Research Institute of Trustworthy Autonomous Systems,Southern University of Science and Technology,Shenzhen,518055,China
4.School of Ophthalmology and Optometry and Eye Hospital,Wenzhou Medical University,Wenzhou,325027,China
5.Singapore Eye Research Institute,Singapore National Eye Centre,Singapore,169856,Singapore
6.Guangdong Provincial Key Laboratory of Brain-inspired Intelligent Computation,Department of Computer Science and Engineering,Southern University of Science and Technology,Shenzhen,518055,China

Jiang，Hongyang,Gao，Mengdi,Huang，Jingqi,et al. DCAMIL: Eye-tracking guided dual-cross-attention multi-instance learning for refining fundus disease detection[J]. Expert Systems with Applications,2024,243.

Jiang，Hongyang,Gao，Mengdi,Huang，Jingqi,Tang，Chen,Zhang，Xiaoqing,&Liu，Jiang.(2024).DCAMIL: Eye-tracking guided dual-cross-attention multi-instance learning for refining fundus disease detection.Expert Systems with Applications,243.

Jiang，Hongyang,et al."DCAMIL: Eye-tracking guided dual-cross-attention multi-instance learning for refining fundus disease detection".Expert Systems with Applications 243(2024).