One step further into the blackbox: a pilot study of how to build more confidence around an AI-based decision system of breast nodule assessment in 2D ultrasound

Dong, Fajin1,2,3,4; She, Ruilian5; Cui, Chen1,2,3; Shi, Siyuan1,2,3; Hu, Xuqiao4; Zeng, Jieying4; Wu, Huaiyu4; Xu, Jinfeng4; Zhang, Yun1,2,3

2021

10.1007/s00330-020-07561-7

EUROPEAN RADIOLOGY   影响因子和分区

0938-7994

1432-1084

Objectives To investigate how a DL model makes decisions in lesion classification with a newly defined region of evidence (ROE) by incorporating "explainable AI" (xAI) techniques. Methods A data set of 785 2D breast ultrasound images acquired from 367 females. The DenseNet-121 was used to classify whether the lesion is benign or malignant. For performance assessment, classification results are evaluated by calculating accuracy, sensitivity, specificity, and receiver operating characteristic for experiments of both coarse and fine regions of interest (ROIs). The area under the curve (AUC) was evaluated, and the true-positive, false-positive, true-negative, and false-negative results with breakdown in high, medium, and low resemblance on test sets were also reported. Results The two models with coarse and fine ROIs of ultrasound images as input achieve an AUC of 0.899 and 0.869, respectively. The accuracy, sensitivity, and specificity of the model with coarse ROIs are 88.4%, 87.9%, and 89.2%, and with fine ROIs are 86.1%, 87.9%, and 83.8%, respectively. The DL model captures ROE with high resemblance of physicians' consideration as they assess the image. Conclusions We have demonstrated the effectiveness of using DenseNet to classify breast lesions with limited quantity of 2D grayscale ultrasound image data. We have also proposed a new ROE-based metric system that can help physicians and patients better understand how AI makes decisions in reading images, which can potentially be integrated as a part of evidence in early screening or triaging of patients undergoing breast ultrasound examinations.

Artificial intelligence Ultrasonography Breast neoplasms

SCI

英语

通讯

Medical Science and Technology Research Foundation of Guangdong[B2019045]

Radiology, Nuclear Medicine & Medical Imaging

Radiology, Nuclear Medicine & Medical Imaging

WOS:000605578600018

SPRINGER

CLINICAL MEDICINE

Web of Science

1.Shandong Univ, Qilu Hosp, Key Lab Cardiovasc Remodeling & Funct Res, Chinese Minist Educ, 107 Wenhuaxi Rd, Jinan 250012, Peoples R China
2.Shandong Univ, Qilu Hosp, Key Lab Cardiovasc Remodeling & Funct Res, Chinese Minist Hlth & State, 107 Wenhuaxi Rd, Jinan 250012, Peoples R China
3.Shandong Univ, Qilu Hosp, Key Lab Cardiovasc Remodeling & Funct Res, Shandong Prov Joint Key Lab Translat Cardiovasc M, 107 Wenhuaxi Rd, Jinan 250012, Peoples R China
4.Jinan Univ, Southern Univ Sci & Technol, Clin Med Coll 2, Dept Ultrasound,Shenzhen Peoples,Affiliated Hosp, Shenzhen 518020, Peoples R China
5.Jinan Univ, Shenzhen Peoples Hosp, Clin Med Coll 2, Dept Obstet & Gynecol, Shenzhen 518020, Peoples R China

Dong, Fajin,She, Ruilian,Cui, Chen,et al. One step further into the blackbox: a pilot study of how to build more confidence around an AI-based decision system of breast nodule assessment in 2D ultrasound[J]. EUROPEAN RADIOLOGY,2021.

Dong, Fajin.,She, Ruilian.,Cui, Chen.,Shi, Siyuan.,Hu, Xuqiao.,...&Zhang, Yun.(2021).One step further into the blackbox: a pilot study of how to build more confidence around an AI-based decision system of breast nodule assessment in 2D ultrasound.EUROPEAN RADIOLOGY.

Dong, Fajin,et al."One step further into the blackbox: a pilot study of how to build more confidence around an AI-based decision system of breast nodule assessment in 2D ultrasound".EUROPEAN RADIOLOGY (2021).