CT2US: Cross-modal transfer learning for kidney segmentation in ultrasound images with synthesized data

Song, Yuxin1,2; Zheng, Jing2,3; Lei, Long3,4; Ni, Zhipeng2,3; Zhao, Baoliang1,3,4,5; Hu, Ying1,3,4,5

2022-05-01

10.1016/j.ultras.2022.106706

ULTRASONICS   影响因子和分区

0041-624X

1874-9968

Accurate segmentation of kidney in ultrasound images is a vital procedure in clinical diagnosis and interventional operation. In recent years, deep learning technology has demonstrated promising prospects in medical image analysis. However, due to the inherent problems of ultrasound images, data with annotations are scarce and arduous to acquire, hampering the application of data-hungry deep learning methods. In this paper, we propose cross-modal transfer learning from computerized tomography (CT) to ultrasound (US) by leveraging annotated data in the CT modality. In particular, we adopt cycle generative adversarial network (CycleGAN) to synthesize US images from CT data and construct a transition dataset to mitigate the immense domain discrepancy between US and CT. Mainstream convolutional neural networks such as U-Net, U-Res, PSPNet, and DeepLab v3+ are pretrained on the transition dataset and then transferred to real US images. We first trained CNN models on a data set composed of 50 ultrasound images and validated them on a validation set composed of 30 ultrasound images. In addition, we selected 82 ultrasound images from another hospital to construct a cross site data set to verify the generalization performance of the models. The experimental results show that with our proposed transfer learning strategy, the segmentation accuracy in dice similarity coefficient (DSC) reaches 0.853 for U-Net, 0.850 for U-Res, 0.826 for PSPNet and 0.827 for DeepLab v3+ on the cross-site test set. Compared with training from scratch, the accuracy improvement was 0.127, 0.097, 0.105 and 0.036 respectively. Our transfer learning strategy effectively improves the accuracy and generalization ability of ultrasound image segmentation model with limited training data.

Cross modality Kidney segmentation Transfer learning Ultrasound imaging

SCI ; EI

英语

其他

National Natural Science Foundation of China[12026604,"U1813204",62003330] ; Key Fundamental Research Program of Shenzhen["JCYJ20200109115201707","JCYJ20200109112818703","JCYJ20200109114233670"]

Acoustics ; Radiology, Nuclear Medicine & Medical Imaging

Acoustics ; Radiology, Nuclear Medicine & Medical Imaging

WOS:000790826400006

ELSEVIER

CLINICAL MEDICINE

Web of Science

1.Chinese Acad Sci, Shenzhen Inst Adv Technol, Beijing 518055, Peoples R China
2.Southern Univ Sci & Technol, Clin Coll Jinan Univ 2, Shenzhen Peoples Hosp, Dept Ultrasound,Affiliated Hosp 1, Shenzhen 518020, Peoples R China
3.Pazhou Lab, Guangzhou 510320, Peoples R China
4.Chinese Acad Sci, Shenzhen Inst Adv Technol, Shenzhen 518055, Peoples R China
5.Univ Chinese Acad Sci, Beijing 100039, Peoples R China

Song, Yuxin,Zheng, Jing,Lei, Long,et al. CT2US: Cross-modal transfer learning for kidney segmentation in ultrasound images with synthesized data[J]. ULTRASONICS,2022,122.

Song, Yuxin,Zheng, Jing,Lei, Long,Ni, Zhipeng,Zhao, Baoliang,&Hu, Ying.(2022).CT2US: Cross-modal transfer learning for kidney segmentation in ultrasound images with synthesized data.ULTRASONICS,122.

Song, Yuxin,et al."CT2US: Cross-modal transfer learning for kidney segmentation in ultrasound images with synthesized data".ULTRASONICS 122(2022).