SLPT: Selective Labeling Meets Prompt Tuning on Label-Limited Lesion Segmentation

Bai, Fan1,2,3; Yan, Ke2,3; Bai, Xiaoyu2,3; Mao, Xinyu1; Yin, Xiaoli4; Zhou, Jingren2,3; Shi, Yu4; Lu, Le2; Meng, Max Q.-H.1,5

10.1007/978-3-031-43895-0_2

2023

26th International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI)

0302-9743

1611-3349

978-3-031-43894-3

MEDICAL IMAGE COMPUTING AND COMPUTER ASSISTED INTERVENTION, MICCAI 2023, PT II

14221

OCT 08-12, 2023

null,Vancouver,CANADA

GEWERBESTRASSE 11, CHAM, CH-6330, SWITZERLAND

SPRINGER INTERNATIONAL PUBLISHING AG

Medical image analysis using deep learning is often challenged by limited labeled data and high annotation costs. Fine-tuning the entire network in label-limited scenarios can lead to overfitting and suboptimal performance. Recently, prompt tuning has emerged as a more promising technique that introduces a few additional tunable parameters as prompts to a task-agnostic pre-trained model, and updates only these parameters using supervision from limited labeled data while keeping the pre-trained model unchanged. However, previous work has overlooked the importance of selective labeling in downstream tasks, which aims to select the most valuable downstream samples for annotation to achieve the best performance with minimum annotation cost. To address this, we propose a framework that combines selective labelingwith prompt tuning (SLPT) to boost performance in limited labels. Specifically, we introduce a feature-aware prompt updater to guide prompt tuning and aTandEm Selective LAbeling (TESLA) strategy. TESLA includes unsupervised diversity selection and supervised selection using prompt-based uncertainty. In addition, we propose a diversified visual prompt tuning strategy to provide multi-promptbased discrepant predictions for TESLA. We evaluate our method on liver tumor segmentation and achieve state-of-the-art performance, outperforming traditional fine-tuning with only 6% of tunable parameters, also achieving 94% of full-data performance by labeling only 5% of the data.

Active Learning Prompt Tuning Segmentation

通讯

英语

EI ; CPCI-S

National Key R&D program of China[2019YFB1312400]

Computer Science ; Radiology, Nuclear Medicine & Medical Imaging

Computer Science, Artificial Intelligence ; Computer Science, Theory & Methods ; Radiology, Nuclear Medicine & Medical Imaging

WOS:001109624900002

Web of Science

1.Department of Electronic Engineering, The Chinese University of Hong Kong, Shatin, Hong Kong
2.DAMO Academy, Alibaba Group, Hangzhou, China
3.Hupan Lab, Hangzhou; 310023, China
4.Department of Radiology, Shengjing Hospital of China Medical University, Shenyang; 110004, China
5.Department of Electronic and Electrical Engineering, Southern University of Science and Technology, Shenzhen, China

Bai, Fan,Yan, Ke,Bai, Xiaoyu,et al. SLPT: Selective Labeling Meets Prompt Tuning on Label-Limited Lesion Segmentation[C]. GEWERBESTRASSE 11, CHAM, CH-6330, SWITZERLAND:SPRINGER INTERNATIONAL PUBLISHING AG,2023.