多任务一致性半监督方法在医学图像分析中的应用

多年来，深度学习理论在医学图像分析领域取得了卓越的进展，但是仍有一些问题需要解决。从数据的角度来说，医学图像数据呈现有标签数据少、无标签数据多的特点。如何使用半监督方法对有标签和无标签的医学图像数据进行有效利用是当前医学图像分析领域的重要问题。近些年，基于自然图像数据集的半监督学习领域出现了很多高效的算法。然而，这些算法对于医学图像数据集并没有表现出与其在自然图像数据集中同样卓越的性能。本文就此问题从多任务协同训练和一致性正则化的思路出发，探索用于医学影像分析具体任务的半监督算法。

本文的主要研究内容和贡献如下：

一是针对心脏左心房分割任务存在标注代价高、数据量少、边界不清晰等特点，本文设计了一种多任务一致性的半监督方法。方法中的主任务是常规的图像分割任务，两个辅助任务为边缘感知预测和中心感知预测任务，通过多任务协同训练提高模型学习数据表征的能力。两个辅助任务的有监督标签是通过对主任务的有监督标签解耦得到。同时方法从任务间一致性正则化的策略出发，通过对两个辅助任务的预测结果进行转换得到组合预测结果，然后拉近组合预测结果与图像分割结果进行之间的距离。此方法可以充分利用来自无标签数据的丰富信息，提升模型的有效性和鲁棒性。最后，实验表明本文所提出的方法在心脏左心房分割数据集上得到了相较于现有半监督方法更加优秀的表现结果，并对预测结果进行可视化显示。

二是针对骨龄评估任务提出了双任务双一致性正则化的半监督骨龄评估方法。方法结合了数据增强一致性和任务间一致性正则化的策略以及单阶段多任务协同训练策略。在双任务协同训练方面，为骨龄评估设计了分类与回归任务协同训练的框架，提供了一种骨龄分类和回归的任务转换方法。在双一致性正则化方面，完成了分类任务和回归任务间一致性正则化方法的具体设计，并提出组合使用数据增强一致性和任务间一致性正则化方法来挖掘无标签数据的信息。实验所提出的半监督框架在骨龄评估问题上实现了相比先进方法更好的性能，同时在对比实验中探究了算法各个部分的作用情况，通过实验证明了本文所提出方法的有效性。

[1] FUJITA H. AI-based computer-aided diagnosis: the latest review to read first[J]. Radiological Physics and Technology, 2020, 13(1): 6-19.
[2] LECUN Y, BOTTOU L, BENGIO Y, et al. Gradient-based learning applied to document recognition[J]. Proceedings of the IEEE, 1998, 86(11): 2278-2324.
[3] KRIZHEVSKY A, SUTSKEVER I, HINTON G E. Imagenet classification with deep convolutional neural networks[C]//Advances in Neural Information Processing Systems. 2012: 1106-1114.
[4] SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[C]//International Conference on Learning Representations. 2015.
[5] SZEGEDY C, LIU W, JIA Y, et al. Going deeper with convolutions[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2015: 1-9.
[6] IOFFE S, SZEGEDY C. Batch normalization: accelerating deep network training by reducing internal covariate shift[C]//International Conference on Machine Learning. PMLR, 2015: 448-456.
[7] SZEGEDY C, VANHOUCKE V, IOFFE S, et al. Rethinking the inception architecture for computer vision[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016: 2818-2826.
[8] HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016: 770-778.
[9] HUANG G, LIU Z, VAN DER MAATEN L, et al. Densely connected convolutional networks [C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2017: 4700-4708.
[10] LUO X, CHEN J, SONG T, et al. Semi-supervised medical image segmentation through sualtask consistency[C]//Proceedings of the AAAI Conference on Artificial Intelligence: volume 35.2021: 8801-8809.
[11] ESCOBAR M, GONZÁLEZ C, TORRES F, et al. Hand pose estimation for pediatric bone age assessment[C]//International Conference on Medical Image Computing and Computer-assisted Intervention. Springer, 2019: 531-539.
[12] LIU Q, YU L, LUO L, et al. Semi-supervised medical image classification with relation-driven self-ensembling model[J]. IEEE Transactions on Medical Imaging, 2020, 39(11): 3429-3440.
[13] LEE D H. Pseudo-label: the simple and effcient semi-supervised learning method for deep neural networks[C]//Workshop on Challenges in Representation Learning, ICML: volume 3.2013.
[14] RASMUS A, BERGLUND M, HONKALA M, et al. Semi-supervised learning with ladder networks[C]//Advances in Neural Information Processing Systems. 2015: 3546-3554.
[15] LAINE S, AILA T. Temporal ensembling for semi-supervised learning[C]//5th International Conference on Learning Representations. 2017.
[16] TARVAINEN A, VALPOLA H. Mean teachers are better role models: weight-averaged consistency targets improve semi-supervised deep learning results[C]//Advances in Neural Information Processing Systems. 2017: 1195-1204.
[17] MIYATO T, MAEDA S I, KOYAMA M, et al. Virtual adversarial training: a regularization method for supervised and semi-supervised learning[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 41(8): 1979-1993.
[18] XIE Q, LUONG M T, HOVY E, et al. Self-training with noisy student improves imagenet classification[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020: 10687-10698.
[19] VERMA V, LAMB A, KANNALA J, et al. Interpolation consistency training for semisupervised learning[C]//Proceedings of the 28th International Joint Conference on Artificial Intelligence. 2019: 3635-3641.
[20] BERTHELOT D, CARLINI N, GOODFELLOW I, et al. Mixmatch: a holistic approach to semisupervised learning[C]//Advances in Neural Information Processing Systems. 2019: 5049-5059.
[21] WANG X, KIHARA D, LUO J, et al. EnAET: a self-trained framework for semi-supervised and supervised learning with ensemble transformations[J]. IEEE Transactions on Image Processing,2020, 30: 1639-1647.
[22] XIE Q, DAI Z, HOVY E, et al. Unsupervised data augmentation for consistency training[J]. Advances in Neural Information Processing Systems, 2020, 33: 6256-6268.
[23] ZHAI X, OLIVER A, KOLESNIKOV A, et al. S4l: self-supervised semi-supervised learning [C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. 2019: 1476-1485.
[24] YU L, WANG S, LI X, et al. Uncertainty-aware self-ensembling model for semi-supervised 3D left atrium segmentation[C]//International Conference on Medical Image Computing and Computer-assisted Intervention. Springer, 2019: 605-613.
[25] BERTHELOT D, CARLINI N, CUBUK E D, et al. Remixmatch: semi-supervised learning with distribution matching and augmentation anchoring[C]//International Conference on Learning Representations. 2019.
[26] CHEN T, KORNBLITH S, SWERSKY K, et al. Big self-supervised models are strong semisupervised learners[J]. Advances in Neural Information Processing Systems, 2020, 33: 22243-22255.
[27] SOHN K, BERTHELOT D, CARLINI N, et al. Fixmatch: simplifying semi-supervised learning with consistency and confidence[J]. Advances in Neural Information Processing Systems, 2020, 33: 596-608.
[28] HU Z, YANG Z, HU X, et al. Simple: similar pseudo label exploitation for semi-supervised classification[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021: 15099-15108.
[29] CUBUK E D, ZOPH B, MANE D, et al. Autoaugment: learning augmentation strategies from data[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019: 113-123.
[30] DEVRIES T, TAYLOR G W. Improved regularization of convolutional neural networks with cutout[J]. CoRR, 2017, abs/1708.04552.
[31] YOU X, PENG Q, YUAN Y, et al. Segmentation of retinal blood vessels using the radial projection and semi-supervised approach[J]. Pattern Recognition, 2011, 44(10-11): 2314-2324.
[32] PORTELA N M, CAVALCANTI G D, REN T I. Semi-supervised clustering for MR brain image segmentation[J]. Expert Systems with Applications, 2014, 41(4): 1492-1497.
[33] XIONG Z, XIA Q, HU Z, et al. A global benchmark of algorithms for segmenting the left atrium from late gadolinium-enhanced cardiac magnetic resonance imaging[J]. Medical Image Analysis, 2021, 67: 101832.
[34] ZHUANG X, SHEN J. Multi-scale patch and multi-modality atlases for whole heart segmentation of mri[J]. Medical Image Analysis, 2016: 77-87.
[35] ZHUANG X. Challenges and methodologies of fully automatic whole heart segmentation: a review[J]. Journal of Healthcare Engineering, 2013, 4(3): 371-407.
[36] TRAN P V. A fully convolutional neural network for cardiac segmentation in short-axis MRI[J]. CoRR, 2016, abs/1604.00494.
[37] XU Z, WU Z, FENG J. CFUN: combining faster R-CNN and U-net network for effcient whole heart segmentation[J]. CoRR, 2018, abs/1812.04914.
[38] KHENED M, KOLLERATHU V A, KRISHNAMURTHI G. Fully convolutional multi-scale residual DenseNets for cardiac segmentation and automated cardiac diagnosis using ensemble of classifiers[J]. Medical Image Analysis, 2019, 51: 21-45.
[39] MILLETARI F, NAVAB N, AHMADI S A. V-net: fully convolutional neural networks for volumetric medical image segmentation[C]//2016 Fourth International Conference on 3D Vision.IEEE, 2016: 565-571.
[40] LI C, XU C, GUI C, et al. Level set evolution without re-initialization: a new variational formulation[C]//2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition: volume 1. IEEE, 2005: 430-436.
[41] WANG Y, WEI X, LIU F, et al. Deep distance transform for tubular structure segmentation in ct scans[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020: 3833-3842.
[42] LI X, YU L, CHEN H, et al. Transformation-consistent self-ensembling model for semisupervised medical image segmentation[J]. IEEE Transactions on Neural Networks and Learning Systems, 2020, 32(2): 523-534.
[43] OUALI Y, HUDELOT C, TAMI M. Semi-supervised semantic segmentation with crossconsistency training[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020: 12674-12684.
[44] RONNEBERGER O, FISCHER P, BROX T. U-net: convolutional networks for biomedical image segmentation[C]//International Conference on Medical Image Computing and Computerassisted Intervention. Springer, 2015: 234-241.
[45] ANTONELLI M, REINKE A, BAKAS S, et al. The medical segmentation decathlon[J]. CoRR, 2021, abs/2106.05735.
[46] QIAN N. On the momentum term in gradient descent learning algorithms[J]. Neural Networks, 1999, 12(1): 145-151.
[47] ROBBINS H, MONRO S. A stochastic approximation method[J]. The Annals of Mathematical Statistics, 1951: 400-407.
[48] DICE L R. Measures of the amount of ecologic association between species[J]. Ecology, 1945, 26(3): 297-302.
[49] PASZKE A, GROSS S, MASSA F, et al. Pytorch: an imperative style, high-performance deep learning library[C]//Advances in Neural Information Processing Systems. 2019: 8024-8035.
[50] GARN S M. Radiographic atlas of skeletal development of the hand and wrist[J]. American Journal of Human Genetics, 1959, 11(3): 282.
[51] 中华人民共和国国家体育总局. 中国青少年儿童手腕骨成熟度及评价方法: 行业标准[Z]. 2016.
[52] TANNER J, HEALY M, GOLDSTEIN H, et al. Assessment of skeletal maturity and prediction of adult height: TW3 method saunders[J]. London, UK, 2001: 1-110.
[53] CICERO M, BILBILY A. Machine learning and the future of radiology: how we won the 2017 rsna ml challenge[Z]. 2017.
[54] IGLOVIKOV V I, RAKHLIN A, KALININ A A, et al. Paediatric bone age assessment using deep convolutional neural networks[C]//Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support: volume 11045. Springer, 2018: 300-308.
[55] PAN X, ZHAO Y, CHEN H, et al. Fully automated bone age assessment on large-scale hand X-ray dataset[J]. International Journal of Biomedical Imaging, 2020: 1-12.
[56] SZEGEDY C, IOFFE S, VANHOUCKE V, et al. Inception-v4, inception-resnet and the impact of residual connections on learning[C]//Thirty-first AAAI Conference on Artificial Intelligence. 2017.
[57] LIU C, XIE H, ZHANG Y. Self-supervised attention mechanism for pediatric bone age assessment with effcient weak annotation[J]. IEEE Transactions on Medical Imaging, 2020, 40(10): 2685-2697.
[58] HALABI S S, PREVEDELLO L M, KALPATHY-CRAMER J, et al. The RSNA pediatric bone age machine learning challenge[J]. Radiology, 2019, 290(2): 498-503.
[59] TSCHANDL P, ROSENDAHL C, KITTLER H. The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions[J]. Scientific Data,2018, 5(1): 1-9.
[60] WANG X, PENG Y, LU L, et al. Chestx-ray8: hospital-scale chest x-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2017: 2097-2106.
[61] KINGMA D P, BA J. Adam: a method for stochastic optimization[C]//International Conference on Learning Representations. 2015.