Embracing imperfect datasets: A review of deep learning solutions for medical image segmentation

Medical Image Analysis - Tập 63 - Trang 101693 - 2020
Nima Tajbakhsh1, Laura Jeyaseelan1, Qian Li1, Jeffrey N. Chiang1, Zhihao Wu1, Xiaowei Ding1
1VoxelCloud, Inc., United States

Tóm tắt

Từ khóa


Tài liệu tham khảo

Abhishek, 2019, Mask2lesion: mask-constrained adversarial skin lesion image synthesis, 71

Agustsson, 2017, Ntire 2017 challenge on single image super-resolution: dataset and study

Alex, 2017, Semisupervised learning using denoising autoencoders for brain lesion detection and segmentation, J. Med. Imaging, 4, 041311, 10.1117/1.JMI.4.4.041311

Alom, M. Z., Hasan, M., Yakopcic, C., Taha, T. M., Asari, V. K., 2018. Recurrent residual convolutional neural network based on u-net (r2u-net) for medical image segmentation. arXiv:1802.06955.

Angermann, C., Haltmeier, M., Steiger, R., Pereverzyev Jr, S., Gizewski, E., 2019. Projection-based 2.5 d u-net architecture for fast volumetric segmentation. arXiv:1902.00347.

Bai, 2019, Self-supervised learning for cardiac mr image segmentation by anatomical position prediction, 541

Bai, 2017, Semi-supervised learning for network-based cardiac mr image segmentation, 253

Bai, 2018, Recurrent neural networks for aortic image sequence segmentation with sparse annotations, 586

Baur, 2017, Semi-supervised deep learning for fully convolutional networks, 311

Bokhorst, J.-M., Pinckaers, H., van Zwam, P., Nagtegaal, I., van der Laak, J., Ciompi, F., 2018. Learning from sparsely annotated data for semantic segmentation in histopathology images.

Bortsova, 2019, Semi-supervised medical image segmentation via learning consistency under transformations, 810

Boykov, 2001, Interactive graph cuts for optimal boundary & region segmentation of objects in nd images, 1, 105

Cai, 2016, Pancreas segmentation in MRI using graph-based decision fusion on convolutional neural networks, 442

Cai, J., Tang, Y., Lu, L., Harrison, A. P., Yan, K., Xiao, J., Yang, L., Summers, R. M., 2018. Accurate weakly supervised deep lesion segmentation on ct scans: self-paced 3d mask generation from recist. arXiv:1801.08614.

Cai, 2019, End-to-end adversarial shape learning for abdomen organ deep segmentation, 124

Campanella, 2019, Clinical-grade computational pathology using weakly supervised deep learning on whole slide images, Nat. Med., 10.1038/s41591-019-0508-1

Can, 2018, Learning to segment medical images with scribble-supervision alone, 236

Chaitanya, 2019, Semi-supervised and task-driven data augmentation, 29

Chartsias, 2017, Adversarial image synthesis for unpaired multi-modal cardiac data, 3

Chartsias, 2018, Factorised spatial representation learning: application in semi-supervised myocardial segmentation, 490

Chen, 2018, Semantic-aware generative adversarial nets for unsupervised domain adaptation in chest x-ray segmentation, 143

Chen, 2019, Synergistic image and feature adaptation: towards cross-modality domain adaptation for medical image segmentation, 33, 865

Chen, C., Ouyang, C., Tarroni, G., Schlemper, J., Qiu, H., Bai, W., Rueckert, D., 2019b. Unsupervised multi-modal style transfer for cardiac mr segmentation. arXiv:1908.07344.

Chen, 2017, Deeplab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs, IEEE Trans. Pattern Anal. Mach. Intell., 40, 834, 10.1109/TPAMI.2017.2699184

Chen, 2019, Multi-task attention-based semi-supervised learning for medical image segmentation, 457

Chen, S., de Bruijne, M., 2018. An end-to-end approach to semantic segmentation with 3d cnn and posterior-crf in medical images. arXiv:1811.03549.

Cheplygina, 2019, Not-so-supervised: a survey of semi-supervised, multi-instance, and transfer learning in medical image analysis, Med Image Anal, 54, 280, 10.1016/j.media.2019.03.009

Christ, 2016, Automatic liver and lesion segmentation in ct using cascaded fully convolutional neural networks and 3d conditional random fields, 415

Çiçek, 2016, 3d u-net: learning dense volumetric segmentation from sparse annotation, 424

Cohen, 2016, Group equivariant convolutional networks, 2990

Cui, 2019, Semi-supervised brain lesion segmentation with an adapted mean teacher model, 554

Dalca, 2018, Anatomical priors in convolutional networks for unsupervised biomedical segmentation, 9290

Dmitriev, 2019, Learning multi-class segmentations from single-class datasets, 9501

Dong, 2017, Automatic brain tumor detection and segmentation using u-net based fully convolutional networks, 506

Dong, 2018, Unsupervised domain adaptation for automatic estimation of cardiothoracic ratio, 544

Dou, Q., Ouyang, C., Chen, C., Chen, H., Glocker, B., Zhuang, X., Heng, P.-A., 2018. Pnp-adanet: plug-and-play adversarial domain adaptation network with a benchmark at cross-modality cardiac segmentation. arXiv:1812.07907.

Duan, 2019, Automatic 3d bi-ventricular segmentation of cardiac images by a shape-refined multi-task deep learning approach, IEEE Trans. Med. Imaging, 38, 2151, 10.1109/TMI.2019.2894322

Feng, 2017, Discriminative localization in CNNS for weakly-supervised segmentation of pulmonary nodules, 568

Fu, 2017, Nuclei segmentation of fluorescence microscopy images using convolutional neural networks, 704

Fu, 2018, Three dimensional fluorescence microscopy image synthesis and segmentation, 2221

Fu, 2018, Joint optic disc and cup segmentation based on multi-label deep network and polar transformation, IEEE Trans. Med. Imaging, 37, 1597, 10.1109/TMI.2018.2791488

Fu, 2016, Deepvessel: retinal vessel segmentation via deep learning and conditional random field, 132

Fu, 2016, Retinal vessel segmentation via deep learning network and fully-connected conditional random fields, 698

Gal, 2016, Dropout as a bayesian approximation: representing model uncertainty in deep learning, 1050

Gao, 2016, Segmentation label propagation using deep convolutional neural networks and dense conditional random field, 1265

Ghafoorian, 2017, Location sensitive deep convolutional neural networks for segmentation of white matter hyperintensities, Sci. Rep., 7, 5110, 10.1038/s41598-017-05300-5

Giger, 2018, Whole brain segmentation and labeling from CT using synthetic MR images, J. Am. Coll. Radiol., 15, 512, 10.1016/j.jacr.2017.12.028

Gildenblat, J., Klaiman, E., 2019. Self-supervised similarity learning for digital pathology. arXiv:1905.08139.

Goodfellow, 2014, Generative adversarial nets, 2672

Gorriz, M., Carlier, A., Faure, E., Giro-i Nieto, X., 2017. Cost-effective active learning for melanoma segmentation. arXiv:1711.09168.

Gu, 2018, Reliable label-Efficient learning for biomedical image recognition, 9294, 1

Guibas, J. T., Virdi, T. S., Li, P. S., 2017. Synthetic medical images from dual generative adversarial networks. arXiv:1709.01872.

Gupta, 2016, Cross modal distillation for supervision transfer, 2827

Harouni, 2018, Universal multi-modal deep network for classification and segmentation of medical images, 2018-April, 872

Harouni, 2018, Universal multi-modal deep network for classification and segmentation of medical images, 872

He, 2017, Mask r-cnn, 2961

He, 2019, Dpa-densebiasnet: semi-supervised 3d fine renal artery segmentation with dense biased network and deep priori anatomy, 139

Hesamian, 2019, Deep learning techniques for medical image segmentation: achievements and challenges, J. Digit. Imaging, 1

Huang, 2018, Multimodal unsupervised image-to-image translation, 172

Huang, Y.-J., Dou, Q., Wang, Z.-X., Liu, L.-Z., Jin, Y., Li, C.-F., Wang, L., Chen, H., Xu, R.-H., 2018b. 3D RoI-aware u-net for accurate and efficient colorectal tumor segmentation. arXiv:1806.10342.

Huo, 2018, Adversarial synthesis learning enables segmentation without target modality ground truth, 1217

Huo, 2018, Synseg-net: synthetic segmentation without target modality ground truth, IEEE Trans. Med. Imaging, 38, 1016, 10.1109/TMI.2018.2876633

Izadyyazdanabadi, 2018, Weakly-supervised learning-based feature localization for confocal laser endomicroscopy glioma images, 300

Jaeger, 2018, Retina u-net: embarrassingly simple exploitation of segmentation supervision for medical object detection

Jamaludin, 2017, Self-supervised Learning for Spinal Mris, 294

Jia, 2017, Constrained deep weak supervision for histopathology image segmentation, IEEE Trans. Med. Imaging, 36, 2376, 10.1109/TMI.2017.2724070

Jin, 2018, Ct-realistic lung nodule simulation from 3d conditional generative adversarial networks for robust lung segmentation, 732

Jung, W., Park, S., Jung, K.-H., Hwang, S. I., 2019. Prostate cancer segmentation using manifold mixup u-net.

Kamnitsas, 2017, Efficient multi-scale 3d cnn with fully connected crf for accurate brain lesion segmentation, Med. Image Anal., 36, 61, 10.1016/j.media.2016.10.004

Karimi, D., Dou, H., Warfield, S. K., Gholipour, A., 2019. Deep learning with noisy labels: exploring techniques and remedies in medical image analysis. arXiv:1912.02911.

Karimi, 2019, Reducing the hausdorff distance in medical image segmentation with convolutional neural networks, IEEE Trans. Med. Imaging

Kendall, 2017, What uncertainties do we need in bayesian deep learning for computer vision?, 5574

Kervadec, 2019, Boundary loss for highly unbalanced segmentation, 285

Krähenbühl, 2011, Efficient inference in fully connected CRFS with gaussian edge potentials, 109

Kuo, 2018, Cost-sensitive active learning for intracranial hemorrhage detection, 715

Larrazabal, 2019, Anatomical priors for image segmentation via post-processing with denoising autoencoders, 585

Li, 2019, Recurrent aggregation learning for multi-view echocardiographic sequences segmentation, 678

Li, 2018, H-Denseunet: hybrid densely connected unet for liver and tumor segmentation from ct volumes, IEEE Trans. Med. Imaging, 37, 2663, 10.1109/TMI.2018.2845918

Li, X., Yu, L., Chen, H., Fu, C.-W., Heng, P.-A., 2019b. Transformation consistent self-ensembling model for semi-supervised medical image segmentation. arXiv:1903.00348.

Li, 2019, Overfitting of neural nets under class imbalance: analysis and improvements for segmentation, 402

Liskowski, 2016, Segmenting retinal blood vessels with deep neural networks, IEEE Trans. Med. Imaging, 35, 2369, 10.1109/TMI.2016.2546227

Litjens, 2017, A survey on deep learning in medical image analysis, Med. Image Anal., 42, 60, 10.1016/j.media.2017.07.005

Liu, 2018, 3d anisotropic hybrid network: Transferring convolutional features from 2d images to 3d anisotropic volumes, 851

Ma, 2019, Neural style transfer improves 3d cardiovascular mr image segmentation on inconsistent data, 128

Mahapatra, 2018, Efficient active learning for image classification and segmentation using a sample selection and conditional generative adversarial network, 580

Matuszewski, 2018, Minimal annotation training for segmentation of microscopy images, 387

Mehta, 2018, Y-net: joint segmentation and classification for diagnosis of breast biopsy images, 893

Menze, 2014, The multimodal brain tumor image segmentation benchmark (brats), IEEE Trans. Med. Imaging, 34, 1993, 10.1109/TMI.2014.2377694

Milletari, 2016, V-net: fully convolutional neural networks for volumetric medical image segmentation, 565

Min, 2019, A two-stream mutual attention network for semi-supervised biomedical segmentation with noisy labels, 33, 4578

Mirikharaji, 2018, Star shape prior in fully convolutional networks for skin lesion segmentation, 737

Mirikharaji, 2019, Learning to segment skin lesions from noisy annotations, 207

Mondal, A. K., Dolz, J., Desrosiers, C., 2018. Few-shot 3d multi-modal medical image segmentation using generative adversarial learning. arXiv:1810.12241.

Monteiro, M., Figueiredo, M. A., Oliveira, A. L., 2018. Conditional random fields as recurrent neural networks for 3d medical imaging segmentation. arXiv:1807.07464.

Myronenko, 2018, 3d mri brain tumor segmentation using autoencoder regularization, 311

Nie, 2018, Asdnet: attention based semi-supervised deep networks for medical image segmentation, 370

Oktay, 2017, Anatomically constrained neural networks (ACNNS): application to cardiac image enhancement and segmentation, IEEE Trans. Med. Imaging, 37, 384, 10.1109/TMI.2017.2743464

Oktay, O., Schlemper, J., Folgoc, L. L., Lee, M., Heinrich, M., Misawa, K., Mori, K., McDonagh, S., Hammerla, N. Y., Kainz, B., Glocker, B., Rueckert, D., 2018. Attention u-net: learning where to look for the pancreas. arXiv:1804.03999.

Ozdemir, 2018, Active learning for segmentation by optimizing content information for maximal entropy, 183

Panfilov, 2019, Improving robustness of deep learning based knee MRI segmentation: mixup and adversarial domain adaptation

Qin, X., 2019. Transfer learning with edge attention for prostate MRI segmentation. arXiv:1912.09847.

Radosavovic, 2018, Data distillation: Towards omni-supervised learning, 4119

Ravishankar, 2017, Learning and incorporating shape models for semantic segmentation, 203

Ronneberger, 2015, U-net: convolutional networks for biomedical image segmentation, 234

Ross, 2018, Exploiting the potential of unlabeled endoscopic video data with self-supervised learning, Int. J. Comput. Assist. Radiol. Surg., 1

Roth, 2015, Deeporgan: multi-level deep convolutional networks for automated pancreas segmentation, 556

Roy, 2018, Concurrent spatial and channel âsqueeze & excitationâin fully convolutional networks, 421

Sakinis, T., Milletari, F., Roth, H., Korfiatis, P., Kostandy, P., Philbrick, K., Akkus, Z., Xu, Z., Xu, D., Erickson, B. J., 2019. Interactive segmentation of medical images through fully convolutional neural networks. arXiv:1903.08205.

Schwing, A. G., Urtasun, R., 2015. Fully connected deep structured networks. arXiv:1503.02351.

Sedai, 2017, Semi-supervised segmentation of optic cup in retinal fundus images using variational autoencoder, 75

Selvaraju, 2017, Grad-cam: visual explanations from deep networks via gradient-based localization, 618

Shin, 2016, Deep convolutional neural networks for computer-aided detection: cnn architectures, dataset characteristics and transfer learning, IEEE Trans. Med. Imaging, 35, 1285, 10.1109/TMI.2016.2528162

Shin, 2018, Medical image synthesis for data augmentation and anonymization using generative adversarial networks, 1

Shorten, 2019, A survey on image data augmentation for deep learning, J. Big Data, 6, 60, 10.1186/s40537-019-0197-0

Silvestri, G., Antiga, L., 2018. Stereology as weak supervision for medical image segmentation.

Sirinukunwattana, 2017, Gland segmentation in colon histology images: the GLaS challenge contest, Med. Image Anal., 35, 489, 10.1016/j.media.2016.08.008

Sourati, 2018, Active deep learning with fisher information for patch-wise semantic segmentation, 83

Sourati, 2019, Intelligent labeling based on fisher information for medical image segmentation using deep learning, IEEE Trans. Med. Imaging, 38, 2642, 10.1109/TMI.2019.2907805

Spitzer, 2018, Improving cytoarchitectonic segmentation of human brain areas with self-supervised siamese networks, 663

Sun, J., Shi, Y., Gao, Y., Wang, L., Zhou, L., Yang, W., Shen, D., 2018. Interactive medical image segmentation via point-based interaction and sequential patch learning. arXiv:1804.10481.

Sun, 2019, Joint cs-mri reconstruction and segmentation with a unified deep network, 492

Taghanaki, S. A., Abhishek, K., Cohen, J. P., Cohen-Adad, J., Hamarneh, G., 2019. Deep semantic segmentation of natural and medical images: a review. arXiv:1910.07655.

Tajbakhsh, 2019, Surrogate supervision for medical image analysis: effective deep learning from limited quantities of labeled data, 1251

Tajbakhsh, N., Lai, B., Ananth, S., Ding, X., 2019b. Errornet: learning error representations from limited data to improve vascular segmentation. arXiv:1910.04814.

Tajbakhsh, 2016, Convolutional neural networks for medical image analysis: full training or fine tuning?, IEEE Trans. Med. Imaging, 35, 1299, 10.1109/TMI.2016.2535302

Tang, 2018, Ct image enhancement using stacked generative adversarial networks and transfer learning for lesion segmentation improvement, 46

Tang, Y., Tang, Y., Xiao, J., Summers, R. M., 2019a. Xlsor: a robust and accurate lung segmentor on chest x-rays using criss-cross attention and customized radiorealistic abnormalities generation. arXiv:1904.09229.

Tang, 2019, Ct-realistic data augmentation using generative adversarial network for robust lymph node segmentation, 10950, 109503V

Valindria, 2018, Multi-modal learning from unpaired images: application to multi-organ segmentation in CT and MRI, Proceedings - 2018 IEEE Winter Conference on Applications of Computer Vision, WACV 2018, 2018-Janua, 547, 10.1109/WACV.2018.00066

Veeling, 2018, Rotation equivariant cnns for digital pathology, 210

Verma, 2019, Manifold mixup: better representations by interpolating hidden states, 6438

Wachinger, 2018, Deepnat: deep convolutional neural network for segmenting neuroanatomy, Neuroimage, 170, 434, 10.1016/j.neuroimage.2017.02.035

Wang, 2018, Interactive medical image segmentation using deep learning with image-specific fine tuning, IEEE Trans. Med. Imaging, 37, 1562, 10.1109/TMI.2018.2791721

Wang, 2018, Deepigeos: a deep interactive geodesic framework for medical image segmentation, IEEE Trans. Pattern Anal. Mach. Intell., 37, 1562

Wang, 2019, Boundary and entropy-driven adversarial learning for fundus image segmentation, 102

Wang, 2019, Patch-based output space adversarial learning for joint optic disc and cup segmentation, IEEE Trans. Med. Imaging, 38, 2485, 10.1109/TMI.2019.2899910

Wang, 2017, A multi-view deep convolutional neural networks for lung nodule segmentation, 1752

Xia, Y., Liu, F., Yang, D., Cai, J., Yu, L., Zhu, Z., Xu, D., Yuille, A., Roth, H., 2020. 3d semi-supervised learning with uncertainty-aware multi-view co-training.

Xia, 2018, Bridging the gap between 2d and 3d organ segmentation with volumetric fusion net, 445

Xu, 2019, Deepatlas: joint semi-supervised learning of image registration and segmentation, 420

Yan, 2018, Deeplesion: automated mining of large-scale lesion annotations and universal lesion detection with deep learning, J. Med. Imaging, 5, 036501, 10.1117/1.JMI.5.3.036501

Yang, 2017, Suggestive annotation: a deep active learning framework for biomedical image segmentation, 399

Yang, 2018, Generalizing deep models for ultrasound image segmentation, 497

Yao, 2019, Integrating 3d geometry of organ for improving medical image segmentation, 318

Yi, 2019, Generative adversarial network in medical imaging: a review, Med. Image Anal., 58, 101552, 10.1016/j.media.2019.101552

Yu, 2019, Uncertainty-aware self-ensembling model for semi-supervised 3d left atrium segmentation, 605

Yu, 2018, Recurrent saliency transformation network: incorporating multi-stage visual cues for small organ segmentation, 8280

Zhang, C., Bengio, S., Hardt, M., Recht, B., Vinyals, O., 2016. Understanding deep learning requires rethinking generalization. arXiv:1611.03530.

Zhang, 2018, Mixup: beyond empirical risk minimization

Zhang, 2018, Self-learning to detect and segment cysts in lung ct images without manual annotation, 1100

Zhang, L., Wang, X., Yang, D., Sanford, T., Harmon, S., Turkbey, B., Roth, H., Myronenko, A., Xu, D., Xu, Z., 2019a. When unseen domain generalization is unnecessary? rethinking data augmentation. arXiv:1906.03347.

Zhang, 2017, Self supervised deep representation learning for fine-grained body part recognition, 578

Zhang, P., Zhong, Y., Deng, Y., Tang, X., Li, X., 2019b. A survey on deep learning of small sample in biomedical image analysis. arXiv:1908.00473.

Zhang, 2018, Task driven generative modeling for unsupervised domain adaptation: application to x-ray image segmentation, 599

Zhang, 2017, Deep adversarial networks for biomedical image segmentation utilizing unannotated images, 408

Zhang, Y., Yang, Q., 2017. A survey on multi-task learning. arXiv:1707.08114.

Zhang, Z., Li, J., Zhong, Z., Jiao, Z., Gao, X., 2019c. A sparse annotation strategy based on attention-guided active learning for 3d medical image segmentation. arXiv:1906.07367.

Zhang, 2018, Translating and segmenting multimodal medical volumes with cycle-and shape-consistency generative adversarial network, 9242

Zhao, 2019, Data augmentation using learned transformations for one-shot medical image segmentation, 8543

Zhao, 2019, Multi-view semi-supervised 3d whole brain segmentation with a self-ensemble network, 256

Zheng, H., Yang, L., Chen, J., Han, J., Zhang, Y., Liang, P., Zhao, Z., Wang, C., Chen, D. Z., 2019. Biomedical image segmentation via representative annotation.

Zheng, 2015, Conditional random fields as recurrent neural networks, 1529

Zhou, 2016, Learning deep features for discriminative localization, 2921

Zhou, 2019, Prior-aware neural network for partially-supervised multi-organ segmentation, 10672

Zhou, Y., Wang, Y., Tang, P., Bai, S., Shen, W., Fishman, E. K., Yuille, A. L., 2018a. Semi-supervised multi-organ segmentation via deep multi-planar co-training. arXiv:1804.02586.

Zhou, 2018, Unet++: a nested U-net architecture for medical image segmentation, 3

Zhou, 2019, Models genesis: generic autodidactic models for 3D medical image analysis

Zhu, 2019, Pick-and-learn: automatic quality evaluation for noisy-labeled image segmentation, 576

Zhu, 2017, Unpaired image-to-image translation using cycle-consistent adversarial networks, 2223

Zhuang, 2019, Evaluation of algorithms for multi-modality whole heart segmentation: an open-access grand challenge, Med. Image Anal., 58, 101537, 10.1016/j.media.2019.101537