Deep Learning in Medical Ultrasound Analysis: A Review

Engineering - Tập 5 Số 2 - Trang 261-275 - 2019
Shengfeng Liu1, Yi Wang1, Xin Yang2, Baiying Lei1, Li Liu1, Shawn Xiang Li1, Dong Ni1, Tianfu Wang1
1National-Regional Key Technology Engineering Laboratory for Medical Ultrasound & Guangdong Provincial Key Laboratory of Biomedical Measurements and Ultrasound Imaging & School of Biomedical Engineering, Health Science Center, Shenzhen University, Shenzhen 518060, China
2Department of Computer Science and Engineering, The Chinese University of Hong Kong, Hong Kong, China

Tóm tắt

Từ khóa


Tài liệu tham khảo

Reddy, 2008, Prenatal imaging: ultrasonography and magnetic resonance imaging, Obstet Gynecol, 112, 145, 10.1097/01.AOG.0000318871.95090.d9

Noble, 2006, Ultrasound image segmentation: a survey, IEEE Trans Med Imaging, 25, 987, 10.1109/TMI.2006.877092

Salomon, 2008, A score-based method for quality control of fetal images at routine second-trimester ultrasound examination, Prenat Diagn, 28, 822, 10.1002/pd.2016

Krizhevsky, 2017, ImageNet classification with deep convolutional neural networks, Commun ACM, 60, 84, 10.1145/3065386

Wang, 2016, A perspective on deep imaging, IEEE Access, 4, 8914, 10.1109/ACCESS.2016.2624938

Russakovsky, 2015, Imagenet large scale visual recognition challenge, Int J Comput Vis, 115, 211, 10.1007/s11263-015-0816-y

Salakhutdinov, 2015, Learning deep generative models, Annu Rev Stat Appl, 2, 361, 10.1146/annurev-statistics-010814-020120

LeCun, 2015, Deep learning, Nature, 521, 436, 10.1038/nature14539

Deng, 2014, Deep learning: methods and applications. Found Trends, Signal Process, 7, 197

Deng, 2013, Recent advances in deep learning for speech research at Microsoft, 8604

Shen, 2017, Deep learning in medical image analysis, Annu Rev Biomed Eng, 19, 221, 10.1146/annurev-bioeng-071516-044442

Greenspan, 2016, Guest editorial deep learning in medical imaging: overview and future promise of an exciting new technique, IEEE Trans Med Imaging, 35, 1153, 10.1109/TMI.2016.2553401

Litjens, 2017, A survey on deep learning in medical image analysis, Med Image Anal, 42, 60, 10.1016/j.media.2017.07.005

Suzuki, 2017, Overview of deep learning in medical imaging, Radiological Phys Technol, 10, 257, 10.1007/s12194-017-0406-5

Ker, 2018, Deep learning applications in medical image analysis, IEEE Access, 6, 9375, 10.1109/ACCESS.2017.2788044

Anas, 2015, Bone enhancement in ultrasound using local spectrum variations for guiding percutaneous scaphoid fracture fixation procedures, Int J CARS, 10, 959, 10.1007/s11548-015-1181-6

Hiramatsu, 2017, Automated detection of masses on whole breast volume ultrasound scanner: false positive reduction using deep convolutional neural network

Bian, 2017, Boundary regularized convolutional neural network for layer parsing of breast anatomy in automated whole breast ultrasound, 259

Shi, 2016, Stacked deep polynomial network based representation learning for tumor classification with small ultrasound image dataset, Neurocomputing, 194, 87, 10.1016/j.neucom.2016.01.074

Azizi, 2015, Ultrasound-based detection of prostate cancer using automatic feature selection with deep belief networks, 70

Yang, 2017, Fine-grained recurrent neural networks for automatic prostate segmentation in ultrasound images, 1633

Wu, 2014, Deep learning based classification of focal liver lesions with contrast-enhanced ultrasound, Optik, 125, 4057, 10.1016/j.ijleo.2014.01.114

Ghesu, 2015, Marginal space deep learning: efficient architecture for detection in volumetric image data, 710

Pereira, 2017, Automated detection of coarctation of aorta in neonates from two-dimensional echocardiograms, J Med Imaging, 4, 10.1117/1.JMI.4.1.014502

Sombune, 2017, Automated embolic signal detection using deep convolutional neural network, 3365

Milletari, 2015, Robust segmentation of various anatomies in 3D ultrasound using hough forests and learned data representations, 111

Lekadir, 2017, A convolutional neural network for automatic characterization of plaque composition in carotid ultrasound, IEEE J Biomed Health Inform, 21, 48, 10.1109/JBHI.2016.2631401

Shin, 2016, Automating carotid intima-media thickness video interpretation with convolutional neural networks, 2526

Ma, 2017, Cascade convolutional neural networks for automatic detection of thyroid nodules in ultrasound images, Med Phys, 44, 1678, 10.1002/mp.12134

Smistad, 2016, Vessel detection in ultrasound images using deep convolutional neural networks, 30

Su, 2017, Detection of lumen and media-adventitia borders in IVUS images using sparse auto-encoder neural network, 1120

Yaqub, 2017, A deep learning solution for automatic fetal neurosonographic diagnostic plane verification using clinical standard constraints, Ultrasound Med Biol, 43, 2925, 10.1016/j.ultrasmedbio.2017.07.013

Huang, 2017, Temporal HeartNet: towards human-level automatic analysis of fetal cardiac screening video, 341

Gao, 2017, Detection and characterization of the fetal heartbeat in free-hand ultrasound sweeps with weakly-supervised two-streams convolutional networks, 305

Qi, 2017, Weakly supervised learning of placental ultrasound images with residual networks, 98

Chen, 2016, Iterative multi-domain regularized deep learning for anatomical structure detection and segmentation from ultrasound images, 487

Ravishankar, 2016, Hybrid approach for automatic segmentation of fetal abdomen from ultrasound images using deep learning, 779

Zhang, 2016, Coarse-to-fine stacked fully convolutional nets for lymph node segmentation in ultrasound images, 443

Ravishankar, 2017, Learning and incorporating shape models for semantic segmentation, 203

Hetherington, 2017, SLIDE: automatic spine level identification system using a deep convolutional neural network, Int J CARS, 12, 1189, 10.1007/s11548-017-1575-8

Golan, 2016, Fully automating Graf’s method for DDH diagnosis using deep convolutional neural networks, 130

Hareendranathan, 2017, Toward automatic diagnosis of hip dysplasia from 2D ultrasound, 982

Burlina, 2017, Automated diagnosis of myositis from muscle ultrasound: exploring the use of machine learning and deep learning methods, PLoS ONE, 12, 10.1371/journal.pone.0184059

Hafiane A, Vieyres P, Delbos A. Deep learning with spatiotemporal consistency for nerve segmentation in ultrasound images. 2017. arXiv:1706.05870.

Fasel, 2010, Deep belief networks for real-time extraction of tongue contours from ultrasound during speech, 1493

Jaumard-Hakoun A, Xu K, Roussel-Ragot P, Dreyfus G, Denby B. Tongue contour extraction from ultrasound images based on deep neural network. 2016. arXiv:1605.05912.

Xu, 2017, Convolutional neural network-based automatic classification of midsagittal tongue gestural targets using B-mode ultrasound images, J Acoust Soc Am, 141, EL531, 10.1121/1.4984122

Chi, 2017, Thyroid nodule classification in ultrasound images by fine-tuning deep convolutional neural network, J Digit Imaging, 30, 477, 10.1007/s10278-017-9997-y

Cheng, 2017, Transfer learning with convolutional neural networks for classification of abdominal ultrasound images, J Digit Imaging, 30, 234, 10.1007/s10278-016-9929-2

Li, 2017, Automatic fetal body and amniotic fluid segmentation from fetal ultrasound images by encoder-decoder network with inner layers, 1485

Fenster, 2001, Three-dimensional ultrasound imaging, Phys Med Biol, 46, R67, 10.1088/0031-9155/46/5/201

Ghesu, 2016, Marginal space deep learning: efficient architecture for volumetric image parsing, IEEE Trans Med Imaging, 35, 1217, 10.1109/TMI.2016.2538802

Akkus, 2017, Deep learning for brain MRI segmentation: state of the art and future directions, J Digit Imaging, 30, 449, 10.1007/s10278-017-9983-4

Xing, 2017, Deep learning in microscopy image analysis: a survey, IEEE Trans Neural Networks Learn Syst, 29, 4550, 10.1109/TNNLS.2017.2766168

Xian, 2018, Automatic breast ultrasound image segmentation: a survey, Pattern Recognit, 79, 340, 10.1016/j.patcog.2018.02.012

He, 2015, Convolutional neural networks at constrained time cost, 5353

Milletari, 2017, Hough-CNN: deep learning for segmentation of deep brain regions in MRI and ultrasound, Comput Vis Image Underst, 164, 92, 10.1016/j.cviu.2017.04.002

Liu, 2017, Learning to diagnose cirrhosis with liver capsule guided ultrasound image classification, Sensors, 17, 149, 10.3390/s17010149

Canziani A, Paszke A, Culurciello E. An analysis of deep neural network models for practical applications. 2016. arXiv:1605.07678.

Chen, 2015, Automatic fetal ultrasound standard plane detection using knowledge transferred recurrent neural networks, 507

Bengio, 1994, Learning long-term dependencies with gradient descent is difficult, IEEE Trans Neural Netw, 5, 157, 10.1109/72.279181

Hochreiter, 1997, Long short-term memory, Neural Comput, 9, 1735, 10.1162/neco.1997.9.8.1735

Cho K, Van Merriënboer B, Gulcehre C, Bahdanau D, Bougares F, Schwenk H. Learning phrase representations using RNN encoder-decoder for statistical machine translation. 2014. arXiv:1406.1078.

Bengio, 2013, Representation learning: a review and new perspectives, IEEE Trans Pattern Anal Mach Intell, 35, 1798, 10.1109/TPAMI.2013.50

Vincent, 2010, Stacked denoising autoencoders: learning useful representations in a deep network with a local denoising criterion, J Mach Learn Res, 11, 3371

Hinton, 2012, A practical guide to training restricted boltzmann machines, 599

Hinton, 2006, Reducing the dimensionality of data with neural networks, Science, 313, 504, 10.1126/science.1127647

Tajbakhsh, 2016, Convolutional neural networks for medical image analysis: full training or fine tuning?, IEEE Trans Med Imaging, 35, 1299, 10.1109/TMI.2016.2535302

Duchi, 2011, Adaptive subgradient methods for online learning and stochastic optimization, J Mach Learn Res, 12, 2121

Sutskever, 2013, On the importance of initialization and momentum in deep learning, 1139

Nair, 2010, Rectified linear units improve restricted boltzmann machines, 807

Glorot, 2011, Deep sparse rectifier neural networks, 315

Goodfellow, 2013, Maxout networks, 1319

Hinton GE, Srivastava N, Krizhevsky A, Sutskever I, Salakhutdinov RR. Improving neural networks by preventing co-adaptation of feature detectors. 2012. arXiv: 1207.0580.

Ioffe, 2015, Batch normalization: accelerating deep network training by reducing internal covariate shift, 448

Pan, 2010, A survey on transfer learning, IEEE Trans Knowl Data Eng, 22, 1345, 10.1109/TKDE.2009.191

Azizi, 2017, Transfer learning from RF to B-mode temporal enhanced ultrasound features for prostate cancer detection, Int J CARS, 12, 1111, 10.1007/s11548-017-1573-x

Chen, 2015, Standard plane localization in fetal ultrasound via domain transferred deep neural networks, IEEE J Biomed Health Inform, 19, 1627, 10.1109/JBHI.2015.2425041

Jia, 2014, Caffe: Convolutional architecture for fast feature embedding, 675

Abadi M, Agarwal A, Barham P, Brevdo E, Chen Z, Citro C, et al. Tensorflow: large-scale machine learning on heterogeneous distributed systems. 2016. arXiv: 1603.04467.

Bastien F, Lamblin P, Pascanu R, Bergstra J, Goodfellow L, Bergeron A, et al. Theano: new features and speed improvements. 2012. arXiv: 1211.5590.

Collobert, 2011, Torch7: a matlab-like environment for machine learning

Chen T, Li M, Li Y, Lin M, Wang N, Wang M. MXNet: a flexible and efficient machine learning library for heterogeneous distributed systems. 2015. arXiv: 1512.01274.

Bahrampour S, Ramakrishnan N, Schott L, Shah M. Comparative study of deep learning software frameworks. 2015. arXiv: 1511.06435.

Giger, 2008, Anniversary paper: history and status of CAD and quantitative image analysis: the role of Medical Physics and AAPM, Med Phys, 35, 5799, 10.1118/1.3013555

Jamieson, 2012, Breast image feature learning with adaptive deconvolutional networks

Liu, 2015, Tumor classification by deep polynomial network and multiple kernel learning on small ultrasound image dataset, 313

Cheng, 2016, Computer-aided diagnosis with deep learning architecture: applications to breast lesions in US images and pulmonary nodules in CT scans, Sci Rep, 6, 24454, 10.1038/srep24454

Zhang, 2016, Deep learning based classification of breast tumors with shear-wave elastography, Ultrasonics, 72, 150, 10.1016/j.ultras.2016.08.004

Han, 2017, A deep learning framework for supporting the classification of breast lesions in ultrasound images, Phys Med Biol, 62, 7714, 10.1088/1361-6560/aa82ec

Antropova, 2017, A deep feature fusion methodology for breast cancer diagnosis demonstrated on three imaging modality datasets, Med Phys, 44, 5162, 10.1002/mp.12453

Ferlay, 2010, Estimates of worldwide burden of cancer in 2008: GLOBOCAN 2008, Int J Cancer, 127, 2893, 10.1002/ijc.25516

Guo, 2017, CEUS-based classification of liver tumors with deep canonical correlation analysis and multi-kernel learning, 1748

Meng, 2017, Liver fibrosis classification based on transfer learning and FCNet for ultrasound images, IEEE Access, 5, 5804

Ma, 2017, A pre-trained convolutional neural network based method for thyroid nodule diagnosis, Ultrasonics, 73, 221, 10.1016/j.ultras.2016.09.011

Liu, 2017, Classification of thyroid nodules in ultrasound images using deep model based transfer learning and hybrid features, 919

Liu, 2017, Feature selection and thyroid nodule classification using transfer learning, 1096

Dudley, 2002, The importance of quality management in fetal measurement, Ultrasound Obstet Gynecol, 19, 190, 10.1046/j.0960-7692.2001.00549.x

Wu, 2017, FUIQA: fetal ultrasound image quality assessment with deep convolutional networks, IEEE Trans Cybern, 47, 1336, 10.1109/TCYB.2017.2671898

Jang J, Kwon JY, Kim B, Lee SM, Park Y, Seo JK. CNN-based estimation of abdominal circumference from ultrasound images. 2017. arXiv: 1702.02741.

Gao, 2016, Describing ultrasound video content using deep convolutional neural networks, 787

Sundaresan, 2017, Automated characterization of the fetal heart in ultrasound images using fully convolutional neural networks, 671

Perrin, 2012, Application of convolutional artificial neural networks to echocardiograms for differentiating congenital heart diseases in a pediatric population

Yu, 2016, Fetal facial standard plane recognition via very deep convolutional networks, 627

Yu, 2018, A deep convolutional neural network-based framework for automatic fetal facial standard plane recognition, IEEE J Biomed Health Inform, 22, 874, 10.1109/JBHI.2017.2705031

Azizi, 2016, Detection of prostate cancer using temporal sequences of ultrasound data: a large clinical feasibility study, Int J CARS, 11, 947, 10.1007/s11548-016-1395-2

Azizi, 2017, Detection and grading of prostate cancer using temporal enhanced ultrasound: combining deep neural networks and tissue mimicking simulations, Int J CARS, 12, 1293, 10.1007/s11548-017-1627-0

Yap, 2018, Automated breast ultrasound lesions detection using convolutional neural networks, IEEE J Biomed Health Inform, 22, 1218, 10.1109/JBHI.2017.2731873

Cao, 2017, Breast tumor detection in ultrasound images using deep learning, 121

Girshick, 2015, Fast R-CNN, 1440

Ren, 2017, Faster R-CNN: Towards real-time object detection with region proposal networks, IEEE Trans Pattern Anal Mach Intell, 39, 1137, 10.1109/TPAMI.2016.2577031

Redmon, 2016, You only look once: unified, real-time object detection, 779

Liu, 2016, SSD: single shot multibox detector, 21

Lipsanen, 2014, KB-R7943, an inhibitor of the reverse Na+/Ca2+ exchanger, does not modify secondary pathology in the thalamus following focal cerebral stroke in rats, Neurosci Lett, 580, 173, 10.1016/j.neulet.2014.08.003

Yang, 2014, Standard plane localization in ultrasound by radial component, 1180

Ni, 2013, Selective search and sequential detection for standard plane localization in ultrasound, 203

Baumgartner, 2016, Real-time standard scan plane detection and localisation in fetal ultrasound using fully convolutional neural networks, 203

Baumgartner, 2017, SonoNet: real-time detection and localisation of fetal standard scan planes in freehand ultrasound, IEEE Trans Med Imaging, 36, 2204, 10.1109/TMI.2017.2712367

Chen, 2014, Fetal abdominal standard plane localization through representation learning with knowledge transfer, 125

Chen, 2017, Ultrasound standard plane detection using a composite neural network framework, IEEE Trans Cybern, 47, 1576, 10.1109/TCYB.2017.2685080

Dezaki, 2017, Deep residual recurrent neural networks for characterization of cardiac cycle phase from echocardiograms, 100

Sofka, 2017, Fully convolutional regression network for accurate detection of measurement points, 258

Ghesu, 2016, An artificial agent for anatomical landmark detection in medical images, 229

Nascimento, 2016, Multi-atlas segmentation using manifold learning with deep belief networks, 867

Ma, 2017, Ultrasound image-based thyroid nodule automatic segmentation using convolutional neural networks, Int J CARS, 12, 1895, 10.1007/s11548-017-1649-7

Singhal, 2017, Automated assessment of endometrium from transvaginal ultrasound using Deep Learned Snake, 83

Bernard, 2007, A RBF-Based multiphase level set method for segmentation in echocardiography using the statistics of the radiofrequency signal

Jacob, 2002, A shape-space-based approach to tracking myocardial borders and quantifying regional left-ventricular function applied in echocardiography, IEEE Trans Med Imaging, 21, 226, 10.1109/42.996341

Carneiro, 2010, Robust left ventricle segmentation from ultrasound data using deep neural networks and efficient search methods, 1085

Carneiro, 2010, Multiple dynamic models for tracking the left ventricle of the heart from ultrasound data using particle filters and deep learning architectures, 2815

Carneiro, 2011, Incremental on-line semi-supervised learning for segmenting the left ventricle of the heart from ultrasound data, 1700

Carneiro, 2012, The use of on-line co-training to reduce the training set size in pattern recognition methods: application to left ventricle segmentation in ultrasound, 948

Carneiro, 2012, The segmentation of the left ventricle of the heart from ultrasound data using deep learning architectures and derivative-based search methods, IEEE Trans Image Process, 21, 968, 10.1109/TIP.2011.2169273

Carneiro, 2013, Combining multiple dynamic models and deep learning architectures for tracking the left ventricle endocardium in ultrasound data, IEEE Trans Pattern Anal Mach Intell, 35, 2592, 10.1109/TPAMI.2013.96

Nascimento, 2017, Deep learning on sparse manifolds for faster object segmentation, IEEE Trans Image Process, 26, 4978, 10.1109/TIP.2017.2725582

Nascimento, 2014, Non-rigid segmentation using sparse low dimensional manifolds and deep belief networks, 288

Raynaud, 2017, Handcrafted features vs. ConvNets in 2D echocardiographic images, 1116

Xiong, 2015, Global supervised descent method, 2664

Yu, 2017, Segmentation of fetal left ventricle in echocardiographic sequences based on dynamic convolutional neural networks, IEEE Trans Biomed Eng, 64, 1886, 10.1109/TBME.2016.2628401

Baka, 2017, Ultrasound aided vertebral level localization for lumbar surgery, IEEE Trans Med Imaging, 36, 2138, 10.1109/TMI.2017.2738612

Wu, 2017, Cascaded fully convolutional networks for automatic prenatal ultrasound image segmentation, 663

Tu, 2010, Auto-context and its application to high-level vision tasks and 3D brain image segmentation, IEEE Trans Pattern Anal Mach Intell, 32, 1744, 10.1109/TPAMI.2009.186

Anas, 2017, Clinical target-volume delineation in prostate brachytherapy using residual neural networks, 365

Zheng, 2015, 3D deep learning for efficient and robust landmark detection in volumetric data, 565

Pesteie, 2018, Automatic localization of the needle target for ultrasound-guided epidural injections, IEEE Trans Med Imaging, 37, 81, 10.1109/TMI.2017.2739110

Nie, 2017, Automatic detection of standard sagittal plane in the first trimester of pregnancy using 3-D ultrasound data, Ultrasound Med Biol, 43, 286, 10.1016/j.ultrasmedbio.2016.08.034

Nie, 2015, A novel method with a deep network and directional edges for automatic detection of a fetal head, 654

Pourtaherian, 2017, Improving needle detection in 3D ultrasound using orthogonal-plane convolutional networks, 610

Zheng, 2008, Four-chamber heart modeling and automatic segmentation for 3-D cardiac CT volumes using marginal space learning and steerable features, IEEE Trans Med Imaging, 27, 1668, 10.1109/TMI.2008.2004421

Looney, 2017, Automatic 3D ultrasound segmentation of the first trimester placenta using deep learning, 279

Kamnitsas, 2017, Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation, Med Image Anal, 36, 61, 10.1016/j.media.2016.10.004

Yang, 2017, Towards automatic semantic segmentation in volumetric ultrasound, 711

Schmidt-Richberg, 2017, Abdomen segmentation in 3D fetal ultrasound using CNN-powered deformable models, 52

Amit, 2017, Classification of breast MRI lesions using small-size training sets: comparison of deep learning approaches

Shin, 2016, Deep convolutional neural networks for computer-aided detection: CNN architectures, dataset characteristics and transfer learning, IEEE Trans Med Imaging, 35, 1285, 10.1109/TMI.2016.2528162

Hadad, 2017, Classification of breast lesions using cross-modal deep learning, 109

Lisa, 2010, Transfer learning, 242