An improved semantic segmentation with region proposal network for cardiac defect interpretation

Siti Nurmaini1, Bayu Adhi Tama2, Muhammad Naufal Rachmatullah3, Annisa Darmawahyuni3, Ade Iriani Sapitri3, Firdaus Firdaus3, Bambang Tutuko3
1Universitas Sriwijaya
2Data Science Group, Institute for Basic Science (IBS), Daejeon, Republic of Korea
3Intelligent System Research Group, Faculty of Computer Science, Universitas Sriwijaya, Palembang, Indonesia

Tóm tắt

Từ khóa


Tài liệu tham khảo

Garcia-Canadilla P, Sanchez-Martinez S, Crispi F, Bijnens B (2020) Machine learning in fetal cardiology: what to expect. Fetal Diagn Ther 47(5):363–372. https://doi.org/10.1159/000505021

Lopez KN, Morris SA, Sexson Tejtel SK, Espaillat A, Salemi JL (2020) US mortality attributable to congenital heart disease across the lifespan from 1999 through 2017 exposes persistent racial/ethnic disparities. Circulation 142(12):1132–1147

Pace ND, Oster ME, Forestieri NE, Enright D, Knight J, Meyer RE (2018) Sociodemographic factors and survival of infants with congenital heart defects. Pediatrics 142(3)

Han B, Tang Y, Qu X, Deng C, Wang X, Li J (2021) Comparison of the 1-year survival rate in infants with congenital heart disease diagnosed by prenatal and postnatal ultrasound: a retrospective study. Medicine 100(4)

Espinoza J (2019) Fetal MRI and prenatal diagnosis of congenital heart defects. Lancet (London, England) 393(10181):1574–1576. https://doi.org/10.1016/S0140-6736(18)32853-8

Kaluva KC, Shanthi C, Thittai AK, Krishnamurthi G (2018) CardioNet: identification of fetal cardiac standard planes from 2D Ultrasound data

Rawat V, Jain A, Shrimali V (2018) Automated techniques for the interpretation of fetal abnormalities: a review. Appl Bionics Biomech 2018

Sundaresan V, Bridge CP, Ioannou C, Noble JA (2017) Automated characterization of the fetal heart in ultrasound images using fully convolutional neural networks. In: 2017 IEEE 14th international symposium on biomedical imaging (ISBI 2017), pp 671–674. https://doi.org/10.1109/ISBI.2017.7950609

Harangi B (2018) Skin lesion classification with ensembles of deep convolutional neural networks. J Biomed Inform 86:25–32. https://doi.org/10.1016/j.jbi.2018.08.006

Yu L, Guo Y, Wang Y, Yu J, Chen P (2016) Segmentation of fetal left ventricle in echocardiographic sequences based on dynamic convolutional neural networks. IEEE Trans Biomed Eng 64(8):1886–1895. https://doi.org/10.1109/TBME.2016.2628401

Ha VK, Ren J-C, Xu X-Y, Zhao S, Xie G, Masero V, Hussain A (2019) Deep learning based single image super-resolution: a survey. Int J Autom Comput 16(4):413–426. https://doi.org/10.1007/s11633-019-1183-x

Patel N, Narasimhan E, Kennedy A (2017) Fetal cardiac US: techniques and normal anatomy correlated with adult CT and MR imaging. Radiographics 37(4):1290–1303

Xia Z, Wang X, Wang C, Zhang C (2018) Subpixel-based accurate and fast dynamic tumor image recognition. J Med Imaging Health Inform 8(5):925–931. https://doi.org/10.1166/jmihi.2018.2390

Xia Z, Wang X, Zhou W, Li R, Wang C, Zhang C (2019) Color medical image lossless watermarking using chaotic system and accurate quaternion polar harmonic transforms. Signal Process 157:108–118. https://doi.org/10.1016/j.sigpro.2018.11.011

Nurmaini S, Rachmatullah MN, Sapitri AI, Darmawahyuni A, Tutuko B, Firdaus F, Partan RU, Bernolian N (2021) Deep learning-based computer-aided fetal echocardiography: application to heart standard view segmentation for congenital heart defects detection. Sensors 21(23):8007

Yang R, Yu Y (2021) Artificial convolutional neural network in object detection and semantic segmentation for medical imaging analysis. Front Oncol 11:573

Rachmatullah MN, Nurmaini S, Sapitri AI, Darmawahyuni A, Tutuko B, Firdaus F (2021) Convolutional neural network for semantic segmentation of fetal echocardiography based on four-chamber view. Bull Electr Eng Inform 10(4):1987–1996

Ma P, Li Q, Li J (2022) Application of artificial intelligence in cardiovascular imaging. J Healthc Eng

Ronneberger O, Fischer P, Brox T (2015) U-net: Convolutional networks for biomedical image segmentation. In: International conference on medical image computing and computer-assisted intervention, pp 234–241.https://doi.org/10.1007/978-3-319-24574-4_28

Hunter LE (2018) Screening views of the fetal heart. In: Fetal cardiology. Springer, Berlin, pp 9–20. https://doi.org/10.1007/978-3-319-77461-9_2

Rueda S, Fathima S, Knight CL, Yaqub M, Papageorghiou AT, Rahmatullah B, Foi A, Maggioni M, Pepe A, Tohka J, Stebbing RV, McManigle JE, Ciurte A, Bresson X, Cuadra MB, Sun C, Ponomarev GV, Gelfand MS, Kazanov MD, Noble JA (2014) Evaluation and comparison of current fetal ultrasound image segmentation methods for biometric measurements: A grand challenge. IEEE Trans Med Imaging 33(4):797–813. https://doi.org/10.1109/TMI.2013.2276943

Gao Y, Noble JA (2017) Detection and characterization of the fetal heartbeat in free-hand ultrasound sweeps with weakly-supervised two-streams convolutional networks. In: International conference on medical image computing and computer-assisted intervention, pp 305–313. https://doi.org/10.1007/978-3-319-66185-8_35

Tajbakhsh N, Shin JY, Gurudu SR, Hurst RT, Kendall CB, Gotway MB, Liang J (2016) Convolutional neural networks for medical image analysis: Full training or fine tuning? IEEE Trans Med Imaging 35(5):1299–1312. https://doi.org/10.1109/TMI.2016.2535302

Xie HN, Wang N, He M, Zhang LH, Cai HM, Xian JB, Lin MF, Zheng J, Yang YZ (2020) Using deep-learning algorithms to classify fetal brain ultrasound images as normal or abnormal. Ultrasound Obstet Gynecol 56(4):579–587. https://doi.org/10.1002/uog.21967

Komatsu M, Sakai A, Komatsu R, Matsuoka R, Yasutomi S, Shozu K, Dozen A, Machino H, Hidaka H, Arakaki T et al (2021) Detection of cardiac structural abnormalities in fetal ultrasound videos using deep learning. Appl Sci 11(1):371. https://doi.org/10.3390/app11010371

Al-Bander B, Alzahrani T, Alzahrani S, Williams BM, Zheng Y (2020) Improving fetal head contour detection by object localisation with deep learning. In: Communications in computer and information science, 1065 CCIS, pp 142–150. https://doi.org/10.1007/978-3-030-39343-4_12

Liu K, Ye Z, Guo H, Cao D, Chen L, Wang FY (2021) FISS GAN: A generative adversarial network for foggy image semantic segmentation. IEEE/CAA J Automatica Sinica 8(8):1428–1439

Sobhaninia Z, Rafiei S, Emami A, Karimi N, Najarian K, Samavi S, Soroushmehr SMR (2019) Fetal ultrasound image segmentation for measuring biometric parameters using multi-task deep learning. In: 2019 41st Annual international conference of the ieee engineering in medicine and biology society (EMBC), pp 6545–6548. arXiv:1909.00273

Senouf O, Vedula S, Zurakhov G, Bronstein A, Zibulevsky M, Michailovich O, Adam D, Blondheim D (2018) High frame-rate cardiac ultrasound imaging with deep learning. In: International conference on medical image computing and computer-assisted intervention, pp 126–134. https://doi.org/10.1007/978-3-030-00928-1_15

Wang C, Pedrycz W, Li Z, Zhou M (2020) Residual-driven fuzzy C-means clustering for image segmentation. IEEE/CAA J Automatica Sinica 8(4):876–889. https://doi.org/10.1109/JAS.2020.1003420

Wang C, Pedrycz W, Yang J, Zhou M, Li Z (2020) Wavelet frame-based fuzzy C-means clustering for segmenting images on graphs. IEEE Trans Cybern 50(9):3938–3949. https://doi.org/10.1109/TCYB.2019.2921779

Zheng Y, Barbu A, Georgescu B, Scheuering M, Comaniciu D (2008) Four-chamber heart modeling and automatic segmentation for 3-D cardiac CT volumes using marginal space learning and steerable features. IEEE Trans Med Imaging 27(11):1668–1681. https://doi.org/10.1109/TMI.2008.2004421

Nakphu N, Dewi DEO, Rizqie MQ, Supriyanto E, Kho DCC, Kadiman S, Rittipravat P et al (2014) Apical four-chamber echocardiography segmentation using Marker-controlled Watershed segmentation. In: 2014 IEEE Conference on biomedical engineering and sciences (IECBES), pp 644–647. https://doi.org/10.1109/IECBES.2014.7047583

Cao Y, McNeillie P, Syeda-Mahmood T (2014) Segmentation of anatomical structures in four-chamber view echocardiogram images. In: 2014 22nd International conference on pattern recognition, pp 568–573. https://doi.org/10.1109/ICPR.2014.108.

Syeda-Mahmood T, Wang Q, McNeillie P, Beymer D, Compas C (2014) Discriminating normal and abnormal left ventricular shapes in four-chamber view 2D echocardiography. In: 2014 IEEE 11th International symposium on biomedical imaging (ISBI), pp 401–404. https://doi.org/10.1109/ISBI.2014.6867893

Kang HC, Kim B, Lee J, Shin J, Shin Y-G (2015) Accurate four-chamber segmentation using gradient-assisted localized active contour model. J Med Imaging Health Inform 5(1):126–137

Zheng Y, Comaniciu D (2014) Marginal space learning for medical image analysis: efficient detection and segmentation of anatomical structures. In: Marginal space learning for medical image analysis: efficient detection and segmentation of anatomical structures, vol 9781493906, Issue Mdl. https://doi.org/10.1007/978-1-4939-0600-0

Jafari MH, Girgis H, Liao Z, Behnami D, Abdi A, Vaseli H et al (2018) A unified framework integrating recurrent fully-convolutional networks and optical flow for segmentation of the left ventricle in echocardiography data. In: Deep learning in medical image analysis and multimodal learning for clinical decision support. Springer, Cham, pp. 29–37. https://doi.org/10.1007/978-3-030-00889-5

Jafari MH, Girgis H, Abdi AH, Liao Z, Pesteie M, Rohling R, Gin K, Tsang T, Abolmaesumi P (2019) Semi-supervised learning for cardiac left ventricle segmentation using conditional deep generative models as Prior University of British Columbia, Vancouver, Canada. Vancouver General Hospital, Vancouver, Canada. In: 2019 IEEE 16th International symposium on biomedical imaging (ISBI 2019), Isbi, pp 649–652. https://doi.org/10.1109/ISBI.2019.8759292

Alsharqi M, Woodward WJ, Mumith JA, Markham DC, Upton R, Leeson P (2018) Artificial intelligence and echocardiography. Echo Res Pract 5(4):R115–R125. https://doi.org/10.1530/ERP-18-0056

Arafati A, Morisawa D, Avendi MR, Amini MR, Assadi RA, Jafarkhani H, Kheradvar A (2020) Generalizable fully automated multi-label segmentation of four-chamber view echocardiograms based on deep convolutional adversarial networks. J R Soc Interface 17(169):20200267. https://doi.org/10.1098/rsif.2020.0267

Painchaud N, Skandarani Y, Judge T, Bernard O, Lalande A, Jodoin P-M (2020) Cardiac segmentation with strong anatomical guarantees. IEEE Trans Med Imaging. https://doi.org/10.1109/TMI.2020.3003240

Leclerc S, Smistad E, Pedrosa J, Ostvik A, Cervenansky F, Espinosa F, Espeland T, Berg E, Jodoin PM, Grenier T, Lartizien C, Dhooge J, Lovstakken L, Bernard O (2019) Deep learning for segmentation using an open large-scale dataset in 2D echocardiography. IEEE Trans Med Imaging 38(9):2198–2210. https://doi.org/10.1109/TMI.2019.2900516

Yang M, Xiao X, Liu Z, Sun L, Guo W, Cui L, Sun D, Zhang P, Yang G (2020) Deep RetinaNet for dynamic left ventricle detection in multiview echocardiography classification. Sci Program

Dong J, Liu S, Wang T (2019) ARVBNet: real-time detection of anatomical structures in fetal ultrasound cardiac four-chamber planes. In: Machine learning and medical engineering for cardiovascular health and intravascular imaging and computer assisted stenting. Springer, Berlin, pp 130–137. https://doi.org/10.1007/978-3-030-33327-0_16

Zhou SK, Park JH, Georgescu B, Comaniciu D, Simopoulos C, Otsuki J (2006) Image-based multiclass boosting and echocardiographic view classification. In: 2006 IEEE computer society conference on computer vision and pattern recognition (CVPR'06), vol 2, pp 1559–1565. https://doi.org/10.1109/CVPR.2006.146

Park JH, Zhou SK, Simopoulos C, Otsuki J, Comaniciu D (2007) Automatic cardiac view classification of echocardiogram. In: 2007 IEEE 11th international conference on computer vision, pp 1–8. https://doi.org/10.1109/ICCV.2007.4408867

Balaji GN, Subashini TS, Chidambaram N (2015) Automatic classification of cardiac views in echocardiogram using histogram and statistical features. Procedia Comput Sci 46:1569–1576. https://doi.org/10.1016/j.procs.2015.02.084

Lili W, Zhongliang F, Pan T (2016) Four-chamber plane detection in cardiac ultrasound images based on improved imbalanced AdaBoost algorithm. In: 2016 IEEE International conference on cloud computing and big data analysis (ICCCBDA), pp 299–303. https://doi.org/10.1109/ICCCBDA.2016.7529574

Khamis H, Zurakhov G, Azar V, Raz A, Friedman Z, Adam D (2017) Automatic apical view classification of echocardiograms using a discriminative learning dictionary. Med Image Anal 36:15–21. https://doi.org/10.1016/j.media.2016.10.007

Lin Z, Le MH, Ni D, Chen S, Li S, Wang T, Lei B (2018) Quality assessment of fetal head ultrasound images based on faster R-CNN. In: Simulation, image processing, and ultrasound systems for assisted diagnosis and navigation. Springer, Berlin, pp 38–46. https://doi.org/10.1007/978-3-030-01045-4_5

Zhou Z, Siddiquee M, Tajbakhsh N, Liang J (2018) UNet++: a nested U-Net architecture for medical image segmentation. In: Deep learning in medical image analysis and multimodal learning for clinical decision support: 4th international workshop, DLMIA 2018, and 8th international workshop, ML-CDS 2018, held in conjunction with MICCAI 2018, Granada, Spain, S..., vol 11045, pp 3–11. https://doi.org/10.1007/978-3-030-00889-5_1

Milletari F, Navab N, Ahmadi SA (2016) V-Net: fully convolutional neural networks for volumetric medical image segmentation. In: Proceedings—2016 4th international conference on 3D vision, 3DV 2016, pp 565–571. https://doi.org/10.1109/3DV.2016.79

Xu L, Liu M, Shen Z, Wang H, Liu X, Wang X, Wang S, Li T, Yu S, Hou M, Guo J, Zhang J, He Y (2020) DW-Net: a cascaded convolutional neural network for apical four-chamber view segmentation in fetal echocardiography. Comput Med Imaging Graph 80:101690. https://doi.org/10.1016/j.compmedimag.2019.101690

Ravishankar H, Prabhu SM, Vaidya V, Singhal N (2016) Hybrid approach for automatic segmentation of fetal abdomen from ultrasound images using deep learning. In: 2016 IEEE 13th international symposium on biomedical imaging (ISBI), pp 779–782. https://doi.org/10.1109/ISBI.2016.7493382

Yap MH, Goyal M, Osman FM, Martí R, Denton E, Juette A, Zwiggelaar R (2019) Breast ultrasound lesions recognition: end-to-end deep learning approaches. J Med Imaging (Bellingham, Wash.), 6(1):011007. https://doi.org/10.1117/1.JMI.6.1.011007

Kopelowitz E, Engelhard G (2019) Lung nodules detection and segmentation using 3D mask-RCNN. ArXiv Preprint arXiv:1907.07676

Anantharaman R, Velazquez M, Lee Y (2018) Utilizing mask R-CNN for detection and segmentation of oral diseases. In: 2018 IEEE international conference on bioinformatics and biomedicine (BIBM), pp 2197–2204. https://doi.org/10.1109/BIBM.2018.8621112

Kowal M, Żejmo M, Skobel M, Korbicz J, Monczak R (2020) Cell Nuclei Segmentation in cytological images using convolutional neural network and seeded watershed algorithm. J Digit Imaging 33(1):231–242. https://doi.org/10.1007/s10278-019-00200-8