ST-V-Net: incorporating shape prior into convolutional neural networks for proximal femur segmentation

Complex & Intelligent Systems - 2023
Chen Zhao1, Joyce H. Keyak2, Jian Tang1, Tadashi Kaneko3, Sundeep Khosla4, Shreyasee Amin5, Elizabeth J. Atkinson6, Lan‐Juan Zhao7, Michael Serou8, Chaoyang Zhang9, Hui Shen7, Hong‐Wen Deng7, Weihua Zhou10
1Department of Applied Computing, Michigan Technological University, 1400 Townsend Dr, Houghton, MI, 49931, USA
2Department of Radiological Sciences, Department of Mechanical and Aerospace Engineering, Department of Biomedical Engineering, and Chao Family Comprehensive Cancer Center, University of California, Irvine, Irvine, CA, 92697, USA
3Department of Radiological Sciences, University of California Irvine, Irvine, CA, 92697, USA
4Division of Endocrinology, Department of Medicine, Mayo Clinic, Rochester, MN, USA
5Division of Epidemiology, Department of Health Sciences Research, and Division of Rheumatology, Department of Medicine, Mayo Clinic, Rochester, MN, USA
6Division of Biomedical Statistics and Informatics, Department of Health Sciences Research, Mayo Clinic, Rochester, MN, USA
7Division of Biomedical Informatics and Genomics, Tulane Center of Biomedical Informatics and Genomics, Deming Department of Medicine, Tulane University, School of Medicine, 1440 Canal Street, Suite 1610, New Orleans, LA, 70112, USA
8Department of Radiology, Tulane University School of Medicine, New Orleans, LA, 70112, USA
9School of Computing Sciences and Computer Engineering, University of Southern Mississippi, Hattiesburg, MS, 39406, USA
10Center of Biocomputing and Digital Health, Institute of Computing and Cybersystems, and Health Research Institute, Michigan Technological University, Houghton, MI, 49931, USA

Tóm tắt

Abstract

We aim to develop a deep-learning-based method for automatic proximal femur segmentation in quantitative computed tomography (QCT) images. We proposed a spatial transformation V-Net (ST-V-Net), which contains a V-Net and a spatial transform network (STN) to extract the proximal femur from QCT images. The STN incorporates a shape prior into the segmentation network as a constraint and guidance for model training, which improves model performance and accelerates model convergence. Meanwhile, a multi-stage training strategy is adopted to fine-tune the weights of the ST-V-Net. We performed experiments using a QCT dataset which included 397 QCT subjects. During the experiments for the entire cohort and then for male and female subjects separately, 90% of the subjects were used in ten-fold stratified cross-validation for training and the rest of the subjects were used to evaluate the performance of models. In the entire cohort, the proposed model achieved a Dice similarity coefficient (DSC) of 0.9888, a sensitivity of 0.9966 and a specificity of 0.9988. Compared with V-Net, the Hausdorff distance was reduced from 9.144 to 5.917 mm, and the average surface distance was reduced from 0.012 to 0.009 mm using the proposed ST-V-Net. Quantitative evaluation demonstrated excellent performance of the proposed ST-V-Net for automatic proximal femur segmentation in QCT images. In addition, the proposed ST-V-Net sheds light on incorporating shape prior to segmentation to further improve the model performance.

Từ khóa


Tài liệu tham khảo

Liu J, Curtis E, Cooper C, Harvey NC (2019) State of the art in osteoporosis risk assessment and treatment. J Endocrinol Investig 2019:1–16

Lang T, Keyak J, Heitz M, Augat P, Lu Y, Mathur A, Genant H (1997) Volumetric quantitative computed tomography of the proximal femur: precision and relation to bone strength. Bone 21:101–108

Carballido-Gamio J, Bonaretti S, Saeed I, Harnish R, Recker R, Burghardt AJ, Keyak JH, Harris T, Khosla S, Lang TF (2015) Automatic multi-parametric quantification of the proximal femur with quantitative computed tomography. Quant Imaging Med Surg 5:552

Keyak J, Sigurdsson S, Karlsdottir G, Oskarsdottir D, Sigmarsdottir A, Kornak J, Harris T, Sigurdsson G, Jonsson B, Siggeirsdottir K (2013) Effect of finite element model loading condition on fracture risk assessment in men and women: the AGES-Reykjavik study. Bone 57:18–29

Johannesdottir F, Allaire B, Bouxsein ML (2018) Fracture prediction by computed tomography and finite element analysis: current and future perspectives. Curr Osteoporos Rep 16:411–422

Younes LB, Nakajima Y, Saito T (2014) Fully automatic segmentation of the femur from 3D-CT images using primitive shape recognition and statistical shape models. Int J Comput Assist Radiol Surg 9:189–196

Xia Y, Fripp J, Chandra SS, Schwarz R, Engstrom C, Crozier S (2013) Automated bone segmentation from large field of view 3D MR images of the hip joint. Phys Med Biol 58:7375

Arezoomand S, Lee W-S, Rakhra KS, Beaulé PE (2015) A 3D active model framework for segmentation of proximal femur in MR images. Int J Comput Assist Radiol Surg 10:55–66

Chandra SS, Xia Y, Engstrom C, Crozier S, Schwarz R, Fripp J (2014) Focused shape models for hip joint segmentation in 3D magnetic resonance images. Med Image Anal 18:567–578

Petroudi S, Loizou C, Pantziaris M, Pattichis C (2012) Segmentation of the common carotid intima-media complex in ultrasound images using active contours. IEEE Trans Biomed Eng 59:3060–3069

Zeng G, Yang X, Li J, Yu L, Heng P-A, Zheng G (2017) 3D U-net with multi-level deep supervision: fully automatic segmentation of proximal femur in 3D MR images. In: International workshop on machine learning in medical imaging. Springer, pp 274–282

Chen F, Liu J, Zhao Z, Zhu M, Liao H (2017) Three-dimensional feature-enhanced network for automatic femur segmentation. IEEE J Biomed Health Inform 23:243–252

Nanda N, Kakkar P, Nagpal S (2019) Computer-aided segmentation of liver lesions in CT scans using cascaded convolutional neural networks and genetically optimised classifier. Arab J Sci Eng 44:4049–4062

Ravishankar H, Venkataramani R, Thiruvenkadam S, Sudhakar P, Vaidya V (2017) Learning and incorporating shape models for semantic segmentation. Springer, Berlin, pp 203–211

Lee MCH, Petersen K, Pawlowski N, Glocker B, Schaap M (2019) Template transformer networks for image segmentation

Jaderberg M, Simonyan K, Zisserman (2015) A Spatial transformer networks. In: Advances in neural information processing systems pp 2017–2025

Cootes TF, Taylor CJ, Cooper DH, Graham J (1995) Active shape models-their training and application. Comput Vis Image Underst 61:38–59

Cootes TF, Edwards GJ, Taylor CJ (1998) Active appearance models. In: European conference on computer vision. Springer, pp 484-498

Riggs BL, Melton LJ III, Robb RA, Camp JJ, Atkinson EJ, Peterson JM, Rouleau PA, McCollough CH, Bouxsein ML, Khosla S (2004) Population-based study of age and sex differences in bone volumetric density, size, geometry, and structure at different skeletal sites. J Bone Miner Res 19:1945–1954

Keyak J, Kaneko T, Khosla S, Amin S, Atkinson E, Lang T, Sibonga J (2020) Hip load capacity and yield load in men and women of all ages. Bone 2020:115321

Seitz P, Ruegsegger P (1983) Fast contour detection algorithm for high precision quantitative CT. IEEE Trans Med Imaging 2:136–141

Bjorck J, Gomes C, Selman B, Weinberger KQ (2018) Understanding batch normalization. arXiv: 180602375

Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R (2014) Dropout: a simple way to prevent neural networks from overfitting. J Mach Learn Res 15:1929–1958

Otsu N (1979) A threshold selection method from gray-level histograms. IEEE Trans Syst Man Cybern 9:62–66

Zhang Y-D, Satapathy SC, Guttery DS, Górriz JM, Wang S-H (2021) Improved breast cancer classification through combining graph convolutional network and convolutional neural network. Inf Process Manage 58:102439

LeCun Y (2015) LeNet-5, convolutional neural networks. http://www.yannlecuncom/exdb/lenet. Vol 20, p 14

Sudre CH, Li W, Vercauteren T, Ourselin S, Cardoso MJ (2017) Generalised dice overlap as a deep learning loss function for highly unbalanced segmentations. In: Deep learning in medical image analysis and multimodal learning for clinical decision support. Springer, pp 240–248

Phaisangittisagul E (2016) An analysis of the regularization between L2 and dropout in single hidden layer neural network. IEEE 2016:174–179

Bock S, Weiß M (2019) A proof of local convergence for the Adam optimizer. IEEE 2019:1–8

Shorten C, Khoshgoftaar TM (2019) A survey on image data augmentation for deep learning. J Big Data 6:60

Deniz CM, Xiang S, Hallyburton RS, Welbeck A, Babb JS, Honig S, Cho K, Chang G (2018) Segmentation of the proximal femur from MR images using deep convolutional neural networks. Sci Rep 8:1–14

Cheng Y, Zhou S, Wang Y, Guo C, Bai J, Tamura S (2013) Automatic segmentation technique for acetabulum and femoral head in CT images. Pattern Recogn 46:2969–2984

Lehmann TM, Gonner C, Spitzer K (2001) Addendum: B-spline interpolation in medical image processing. IEEE Trans Med Imaging 20:660–665

Rueckert D, Aljabar P, Heckemann RA, Hajnal JV, Hammers A (2006) Diffeomorphic registration using B-splines. In Springer, pp 702–709

Thông tin
Thông tin xuất bản

Complex & Intelligent Systems

Thông tin tác giả