CTUNet: automatic pancreas segmentation using a channel-wise transformer and 3D U-Net
Tóm tắt
Diabetes, pancreatic cancer, and pancreatitis are all diseases of the pancreas, which seriously threaten people’s lives. The pancreas has a special anatomical structure, its size, shape, and position are variable, and it is highly similar to other surrounding deep abdominal tissues, so achieving accurate segmentation is still one of the most challenging tasks in the field of medical image segmentation. We propose a new network CTUNet that combines Transformer and 3D U-Net, which can achieve high-precision automatic segmentation of the pancreas. We deploy the Transformer on skip connections to coordinate global explicit features and guide the network learning. In view of pancreas reciprocity and shape variability, we design a Pancreas Attention module and add it to each encoder to further enhance the ability to extract context information and learn distinct features. In addition, in the decoder, we use a novel Feature Concatenation module with an attention mechanism to further promote the fusion of different levels of features and alleviate the problem of loss of down-sampling feature information. We train and test our model on the NIH dataset and evaluate with Dice Similarity Coefficient, Jaccard Index, Precision, and Recall. Experimental results show that our proposed model outperforms most existing pancreas segmentation methods.
Tài liệu tham khảo
citation_journal_title=Gastroenterology; citation_title=Burden of gastrointestinal, liver, and pancreatic diseases in the United States; citation_author=AF Peery, SD Crockett, AS Barritt, ES Dellon, S Eluri, LM Gangarosa, ET Jensen, JL Lund, S Pasricha, T Runge; citation_volume=149; citation_issue=7; citation_publication_date=2015; citation_pages=1731-1741; citation_doi=10.1053/j.gastro.2015.08.045; citation_id=CR1
Dmitriev, K., Gutenko, I., Nadeem, S., Kaufman, A.: Pancreas and cyst segmentation. In: Medical Imaging 2016: Image Processing, vol. 9784, pp. 628–634. SPIE (2016)
citation_journal_title=IEEE Trans. Image Process.; citation_title=A bottom-up approach for pancreas segmentation using cascaded superpixels and (deep) image patch labeling; citation_author=A Farag, L Lu, HR Roth, J Liu, E Turkbey, RM Summers; citation_volume=26; citation_issue=1; citation_publication_date=2016; citation_pages=386-399; citation_doi=10.1109/TIP.2016.2624198; citation_id=CR3
citation_journal_title=Int. J. Comput. Assist. Radiol. Surg.; citation_title=A geometric method for the detection and correction of segmentation leaks of anatomical structures in volumetric medical images; citation_author=A Kronman, L Joskowicz; citation_volume=11; citation_issue=3; citation_publication_date=2016; citation_pages=369-380; citation_doi=10.1007/s11548-015-1285-z; citation_id=CR4
Roth, H.R., Lu, L., Farag, A., Shin, H.-C., Liu, J., Turkbey, E.B., Summers, R.M.: Deeporgan: Multi-level deep convolutional networks for automated pancreas segmentation. In: International Conference on Medical Image Computing and Computer-assisted Intervention, pp. 556–564. Springer (2015)
citation_journal_title=Vis. Comput.; citation_title=A systematic review on application of deep learning in digestive system image processing; citation_author=H Zhuang, J Zhang, F Liao; citation_volume=1; citation_publication_date=2021; citation_pages=1-16; citation_id=CR6
Li, J., Gsaxner, C., Pepe, A., Schmalstieg, D., Kleesiek, J., Egger, J.: Sparse convolutional neural networks for medical image analysis (2022)
Zhu, Z., Xia, Y., Shen, W., Fishman, E., Yuille, A.: A 3d coarse-to-fine framework for volumetric medical image segmentation. In: 2018 International Conference on 3D Vision (3DV), pp. 682–690. IEEE (2018)
Çiçek, Ö., Abdulkadir, A., Lienkamp, S.S., Brox, T., Ronneberger, O.: 3d u-net: learning dense volumetric segmentation from sparse annotation. In: International Conference on Medical Image Computing and Computer-assisted Intervention, pp. 424–432. Springer (2016)
Lei, T., Wang, R., Wan, Y., Du, X., Meng, H., Nandi, A.K.: Medical image segmentation using deep learning: a Survey (2020)
Zhou, Y., Xie, L., Shen, W., Wang, Y., Fishman, E.K., Yuille, A.L.: A fixed-point model for pancreas segmentation in abdominal ct scans. In: International Conference on Medical Image Computing and Computer-assisted Intervention, pp. 693–701. Springer (2017)
Li, J., Chen, J., Tang, Y., Landman, B.A., Zhou, S.K.: Transforming medical imaging with transformers? A comparative review of key properties, current progresses, and future perspectives. arXiv preprint
arXiv:2206.01136
(2022)
citation_journal_title=Vis. Comput.; citation_title=Contour-aware semantic segmentation network with spatial attention mechanism for medical image; citation_author=Z Cheng, A Qu, X He; citation_volume=1; citation_publication_date=2021; citation_pages=1-14; citation_id=CR13
citation_journal_title=Plos One; citation_title=Efficient, high-performance semantic segmentation using multi-scale feature extraction; citation_author=M Knolle, G Kaissis, F Jungmann, S Ziegelmayer, D Sasse, M Makowski, D Rueckert, R Braren; citation_volume=16; citation_issue=8; citation_publication_date=2021; citation_pages=0255397; citation_doi=10.1371/journal.pone.0255397; citation_id=CR14
Ronneberger, O., Fischer, P., Brox, T.: U-net: convolutional networks for biomedical image segmentation. In: International Conference on Medical Image Computing and Computer-assisted Intervention, pp. 234–241. Springer (2015)
citation_journal_title=Vis. Comput.; citation_title=Frnet: an end-to-end feature refinement neural network for medical image segmentation; citation_author=D Wang, G Hu, C Lyu; citation_volume=37; citation_issue=5; citation_publication_date=2021; citation_pages=1101-1112; citation_doi=10.1007/s00371-020-01855-z; citation_id=CR16
Zhao, T., Cao, K., Yao, J., Nogues, I., Lu, L., Huang, L., Xiao, J., Yin, Z., Zhang, L.: 3d graph anatomy geometry-integrated network for pancreatic mass segmentation, diagnosis, and quantitative patient management. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13743–13752 (2021)
Chen, J., Lu, Y., Yu, Q., Luo, X., Adeli, E., Wang, Y., Lu, L., Yuille, A.L., Zhou, Y.: Transunet: Transformers make strong encoders for medical image segmentation. arXiv preprint
arXiv:2102.04306
(2021)
citation_journal_title=The Lancet; citation_title=Pancreatic cancer; citation_author=A Vincent, J Herman, R Schulick, RH Hruban, M Goggins; citation_volume=378; citation_issue=9791; citation_publication_date=2011; citation_pages=607-620; citation_doi=10.1016/S0140-6736(10)62307-0; citation_id=CR19
Oktay, O., Schlemper, J., Folgoc, L.L., Lee, M., Heinrich, M., Misawa, K., Mori, K., McDonagh, S., Hammerla, N.Y., Kainz, B., et al.: Attention u-net: Learning where to look for the pancreas. arXiv preprint
arXiv:1804.03999
(2018)
citation_journal_title=Vis. Comput.; citation_title=Aggregated squeeze-and-excitation transformations for densely connected convolutional networks; citation_author=M Yang, T Ma, Q Tian, Y Tian, A Al-Dhelaan, M Al-Dhelaan; citation_volume=1; citation_publication_date=2021; citation_pages=1-14; citation_id=CR21
Chen, Y., Dai, X., Liu, M., Chen, D., Yuan, L., Liu, Z.: Dynamic convolution: attention over convolution kernels. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11030–11039 (2020)
citation_journal_title=IEEE Trans. Pattern Anal. Mach. Intell.; citation_title=Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs; citation_author=L-C Chen, G Papandreou, I Kokkinos, K Murphy, AL Yuille; citation_volume=40; citation_issue=4; citation_publication_date=2017; citation_pages=834-848; citation_doi=10.1109/TPAMI.2017.2699184; citation_id=CR23
Zagoruyko, S., Komodakis, N.: Wide residual networks. arXiv preprint
arXiv:1605.07146
(2016)
citation_journal_title=IEEE Access; citation_title=Automatic pancreas segmentation using double adversarial networks with pyramidal pooling module; citation_author=M Li, F Lian, S Guo; citation_volume=9; citation_publication_date=2021; citation_pages=140965-140974; citation_doi=10.1109/ACCESS.2021.3118718; citation_id=CR25
citation_journal_title=Med. Image Anal.; citation_title=Pancreas segmentation using a dual-input v-mesh network; citation_author=Y Wang, G Gong, D Kong, Q Li, J Dai, H Zhang, J Qu, X Liu, J Xue; citation_volume=69; citation_publication_date=2021; citation_doi=10.1016/j.media.2021.101958; citation_id=CR26
citation_journal_title=Adv. Neural Inf. Process. Syst.; citation_title=Attention is all you need; citation_author=A Vaswani, N Shazeer, N Parmar, J Uszkoreit, L Jones, AN Gomez, Ł Kaiser, I Polosukhin; citation_volume=30; citation_publication_date=2017; citation_pages=1; citation_id=CR27
Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., Zagoruyko, S.: End-to-end object detection with transformers. In: European Conference on Computer Vision, pp. 213–229. Springer (2020)
Cao, H., Wang, Y., Chen, J., Jiang, D., Zhang, X., Tian, Q., Wang, M.: Swin-unet: Unet-like pure transformer for medical image segmentation. arXiv preprint
arXiv:2105.05537
(2021)
Wang, H., Cao, P., Wang, J., Zaiane, O.R.: Uctransnet: Rethinking the skip connections in u-net from a channel-wise perspective with transformer. arXiv preprint
arXiv:2109.04335
(2021)
Roth, H.R., Farag, A., Lu, L., Turkbey, E.B., Summers, R.M.: Deep convolutional networks for pancreas segmentation in ct imaging. In: Medical Imaging 2015: Image Processing, vol. 9413, p. 94131. International Society for Optics and Photonics (2015)
Cai, J., Lu, L., Xie, Y., Xing, F., Yang, L.: Improving deep pancreas segmentation in ct and mri images via recurrent neural contextual learning and direct loss function. arXiv preprint
arXiv:1707.04912
(2017)
Zhou, Y., Xie, L., Shen, W., Fishman, E., Yuille, A.: Pancreas segmentation in abdominal ct scan: a coarse-to-fine approach. arXiv preprint
arXiv:1612.08230
(2016)
Yu, Q., Xie, L., Wang, Y., Zhou, Y., Fishman, E.K., Yuille, A.L.: Recurrent saliency transformation network: Incorporating multi-stage visual cues for small organ segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 8280–8289 (2018)
citation_journal_title=IEEE Access; citation_title=Pancreatic segmentation via ringed residual u-net; citation_author=L Lu, L Jian, J Luo, B Xiao; citation_volume=7; citation_publication_date=2019; citation_pages=172871-172878; citation_doi=10.1109/ACCESS.2019.2956550; citation_id=CR35
Zhou, Z., Siddiquee, M.M.R., Tajbakhsh, N., Liang, J.: Unet++: A nested u-net architecture for medical image segmentation. In: 4th Deep Learning in Medical Image Analysis (DLMIA) Workshop (2018)
Wang, Z.-H., Liu, Z., Song, Y.-Q., Zhu, Y.: Densely connected deep u-net for abdominal multi-organ segmentation. In: 2019 IEEE International Conference on Image Processing (ICIP), pp. 1415–1419 (2019). IEEE
Wang, W., Song, Q., Feng, R., Chen, T., Chen, J., Chen, D.Z., Wu, J.: A fully 3d cascaded framework for pancreas segmentation. In: 2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI), pp. 207–211 (2020). IEEE
citation_journal_title=IEEE Trans. Cybern.; citation_title=Cascaded multitask 3-d fully convolutional networks for pancreas segmentation; citation_author=J Xue, K He, D Nie, E Adeli, Z Shi, S-W Lee, Y Zheng, X Liu, D Li, D Shen; citation_volume=51; citation_issue=4; citation_publication_date=2019; citation_pages=2153-2165; citation_doi=10.1109/TCYB.2019.2955178; citation_id=CR39
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al.: An image is worth 16x16 words: transformers for image recognition at scale. arXiv preprint
arXiv:2010.11929
(2020)
citation_journal_title=Med. Image Anal.; citation_title=Attention gated networks: learning to leverage salient regions in medical images; citation_author=J Schlemper, O Oktay, M Schaap, M Heinrich, B Kainz, B Glocker, D Rueckert; citation_volume=53; citation_publication_date=2019; citation_pages=197-207; citation_doi=10.1016/j.media.2019.01.012; citation_id=CR41
citation_journal_title=Vis. Comput.; citation_title=X-net: a dual encoding-decoding method in medical image segmentation; citation_author=Y Li, Z Wang, L Yin, Z Zhu, G Qi, Y Liu; citation_volume=1; citation_publication_date=2021; citation_pages=1-11; citation_id=CR42
Sha, Y., Zhang, Y., Ji, X., Hu, L.: Transformer-unet: Raw image processing with unet. arXiv preprint
arXiv:2109.08417
(2021)
Rickmann, A.-M., Roy, A.G., Sarasua, I., Wachinger, C.: Recalibrating 3d convnets with project & excite. IEEE Trans. Med. Imag. 39(7), 2461–2471 (2020)
Andonie, R.: Hyperparameter optimization in learning systems. J. Membr. Comput. 1(4), 279–291 (2019)
Sudre, C.H., Li, W., Vercauteren, T., Ourselin, S., Cardoso, M.J.: Generalised dice overlap as a deep learning loss function for highly unbalanced segmentations. In: International Workshop on Deep Learning in Medical Image Analysis International Workshop on Multimodal Learning for Clinical Decision Support (2017)
citation_journal_title=Med. Phys.; citation_title=Anatomynet: deep learning for fast and fully automated whole-volume segmentation of head and neck anatomy; citation_author=W Zhu, Y Huang, L Zeng, X Chen, Y Liu, Z Qian, N Du, W Fan, X Xie; citation_volume=46; citation_issue=2; citation_publication_date=2019; citation_pages=576-589; citation_doi=10.1002/mp.13300; citation_id=CR47
Isensee, F., Jaeger, P.F., Kohl, S.A., Petersen, J., Maier-Hein, K.H.: nnu-net: a self-configuring method for deep learning-based biomedical image segmentation. Nat. Methods 18(2), 203–211 (2021)