An automatic nuclei segmentation method based on deep convolutional neural networks for histopathology images
Tóm tắt
Since nuclei segmentation in histopathology images can provide key information for identifying the presence or stage of a disease, the images need to be assessed carefully. However, color variation in histopathology images, and various structures of nuclei are two major obstacles in accurately segmenting and analyzing histopathology images. Several machine learning methods heavily rely on hand-crafted features which have limitations due to manual thresholding. To obtain robust results, deep learning based methods have been proposed. Deep convolutional neural networks (DCNN) used for automatically extracting features from raw image data have been proven to achieve great performance. Inspired by such achievements, we propose a nuclei segmentation method based on DCNNs. To normalize the color of histopathology images, we use a deep convolutional Gaussian mixture color normalization model which is able to cluster pixels while considering the structures of nuclei. To segment nuclei, we use Mask R-CNN which achieves state-of-the-art object segmentation performance in the field of computer vision. In addition, we perform multiple inference as a post-processing step to boost segmentation performance. We evaluate our segmentation method on two different datasets. The first dataset consists of histopathology images of various organ while the other consists histopathology images of the same organ. Performance of our segmentation method is measured in various experimental setups at the object-level and the pixel-level. In addition, we compare the performance of our method with that of existing state-of-the-art methods. The experimental results show that our nuclei segmentation method outperforms the existing methods. We propose a nuclei segmentation method based on DCNNs for histopathology images. The proposed method which uses Mask R-CNN with color normalization and multiple inference post-processing provides robust nuclei segmentation results. Our method also can facilitate downstream nuclei morphological analyses as it provides high-quality features extracted from histopathology images.
Tài liệu tham khảo
Ho J, Ahlers SM, Stratman C, Aridor O, Pantanowitz L, Fine JL, Kuzmishin JA, Montalto MC, Parwani AV. Can digital pathology result in cost savings? a financial projection for digital pathology implementation at a large integrated health care organization. J Pathol Informa. 2014; 5:33.
Otsu N. A threshold selection method from gray-level histograms. IEEE Trans Syst Man Cybern. 1979; 9(1):62–6.
Yang X, Li H, Zhou X. Nuclei segmentation using marker-controlled watershed, tracking using mean-shift, and kalman filter in time-lapse microscopy. IEEE Trans Circ Syst I Regular Pap. 2006; 53(11):2405–14.
Filipczuk P, Kowal M, Obuchowicz A. Automatic breast cancer diagnosis based on k-means clustering and adaptive thresholding hybrid segmentation. In: Image Processing and Communications Challenges 3. Springer: 2011. p. 295–302. https://doi.org/10.1007/978-3-642-23154-4_33.
Rother C, Kolmogorov V, Blake A. Grabcut: Interactive foreground extraction using iterated graph cuts. In: ACM Transactions on Graphics (TOG). New York: ACM: 2004. p. 309–14.
Al-Kofahi Y, Lassoued W, Lee W, Roysam B. Improved automatic detection and segmentation of cell nuclei in histopathology images. IEEE Trans Biomed Eng. 2010; 57(4):841–52.
Veta M, Van Diest PJ, Kornegoor R, Huisman A, Viergever MA, Pluim JP. Automatic nuclei segmentation in h&e stained breast cancer histopathology images. PloS ONE. 2013; 8(7):70221.
Vahadane A, Sethi A. Towards generalized nuclear segmentation in histological images. In: Bioinformatics and Bioengineering (BIBE), 2013 IEEE 13th International Conference On. IEEE: 2013. p. 1–4. https://doi.org/10.1109/bibe.2013.6701556.
Chang H, Han J, Borowsky A, Loss L, Gray JW, Spellman PT, Parvin B. Invariant delineation of nuclear architecture in glioblastoma multiforme for clinical and molecular association. IEEE Trans Med Imaging. 2013; 32(4):670–82.
Kong H, Gurcan M, Belkacem-Boussaid K. Partitioning histopathological images: an integrated framework for supervised color-texture segmentation and cell splitting. IEEE Trans Med Imaging. 2011; 30(9):1661–77.
Plissiti ME, Nikou C. Overlapping cell nuclei segmentation using a spatially adaptive active physical model. IEEE Trans Image Process. 2012; 21(11):4568–80.
Zhang M, Wu T, Bennett KM. Small blob identification in medical images using regional features from optimum scale. IEEE Trans Biomed Eng. 2015; 62(4):1051–62.
Xing F, Xie Y, Yang L. An automatic learning-based framework for robust nucleus segmentation. IEEE Trans Med Imaging. 2016; 35(2):550–66.
Sirinukunwattana K, Raza SEA, Tsang Y-W, Snead DR, Cree IA, Rajpoot NM. Locality sensitive deep learning for detection and classification of nuclei in routine colon cancer histology images. IEEE Trans Med Imaging. 2016; 35(5):1196–206.
Song Y, Zhang L, Chen S, Ni D, Lei B, Wang T. Accurate segmentation of cervical cytoplasm and nuclei based on multiscale convolutional network and graph partitioning. IEEE Trans Biomed Eng. 2015; 62(10):2421–33.
Irshad H, Veillard A, Roux L, Racoceanu D. Methods for nuclei detection, segmentation, and classification in digital histopathology: a review—current status and future potential. IEEE Rev Biomed Eng. 2014; 7:97–114.
Xing F, Yang L. Robust nucleus/cell detection and segmentation in digital pathology and microscopy images: a comprehensive review. IEEE Rev Biomed Eng. 2016; 9:234–63.
Long J, Shelhamer E, Darrell T. Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE: 2015. p. 3431–40. https://doi.org/10.1109/cvpr.2015.7298965.
Naylor P, Laé M, Reyal F, Walter T. Nuclei segmentation in histopathology images using deep neural networks. In: Biomedical Imaging (ISBI 2017), 2017 IEEE 14th International Symposium On. IEEE: 2017. p. 933–6. https://doi.org/10.1109/isbi.2017.7950669.
Kumar N, Verma R, Sharma S, Bhargava S, Vahadane A, Sethi A. A dataset and a technique for generalized nuclear segmentation for computational pathology. IEEE Trans Med Imaging. 2017; 36(7):1550–60.
Ronneberger O, Fischer P, Brox T. U-net: Convolutional networks for biomedical image segmentation. In: International Conference on Medical Image Computing and Computer-assisted Intervention. Springer: 2015. p. 234–41. https://doi.org/10.1007/978-3-319-24574-4_28.
Cui Y, Zhang G, Liu Z, Xiong Z, Hu J. A deep learning algorithm for one-step contour aware nuclei segmentation of histopathological images. arXiv preprint. 2018. arXiv:1803.02786.
Johnson JW. Adapting mask-rcnn for automatic nucleus segmentation. arXiv preprint. 2018. arXiv:1805.00500.
Zanjani FG, Zinger S, Bejnordi BE, van der Laak JA, et al.Histopathology Stain-Color Normalization Using Deep Generative Models. Medical Imaging with Deep Learning, (Midl):1–11. 2018.
Ballarò B, Florena AM, Franco V, Tegolo D, Tripodo C, Valenti C. An automated image analysis methodology for classifying megakaryocytes in chronic myeloproliferative disorders. Med Image Anal. 2008; 12(6):703–12.
Kayser K, Görtler J, Metze K, Goldmann T, Vollmer E, Mireskandari M, Kosjerina Z, Kayser G. How to measure image quality in tissue-based diagnosis (diagnostic surgical pathology). In: Diagnostic Pathology. BioMed Central: 2008. p. 11. https://doi.org/10.1186/1746-1596-3-s1-s11.
Vahadane A, Peng T, Sethi A, Albarqouni S, Wang L, Baust M, Steiger K, Schlitter AM, Esposito I, Navab N. Structure-preserving color normalization and sparse stain separation for histological images. IEEE Trans Med Imaging. 2016; 35(8):1962–71.
Khan AM, Rajpoot N, Treanor D, Magee D. A nonlinear mapping approach to stain normalization in digital histopathology images using image-specific color deconvolution. IEEE Trans Biomed Eng. 2014; 61(6):1729–38.
Zarella MD, Yeoh C, Breen DE, Garcia FU. An alternative reference space for h&e color normalization. PloS ONE. 2017; 12(3):0174489.
Kingma DP, Ba J. Adam: A method for stochastic optimization. arXiv preprint. 2014. arXiv:1412.6980.
He K, Gkioxari G, Dollár P, Girshick R. Mask r-cnn. In: Computer Vision (ICCV), 2017 IEEE International Conference On. Piscataway: IEEE: 2017. p. 2980–8.
Ren S, He K, Girshick R, Sun J. Faster r-cnn: Towards real-time object detection with region proposal networks. In: Advances in Neural Information Processing Systems. Institute of Electrical and Electronics Engineers (IEEE): 2015. p. 91–9. https://doi.org/10.1109/tpami.2016.2577031.
Lin T-Y, Dollár P, Girshick R, He K, Hariharan B, Belongie S. Feature pyramid networks for object detection. In: CVPR. IEEE: 2017. p. 4. https://doi.org/10.1109/cvpr.2017.106.
Veta M, Heng YJ, Stathonikos N, Bejnordi BE, Beca F, Wollmann T, Rohr K, Shah MA, Wang D, Rousson M, et al.Predicting breast tumor proliferation from whole-slide images: the tupac16 challenge. arXiv preprint. 2018. arXiv:1807.08284.
Carpenter AE, Jones TR, Lamprecht MR, Clarke C, Kang IH, Friman O, Guertin DA, Chang JH, Lindquist RA, Moffat J, et al.Cellprofiler: image analysis software for identifying and quantifying cell phenotypes. Genome Biol. 2006; 7(10):100.
Dong F, Irshad H, Oh E-Y, Lerwill MF, Brachtel EF, Jones NC, Knoblauch NW, Montaser-Kouhsari L, Johnson NB, Rao LK, et al.Computational pathology to discriminate benign from malignant intraductal proliferations of the breast. PloS ONE. 2014; 9(12):114885.
Noh H, Hong S, Han B. Learning deconvolution network for semantic segmentation. In: Proceedings of the IEEE International Conference on Computer Vision. IEEE: 2015. p. 1520–8. https://doi.org/10.1109/iccv.2015.178.