Deep learning for digital pathology image analysis: A comprehensive tutorial with selected use cases

Gurcan, 2009, Histopathological image analysis: A review, IEEE Rev Biomed Eng, 2, 147, 10.1109/RBME.2009.2034865 Veta, 2014, Breast cancer histopathology image analysis: A review, IEEE Trans Biomed Eng, 61, 1400, 10.1109/TBME.2014.2303852 Bhargava, 2016, A review of emerging themes in image informatics and molecular analysis for digital pathology, Annu Rev Biomed Eng, 18 Lewis, 2014, A quantitative histomorphometric classifier (QuHbIC) identifies aggressive versus indolent p16-positive oropharyngeal squamous cell carcinoma, Am J Surg Pathol, 38, 128, 10.1097/PAS.0000000000000086 Basavanhally, 2011, Multi-field-of-view strategy for image-based outcome prediction of multi-parametric estrogen receptor-positive breast cancer histopathology: Comparison to oncotype DX, J Pathol Inform, 2, S1, 10.4103/2153-3539.92027 Veta, 2015, Assessment of algorithms for mitosis detection in breast cancer histopathology images, Med Image Anal, 20, 237, 10.1016/j.media.2014.11.010 Roux, 2013, Mitosis detection in breast cancer histological images An ICPR 2012 contest, J Pathol Inform, 4, 8, 10.4103/2153-3539.112693 Ciresan, 2013, Mitosis detection in breast cancer histology images with deep neural networks, Med Image Comput Comput Assist Interv, 16(Pt 2), 411 Cruz-Roa A, Basavanhally A, González F, Gilmore H, Feldman M, Ganesan S, et al. Automatic detection of invasive ductal carcinoma in whole slide images with convolutional neural networks. In: SPIE Medical Imaging. Vol. 9041.;2014. p. 904103-904103-15. Chen T, Chefd’hotel C. Deep learning based automatic immune cell detection for immunohistochemistry images. In: Wu G, Zhang D, Zhou L, editors. Machine Learning in Medical Imaging. (Lecture Notes in Computer Science). Vol. 8679.: Springer International Publishing; 2014. p. 17-24. Goodfellow IJ, Warde-Farley D, Lamblin P, et al. Pylearn2: A machine learning research library. arXiv preprint arXiv: 1308.4214; 2013. Bastien F, Lamblin P, Pascanu R, Bergstra J, Goodfellow IJ, Bergeron A, et al. “Theano: New Features and Speed Improvements.” Deep Learning and Unsupervised Feature Learning NIPS 2012 Workshop; 2012. LeCun Y, Bottou L, Orr G, Muller K. Efficient backprop. In: Orr G, Müller KR, editors. Neural Networks: Tricks of the Trade. Springer; 1998. 2012, Vol. 7700 Ciresan D, Giusti A, Gambardella LM, Schmidhuber J. Deep neural networks segment neuronal membranes in electron microscopy images. In: Pereira F, Burges C, Bottou L, Weinberger K, editors. Advances in Neural Information Processing Systems 25. Curran Associates, Inc.; 2012. p. 2843-51. Kainz P, Pfeiffer M, Urschler M. Semantic Segmentation of Colon Glands with Deep Convolutional Neural Networks and Total Variation Segmentation. CoRR, Vol. abs/1511.06919; 2015. Maqlin, 2015, Automated nuclear pleomorphism scoring in breast cancer histopathology images using deep neural networks, Vol. 9468, 269 Sirinukunwattana K, Raza S, Tsang YW, Snead D, Cree I, Rajpoot N. Locality sensitive deep learning for detection and classification of nuclei in routine colon cancer histology images. IEEE Trans Med Imaging 2016. Zhou Y, Chang H, Barner KE, Parvin B. Nuclei Segmentation via Sparsity Constrained Convolutional Regression. In: Biomedical Imaging (ISBI), 2015 IEEE 12th International Symposium on; April, 2015. p. 1284-7. Xu, 2016, Stacked sparse autoencoder (SSAE) for nuclei detection on breast cancer histopathology images, IEEE Trans Med Imaging, 35, 119, 10.1109/TMI.2015.2458702 Xu Y, Jia Z, Ai Y, Zhang F, Lai M, Chang EIC. Deep Convolutional Activation Features for Large Scale Brain Tumor Histopathology Image Classification and Segmentation. In: Acoustics, Speech and Signal Processing (ICASSP), 2015 IEEE International Conference on; April, 2015. p. 947-51. Sirinukunwattana, 2015, A spatially constrained deep learning framework for detection of epithelial tumor nuclei in cancer histology images, 154 Xu, 2016, A deep convolutional neural network for segmenting and classifying epithelial and stromal regions in histopathological images, Neurocomputing, 191, 214, 10.1016/j.neucom.2016.01.034 Ertosun, 2015, Automated grading of gliomas using deep learning in digital pathology images: A modular approach with ensemble of convolutional neural networks, AMIA Annu Symp Proc, 2015, 1899 Jia Y, Shelhamer E, Donahue J, Karayev S, Long J, Girshick R, et al. Caffe: Convolutional Architecture for Fast Feature Embedding. arXiv preprint arXiv: 1408.5093; 2014. Krizhevsky Krizhevsky, 2012, Imagenet classification with deep convolutional neural networks, 1097 Janowczyk Gurcan MN, Madabhushi A, Rajpoot N. Pattern recognition in histopathological images: An ICPR 2010 contest. In: Ünay D, Çataltepe Z, Aksoy S, editors. Recognizing Patterns in Signals, Speech, Images and Videos. (Lecture Notes in Computer Science). Vol. 6388: Springer Berlin Heidelberg; 2010. p. 226-34. Doyle, 2012, A boosted Bayesian multiresolution classifier for prostate cancer detection from digitized needle biopsies, IEEE Trans Biomed Eng, 59, 1205, 10.1109/TBME.2010.2053540 Krizhevsky, 2009 LeCun, 2015, Deep learning, Nature, 521, 436, 10.1038/nature14539 Lee, 2009, Convolutional Deep Belief Networks for Scalable Unsupervised Learning of Hierarchical Representations, 609 LeCun Y, Kavukcuoglu K, Farabet C. Convolutional Networks and Applications in Vision. In: International Symposium on Circuits and Systems (ISCAS 2010), May 30 – June 2, 2010, Paris, France; 2010. p. 253-6. Nair, 2010, Rectified linear units improve restricted boltzmann machines, 807 Dahl GE, Sainath TN, Hinton GE. Improving Deep Neural Networks for LVCSR Using Rectified Linear Units and Dropout. In: IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2013, Vancouver, BC, Canada; 26-31 May, 2013. p. 8609-13. Glorot X, Bordes A, Bengio Y. Deep sparse rectifier neural networks. In: Gordon GJ, Dunson DB, editors. Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics (AISTATS-11). Vol. 15. Workshop and Conference Proceedings; 2011. p. 315-23. [Journal of Machine Learning Research]. Srivastava, 2014, Dropout: A simple way to prevent neural networks from overfitting, J Mach Learn Res, 15, 1929 Duchi, 2011, Adaptive subgradient methods for online learning and stochastic optimization, J Mach Learn Res, 12, 2121 Genestie, 1998, Comparison of the prognostic value of scarff-bloom-richardson and nottingham histological grades in a series of 825 cases of breast cancer: Major importance of the mitotic count as a component of both grading systems, Anticancer Res, 18, 571 Humphrey, 2004, Gleason grading and prognostic factors in carcinoma of the prostate, Mod Pathol, 17, 292, 10.1038/modpathol.3800054 Irshad, 2014, Methods for nuclei detection, segmentation, and classification in digital histopathology: A review-current status and future potential, IEEE Rev Biomed Eng, 7, 97, 10.1109/RBME.2013.2295804 Ruifrok, 2001, Quantification of histochemical staining by color deconvolution, Anal Quant Cytol Histol, 23, 291 Beck, 2011, Systematic analysis of breast cancer morphology uncovers stromal features associated with survival, Sci Transl Med, 3, 108ra113, 10.1126/scitranslmed.3002564 Basavanhally A, Yu E, Xu J, Ganesan S, Feldman M, Tomaszewski J, et al. Incorporating domain knowledge for tubule detection in breast histopathology using o’callaghan neighborhoods. In: SPIE Medical Imaging. (Computer-Aided Diagnosis). Vol. 7963. SPIE; 2011. p. 796310. Sirinukunwattana K, Snead D, Rajpoot N. A Random Polygons Model of Glandular Structures in Colon Histology Images. In: Biomedical Imaging (ISBI), 2015 IEEE 12th International Symposium on; April, 2015. p. 1526-9. Fatakdawala, 2010, Expectation-maximization-driven geodesic active contour with overlap resolution (EMaGACOR): Application to lymphocyte segmentation on breast cancer histopathology, IEEE Trans Biomed Eng, 57, 1676, 10.1109/TBME.2010.2041232 Arteta C, Lempitsky V, Noble JA, Zisserman A. Learning to detect cells using non-overlapping extremal regions. In: Ayache N, editor. International Conference on Medical Image Computing and Computer Assisted Intervention. (Lecture Notes in Computer Science). MICCAI, Springer; 2012. p. 348-56. Chang H, Loss L, Parvin B. Nuclear Segmentation in H and E Sections via Multi-reference Graph-cut (mrgc). International Symposium Biomedical Imaging; 2012. Wang, 2014, Mitosis detection in breast cancer pathology images by combining handcrafted and convolutional neural network features, J Med Imaging (Bellingham), 1, 034003, 10.1117/1.JMI.1.3.034003 Orlov, 2008, WND-CHARM: Multi-purpose image classification using compound image transforms, Pattern Recognit Lett, 29, 1684, 10.1016/j.patrec.2008.04.013 Zeiler MD, Fergus R. Visualizing and Understanding Convolutional Networks. In: Computer Vision – ECCV 2014 – 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part I; 2014. p. 818-33. Erhan D, Bengio Y, Courville A, Vincent P. Visualizing Higher-layer Features of a Deep Network. Tech. Rep. 1341, University of Montreal, June 2009. ICML 2009 Workshop on Learning Feature Hierarchies, Montréal, Canada; 2009. Janowczyk A, Doyle S, Gilmore H, Madabhushi A. A resolution adaptive deep hierarchical (radhical) learning scheme applied to nuclear segmentation of digital pathology images. In: Computer Methods in Biomechanics and Biomedical Engineering: Imaging & Visualization. 2016.