Artificial intelligence in radiology

Editors, N. Auspicious machine learning. Nat. Biomed. Engineer. 1, 0036 (2017).

Mnih, V. et al. Human-level control through deep reinforcement learning. Nature 518, 529–533 (2015).

Moravcík, M. et al. DeepStack: Expert-level artificial intelligence in heads-up no-limit poker. Science 356, 508–513 (2017).

Xiong, W. et al. Toward human parity in conversational speech recognition. IEEE/ACM Trans. Audio Speech Language Process. 25, 2410–2423 (2017).

Pendleton, S. D. et al. Perception, planning, control, and coordination for autonomous vehicles. Machines 5, 6 (2017).

Silver, D. et al. Mastering the game of Go with deep neural networks and tree search. Nature 529, 484–489 (2016).

Grace, K., Salvatier, J., Dafoe, A., Zhang, B. & Evans, O. When will AI exceed human performance? Evidence from AI experts. Preprint at arXiv, 1705.08807 (2017).

Rusk, N. Deep learning. Nat. Methods 13, 35–35 (2015).

Esteva, A. et al. Dermatologist-level classification of skin cancer with deep neural networks. Nature 542, 115–118 (2017).

Gulshan, V. et al. Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. JAMA 316, 2402–2410 (2016).

Aerts, H. J. W. L. The potential of radiomic-based phenotyping in precision medicine: a review. JAMA Oncol. 2, 1636–1642 (2016).

Kumar, V. et al. Radiomics: the process and the challenges. Magn. Reson. Imag. 30, 1234–1248 (2012).

Lambin, P. et al. Radiomics: extracting more information from medical images using advanced feature analysis. Eur. J. Cancer 48, 441–446 (2012).

Litjens, G. et al. A survey on deep learning in medical image analysis. Med. Image Anal. 42, 60–88 (2017).

Kolossváry, M., Kellermayer, M., Merkely, B. & Maurovich-Horvat, P. Cardiac computed tomography radiomics: a comprehensive review on radiomic techniques. J. Thorac. Imag. 33, 26–34 (2018).

Aerts, H. J. W. L. et al. Decoding tumour phenotype by noninvasive imaging using a quantitative radiomics approach. Nat. Commun. 5, 4006 (2014).

Coroller, T. P. et al. CT-based radiomic signature predicts distant metastasis in lung adenocarcinoma. Radiother. Oncol. 114, 345–350 (2015).

Wu, W. et al. Exploratory study to identify radiomics classifiers for lung cancer histology. Front. Oncol. 6, 71 (2016).

Huynh, E. et al. Associations of radiomic data extracted from static and respiratory-gated CT scans with disease recurrence in lung cancer patients treated with SBRT. PLoS ONE 12, e0169172 (2017).

Rios Velazquez, E. et al. Somatic mutations drive distinct imaging phenotypes in lung cancer. Cancer Res. 77, 3922–3930 (2017).

Grossmann, P. et al. Defining the biological basis of radiomic phenotypes in lung cancer. eLife 6, e23421 (2017).

Parmar, C., Grossmann, P., Bussink, J., Lambin, P. & Aerts, H. J. W. L. Machine learning methods for quantitative radiomic biomarkers. Sci. Rep. 5, 13087 (2015).

O’Connor, J. P. B. et al. Imaging biomarker roadmap for cancer studies. Nat. Rev. Clin. Oncol. 14, 169–186 (2017).

Boland, G. W. L., Guimaraes, A. S. & Mueller, P. R. The radiologist’s conundrum: benefits and costs of increasing CT capacity and utilization. Eur. Radiol. 19, 9–12 (2009).

McDonald, R. J. et al. The effects of changes in utilization and technological advancements of cross-sectional imaging on radiologist workload. Acad. Radiol. 22, 1191–1198 (2015).

Fitzgerald, R. Error in radiology. Clin. Radiol. 56, 938–946 (2001).

Ledley, R. S. & Lusted, L. B. Reasoning foundations of medical diagnosis; symbolic logic, probability, and value theory aid our understanding of how physicians reason. Science 130, 9–21 (1959).

Lodwick, G. S., Keats, T. E. & Dorst, J. P. The coding of Roentgen images for computer analysis as applied to lung cancer. Radiology 81, 185–200 (1963).

Ambinder, E. P. A history of the shift toward full computerization of medicine. J. Oncol. Pract. 1, 54–56 (2005).

Haug, P. J. Uses of diagnostic expert systems in clinical care. Proc. Annu. Symp. Comput. Appl. Med. Care, 379–383 (1993).

Castellino, R. A. Computer aided detection (CAD): an overview. Cancer Imag. 5, 17–19 (2005).

Shen, D., Wu, G. & Suk, H.-I. Deep learning in medical image analysis. Annu. Rev. Biomed. Eng. 19, 221–248 (2017).

Veeraraghavan, H. MO-A-207B-01: Radiomics: Segmentation & feature extraction techniques. Med. Phys. 43, 3694–3694 (2016).

Paul, R. et al. Deep feature transfer learning in combination with traditional features predicts survival among patients with lung adenocarcinoma. Tomography 2, 388–395 (2016).

Cheng, J.-Z. et al. Computer-aided diagnosis with deep learning architecture: applications to breast lesions in US images and pulmonary nodules in CT scans. Sci. Rep. 6, 24454 (2016).

Chen, H., Zheng, Y., Park, J.-H., Heng, P.-A. & Zhou, S. K. in Medical Image Computing and Computer-Assisted Intervention — MICCAI 2016 487–495 (Athens, Greece, 2016).

Ghafoorian, M. et al. Location sensitive deep convolutional neural networks for segmentation of white matter hyperintensities. Sci. Rep. 7, 5110 (2017).

Wang, H. et al. Comparison of machine learning methods for classifying mediastinal lymph node metastasis of non-small cell lung cancer from 18F-FDG PET/CT images. EJNMMI Res. 7, 11 (2017).

van Ginneken, B., Schaefer-Prokop, C. M. & Prokop, M. Computer-aided diagnosis: how to move from the laboratory to the clinic. Radiology 261, 719–732 (2011).

Nagaraj, S., Rao, G. N. & Koteswararao, K. The role of pattern recognition in computer-aided diagnosis and computer-aided detection in medical imaging: a clinical validation. Int. J. Comput. Appl. 8, 18–22 (2010).

Cole, E. B. et al. Impact of computer-aided detection systems on radiologist accuracy with digital mammography. AJR Am. J. Roentgenol. 203, 909–916 (2014).

Lehman, C. D. et al. Diagnostic accuracy of digital screening mammography with and without computer-aided detection. JAMA Intern. Med. 175, 1828–1837 (2015).

Huang, X., Shan, J. & Vaidya, V. in 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017) 379–383 (Melbourne, Australia, 2017).

Tsehay, Y. K. et al. in Proceedings of SPIE https://doi.org/10.1117/12.2254423 (2017).

Kooi, T. et al. Large scale deep learning for computer aided detection of mammographic lesions. Med. Image Anal. 35, 303–312 (2017).

Sharma, N. & Aggarwal, L. M. Automated medical image segmentation techniques. J. Med. Phys. 35, 3–14 (2010).

Haralick, R. M. & Shapiro, L. G. Image segmentation techniques. Computer Vision Graph. Image Process. 29, 100–132 (1985).

Pham, D. L., Xu, C. & Prince, J. L. Current methods in medical image segmentation. Annu. Rev. Biomed. Eng. 2, 315–337 (2000).

Grau, V., Mewes, A. U. J., Alcañiz, M., Kikinis, R. & Warfield, S. K. Improved watershed transform for medical image segmentation using prior information. IEEE Trans. Med. Imag. 23, 447–458 (2004).

Parisot, S. et al. A probabilistic atlas of diffuse WHO grade II glioma locations in the brain. PLoS ONE 11, e0144200 (2016).

Ghose, S. et al. in 2012 19th IEEE International Conference on Image Processing 541–544 (Orlando, FL, USA, 2012).

Han, X. et al. Atlas-based auto-segmentation of head and neck CT images. Med. Image Comput. Comput. Assist. Interv. 11, 434–441 (2008).

Long, J., Shelhamer, E. & Darrell, T. in 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 3431–3440 (Boston, MA, USA, 2015).

Ronneberger, O., Fischer, P. & Brox, T. U. in Medical Image Computing and Computer-Assisted Intervention — MICCAI 2015 234–241 (Munich, Germany, 2015).

Moeskops, P. et al. in Medical Image Computing and Computer-Assisted Intervention — MICCAI 2016 478–486 (Athens, Greece, 2016).

de Brebisson, A. & Montana, G. in 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) 20–28 (Boston, MA, USA, 2015).

Cioffi, U., Raveglia, F., De Simone, M. & Baisi, A. Ground-glass opacities: a curable disease but a big challenge for surgeons. J. Thorac. Cardiovasc. Surg. 154, 375–376 (2017).

Champaign, J. L. & Cederbom, G. J. Advances in breast cancer detection with screening mammography. Ochsner J. 2, 33–35 (2000).

Shiraishi, J., Li, Q., Appelbaum, D. & Doi, K. Computer-aided diagnosis and artificial intelligence in clinical imaging. Semin. Nucl. Med. 41, 449–462 (2011).

Ayer, T., Ayvaci, M. U., Liu, Z. X., Alagoz, O. & Burnside, E. S. Computer-aided diagnostic models in breast cancer screening. Imag. Med. 2, 313–323 (2010).

Zhang, J., Wang, Y., Yu, B., Shi, X. & Zhang, Y. Application of computer-aided diagnosis to the sonographic evaluation of cervical lymph nodes. Ultrason. Imag. 38, 159–171 (2016).

Giannini, V. et al. A fully automatic computer aided diagnosis system for peripheral zone prostate cancer detection using multi-parametric magnetic resonance imaging. Comput. Med. Imaging Graph. 46, 219–226 (2015).

El-Baz, A. et al. Computer-aided diagnosis systems for lung cancer: challenges and methodologies. Int. J. Biomed. Imag. 2013, 942353 (2013).

Edey, A. J. & Hansell, D. M. Incidentally detected small pulmonary nodules on CT. Clin. Radiol. 64, 872–884 (2009).

Mirsadraee, S., Oswal, D., Alizadeh, Y., Caulo, A. & van Beek, E. Jr. The7th lung cancer TNM classification and staging system: review of the changes and implications. World J. Radiol. 4, 128–134 (2012).

Sohn, K., Shang, W. & Lee, H. in Advances in Neural Information Processing Systems 27 (NIPS 2014) (eds Ghahramani, Z., Welling, M., Cortes, C., Lawrence, N. D. & Weinberger, K. Q.) 2141–2149 (Montreal, Canada, 2014).

Litjens, G. et al. Deep learning as a tool for increased accuracy and efficiency of histopathological diagnosis. Sci. Rep. 6, 26286 (2016).

Cruz-Roa, A. et al. Accurate and reproducible invasive breast cancer detection in whole-slide images: A Deep Learning approach for quantifying tumor extent. Sci. Rep. 7, 46450 (2017).

Jaffe, C. C. Measures of response: RECIST, WHO, and new alternatives. J. Clin. Oncol. 24, 3245–3251 (2006).

Thiesse, P. et al. Response rate accuracy in oncology trials: reasons for interobserver variability. Groupe Français d’Immunothérapie of the Fédération Nationale des Centres de Lutte Contre le Cancer. J. Clin. Oncol. 15, 3507–3514 (1997).

Khorasani, R., Erickson, B. J. & Patriarche, J. New opportunities in computer-aided diagnosis: change detection and characterization. J. Am. Coll. Radiol. 3, 468–469 (2006).

Patriarche, J. W. & Erickson, B. J. Part 1. Automated change detection and characterization in serial MR studies of brain-tumor patients. J. Digit. Imag. 20, 203–222 (2007).

Pan, X., Sidky, E. Y. & Vannier, M. Why do commercial CT scanners still employ traditional, filtered back-projection for image reconstruction? Inverse Probl. 25, 1230009 (2009).

Pipatsrisawat, T., Gacic, A., Franchetti, F., Puschel, M. & Moura, J. M. F. in Proceedings. (ICASSP ‘05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005 v/153–v/156 (Philadelphia, PA, USA, 2005).

Zhu, B., Liu, J. Z., Cauley, S. F., Rosen, B. R. & Rosen, M. S. Image reconstruction by domain-transform manifold learning. Nature 555, 487–492 (2018).

Hammernik, K., Würfl, T., Pock, T. & Maier, A. A. in Bildverarbeitung für die Medizin 2017 (eds Maier-Hein, K., Deserno, T., Handels, H. & Tolxdorff, T.) 92–97 (Springer, Berlin, Heidelberg, 2017).

Gjesteby, L. et al. in Developments in X-Ray Tomography XI 10391-31 (San Diego, CA, USA, 2017).

El-Gamal, F. E.-Z. A., Elmogy, M. & Atwan, A. Current trends in medical image registration and fusion. Egypt. Informat. J. 17, 99–124 (2016).

Yang, X., Kwitt, R., Styner, M. & Niethammer, M. Quicksilver: fast predictive image registration — a deep learning approach. Neuroimage 158, 378–396 (2017).

Ngiam, J. et al. in Proceedings of the 28th International Conference on Machine Learning 689–696 (Bellevue, WA, USA, 2011).

Yankeelov, T. E., Abramson, R. G. & Quarles, C. C. Quantitative multimodality imaging in cancer research and therapy. Nat. Rev. Clin. Oncol. 11, 670–680 (2014).

Johnson, A. J., Chen, M. Y. M., Zapadka, M. E., Lyders, E. M. & Littenberg, B. Radiology report clarity: a cohort study of structured reporting compared with conventional dictation. J. Am. Coll. Radiol. 7, 501–506 (2010).

Levy, M. A. & Rubin, D. L. Tool support to enable evaluation of the clinical response to treatment. AMIA Annu. Symp. Proc. 2008, 399–403 (2008).

European Society of Radiology (ESR). Good practice for radiological reporting. Guidelines from the European Society of Radiology (ESR). Insights Imag. 2, 93–96 (2011).

Folio, L. R. et al. Quantitative radiology reporting in oncology: survey of oncologists and radiologists. AJR Am. J. Roentgenol. 205, W233–W243 (2015).

Karpathy, A. & Fei-Fei, L. Deep visual-semantic alignments for generating image descriptions. IEEE Trans. Pattern Anal. Mach. Intell. 39, 664–676 (2017).

Shin, H.-C. et al. in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2497–2506 (Las Vegas, NV, USA, 2016).

Lee, J.-G. et al. Deep learning in medical imaging: general overview. Kor. J. Radiol. 18, 570–584 (2017).

OECD. Computed tomography (CT) exams. https://doi.org/10.1787/3c994537-en (2018).

OECD. Magnetic resonance imaging (MRI) exams. https://doi.org/10.1787/1d89353f-en (2018).

Bryan, S. et al. Radiology report times: impact of picture archiving and communication systems. AJR Am. J. Roentgenol. 170, 1153–1159 (1998).

Mansoori, B., Erhard, K. K. & Sunshine, J. L. Picture Archiving and Communication System (PACS) implementation, integration and benefits in an integrated health system. Acad. Radiol. 19, 229–235 (2012).

Lemke, H. U. PACS developments in Europe. Comput. Med. Imag. Graph. 27, 111–120 (2003).

Mendel, J. B. & Schweitzer, A. L. PACS for the developing world. J. Global Radiol. 1, 5 (2015).

Goodfellow, I. et al. in Advances in Neural Information Processing Systems 27 (NIPS 2014) (eds Ghahramani, Z., Welling, M., Cortes, C., Lawrence, N. D. & Weinberger, K. Q.) 2672–2680 (Montreal, Canada, 2014).

Kingma, D. P. & Welling, M. Auto-encoding variational Bayes. Preprint at arXiv, 1312.6114 (2013).

Kamnitsas, K. et al. in Information Processing in Medical Imaging 597–609 (Springer, Cham, 2017).

Kallenberg, M. et al. Unsupervised deep learning applied to breast density segmentation and mammographic risk scoring. IEEE Trans. Med. Imag. 35, 1322–1331 (2016).

Zhang, P., Wang, F. & Zheng, Y. in 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017) 578–582 (Melbourne, Australia, 2017).

Clark, K. et al. The Cancer Imaging Archive (TCIA): maintaining and operating a public information repository. J. Digit. Imag. 26, 1045–1057 (2013).

Wang, G. A. Perspective on deep imaging. IEEE Access 4, 8914–8924 (2016).

Ford, R. A., Price, W. & Nicholson, I. I. Privacy and accountability in black-box medicine. Mich. Telecomm. Tech. L. Rev. 23, 1 (2016).

Selbst, A. D. & Powles, J. Meaningful information and the right to explanation. Int. Data Privacy Law 7, 233–242 (2017).

Imming, P., Sinning, C. & Meyer, A. Drugs, their targets and the nature and number of drug targets. Nat. Rev. Drug Discov. 5, 821–834 (2006).

Mehlhorn, H. et al. in Encyclopedia of Parasitology 3rd edn (ed. Mehlhorn, H.) 400–402 (Springer, Berlin, Heidelberg, 2008).

Shokri, R. & Shmatikov, V. in Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security 1310–1321 (Denver, CO, USA, 2015).

Phong, L. T., Aono, Y., Hayashi, T., Wang, L. & Moriai, S. in Applications and Techniques in Information Security. 8th International Conference, ATIS 2017 (eds Batten, L., Kim, D. S., Zhang, X. & Li, G.) 719, 100–110 (Auckland, New Zealand, 2017).

McMahan, H. B., Moore, E., Ramage, D., Hampson, S. & y Arcas, B. A. in Proceedings of the 20th International Conference on Artificial Intelligence and Statistics (AISTATS) 1273–1282 (Fort Lauderdale, FL, USA, 2017).

Gilad-Bachrach, R. et al. in Proceedings of the 33rd International Conference on Machine Learning 201–210 (New York, NY, USA, 2016).

Cahan, A. & Cimino, J. J. A. Learning health care system using computer-aided diagnosis. J. Med. Internet Res. 19, e54 (2017).

LeCun, Y., Bengio, Y. & Hinton, G. Deep learning. Nature 521, 436–444 (2015).

Miotto, R., Wang, F., Wang, S., Jiang, X. & Dudley, J. T. Deep learning for healthcare: review, opportunities and challenges. Brief. Bioinform. https://doi.org/10.1093/bib/bbx044 (2017).

Kevin Zhou, S., Greenspan, H. & Shen, D. Deep Learning for Medical Image Analysis. (Academic Press, 2017).

Shin, H.-C. et al. Deep convolutional neural networks for computer-aided detection: CNN architectures, dataset characteristics and transfer learning. IEEE Trans. Med. Imag. 35, 1285–1298 (2016).

Shin, Y. & Balasingham, I. in 2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) 3277–3280 (Jeju Island, Korea, 2017).

Orringer, D. A. et al. Rapid intraoperative histology of unprocessed surgical specimens via fibre-laser-based stimulated Raman scattering microscopy. Nat. Biomed. Eng. 1, 0027 (2017).

Albarqouni, S. et al. AggNet: deep learning from crowds for mitosis detection in breast cancer histology images. IEEE Trans. Med. Imag. 35, 1313–1321 (2016).

Djuric, U., Zadeh, G., Aldape, K. & Diamandis, P. Precision histology: how deep learning is poised to revitalize histomorphology for personalized cancer care. Precision Oncol. 1, 22 (2017).

Janowczyk, A. & Madabhushi, A. Deep learning for digital pathology image analysis: a comprehensive tutorial with selected use cases. J. Pathol. Inform. 7, 29 (2016).

Bejnordi, B. E. et al. in 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017) 929–932 (Melbourne, Australia, 2017).

Yuan, Y. et al. DeepGene: an advanced cancer type classifier based on deep learning and somatic point mutations. BMC Bioinform. 17, 476 (2016).

Alipanahi, B., Delong, A., Weirauch, M. T. & Frey, B. J. Predicting the sequence specificities of DNA- and RNA-binding proteins by deep learning. Nat. Biotechnol. 33, 831–838 (2015).