Artificial Intelligence for Ultrasound Informative Image Selection of Metacarpal Head Cartilage. A Pilot Study

Frontiers in Medicine - Tập 8
Edoardo Cipolletta1, Maria Chiara Fiorentino2, Sara Moccia3,2, Irene Guidotti2, Walter Grassi1, Emilio Filippucci1, Emanuele Frontoni2
1Rheumatology Unit, Department of Clinical and Molecular Sciences, Polytechnic University of Marche, Ancona, Italy
2Department of Information Engineering, Polytechnic University of Marche, Ancona, Italy
3Department of Advanced Robotics, Italian Institute of Technology, Genoa, Italy

Tóm tắt

Objectives:This study aims to develop an automatic deep-learning algorithm, which is based on Convolutional Neural Networks (CNNs), for ultrasound informative-image selection of hyaline cartilage at metacarpal head level. The algorithm performance and that of three beginner sonographers were compared with an expert assessment, which was considered the gold standard.Methods:The study was divided into two steps. In the first one, an automatic deep-learning algorithm for image selection was developed using 1,600 ultrasound (US) images of the metacarpal head cartilage (MHC) acquired in 40 healthy subjects using a very high-frequency probe (up to 22 MHz). The algorithm task was to identify US images defined informative as they show enough information to fulfill the Outcome Measure in Rheumatology US definition of healthy hyaline cartilage. The algorithm relied on VGG16 CNN, which was fine-tuned to classify US images in informative and non-informative ones. A repeated leave-four-subject out cross-validation was performed using the expert sonographer assessment as gold-standard. In the second step, the expert assessed the algorithm and the beginner sonographers' ability to obtain US informative images of the MHC.Results:The VGG16 CNN showed excellent performance in the first step, with a mean area (AUC) under the receiver operating characteristic curve, computed among the 10 models obtained from cross-validation, of 0.99 ± 0.01. The model that reached the best AUC on the testing set, which we named “MHC identifier 1,” was then evaluated by the expert sonographer. The agreement between the algorithm, and the expert sonographer was almost perfect [Cohen's kappa: 0.84 (95% confidence interval: 0.71–0.98)], whereas the agreement between the expert and the beginner sonographers using conventional assessment was moderate [Cohen's kappa: 0.63 (95% confidence interval: 0.49–0.76)]. The conventional obtainment of US images by beginner sonographers required 6.0 ± 1.0 min, whereas US videoclip acquisition by a beginner sonographer lasted only 2.0 ± 0.8 min.Conclusion:This study paves the way for the automatic identification of informative US images for assessing MHC. This may redefine the US reliability in the evaluation of MHC integrity, especially in terms of intrareader reliability and may support beginner sonographers during US training.

Từ khóa


Tài liệu tham khảo

Fox, 2009, The basic science of articular cartilage: structure, composition, and function, Sports Health, 1, 461, 10.1177/1941738109350438

Pap, 2015, Cartilage damage in osteoarthritis and rheumatoid arthritis-two unequal siblings, Nat Rev Rheumatol, 11, 606, 10.1038/nrrheum.201595

Navarro-compán, 2014, Relationship between types of radiographic damage and disability in patients with rheumatoid arthritis in the EURIDISS cohort: a longitudinal study, Rheumatology, 54, 83, 10.1093/rheumatology/keu284

Mandl, 2015, Relationship between radiographic joint space narrowing, sonographic cartilage thickness and anatomy in rheumatoid arthritis and control joints, Ann Rheum Dis, 74, 2022, 10.1136/annrheumdis-2014-205585

Døhn, 2006, Are bone erosions detected by magnetic resonance imaging and ultrasonography true erosions? A comparison with computed tomography in rheumatoid arthritis metacarpophalangeal joints, Arthritis Res Ther, 8, R110, 10.1186/ar1995

Wakefield, 2000, The value of sonography in the detection of bone erosions in patients with rheumatoid arthritis: a comparison with conventional radiography, Arthritis Rheum, 43, 2762, 10.1002/1529-0131(200012)43:12<2762::AID-ANR16>3.0.CO;2

Szkudlarek, 2006, Ultrasonography of the metacarpophalangeal and proximal interphalangeal joints in rheumatoid arthritis: a comparison with magnetic resonance imaging, conventional radiography and clinical examination, Arthritis Res Ther, 8, R52, 10.1186/ar1904

Scheel, 2006, Prospective 7 year follow up imaging study comparing radiography, ultrasonography, and magnetic resonance imaging in rheumatoid arthritis finger joints, Ann Rheum Dis, 65, 595, 10.1136/ard.2005041814

Funck-Brentano, 2009, Benefits of ultrasonography in the management of early arthritis: a cross-sectional study of baseline data from the ESPOIR cohort, Rheumatology, 48, 1515, 10.1093/rheumatology/kep279

Wiell, 2007, Ultrasonography, magnetic resonance imaging, radiography, and clinical assessment of inflammatory and destructive changes in fingers and toes of patients with psoriatic arthritis, Arthritis Res Ther, 9, R119, 10.1186/ar2327

Hurnakova, 2019, Prevalence and distribution of cartilage damage at the metacarpal head level in rheumatoid arthritis and osteoarthritis: an ultrasound study, Rheumatology, 58, 1206, 10.1093/rheumatology/key443

Möller, 2009, Measuring finger joint cartilage by ultrasound as a promising alternative to conventional radiograph imaging, Arthritis Rheum, 61, 435, 10.1002/art24424

Filippucci, 2010, Interobserver reliability of ultrasonography in the assessment of cartilage damage in rheumatoid arthritis, Ann Rheum Dis, 69, 1845, 10.1136/ard.2009125179

Cipolletta, 2020, The reliability of ultrasound in the assessment of hyaline cartilage in rheumatoid and healthy metacarpal heads, Ultraschall Med, 10.1055/a-1285-4602

Iagnocco, 2012, The reliability of musculoskeletal ultrasound in the detection of cartilage abnormalities at the metacarpo-phalangeal joints, Osteoarthritis Cartilage, 20, 1142, 10.1016/j.joca.2012.07003

Cipolletta, 2019, FRI0634 standard reference values of metacarpal head cartilage thickness measurement by ultrasound in healthy subjects, Ann Rheum Dis, 78, 1014, 10.1136/annrheumdis-2019-eular5807

Cipolletta, 2020, Prevalence and distribution of cartilage and bone damage at metacarpal head in healthy subjects, Clin Exp Rheumatol

Hammer, 2016, Global ultrasound assessment of structural lesions in osteoarthritis: a reliability study by the OMERACT ultrasonography group on scoring cartilage and osteophytes in finger joints, Ann Rheum Dis, 75, 402, 10.1136/annrheumdis-2014-206289

Gutiérrez, 2013, Ultrasound learning curve in gout: a disease-oriented training program, Arthritis Care Res, 65, 1265, 10.1002/acr22009

Gutierrez, 2011, Inter-observer reliability of high-resolution ultrasonography in the assessment of bone erosions in patients with rheumatoid arthritis: experience of an intensive dedicated training programme, Rheumatology, 50, 373, 10.1093/rheumatology/keq320

Filippucci, 2007, E-learning in ultrasonography: a web-based approach, Ann Rheum Dis, 66, 962, 10.1136/ard.2006064568

Akkus, 2019, A survey of deep-learning applications in ultrasound: artificial intelligence–powered ultrasound for improving clinical workflow, J Am Coll Radiol, 16, 1318, 10.1016/j.jacr.2019.06004

Liu, 2019, Deep learning in medical ultrasound analysis: a review, Engineering, 5, 261, 10.1016/j.eng.2018.11020

Litjens, 2017, A survey on deep learning in medical image analysis, Med Image Anal, 42, 60, 10.1016/j.media.2017.07005

Stoel, 2020, Use of artificial intelligence in imaging in rheumatology – current status and future perspectives, RMD Open, 6, e001063, 10.1136/rmdopen-2019-001063

Zaffino, 2020, A review on advances in intra-operative imaging for surgery and therapy: imagining the operating room of the future, Ann Biomed Eng, 48, 2171, 10.1007/s10439-020-02553-6

Moccia, 2018, Blood vessel segmentation algorithms - review of methods, datasets and evaluation metrics, Comput Methods Programs Biomed, 158, 71, 10.1016/j.cmpb.2018.02001

Chang, 2016, Computer-aided diagnosis of different rotator cuff lesions using shoulder musculoskeletal ultrasound, Ultrasound Med Biol, 42, 2315, 10.1016/j.ultrasmedbio.2016.05016

Klauser, 2011, Contrast-enhanced ultrasonography for the detection of joint vascularity in arthritis - subjective grading versus computer-aided objective quantification, Ultraschall der Medizin, 32, E31, 10.1055/s-0031-1281671

Chang, 2019, Quantitative diagnosis of rotator cuff tears based on sonographic pattern recognition, PLoS ONE, 14, e0212741, 10.1371/journal.pone0212741

Andersen, 2019, Neural networks for automatic scoring of arthritis disease activity on ultrasound images, RMD Open, 5, e000891, 10.1136/rmdopen-2018-000891

Christensen, 2020, Applying cascaded convolutional neural network design further enhances automatic scoring of arthritis disease activity on ultrasound images from rheumatoid arthritis patients, Ann Rheum Dis, 79, 1189, 10.1136/annrheumdis-2019-216636

Fiorentino, 2019, A learning approach for informative-frame selection in US rheumatology images, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 228

Esteva, 2017, Dermatologist-level classification of skin cancer with deep neural networks, Nature, 542, 115, 10.1038/nature21056

Tajbakhsh, 2016, Convolutional neural networks for medical image analysis: full training or fine tuning?, IEEE Trans Med Imaging, 35, 1299, 10.1109/TMI.20162535302

Mandl, 2019, Development of semiquantitative ultrasound scoring system to assess cartilage in rheumatoid arthritis, Rheumatology, 58, 1802, 10.1093/rheumatology/kez153

Torp-Pedersen, 2010, Articular Cartilage thickness measured with US is not as easy as it appears: a systematic review of measurement techniques and image interpretation, Ultraschall der Medizin - Eur J Ultrasound, 32, 54, 10.1055/s-0029-1245386

Möller, 2017, The 2017 EULAR standardised procedures for ultrasound imaging in rheumatology, Ann Rheum Dis, 76, 1974, 10.1136/annrheumdis-2017-211585

Patrini, 2020, Transfer learning for informative-frame selection in laryngoscopic videos through learned features, Med Biol Eng Comput, 58, 1225, 10.1007/s11517-020-02127-7

Landis, 1977, The measurement of observer agreement for categorical data, Biometrics, 33, 159, 10.2307/2529310

Filippucci, 2019, Ultrasound imaging in rheumatoid arthritis, Radiol Med, 124, 1087, 10.1007/s11547-019-01002-2

Ruta, 2015, Pan-American League Against Rheumatisms (PANLAR) Ultrasound Study Group. General applications of ultrasound in rheumatology: why we need it in our daily practice, J Clin Rheumatol, 21, 133, 10.1097/RHU0000000000000230

Naredo, 2010, Current state of musculoskeletal ultrasound training and implementation in Europe: results of a survey of experts and scientific societies, Rheumatology, 49, 2438, 10.1093/rheumatology/keq243

Bruyn, 2019, OMERACT definitions for ultrasonographic pathologies and elementary lesions of rheumatic disorders 15 years on, J Rheumatol, 46, 1388, 10.3899/jrheum181095

Davis, 2020, Artificial intelligence and echocardiography: a primer for cardiac sonographers, J Am Soc Echocardiogr, 33, 1061, 10.1016/j.echo.2020.04025

Moccia, 2018, Learning-based classification of informative laryngoscopic frames, Comput Methods Programs Biomed, 158, 21, 10.1016/j.cmpb.2018.01030

Martel-Pelletier, 2016, Osteoarthritis, Nat Rev Dis Primers, 2, 16072, 10.1038/nrdp.201672

Thông tin
Thông tin xuất bản

Frontiers in Medicine

Tập 8

Thông tin tác giả