Autonomous Structural Visual Inspection Using Region‐Based Deep Learning for Detecting Multiple Damage Types
Tóm tắt
Computer vision‐based techniques were developed to overcome the limitations of visual inspection by trained human resources and to detect structural damage in images remotely, but most methods detect only specific types of damage, such as concrete or steel cracks. To provide quasi real‐time simultaneous detection of multiple types of damages, a Faster Region‐based Convolutional Neural Network (Faster R‐CNN)‐based structural visual inspection method is proposed. To realize this, a database including 2,366 images (with 500 × 375 pixels) labeled for five types of damages—concrete crack, steel corrosion with two levels (medium and high), bolt corrosion, and steel delamination—is developed. Then, the architecture of the Faster R‐CNN is modified, trained, validated, and tested using this database. Results show 90.6%, 83.4%, 82.1%, 98.1%, and 84.7% average precision (AP) ratings for the five damage types, respectively, with a mean AP of 87.8%. The robustness of the trained Faster R‐CNN is evaluated and demonstrated using 11 new 6,000 × 4,000‐pixel images taken of different structures. Its performance is also compared to that of the traditional CNN‐based method. Considering that the proposed method provides a remarkably fast test speed (0.03 seconds per image with 500 × 375 resolution), a framework for quasi real‐time damage detection on video using the trained networks is developed.
Từ khóa
Tài liệu tham khảo
Adeli H., 2009, Intelligent Infrastructure—Neural Networks, Wavelets, and Chaos Theory for Intelligent Transportation Systems and Smart Structures
Ciresan D. C. Meier U. Masci J. Maria Gambardella L. &Schmidhuber J.(2011) Flexible high performance convolutional neural networks for image classification inProceedings of International Joint Conference on Artificial Intelligence (IJCAI) Barcelona Spain 16–22 July 2011 1237–42.
Everingham M. Zisserman A. Williams C. K. VanGool L. Allan M. Bishop C. M. Chapelle O. Dalal N. Deselaers T.&Dorkó G.(2007) The PASCAL Visual Object Classes Challenge 2007 (VOC2007) Results. Available at:http://host.robots.ox.ac.uk/pascal/VOC/voc2007/ accessed June 2017.
Fan Q. Brown L.&Smith J.(2016) A closer look at Faster R‐CNN for vehicle detection inProceedings of 2016 IEEE Intelligent Vehicles Symposium
(IV) Gothenburg Sweden 19-22 June 2016 124-29.
Girshick R.(2015) Fast R‐CNN inProceedings of the IEEE International Conference on Computer Vision Santiago Chile 07–13 December 2015 1440–48.
Girshick R. Donahue J. Darrell T.&Malik J.(2014) Rich feature hierarchies for accurate object detection and semantic segmentation inProceedings of the IEEE Conference on Computer Vision and Pattern Recognition Columbus OH 23–28 June 2014 580–87.
He K. Zhang X. Ren S.&Sun J.(2014) Spatial pyramid pooling in deep convolutional networks for visual recognition inProceedings of the 13th European Conference on Computer Vision Zurich Switzerland 6–12 September 2014 346–61.
He K. Zhang X. Ren S.&Sun J.(2016) Deep residual learning for image recognition inProceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Las Vegas NV 12 December 2016 770–78.
Krizhevsky A. Sutskever I.&Hinton G. E.(2012) Imagenet classification with deep convolutional neural networks inProceedings of the Neural Information Processing Systems Conference Stateline NV 3–8 December 2012.
Li C. Kang Q. Ge G. Song Q. Lu H.&Cheng J.(2016) Deep: learning deep binary encoding for multi‐label classification inProceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) Las Vegas NV 26 June–1 July 2016 744–51.
Nair V.&Hinton G. E.(2010) Rectified linear units improve restricted Boltzmann machines inProceedings of the 27th International Conference on Machine Learning (ICML‐10) Haifa Israel 21–24 June 2010 807–14.
Park J. Kim T.&Kim J.(2015) Image‐based bolt‐loosening detection technique of bolt joint in steel bridges inProceedings of the 6th International Conference on Advances in Experimental Structural Engineering (6AESE) Urbana–Champaign IL 1–2 August 2015.
Ryan T. Mann J. Chill Z.&Ott B.(2012) Bridge Inspector's Reference Manual (BIRM) Report Federal Highway Administration (FHWA) Report No. FHWA NHI 12–049 2012.
Sermanet P. Eigen D. Zhang X. Mathieu M. Fergus R.&Lecun Y.(2014) Overfeat: integrated recognition localization and detection using convolutional networks inProceedings of the International Conference on Learning Representations (ICLR2014) Banff Canada 14–16 April 2014.
Simonyan K.&Zisserman A.(2014) Very deep convolutional networks for large‐scale image recognition inProceedings of the International Conference on Learning Representations (ICLR) San Diego CA 7–9 May 2015.
Szegedy C. Liu W. Jia Y. Sermanet P. Reed S. Anguelov D. Erhan D. Vanhoucke V.&Rabinovich A.(2015) Going deeper with convolutions inProceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Boston MA 7–12 June 2015 1–9.