Autonomous Structural Visual Inspection Using Region‐Based Deep Learning for Detecting Multiple Damage Types

Computer-Aided Civil and Infrastructure Engineering - Tập 33 Số 9 - Trang 731-747 - 2018
Young‐Jin Cha1, Wooram Choi1, G. Edward Suh1, Sadegh Mahmoudkhani1, Oral Büyüköztürk2
1Department of Civil Engineering, University of Manitoba, Winnipeg, MB, Canada
2Department of Civil and Environmental Engineering, Massachusetts Institute of Technology, Cambridge, MA, USA

Tóm tắt

AbstractComputer vision‐based techniques were developed to overcome the limitations of visual inspection by trained human resources and to detect structural damage in images remotely, but most methods detect only specific types of damage, such as concrete or steel cracks. To provide quasi real‐time simultaneous detection of multiple types of damages, a Faster Region‐based Convolutional Neural Network (Faster R‐CNN)‐based structural visual inspection method is proposed. To realize this, a database including 2,366 images (with 500 × 375 pixels) labeled for five types of damages—concrete crack, steel corrosion with two levels (medium and high), bolt corrosion, and steel delamination—is developed. Then, the architecture of the Faster R‐CNN is modified, trained, validated, and tested using this database. Results show 90.6%, 83.4%, 82.1%, 98.1%, and 84.7% average precision (AP) ratings for the five damage types, respectively, with a mean AP of 87.8%. The robustness of the trained Faster R‐CNN is evaluated and demonstrated using 11 new 6,000 × 4,000‐pixel images taken of different structures. Its performance is also compared to that of the traditional CNN‐based method. Considering that the proposed method provides a remarkably fast test speed (0.03 seconds per image with 500 × 375 resolution), a framework for quasi real‐time damage detection on video using the trained networks is developed.

Từ khóa


Tài liệu tham khảo

10.1061/(ASCE)0887-3801(2003)17:4(255)

Adeli H., 2009, Intelligent Infrastructure—Neural Networks, Wavelets, and Chaos Theory for Intelligent Transportation Systems and Smart Structures

10.1111/mice.12263

10.1016/j.autcon.2016.06.008

10.1016/j.autcon.2011.12.001

Ciresan D. C. Meier U. Masci J. Maria Gambardella L. &Schmidhuber J.(2011) Flexible high performance convolutional neural networks for image classification inProceedings of International Joint Conference on Artificial Intelligence (IJCAI) Barcelona Spain 16–22 July 2011 1237–42.

10.1111/j.1467-8667.2011.00736.x

10.1007/s11263-009-0275-4

Everingham M. Zisserman A. Williams C. K. VanGool L. Allan M. Bishop C. M. Chapelle O. Dalal N. Deselaers T.&Dorkó G.(2007) The PASCAL Visual Object Classes Challenge 2007 (VOC2007) Results. Available at:http://host.robots.ox.ac.uk/pascal/VOC/voc2007/ accessed June 2017.

Fan Q. Brown L.&Smith J.(2016) A closer look at Faster R‐CNN for vehicle detection inProceedings of 2016 IEEE Intelligent Vehicles Symposium

(IV) Gothenburg Sweden 19-22 June 2016 124-29.

10.1016/j.aei.2012.06.005

Girshick R.(2015) Fast R‐CNN inProceedings of the IEEE International Conference on Computer Vision Santiago Chile 07–13 December 2015 1440–48.

Girshick R. Donahue J. Darrell T.&Malik J.(2014) Rich feature hierarchies for accurate object detection and semantic segmentation inProceedings of the IEEE Conference on Computer Vision and Pattern Recognition Columbus OH 23–28 June 2014 580–87.

10.1023/A:1022508121821

He K. Zhang X. Ren S.&Sun J.(2014) Spatial pyramid pooling in deep convolutional networks for visual recognition inProceedings of the 13th European Conference on Computer Vision Zurich Switzerland 6–12 September 2014 346–61.

He K. Zhang X. Ren S.&Sun J.(2016) Deep residual learning for image recognition inProceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Las Vegas NV 12 December 2016 770–78.

10.1016/j.aei.2011.01.002

Krizhevsky A. Sutskever I.&Hinton G. E.(2012) Imagenet classification with deep convolutional neural networks inProceedings of the Neural Information Processing Systems Conference Stateline NV 3–8 December 2012.

Li C. Kang Q. Ge G. Song Q. Lu H.&Cheng J.(2016) Deep: learning deep binary encoding for multi‐label classification inProceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) Las Vegas NV 26 June–1 July 2016 744–51.

10.1016/j.autcon.2016.08.008

10.1515/revneuro-2016-0029

Nair V.&Hinton G. E.(2010) Rectified linear units improve restricted Boltzmann machines inProceedings of the 27th International Conference on Machine Learning (ICML‐10) Haifa Israel 21–24 June 2010 807–14.

10.1111/j.1467-8667.2011.00716.x

10.1111/j.1467-8667.2012.00790.x

Park J. Kim T.&Kim J.(2015) Image‐based bolt‐loosening detection technique of bolt joint in steel bridges inProceedings of the 6th International Conference on Advances in Experimental Structural Engineering (6AESE) Urbana–Champaign IL 1–2 August 2015.

10.14359/51689560

10.1109/TPAMI.2016.2577031

10.1007/s11263-015-0816-y

Ryan T. Mann J. Chill Z.&Ott B.(2012) Bridge Inspector's Reference Manual (BIRM) Report Federal Highway Administration (FHWA) Report No. FHWA NHI 12–049 2012.

Sermanet P. Eigen D. Zhang X. Mathieu M. Fergus R.&Lecun Y.(2014) Overfeat: integrated recognition localization and detection using convolutional networks inProceedings of the International Conference on Learning Representations (ICLR2014) Banff Canada 14–16 April 2014.

10.3846/13923730.2016.1157095

Simonyan K.&Zisserman A.(2014) Very deep convolutional networks for large‐scale image recognition inProceedings of the International Conference on Learning Representations (ICLR) San Diego CA 7–9 May 2015.

10.1111/1467-8667.00302

Szegedy C. Liu W. Jia Y. Sermanet P. Reed S. Anguelov D. Erhan D. Vanhoucke V.&Rabinovich A.(2015) Going deeper with convolutions inProceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Boston MA 7–12 June 2015 1–9.

10.1007/s11263-013-0620-5

10.1061/(ASCE)CP.1943-5487.0000451

10.1111/mice.12141

10.1111/j.1467-8667.2010.00674.x

10.1111/mice.12042

10.1007/978-3-319-10590-1_53

Thông tin
Thông tin xuất bản

Computer-Aided Civil and Infrastructure Engineering

Tập 33 Số 9

731-747

Thông tin tác giả