Enclosing contour tracking of highway construction equipment based on orientation-aware bounding box using UAV

Journal of Infrastructure Preservation and Resilience
Yapeng Guo1, Yang Xu2, Zhonglong Li1, Hui Li2, Shunlong Li1
1School of Transportation Science and Engineering, Harbin Institute of Technology, Harbin 150090, China
2School of Civil Engineering, Harbin Institute of Technology, Harbin 150090, China

Tóm tắt

AbstractConstruction equipment tracking of highway construction site can obtain the spatiotemporal location in real time and provide data basis for construction risk control. The complete 2D moving of construction equipment in surveillance videos could be spatially represented by the translation, rotation and size change of corresponding images. To describe the temporal relationships of these variables, this study proposes a construction equipment enclosing contour tracking method based on orientation-aware bounding box (OABB), where UAV surveillance videos are employed to alleviate the occlusion problem. The method balances the rotation insensitivity of horizontal bounding box and the complexity of pixel-level segmented contour, which has three modules. The first module integrates OABB into a deep learning detector to provide detected contours. The second module updates OABBs with Kalman prediction to output tracked contours. The third module manages IDs of multiple tracked contours for construction equipment motions. Five in-situ UAV videos including 4325 frames were employed as the evaluation dataset. The tracking performance achieved 2.657 degrees in angle error, 97.523% in MOTA and 83.243% in MOTP.

Từ khóa


Tài liệu tham khảo

Bang S, Hong Y, Kim H (2021) Proactive proximity monitoring with instance segmentation and unmanned aerial vehicle-acquired video-frame prediction. Comput-Aided Civ Infrastructure Eng 36(6):800–816. https://doi.org/10.1111/mice.12672

Brilakis I, Park M-W, Jog G (2011) Automated vision tracking of project related entities. Adv Eng Inform 25(4):713–724. https://doi.org/10.1016/j.aei.2011.01.003

Sherafat B, Ahn Changbum R, Akhavian R, Behzadan Amir H, Golparvar-Fard M, Kim H, Lee Y-C, Rashidi A, Azar Ehsan R (2020) Automated methods for activity recognition of construction workers and equipment: state-of-the-art review. J Constr Eng Manag 146(6):03120002. https://doi.org/10.1061/(ASCE)CO.1943-7862.0001843

Teizer J (2015) Status quo and open challenges in vision-based sensing and tracking of temporary resources on infrastructure construction sites. Adv Eng Inform 29(2):225–238. https://doi.org/10.1016/j.aei.2015.03.006

Yang J, Park M-W, Vela PA, Golparvar-Fard M (2015) Construction performance monitoring via still images, time-lapse photos, and video streams: Now, tomorrow, and the future. Adv Eng Inform 29(2):211–224. https://doi.org/10.1016/j.aei.2015.01.011

Guo H, Yu Y, Skitmore M (2017) Visualization technology-based construction safety management: a review. Autom Constr 73:135–144. https://doi.org/10.1016/j.autcon.2016.10.004

Park M-W, Makhmalbaf A, Brilakis I (2011) Comparative study of vision tracking methods for tracking of construction site resources. Autom Constr 20(7):905–915. https://doi.org/10.1016/j.autcon.2011.03.007

Seo J, Han S, Lee S, Kim H (2015) Computer vision techniques for construction safety and health monitoring. Adv Eng Inform 29(2):239–251. https://doi.org/10.1016/j.aei.2015.02.001

Xu S, Wang J, Shou W, Ngo T, Sadick A-M, Wang X (2021) Computer Vision techniques in construction: a critical review. Arch Comput Methods Eng 28(5):3383–3397. https://doi.org/10.1007/s11831-020-09504-3

Arslan M, Cruz C, Roxin A-M, Ginhac D (2018) Spatio-temporal analysis of trajectories for safer construction sites. Smart Sustainable Built Environ 7(1):80–100. https://doi.org/10.1108/SASBE-10-2017-0047

Lu M, Chen W, Shen X, Lam H-C, Liu J (2007) Positioning and tracking construction vehicles in highly dense urban areas and building construction sites. Autom Constr 16(5):647–656. https://doi.org/10.1016/j.autcon.2006.11.001

Song J, Haas Carl T, Caldas Carlos H (2006) Tracking the Location of Materials on Construction Job Sites. J Constr Eng Manag 132(9):911–918. https://doi.org/10.1061/(ASCE)0733-9364(2006)132:9(911)

Kim D, Liu M, Lee S, Kamat VR (2019) Remote proximity monitoring between mobile construction resources using camera-mounted UAVs. Autom Constr 99:168–182. https://doi.org/10.1016/j.autcon.2018.12.014

Tang S, Golparvar-Fard M, Naphade M, Gopalakrishna Murali M (2020) Video-Based Motion Trajectory Forecasting Method for Proactive Construction Safety Monitoring Systems. J Comput Civ Eng 34(6):04020041. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000923

Zhao Y, Chen Q, Cao W, Yang J, Gui G (2019) Deep learning for risk detection and trajectory tracking at construction sites. IEEE Access 7:30905–30912. https://doi.org/10.1109/ACCESS.2019.2902658

Zhu Z, Ren X, Chen Z (2016) Visual tracking of construction jobsite workforce and equipment with particle filtering. J Comput Civ Eng 30(6):04016023. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000573

Jog GM, Brilakis IK, Angelides DC (2011) Testing in harsh conditions: Tracking resources on construction sites with machine vision. Autom Constr 20(4):328–337. https://doi.org/10.1016/j.autcon.2010.11.003

Zhu Z, Park M-W, Koch C, Soltani M, Hammad A, Davari K (2016) Predicting movements of onsite workers and mobile equipment for enhancing construction site safety. Autom Constr 68:95–101. https://doi.org/10.1016/j.autcon.2016.04.009

Chen C, Zhu Z, Hammad A (2020) Automated excavators activity recognition and productivity analysis from construction site surveillance videos. Autom Constr 110:103045. https://doi.org/10.1016/j.autcon.2019.103045

Kim J, Chi S (2017) Adaptive detector and tracker on construction sites using functional integration and online learning. J Comput Civ Eng 31(5):04017026. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000677

Kim J, Chi S (2020) Multi-camera vision-based productivity monitoring of earthmoving operations. Autom Constr 112:103121. https://doi.org/10.1016/j.autcon.2020.103121

Xiao B, Kang S-C (2021) Vision-Based Method Integrating Deep Learning Detection for Tracking Multiple Construction Machines. J Comput Civ Eng 35(2):04020071. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000957

Xiao B, Lin Q, Chen Y (2021) A vision-based method for automatic tracking of construction machines at nighttime based on deep learning illumination enhancement. Autom Constr 127:103721. https://doi.org/10.1016/j.autcon.2021.103721

Zhu Z, Ren X, Chen Z (2017) Integrated detection and tracking of workforce and equipment from construction jobsite videos. Autom Constr 81:161–171. https://doi.org/10.1016/j.autcon.2017.05.005

Wang Z, Zhang Q, Yang B, Wu T, Lei K, Zhang B, Fang T (2021) Vision-based framework for automatic progress monitoring of precast walls by using surveillance videos during the construction phase. J Comput Civ Eng 35(1):04020056. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000933

Guo Y, Xu Y, Li S (2020) Dense construction vehicle detection based on orientation-aware feature fusion convolutional neural network. Autom Constr 112:103124. https://doi.org/10.1016/j.autcon.2020.103124

Ham Y, Han KK, Lin JJ, Golparvar-Fard M (2016) Visual monitoring of civil infrastructure systems via camera-equipped Unmanned Aerial Vehicles (UAVs): a review of related works. Vis Eng 4(1):1. https://doi.org/10.1186/s40327-015-0029-z

Chen C, Zhong J, Tan Y (2019) Multiple-oriented and small object detection with convolutional neural networks for aerial image. Remote Sensing 11(18):2176. https://doi.org/10.3390/rs11182176

Ma J, Zhou Z, Wang B, Zong H, Wu F (2019) Ship detection in optical satellite images via directional bounding boxes based on ship center and orientation prediction. Remote Sensing 11(18):2173. https://doi.org/10.3390/rs11182173

Zhou X, Wang D, Krhenbühl P (2019) Objects as Points. arXiv. https://arxiv.org/abs/1904.07850v2

An X, Zhou L, Liu Z, Wang C, Li P, Li Z (2021) Dataset and benchmark for detecting moving objects in construction sites. Autom Constr 122:103482. https://doi.org/10.1016/j.autcon.2020.103482

Bewley A, Ge Z, Ott L, Ramos F, Upcroft B Simple online and realtime tracking. In: 2016 IEEE international conference on image processing (ICIP), 2016. IEEE, pp 3464–3468 https://doi.org/10.1109/ICIP.2016.7533003

Milan A, Leal-Taixé L, Reid I, Roth S, Schindler K (2016) MOT16: A benchmark for multi-object tracking. arXiv preprint arXiv:160300831. https://arxiv.org/abs/1603.00831

Thông tin
Thông tin xuất bản

Journal of Infrastructure Preservation and Resilience

Thông tin tác giả