Fast Estimation of Loader’s Shovel Load Volume by 3D Reconstruction of Material Piles

Springer Science and Business Media LLC
Binyun Wu1, Shaojie Wang1, Haiying Lin1, Shijiang Li1, Liang Hou1
1Pen-Tung Sah Institute of Micro-Nano Science and Technology, Xiamen University, Xiamen 361102, China

Tóm tắt

AbstractFast and accurate measurement of the volume of earthmoving materials is of great significance for the real-time evaluation of loader operation efficiency and the realization of autonomous operation. Existing methods for volume measurement, such as total station-based methods, cannot measure the volume in real time, while the bucket-based method also has the disadvantage of poor universality. In this study, a fast estimation method for a loader’s shovel load volume by 3D reconstruction of material piles is proposed. First, a dense stereo matching method (QORB–MAPM) was proposed by integrating the improved quadtree ORB algorithm (QORB) and the maximum a posteriori probability model (MAPM), which achieves fast matching of feature points and dense 3D reconstruction of material piles. Second, the 3D point cloud model of the material piles before and after shoveling was registered and segmented to obtain the 3D point cloud model of the shoveling area, and the Alpha-shape algorithm of Delaunay triangulation was used to estimate the volume of the 3D point cloud model. Finally, a shovel loading volume measurement experiment was conducted under loose-soil working conditions. The results show that the shovel loading volume estimation method (QORB–MAPM VE) proposed in this study has higher estimation accuracy and less calculation time in volume estimation and bucket fill factor estimation, and it has significant theoretical research and engineering application value.

Từ khóa


Tài liệu tham khảo

S Dadhich, U Bodin, U Andersson. Key challenges in automation of earth-moving machines. Automation in Construction, 2016, 68: 212–222.

D Pratt. Fundamentals of construction estimating. Boston: Cengage Learning, 2010.

M Bügler, A Borrmann, G Ogunmakin, et al. Fusion of photogrammetry and video analysis for productivity assessment of earthwork processes. Computer-Aided Civil and Infrastructure Engineering, 2017, 32(2): 107–123.

M Savia, H N Koivo. Neural-network-based payload determination of a moving loader. Control Engineering Practice, 2004, 12(5): 555–561.

M Yakar, H M Yılmaz, Ö Mutluoǧlu. Close range photogrammetry and robotic total station in volume calculation. International Journal of the Physical Sciences, 2010, 5(2): 86–96.

H He, T Chen, H Zeng, et al. Ground control point-free unmanned aerial vehicle-based photogrammetry for volume estimation of stockpiles carried on barges. Sensors, 2019, 19(16): 3534.

H Anwar, S M Abbas, A Muhammad, et al. Volumetric estimation of contained soil using 3D sensors. Commercial Vehicle Technology Symposium, 2014: 11–13.

J Guevara, T Arevalo-Ramirez, F Yandun, et al. Point cloud-based estimation of effective payload volume for earthmoving loaders. Automation in Construction, 2020, 117: 103207.

J X Lu, Q S Bi, Y N Li, et al. Estimation of fill factor for earth-moving machines based on 3D point clouds. Measurement, 2020, 165: 108114.

J X Lu, Z W Yao, Q S Bi, et al. A neural network–based approach for fill factor estimation and bucket detection on construction vehicles. Computer-Aided Civil and Infrastructure Engineering, 2021, 36: 1600–1618.

Y Arayici. An approach for real world data modelling with the 3D terrestrial laser scanner for built environment. Automation in Construction, 2007, 16(6): 816–829.

M Golparvar-Fard, J Bohn, J Teizer, et al. Evaluation of image-based modeling and laser scanning accuracy for emerging automated performance monitoring techniques. Automation in Construction, 2011, 20(8): 1143–1155.

M Yakar, H M Yilmaz, O Mutluoglu. Performance of photogrammetric and terrestrial laser scanning methods in volume computing of excavtion and filling areas. Arabian Journal for Science and Engineering, 2013, 39(1): 387–394.

Z Ma, S Liu. A review of 3D reconstruction techniques in civil engineering and their applications. Advanced Engineering Informatics, 2018, 37, 163–174.

C Sung, P Y Kim. 3D terrain reconstruction of construction sites using a stereo camera. Automation in Construction, 2016, 64: 65–77.

H Hirschmuller. Stereo processing by semiglobal matching and mutual information. IEEE Transaction on Pattern Analysis and Machine Intelligence, 2008, 30(2): 328–341.

D G Lowe. Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 2004, 60(2): 91–110.

H Bay, A Ess, T Tuytelaars, et al. Speeded-up robust features (SURF). Computer Vision and Image Understanding, 2008, 110(3): 346–359.

E Rublee, V Rabaud, K Konolige, et al. ORB: an efficient alternative to SIFT or SURF. IEEE International Conference on Computer Vision, 2011: 2564-2571.

J Jiao, B Zhao, S Wu. A speed-up and robust image registration algorithm based on FAST. IEEE International Conference on Computer Science & Automation Engineering. 2011: 10–12.

M Calonder, V Lepetit, C Strecha, et al. BRIEF: Binary robust independent elementary features. Proceedings of the 11th European Conference on Computer Vision. Heraklion, Creece. 2010, 6314: 778–792.

U Chaudhuri, B Banerjee, A Bhattacharya, et al. CMIR-NET: A deep learning based model for cross-modal retrieval in remote sensing. Pattern Recognition. Letters, 2010, 131: 456–462.

Y S Li, Y J Zhang, X Huang, et al. Large-scale remote sensing image retrieval by deep hashing neural networks. IEEE Transaction on Geoscience and Remote Sensing, 2017, 56(2): 950–965.

R Z Wang, J C Yan, X K Yang. Learning combinatorial embedding networks for deep graph matching. in: IEEE International Conference on Computer Vision, 2019: 3056–3065.

B Jiang, P F Sun, B Luo. GLMNet: Graph learning-matching convolutional networks for feature matching. Pattern Recognition, 2022, 121: 108167.

P Sarlin, D DeTone, T Malisiewicz, et al. Superglue: learning feature matching with graph neural networks. IEEE Conference on Computer Vision and Pattern Recognition, 2020: 4937–4946.

C Liguori, A Paolillo, A Pietrosanto. An on-line stereo-vision system for dimensional measurements of rubber extrusions. Measurement, 2004, 35(3): 221–231.

T Zhang, J H Liu, S L Liu, et al. A 3D reconstruction method for pipeline inspection based on multi-vision. Measurement, 2017, 98: 35–48.

G F Xiao, Y T Li, Q X Xia, et al. Research on the on-line dimensional accuracy measurement method of conical spun workpieces based on machine vision technology. Measurement, 2019, 148: 106881.

J Miller, J Morgenroth, C Gomez. 3D modelling of individual trees using a handheld camera: Accuracy of height, diameter and volume estimates. Urban Forestry & Urban Greening, 2015, 14(4): 932-940.

P Muñoz-Benavent, G Andreu-García, J M Valiente-González, et al. Enhanced fish bending model for automatic tuna sizing using computer vision. Computers and Electronics in Agriculture, 2018, 150: 52–61.

S Barone, A Paoli, A V Razionale. Shape measurement by a multi-view methodology based on the remote tracking of a 3D optical scanner. Optics and Lasers in Engineering, 2012, 50(3): 380–390.

M R Yao. Research on 3D vision measurement technology of aeroengine blade profile. Harbin: Harbin Institute of Technology, 2019. (in Chinese)

J R Borthwick. Mining haul truck pose estimation and load profiling using stereo vision. Vancouver: University of British Columbia, 2009.

M Yakar, H M Yilmaz, O Mutluoglu. Performance of photogrammetric and terrestrial laser scanning methods in volume computing of excavation and filling areas. Arabian Journal for Science & Engineering, 2014, 39(1): 387-394.

L Fu, J Zhu, W L Li, et al. Fast estimation method of volumes of landslide deposit by the 3D reconstruction of smartphone images. Landslides, 2021, 18(9): 3269–3278.

R Mur-Artal, J M M Montiel, J D Tardos. ORB-SLAM: A versatile and accurate monocular SLAM system. IEEE Transactions on Robotics, 2015, 31(5): 1147–1163.

R Mur-Artal, J D Tardos. ORB-SLAM2: An open-source SLAM system for monocular, stereo, and RGB-D cameras. IEEE Transactions on Robotics, 2017, 33(5): 1255–1262

Thông tin
Thông tin xuất bản

Springer Science and Business Media LLC

Thông tin tác giả