End-to-end point cloud-based segmentation of building members for automating dimensional quality control

N.Z.S. Iso, ISO 8402:1994 Australian/New Zealand Standard Quality management and quality assurance—Vocabulary, 1994. [Online]. Available: https://www.saiglobal.com/pdftemp/previews/osh/as/as8000/8400/8402.pdf. N. Johnston, S. Reid, An Examination of Building Defects in Residential Multi-owned Properties, 2019, pp. 1–62. Infrastructure, A comprehensive assesment of Americas's Infrastructure, Asce, 2017, pp. 1-112. [Online]. Available: https://www.infrastructurereportcard.org/. Lin, 2021, Bridge Inspection with Aerial Robots: Automating the Entire Pipeline of Visual Data Capture, 3D Mapping, Defect Detection, Analysis, and Reporting, J. Comput. Civ. Eng., 35, 04020064, 10.1061/(ASCE)CP.1943-5487.0000954 Arashpour, 2019, Performance-based control of variability and tolerance in off-site manufacture and assembly: optimization of penalty on poor production quality, Constr. Manag. Econ., 1 Tavakolan, 2022, “A parallel computing simulation-based multi-objective optimization framework for economic analysis of building energy retrofit: A case study in Iran, J. Build. Eng., 45 Kim, 2019, Non-contact sensing based geometric quality assessment of buildings and civil structures: A review, Autom. Constr., 100, 163, 10.1016/j.autcon.2019.01.002 Wang, 2008, Enhancing construction quality inspection and management using RFID technology, Autom. Constr., 17, 467, 10.1016/j.autcon.2007.08.005 B.M. Phares, G.A. Washer, D.D. Rolander, B.A. Graybeal, M. Moore, Routine Highway Bridge Inspection Condition Documentation Accuracy and Reliability, J. Bridge Eng. 9(4) (2004) 403–413, doi: 10.1061/(asce)1084-0702(2004)9:4(403). Paneru, 2021, Computer vision applications in construction: Current state, opportunities & challenges, Autom. Constr., 132, 10.1016/j.autcon.2021.103940 Wang, 2019, Applications of 3D point cloud data in the construction industry: A fifteen-year review from 2004 to 2018, Adv. Eng. Inf., 39, 306, 10.1016/j.aei.2019.02.007 Yu, 2019, “Accurate 3D Shape, Displacement and Deformation Measurement Using a Smartphone,” (in eng), Sens. (Basel, Switzerland), 19, 719, 10.3390/s19030719 Abolhasannejad, 2018, “Developing an Optical Image-Based Method for Bridge Deformation Measurement Considering Camera Motion,” (in eng), Sens. (Basel, Switzerland), 18, 2754, 10.3390/s18092754 Wojtkowska, 2021, Validation of terrestrial laser scanning and artificial intelligence for measuring deformations of cultural heritage structures, Measurement, 167, 10.1016/j.measurement.2020.108291 Poullis, 2013, A Framework for Automatic Modeling from Point Cloud Data, IEEE Trans. Pattern Anal. Mach. Intell., 35, 2563, 10.1109/TPAMI.2013.64 Mirzaei, 2022, 3D point cloud data processing with machine learning for construction and infrastructure applications: A comprehensive review, Adv. Eng. Inf., 51, 10.1016/j.aei.2021.101501 Arayici, 2007, An approach for real world data modelling with the 3D terrestrial laser scanner for built environment, Autom. Constr., 16, 816, 10.1016/j.autcon.2007.02.008 Pătrăucean, 2015, State of research in automatic as-built modelling, Adv. Eng. Inf., 29, 162, 10.1016/j.aei.2015.01.001 Arashpour, 2013, A new approach for modelling variability in residential construction projects, Australasian J. Constr. Econom. Build., 13, 83 Jung, 2014, Productive modeling for development of as-built BIM of existing indoor structures, Autom. Constr., 42, 68, 10.1016/j.autcon.2014.02.021 M. Arashpour, J. Lamborn, P. Farzanehfar, Optimising collaborative learning and group work amongst tertiary students, in: 10th International Structural Engineering and Construction Conference, ISEC 2019, 2019: ISEC Press, doi: https://doi.org/10.14455/ISEC.res.2019.121. M. Arashpour, J. Lamborn, P. Farzanehfar, Group Dynamics in Higher Education: Impacts of Gender Inclusiveness and Selection Interventions on Collaborative Learning, in: Claiming Identity Through Redefined Teaching in Construction Programs: IGI Global, 2020, pp. 42-60. Iman Zolanvari, 2016, Slicing Method for curved façade and window extraction from point clouds, ISPRS J. Photogramm. Remote Sens., 119, 334, 10.1016/j.isprsjprs.2016.06.011 Xiao, 2017, User-Guided Dimensional Analysis of Indoor Building Environments from Single Frames of RGB-D Sensors, J. Comput. Civ. Eng., 31, 04017006, 10.1061/(ASCE)CP.1943-5487.0000648 Arashpour, 2014, “Framework for improving workflow stability: Deployment of optimized capacity buffers in a synchronized construction production,” (in English), Can. J. Civ. Eng., 41, 995, 10.1139/cjce-2014-0199 Riveiro, 2013, Validation of terrestrial laser scanning and photogrammetry techniques for the measurement of vertical underclearance and beam geometry in structural inspection of bridges, Measurement, 46, 784, 10.1016/j.measurement.2012.09.018 Dai, 2011, Photogrammetry Assisted Measurement of Interstory Drift for Rapid Post-disaster Building Damage Reconnaissance, J. Nondestr. Eval., 30, 201, 10.1007/s10921-011-0108-6 Truong-Hong, 2022, Extracting structural components of concrete buildings from laser scanning point clouds from construction sites, Adv. Eng. Inf., 51, 10.1016/j.aei.2021.101490 Kardovskyi, 2021, Artificial intelligence quality inspection of steel bars installation by integrating mask R-CNN and stereo vision, Autom. Constr., 130, 10.1016/j.autcon.2021.103850 Kwon, 2014, A defect management system for reinforced concrete work utilizing BIM, image-matching and augmented reality, Autom. Constr., 46, 74, 10.1016/j.autcon.2014.05.005 Arashpour, 2021, Scene understanding in construction and buildings using image processing methods: A comprehensive review and a case study, J. Build. Eng., 33 Iglesias, 2018, Automated vision system for quality inspection of slate slabs, Comput Ind, 99, 119, 10.1016/j.compind.2018.03.030 Anil, 2013, Deviation analysis method for the assessment of the quality of the as-is Building Information Models generated from point cloud data, Autom. Constr., 35, 507, 10.1016/j.autcon.2013.06.003 Kim, 2016, Automated dimensional quality assurance of full-scale precast concrete elements using laser scanning and BIM, Autom. Constr., 72, 102, 10.1016/j.autcon.2016.08.035 Kim, 2019, A mirror-aided laser scanning system for geometric quality inspection of side surfaces of precast concrete elements, Measurement, 141, 420, 10.1016/j.measurement.2019.04.060 Wang, 2019, Computational Methods of Acquisition and Processing of 3D Point Cloud Data for Construction Applications, Arch. Comput. Meth. Eng., 27, 479, 10.1007/s11831-019-09320-4 Maalek, 2019, Automatic Recognition of Common Structural Elements from Point Clouds for Automated Progress Monitoring and Dimensional Quality Control in Reinforced Concrete Construction, Remote Sens. (Basel), 11, 1102, 10.3390/rs11091102 Romero-Jarén, 2021, Automatic segmentation and classification of BIM elements from point clouds, Autom. Constr., 124, 10.1016/j.autcon.2021.103576 Fotsing, 2021, Iterative closest point for accurate plane detection in unorganized point clouds, Autom. Constr., 125, 10.1016/j.autcon.2021.103610 Fischler, 1981, Random sample consensus, Commun. ACM, 24, 381, 10.1145/358669.358692 Aitelkadi, 2013, Segmentation of heritage building by means of geometric and radiometric components from terrestrial laser scanning, ISPRS Ann. Photogramm., Remote Sens. Spatial Inf. Sci., II-5/W1, 1, 10.5194/isprsannals-II-5-W1-1-2013 Xu, 2015, Investigation on the Weighted RANSAC Approaches for Building Roof Plane Segmentation from LiDAR Point Clouds, Remote Sens. (Basel), 8, 5, 10.3390/rs8010005 Previtali, 2014, Automatic façade modelling using point cloud data for energy-efficient retrofitting, Appl. Geomatics, 6, 95, 10.1007/s12518-014-0129-9 A.P. Dal Poz, M.S. Yano, Ransac-Based Segmentation for Building Roof Face Detection in Lidar Point Cloud, in: presented at the IGARSS 2018 - 2018 IEEE International Geoscience and Remote Sensing Symposium, 2018/07, 2018. [Online]. Available: http://dx.doi.org/10.1109/igarss.2018.8518502. Miller, 2018, A review of unsupervised statistical learning and visual analytics techniques applied to performance analysis of non-residential buildings, Renew. Sustain. Energy Rev., 81, 1365, 10.1016/j.rser.2017.05.124 Aldoma, 2012, Tutorial: Point Cloud Library: Three-Dimensional Object Recognition and 6 DOF Pose Estimation, IEEE Rob. Autom. Mag., 19, 80, 10.1109/MRA.2012.2206675 Czerniawski, 2018, 6D DBSCAN-based segmentation of building point clouds for planar object classification, Autom. Constr., 88, 44, 10.1016/j.autcon.2017.12.029 Aljumaily, 2017, Urban Point Cloud Mining Based on Density Clustering and MapReduce, J. Comput. Civ. Eng., 31, 04017021, 10.1061/(ASCE)CP.1943-5487.0000674 Lu, 2018, Detection of Structural Components in Point Clouds of Existing RC Bridges, Comput. Aided Civ. Inf. Eng., 34, 191, 10.1111/mice.12407 Ahmed, 2014, Automatic Detection of Cylindrical Objects in Built Facilities, J. Comput. Civ. Eng., 28, 04014009, 10.1061/(ASCE)CP.1943-5487.0000329 Mura, 2016, Piecewise-planar Reconstruction of Multi-room Interiors with Arbitrary Wall Arrangements, Comput. Graphics Forum, 35, 179, 10.1111/cgf.13015 Dimitrov, 2015, Segmentation of building point cloud models including detailed architectural/structural features and MEP systems, Autom. Constr., 51, 32, 10.1016/j.autcon.2014.12.015 Khaloo, 2017, Robust normal estimation and region growing segmentation of infrastructure 3D point cloud models, Adv. Eng. Inf., 34, 1, 10.1016/j.aei.2017.07.002 Koo, 2021, Automatic classification of wall and door BIM element subtypes using 3D geometric deep neural networks, Adv. Eng. Inf., 47, 10.1016/j.aei.2020.101200 Arashpour, 2022, Computer vision for anatomical analysis of equipment in civil infrastructure projects: Theorizing the development of regression-based deep neural networks, Autom. Constr., 137, 10.1016/j.autcon.2022.104193 Arashpour, 2022, Predicting individual learning performance using machine-learning hybridized with the teaching-learning-based optimization, Comput. Appl. Eng. Educ. S. Wang, S. Suo, W.-C. Ma, A. Pokrovsky, R. Urtasun, Deep Parametric Continuous Convolutional Neural Networks, in: presented at the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018/06, 2018. [Online]. Available: http://dx.doi.org/10.1109/cvpr.2018.00274. LeCun, 1989, Backpropagation Applied to Handwritten Zip Code Recognition, Neural Comput., 1, 541, 10.1162/neco.1989.1.4.541 Bello, 2020, Review: Deep Learning on 3D Point Clouds, Remote Sens. (Basel), 12, 1729, 10.3390/rs12111729 Coudron, 2020, “Semantic Extraction of Permanent Structures for the Reconstruction of Building Interiors from Point Clouds,” (in eng), Sensors (Basel, Switzerland), 20, 6916, 10.3390/s20236916 W. Zhirong, et al., 3D ShapeNets: A deep representation for volumetric shapes,“ presented at the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015/06, 2015. [Online]. Available: http://dx.doi.org/10.1109/cvpr.2015.7298801. C.R. Qi, L. Yi, H. Su, L.J. Guibas, PointNet++: Deep hierarchical feature learning on point sets in a metric space, Advances in Neural Information Processing Systems, vol. 2017-Decem, pp. 5100-5109, 2017. X. Wang, S. Liu, X. Shen, C. Shen, J. Jia, Associatively Segmenting Instances and Semantics in Point Clouds, in: presented at the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019/06, 2019. [Online]. Available: http://dx.doi.org/10.1109/cvpr.2019.00422. D. Maturana, S. Scherer, VoxNet: A 3D Convolutional Neural Network for real-time object recognition, in: presented at the 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2015/09, 2015. [Online]. Available: http://dx.doi.org/10.1109/iros.2015.7353481. Zhao, 2018, “Dense RGB-D Semantic Mapping with Pixel-Voxel Neural Network,” (in eng), Sensors (Basel, Switzerland), 18, 3099, 10.3390/s18093099 B.-S. Kim, P. Kohli, S. Savarese, 3D Scene Understanding by Voxel-CRF, in: presented at the 2013 IEEE International Conference on Computer Vision, 2013/12, 2013. [Online]. Available: http://dx.doi.org/10.1109/iccv.2013.180. Xu, 2017, Segmentation of building roofs from airborne LiDAR point clouds using robust voxel-based region growing, Remote Sens. Lett., 8, 1062, 10.1080/2150704X.2017.1349961 Wang, 2018, “Voxel segmentation-based 3D building detection algorithm for airborne LIDAR data,” (in eng), PLoS One, 13, e0208996, 10.1371/journal.pone.0208996 H. Su, S. Maji, E. Kalogerakis, E. Learned-Miller, Multi-view Convolutional Neural Networks for 3D Shape Recognition, in: presented at the 2015 IEEE International Conference on Computer Vision (ICCV), 2015/12, 2015. [Online]. Available: http://dx.doi.org/10.1109/iccv.2015.114. R. Q. Charles, H. Su, M. Kaichun, L. J. Guibas, PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation, in: presented at the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017/07, 2017. [Online]. Available: http://dx.doi.org/10.1109/cvpr.2017.16. C. Xiang, C.R. Qi, B. Li, Generating 3D Adversarial Point Clouds, in: presented at the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019 [Online]. Available: http://dx.doi.org/10.1109/cvpr.2019.00935. Y. Li, R. Bu, M. Sun, W. Wu, X. Di, B. Chen, PointCNN: Convolution On X-Transformed Points, 2018. [Online]. Available: http://arxiv.org/abs/1801.07791. Wang, 2019, Dynamic Graph CNN for Learning on Point Clouds, ACM Trans. Graph., 38, 1, 10.1145/3326362 E. Agapaki, I. Brilakis, CLOI-NET: Class segmentation of industrial facilities’ point cloud datasets, Adv. Eng. Inform. 45 (2020) p. 101121, doi: 10.1016/j.aei.2020.101121. Ma, 2020, Semantic segmentation of point clouds of building interiors with deep learning: Augmenting training datasets with synthetic BIM-based point clouds, Autom. Constr., 113, 10.1016/j.autcon.2020.103144 Perez-Perez, 2021, Scan2BIM-NET: Deep Learning Method for Segmentation of Point Clouds for Scan-to-BIM, J. Constr. Eng. Manag., 147, 04021107, 10.1061/(ASCE)CO.1943-7862.0002132 A. Smith, R. Sarlo, Automated extraction of structural beam lines and connections from point clouds of steel buildings, Computer-Aided Civil and Infrastructure Engineering, https://doi.org/10.1111/mice.12699 vol. 37, no. 1, pp. 110-125, 2022/01/01 2022, doi: https://doi.org/10.1111/mice.12699. Wang, 2021, Fully automated generation of parametric BIM for MEP scenes based on terrestrial laser scanning data, Autom. Constr., 125, 10.1016/j.autcon.2021.103615 Kim, 2020, Automated dimensional quality assessment for formwork and rebar of reinforced concrete components using 3D point cloud data, Autom. Constr., 112, 10.1016/j.autcon.2020.103077 Rausch, 2019, Monte Carlo simulation for tolerance analysis in prefabrication and offsite construction, Autom. Constr., 103, 300, 10.1016/j.autcon.2019.03.026 Puri, 2018, Assessment of compliance of dimensional tolerances in concrete slabs using TLS data and the 2D continuous wavelet transform, Autom. Constr., 94, 62, 10.1016/j.autcon.2018.06.004 Kim, 2014, Automated dimensional quality assessment of precast concrete panels using terrestrial laser scanning, Autom. Constr., 45, 163, 10.1016/j.autcon.2014.05.015 Wang, 2017, Automated Estimation of Reinforced Precast Concrete Rebar Positions Using Colored Laser Scan Data, Comput. Aided Civ. Inf. Eng., 32, 787, 10.1111/mice.12293 Nahangi, 2015, Automated assembly discrepancy feedback using 3D imaging and forward kinematics, Autom. Constr., 56, 36, 10.1016/j.autcon.2015.04.005 Bosché, 2010, Automated recognition of 3D CAD model objects in laser scans and calculation of as-built dimensions for dimensional compliance control in construction, Adv. Eng. Inf., 24, 107, 10.1016/j.aei.2009.08.006 Li, 2017, An Improved RANSAC for 3D Point Cloud Plane Segmentation Based on Normal Distribution Transformation Cells, Remote Sens. (Basel), 9, 433, 10.3390/rs9050433 M. Ester, H.-P. Kriegel, J. Sander, X. Xu, A Density-Based Algorithm for Discovering Clusters in Large Spatial Databases with Noise, in: Proceedings of the Second International Conference on Knowledge Discovery and Data Mining (KDD-96), 1996. [Online]. Available: www.aaai.org. [Online]. Available: www.aaai.org. Akbari, 2016, Automated Determination of the Input Parameter of DBSCAN Based on Outlier Detection, 280 Z. Yu, T. Wang, T. Guo, H. Li, J. Dong, Robust point cloud normal estimation via neighborhood reconstruction, Adv. Mech. Eng. 11(4) (2019) p. 1687814019836043, doi: 10.1177/1687814019836043. in AS/NZS 5131:2016 - Structural steelwork - Fabrication and erection, ed: Standards Australia / Standards New Zealand, 2016. Truong-Hong, 2015, Quantitative evaluation strategies for urban 3D model generation from remote sensing data, Comput. Graph., 49, 82, 10.1016/j.cag.2015.03.001 Rutzinger, 2009, A Comparison of Evaluation Techniques for Building Extraction From Airborne Laser Scanning, IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens., 2, 11, 10.1109/JSTARS.2009.2012488 Guo, 2020, Geometric quality inspection of prefabricated MEP modules with 3D laser scanning, Autom. Constr., 111, 10.1016/j.autcon.2019.103053