Bridging the gap: Neuro-Symbolic Computing for advanced AI applications in construction

Badreddine S, d’Avila Garcez A, Serafini L, Spranger M (2022). Logic tensor networks. Artificial Intelligence, 303: 103649

Braun A, Tuttas S, Borrmann A, Stilla U (2020). Improving progress monitoring by fusing point clouds, semantic data and computer vision. Automation in Construction, 116: 103210

Brilakis I, Park M W, Jog G (2011). Automated vision tracking of project related entities. Advanced Engineering Informatics, 25(4): 713–724

Brown T B, Mann B, Ryder N, Subbiah M, Kaplan J D, Dhariwal P, Neelakantan A, Shyam P, Sastry G, Askell A, et al. (2020). Language models are few-shot learners. In: Proceedings of the 34th International Conference on Neural Information Processing Systems. Vancouver, BC: Curran Associates Inc., 1877–1901

Carion N, Massa F, Synnaeve G, Usunier N, Kirillov A, Zagoruyko S (2020). End-to-end object detection with transformers. In: Proceedings of the 16th European Conference. Glasgow: Springer-Verlag, 213–229

Chi S, Caldas C H (2011). Automated object identification using optical video cameras on construction sites. Computer-Aided Civil and Infrastructure Engineering, 26(5): 368–380

Chi S, Caldas C (2012). Image-based safety assessment: Automated spatial safety risk identification of earthmoving and surface mining activities. Journal of Construction Engineering and Management, 138(3): 341–351

Chi S, Caldas C H, Kim D Y (2009). A methodology for object identification and tracking in construction based on spatial modeling and image matching techniques. Computer-Aided Civil and Infrastructure Engineering, 24(3): 199–211

Cohen W W, Yang F, Mazaitis K R (2017). TensorLog: Deep learning meets probabilistic DBs. arXiv preprint, arXiv:1707.05390

Cussens J (2001). Parameter estimation in stochastic logic programs. Machine Learning, 44(3): 245–271

d’Avila Garcez A, Lamb L C (2023). Neurosymbolic AI: The 3rd wave. Artificial Intelligence Review, 56(11): 12387–12406

Daniele A, Serafini L (2019). Knowledge enhanced neural networks. In: Proceedings of the 16th Pacific Rim International Conference on Artificial Intelligence: Trends in Artificial Intelligence. Cuvu, Yanuca Island: Springer Cham, 542–554

Ding L, Fang W, Luo H, Love P E D, Zhong B, Ouyang X (2018). A deep hybrid learning model to detect unsafe behavior: Integrating convolution neural networks and long short-term memory. Automation in Construction, 86: 118–124

Du Y, Fu Z, Liu Q, Wang Y (2022). Visual grounding with transformers. In: Proceedings of the IEEE International Conference on Multimedia and Expo (ICME). Taipei: IEEE, 1–6

Fang Q, Li H, Luo X, Ding L, Luo H, Rose T M, An W (2018a). Detecting non-hardhat-use by a deep learning method from far-field surveillance videos. Automation in Construction, 85: 1–9

Fang Q, Li H, Luo X, Ding L, Rose T M, An W, Yu Y (2018b). A deep learning-based method for detecting non-certified work on construction sites. Advanced Engineering Informatics, 35: 56–68

Fang W, Ding L, Luo H, Love P E D (2018c). Falls from heights: A computer vision-based approach for safety harness detection. Automation in Construction, 91: 53–61

Fang W, Love P E D, Ding L, Xu S, Kong T, Li H (2021). Computer vision and deep learning to manage safety in construction: Matching images of unsafe behavior and semantic rules. IEEE Transactions on Engineering Management, 70(12): 4120–4132

Garnelo M, Shanahan M (2019). Reconciling deep learning with symbolic artificial intelligence: Representing objects and relations. Current Opinion in Behavioral Sciences, 29: 17–23

Girdhar R, Gkioxari G, Torresani L, Paluri M, Tran D (2018). Detectand- track: Efficient pose estimation in videos. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, UT: IEEE, 350–359

Girshick R (2015). Fast R-CNN. In: Proceedings of the IEEE International Conference on Computer Vision. Santiago: IEEE, 1440–1448

Gong J, Caldas C H (2010). Computer vision-based video interpretation model for automated productivity analysis of construction operations. Journal of Computing in Civil Engineering, 24(3): 252–263

He K, Gkioxari G, Dollár P, Girshick R (2017). Mask R-CNN. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV). Venice: IEEE, 2980–2988

Hochreiter S, Schmidhuber J (1997). Long short-term memory. Neural Computation, 9(8): 1735–1780

Hu Q, Yang B, Xie L, Rosa S, Guo Y, Wang Z, Trigoni N, Markham A (2020). Randla-net: Efficient semantic segmentation of large-scale point clouds. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Seattle, WA IEEE, 11105–11114

Huang Y, Wu Q, Wang W, Wang L (2020). Image and sentence matching via semantic concepts and order learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 42(3): 636–650

Jain A, Gkanatsios N, Mediratta I, Fragkiadaki K (2022). Bottom up top down detection transformers for language grounding in images and point clouds. In: Proceedings of the 17th European Conference on Computer Vision. Tel Aviv: Springer-Verlag, 417–433

Kahneman D, Hinton G, Bengio Y, LeCun Y, Rossi F (2020). AAAI-20 fireside chat with Daniel Kahneman

Kim J, Chung D, Kim Y, Kim H (2022). Deep learning-based 3D reconstruction of scaffolds using a robot dog. Automation in Construction, 134: 104092

Kim P S, Lee D G, Lee S W (2018). Discriminative context learning with gated recurrent unit for group activity recognition. Pattern Recognition, 76: 149–161

Kimmig A, Mihalkova L, Getoor L (2015). Lifted graphical models: A survey. Machine Learning, 99(1): 1–45

Kimmig A, van den Broeck G, de Raedt L (2011). An algebraic prolog for reasoning about possible worlds. In: Proceedings of the 25th AAAI Conference on Artificial Intelligence. San Francisco, CA: AAAI Press, 209–214

Krizhevsky A, Sutskever I, Hinton G E (2012). ImageNet classification with deep convolutional neural networks. In: Proceedings of the 25th International Conference on Neural Information Processing Systems. Lake Tahoe, NV: Curran Associates Inc., 1097–1105

LeCun Y, Bengio Y, Hinton G (2015). Deep learning. Nature, 521(7553): 436–444

Lei L, Zhou Y, Luo H, Love P E D (2019). A CNN-based 3D patch registration approach for integrating sequential models in support of progress monitoring. Advanced Engineering Informatics, 41: 100923

Li Y, Wei H, Han Z, Huang J, Wang W (2020). Deep learning-based safety helmet detection in engineering management based on convolutional neural networks. Advances in Civil Engineering, 9703560

Luo X, Li H, Cao D, Dai F, Seo J, Lee S (2018). Recognizing diverse construction activities in site images via relevance networks of construction-related objects detected by convolutional neural networks. Journal of Computing in Civil Engineering, 32(3): 04018012

Luo X, Li H, Yang X, Yu Y, Cao D (2019). Capturing and understanding workers’ activities in far-field surveillance videos with deep action recognition and Bayesian nonparametric learning. Computer-Aided Civil and Infrastructure Engineering, 34(4): 333–351

Luo X, Li H, Yu Y, Zhou C, Cao D (2020). Combining deep features and activity context to improve recognition of activities of workers in groups. Computer-Aided Civil and Infrastructure Engineering, 35(9): 965–978

Manhaeve R, Dumančić S, Kimmig A, Demeester T, de Raedt L (2021). Neural probabilistic logic programming in DeepProbLog. Artificial Intelligence, 298: 103504

Mao J, Gan C, Kohli P, Tenenbaum J B, Wu J (2019). The neurosymbolic concept learner: Interpreting scenes, words, and sentences from natural supervision. In: Proceedings of the 7th International Conference on Learning Representations. New Orleans, LA: Open-Review.net

Qu M, Tang J (2019). Probabilistic logic neural networks for reasoning. In: Proceedings of the 33rd International Conference on Neural Information Processing Systems. Vancouver, BC: Curran Associates Inc., 7712–7722

Radford A, Narasimhan K, Salimans T, Sutskever I (2018). Improving language understanding by generative pre-training. OpenAI

Ren S, He K, Girshick R, Sun J (2017). Faster R-CNN: Towards real-time object detection with region proposal networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(6): 1137–1149

Richardson M, Domingos P (2006). Markov logic networks. Machine Learning, 62(1–2): 107–136

Riegel R, Gray A, Luus F, Khan N, Makondo N, Yunus Akhalwaya I, Qian H, Fagin R, Barahona F, Sharma U, Ikbal S, Karanam H, Neelam S, Likhyani A, Srivastava S (2020). Logical neural networks. arXiv preprint, arXiv:2006.13155

Rodríguez-Gonzálvez P, Rodríguez-Martín M, Ramos L F, González-Aguilera D (2017). 3D reconstruction methods and quality assessment for visual inspection of welds. Automation in Construction, 79: 49–58

Seo J, Han S, Lee S, Kim H (2015). Computer vision techniques for construction safety and health monitoring. Advanced Engineering Informatics, 29(2): 239–251

Sourek G, Aschenbrenner V, Zelezny F, Schockaert S, Kuzelka O (2018). Lifted relational neural networks: Efficient learning of latent relational structures. Journal of Artificial Intelligence Research, 62: 69–100

Teizer J (2015). Status quo and open challenges in vision-based sensing and tracking of temporary resources on infrastructure construction sites. Advanced Engineering Informatics, 29(2): 225–238

Teizer J, Allread B S, Fullerton C E, Hinze J (2010). Autonomous proactive real-time construction worker and equipment operator proximity safety alert system. Automation in Construction, 19(5): 630–640

Tran S N, d’Avila Garcez A S (2018). Deep logic networks: Inserting and extracting knowledge from deep belief networks. IEEE Transactions on Neural Networks and Learning Systems, 29(2): 246–258

Vahdatikhaki F, Hammad A (2015). Risk-based look-ahead workspace generation for earthwork equipment using near real-time simulation. Automation in Construction, 58: 207–220

Wang Q (2019). Automatic checks from 3D point cloud data for safety regulation compliance for scaffold work platforms. Automation in Construction, 104: 38–51

Wang Q, Mao Z, Wang B, Guo L (2017). Knowledge graph embedding: A survey of approaches and applications. IEEE Transactions on Knowledge and Data Engineering, 29(12): 2724–2743

Wu F, Jin G, Gao M, He Z, Yang Y (2019). Helmet detection based on improved YOLO v3 deep model. In: Proceedings of the 16th International Conference on Networking, Sensing and Control (ICNSC). Banff, AB: IEEE, 363–368

Wu H, Zhong B, Li H, Love P E D, Pan X, Zhao N (2021). Combining computer vision with semantic reasoning for on-site safety management in construction. Journal of Building Engineering, 42: 103036

Yang J, Park M W, Vela P A, Golparvar-Fard M (2015). Construction performance monitoring via still images, time-lapse photos, and video streams: Now, tomorrow, and the future. Advanced Engineering Informatics, 29(2): 211–224

Yang X, Li H, Yu Y, Luo X, Huang T, Yang X (2018). Automatic pixel-level crack detection and measurement using fully convolutional network. Computer-Aided Civil and Infrastructure Engineering, 33(12): 1090–1109

Zhang K, Zhang Z, Li Z, Qiao Y (2016). Joint face detection and alignment using multitask cascaded convolutional networks. IEEE Signal Processing Letters, 23(10): 1499–1503

Zhong B, Li H, Luo H, Zhou J, Fang W, Xing X (2020). Ontology-based semantic modeling of knowledge in construction: Classification and identification of hazards implied in images. Journal of Construction Engineering and Management, 146(4): 04020013

Zhu X, Su W, Lu L, Li B, Wang X, Dai J (2020). Deformable DETR: Deformable transformers for end-to-end object detection. arXiv preprint, arXiv:2010.04159