Winding pathway understanding based on angle projections in a field environment

Springer Science and Business Media LLC - Tập 53 - Trang 16859-16874 - 2022
Luping Wang1, Hui Wei2
1Laboratory of 3D Scene Understanding and Visual Navigation, School of Mechanical Engineering, University of Shanghai for Science and Technology, Shangh ai, China
2Laboratory of Algorithms for Cognitive Models, School of Computer Science, Fudan University, Shanghai, China

Tóm tắt

Scene understanding is a core problem for autonomous navigation. However, its implementation is frustrated by a variety of unsettled issues, such as understanding winding pathways in unknown, dynamic, and field environments. Traditional three-dimensional (3D) estimation from 3D point clouds or fused data is memory intensive and energy-consuming, which makes these approaches less reliable in a resource-constrained field robot system with limited computation, memory, and energy resources. In this study, we present a methodology to understand winding field pathways and reconstruct them in a 3D environment, using a low-cost monocular camera without prior training. Winding angle projections are assigned to clusters. By composing subclusters, candidate surfaces are shaped. Based on geometric inferences of integrity and orientation, a field pathway can be approximately understood and reconstructed using straight and winding surfaces in a 3D scene. With the use of geometric inference, no prior training is needed, and the approach is robust to colour and illumination. The percentage of incorrectly classified pixels was compared to the ground truth. Experimental results demonstrated that the method can successfully understand winding pathways, meeting the requirements for robot navigation in an unstructured environment.

Tài liệu tham khảo

GIBSON EJ, WALK RD (1960) The visual cliff. Sci Am 202:64–71 Koenderink JJ, Doorn AJV, Kappers AM (1996) Pictorial surface attitude and local depth comparisons. Percept Psychophys. 58(2):163–173 Wei H, Wang L (2018) Understanding of indoor scenes based on projection of spatial rectangles. Pattern Recognit 81:497–514 Magerand L, Del Bue A (2020) Revisiting projective structure from motion: a robust and efficient incremental solution. IEEE Trans Pattern Anal Mach Intell 42(2):430–443 Bescós B, Cadena C, Neira J (2021) Empty Cities: a dynamic-object-invariant space for visual SLAM. IEEE Trans Robotics 37(2):433–451 Cavagna A, Melillo S, Parisi L, Ricci-Tersenghi F (2021) SpaRTA tracking across occlusions via partitioning of 3d clouds of points. IEEE Trans Pattern Anal Machine Intell 43(4):1394–1403 Bello SA, Wang C, Wambugu NM, Adam JM (2021) FFPointNet: Local and global fused feature for 3D point clouds analysis. Neurocomputing 461:55–62 Wolf P, Berns K (2021) Data-fusion for robust off-road perception considering data quality of uncertain sensors. In: IEEE/RSJ International conference on intelligent robots and systems, IROS, Prague, Czech Republic, pp 6876–6883 Wei H, Wang L (2018) Visual navigation using projection of spatial right-angle in indoor environment. IEEE Trans Image Process 27(7):3164–3177 Wang L, Wei H (2020) Understanding of curved corridor scenes based on projection of spatial right-angles. IEEE Trans Image Process 29:9345–9359 Wang Z (2022) Recognition of occluded objects by slope difference distribution features. Appl Soft Comput 120:108622 Wang Z (2022) Automatic and robust hand gesture recognition by SDD features based model matching. Appl Intell Wang Z (2020) Robust three-dimensional face reconstruction by one-shot structured light line pattern. Opt Lasers Eng 124:105798 Wang L, Wei H (2020) Avoiding non-Manhattan obstacles based on projection of spatial corners in indoor environment. IEEE/CAA Journal of Automatica Sinica 7:1190–1200 Wang L, Wei H (2020) Understanding of wheelchair ramp scenes for disabled people with visual impairments. Eng Appl Artif Intell 90:103569 Wang L, Wei H (2021) Indoor scene understanding based on manhattan and non-manhattan projection of spatial right-angles. J Vis Commun Image Represent 80:103307 Hedau V, Hoiem D, Forsyth D (2009) Recovering the spatial layout of cluttered rooms. In: ICCV, pp 1849–1856 Zhang Y, Yu F, Song S, Xu P, Seff A (2016) Largescale scene understanding challenge: Room layout estimation Cordts M, Omran M, Ramos S, Rehfeld T, Enzweiler M, Benenson R et al (2016) The Cityscapes Dataset for semantic urban scene understanding. In: IEEE Conference on computer vision and pattern recognition (CVPR), pp 3213–3223 Badrinarayanan V, Kendall A, Cipolla R (2017) SegNet: a deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans Pattern Anal Machine Intell 39(12):2481–2495 Chen Y, Li W, Van Gool L (2018) ROAD: reality oriented adaptation for semantic segmentation of urban scenes. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR) Zhang Y, David P, Foroosh H, Gong B (2020) A curriculum domain adaptation approach to the semantic segmentation of urban scenes. IEEE Trans Pattern Anal Machine Intell 42(8):1823–1841 Romera E, Alvarez J M, Bergasa L M, Arroyo R (2018) ERFNet: efficient residual factorized ConvNet for real-time semantic segmentation. IEEE Trans Intell Transp Syst 19(1):263–272 Wang W, Zhou T, Yu F, Dai J, Konukoglu E, Gool LV (2021) Exploring cross-image pixel contrast for semantic segmentation. In: IEEE/CVF international conference on computer vision (ICCV), pp 7283–7293 Zhou T, Wang W, Konukoglu E, Gool LV (2022) Rethinking semantic segmentation: a prototype view accepted to CVPR 2022. arXiv:2203.15102 Lateef F, Kas M, Ruichek Y (2021) Saliency heat-map as visual attention for autonomous driving using generative adversarial network (GAN).IEEE Trans Intell Transp Syst, pp 1–14 Chiaroni F, Rahal MC, Hueber N, Dufaux F (2021) Self-supervised learning for autonomous vehicles perception: a conciliation between analytical and learning methods. IEEE Signal Process Mag 38 (1):31–41 Roncancio H, Becker M, Broggi A, Cattani S (2014) Traversability analysis using terrain mapping and online-trained terrain type classifier. In: IEEE Intelligent vehicles symposium proceedings, Dearborn, MI, USA, June 8-11, 2014, pp 1239–1244 Wang L, Wei H (2021) Curved alleyway understanding based on monocular vision in street scenes. IEEE Trans Intell Transp Syst, pp 1–20 Arena P, Blanco CF, Li Noce A, Taffara S, Patane L (2020) Learning traversability map of different robotic platforms for unstructured terrains path planning. In: International joint conference on neural networks (IJCNN), pp 1–8 Fan DD, Agha-mohammadi Aa, Theodorou EA (2022) Learning risk-aware Costmaps for Traversability in challenging environments. IEEE Robot Autom Lett 7(1):279–286 Nikolovski G, Reke M, Elsen I, Schiffer S (2021) Machine learning based 3D object detection for navigation in unstructured environments. In: IEEE Intelligent vehicles symposium workshops (IV Workshops), pp 236–242 Wigness M, Rogers JG (2017) Unsupervised semantic scene labeling for streaming data. In: IEEE Conference on computer vision and pattern recognition (CVPR), pp 5910–5919 Humblot-Renaux G, Marchegiani L, Moeslund TB, Gade R (2022) Navigation-oriented scene understanding for robotic autonomy: learning to segment driveability in egocentric images. IEEE Robot Autom Lett 7(2):2913–2920 Holder CJ, Drive Breckon TP (2021) Learning to end-to-end off-road path prediction. IEEE Intell Transp Syst Mag 13(2):217–221 Baheti B, Innani S, Gajre SS, Talbar SN (2020) Semantic scene segmentation in unstructured environment with modified DeepLabV3+. Pattern Recognit Lett 138:223–229 Bosilj P, Aptoula E, Duckett T, Cielniak G (2020) Transfer learning between crop types for semantic segmentation of crops versus weeds in precision agriculture. J Field Robot 37(1):7–19 Gao B, Hu S, Zhao X, Zhao H (2021) Fine-grained off-road semantic segmentation and mapping via contrastive learning. In: IEEE/RSJ international conference on intelligent robots and systems, IROS, Prague, Czech Republic, pp 5950–5957 Tassis LM, Tozzi de Souza JE, Krohling RA (2021) A deep learning approach combining instance and semantic segmentation to identify diseases and pests of coffee leaves from in-field images. Comput Electron Agric 186:106191 Zurn J, Burgard W, Valada A (2021) Self-supervised visual terrain classification from unsupervised acoustic feature learning. IEEE Trans Robot 37(2):466–481 Bernuy F, Ruiz-del-Solar J (2015) Semantic mapping of large-scale outdoor scenes for autonomous off-road driving. In: IEEE international conference on computer vision workshop, ICCV workshops, Santiago, Chile. IEEE Computer Society, pp 124– 130 Dong W, Roy P, Isler V (2020) Semantic mapping for orchard environments by merging two-sides reconstructions of tree rows. Journal of Field Robotics 37(1):97–121 Maturana D, Chou P, Uenoyama M, Scherer SA (2017) Real-time semantic mapping for autonomous off-road navigation. In: Field and service robotics, results of the 11th international conference, FSR, Zurich, Switzerland, vol 5, pp 335–350 Yang Y, Tang D, Wang D, Song W, Wang J, Fu M (2020) Multi-camera visual SLAM for off-road navigation. Robotics Auton Syst 128:103505 Arbelaez P, FC Maire M (2019) From contours to regions: An empirical evaluation. In: CVPR, pp 2294–2301 Rother C (2000) A new approach for vanishing point detection in architectural environments. In: Mirmehdi M, Thomas B T (eds) Proceedings of the british machine vision conference, BMVC, Bristol, UK. British machine vision association, pp 1–10 García-Faura Á, Martínez FF, Kleinlein R, San-Segundo-Hernández R, Díaz-de-María F (2019) A multi-threshold approach and a realistic error measure for vanishing point detection in natural landscapes. Eng Appl Artif Intell 85:713–726 Xiao J, Hays J, Ehinger K, Oliva A, Torralba ASUN (2010) Database: large-scale scene recognition from Abbey to Zoo. CVPR, pp 485–3492 Metzger KA, Mortimer P, Wuensche H (2020) A fine-grained dataset and its efficient semantic segmentation for unstructured driving scenarios. In: 25th International conference on pattern recognition, ICPR, Virtual Event / Milan, Italy, pp 7892–7899 Holder CJ, Breckon TP (2021) Learning to drive: end-to-end off-road path prediction. IEEE Intell Trans Syst Mag 13(2):217–221 Geiger A, Lenz P, Stiller C, Urtasun R (2013) Vision meets robotics: the KITTI dataset. Int J Robotics Res 32(11):1231–1237 Wigness MB, Eum S, Rogers JG, Han D, Kwon H (2019) A RUGD Dataset for autonomous navigation and visual perception in unstructured outdoor environments. In: IEEE/RSJ International conference on intelligent robots and systems, IROS, Macau, SAR, China, pp 5000–5007 Badrinarayanan V, Kendall A, Cipolla R (2017) SegNet: A deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans Pattern Anal Mach Intell 39(12):2481–2495