Winding pathway understanding based on angle projections in a field environment
Tóm tắt
Scene understanding is a core problem for autonomous navigation. However, its implementation is frustrated by a variety of unsettled issues, such as understanding winding pathways in unknown, dynamic, and field environments. Traditional three-dimensional (3D) estimation from 3D point clouds or fused data is memory intensive and energy-consuming, which makes these approaches less reliable in a resource-constrained field robot system with limited computation, memory, and energy resources. In this study, we present a methodology to understand winding field pathways and reconstruct them in a 3D environment, using a low-cost monocular camera without prior training. Winding angle projections are assigned to clusters. By composing subclusters, candidate surfaces are shaped. Based on geometric inferences of integrity and orientation, a field pathway can be approximately understood and reconstructed using straight and winding surfaces in a 3D scene. With the use of geometric inference, no prior training is needed, and the approach is robust to colour and illumination. The percentage of incorrectly classified pixels was compared to the ground truth. Experimental results demonstrated that the method can successfully understand winding pathways, meeting the requirements for robot navigation in an unstructured environment.
Tài liệu tham khảo
GIBSON EJ, WALK RD (1960) The visual cliff. Sci Am 202:64–71
Koenderink JJ, Doorn AJV, Kappers AM (1996) Pictorial surface attitude and local depth comparisons. Percept Psychophys. 58(2):163–173
Wei H, Wang L (2018) Understanding of indoor scenes based on projection of spatial rectangles. Pattern Recognit 81:497–514
Magerand L, Del Bue A (2020) Revisiting projective structure from motion: a robust and efficient incremental solution. IEEE Trans Pattern Anal Mach Intell 42(2):430–443
Bescós B, Cadena C, Neira J (2021) Empty Cities: a dynamic-object-invariant space for visual SLAM. IEEE Trans Robotics 37(2):433–451
Cavagna A, Melillo S, Parisi L, Ricci-Tersenghi F (2021) SpaRTA tracking across occlusions via partitioning of 3d clouds of points. IEEE Trans Pattern Anal Machine Intell 43(4):1394–1403
Bello SA, Wang C, Wambugu NM, Adam JM (2021) FFPointNet: Local and global fused feature for 3D point clouds analysis. Neurocomputing 461:55–62
Wolf P, Berns K (2021) Data-fusion for robust off-road perception considering data quality of uncertain sensors. In: IEEE/RSJ International conference on intelligent robots and systems, IROS, Prague, Czech Republic, pp 6876–6883
Wei H, Wang L (2018) Visual navigation using projection of spatial right-angle in indoor environment. IEEE Trans Image Process 27(7):3164–3177
Wang L, Wei H (2020) Understanding of curved corridor scenes based on projection of spatial right-angles. IEEE Trans Image Process 29:9345–9359
Wang Z (2022) Recognition of occluded objects by slope difference distribution features. Appl Soft Comput 120:108622
Wang Z (2022) Automatic and robust hand gesture recognition by SDD features based model matching. Appl Intell
Wang Z (2020) Robust three-dimensional face reconstruction by one-shot structured light line pattern. Opt Lasers Eng 124:105798
Wang L, Wei H (2020) Avoiding non-Manhattan obstacles based on projection of spatial corners in indoor environment. IEEE/CAA Journal of Automatica Sinica 7:1190–1200
Wang L, Wei H (2020) Understanding of wheelchair ramp scenes for disabled people with visual impairments. Eng Appl Artif Intell 90:103569
Wang L, Wei H (2021) Indoor scene understanding based on manhattan and non-manhattan projection of spatial right-angles. J Vis Commun Image Represent 80:103307
Hedau V, Hoiem D, Forsyth D (2009) Recovering the spatial layout of cluttered rooms. In: ICCV, pp 1849–1856
Zhang Y, Yu F, Song S, Xu P, Seff A (2016) Largescale scene understanding challenge: Room layout estimation
Cordts M, Omran M, Ramos S, Rehfeld T, Enzweiler M, Benenson R et al (2016) The Cityscapes Dataset for semantic urban scene understanding. In: IEEE Conference on computer vision and pattern recognition (CVPR), pp 3213–3223
Badrinarayanan V, Kendall A, Cipolla R (2017) SegNet: a deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans Pattern Anal Machine Intell 39(12):2481–2495
Chen Y, Li W, Van Gool L (2018) ROAD: reality oriented adaptation for semantic segmentation of urban scenes. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR)
Zhang Y, David P, Foroosh H, Gong B (2020) A curriculum domain adaptation approach to the semantic segmentation of urban scenes. IEEE Trans Pattern Anal Machine Intell 42(8):1823–1841
Romera E, Alvarez J M, Bergasa L M, Arroyo R (2018) ERFNet: efficient residual factorized ConvNet for real-time semantic segmentation. IEEE Trans Intell Transp Syst 19(1):263–272
Wang W, Zhou T, Yu F, Dai J, Konukoglu E, Gool LV (2021) Exploring cross-image pixel contrast for semantic segmentation. In: IEEE/CVF international conference on computer vision (ICCV), pp 7283–7293
Zhou T, Wang W, Konukoglu E, Gool LV (2022) Rethinking semantic segmentation: a prototype view accepted to CVPR 2022. arXiv:2203.15102
Lateef F, Kas M, Ruichek Y (2021) Saliency heat-map as visual attention for autonomous driving using generative adversarial network (GAN).IEEE Trans Intell Transp Syst, pp 1–14
Chiaroni F, Rahal MC, Hueber N, Dufaux F (2021) Self-supervised learning for autonomous vehicles perception: a conciliation between analytical and learning methods. IEEE Signal Process Mag 38 (1):31–41
Roncancio H, Becker M, Broggi A, Cattani S (2014) Traversability analysis using terrain mapping and online-trained terrain type classifier. In: IEEE Intelligent vehicles symposium proceedings, Dearborn, MI, USA, June 8-11, 2014, pp 1239–1244
Wang L, Wei H (2021) Curved alleyway understanding based on monocular vision in street scenes. IEEE Trans Intell Transp Syst, pp 1–20
Arena P, Blanco CF, Li Noce A, Taffara S, Patane L (2020) Learning traversability map of different robotic platforms for unstructured terrains path planning. In: International joint conference on neural networks (IJCNN), pp 1–8
Fan DD, Agha-mohammadi Aa, Theodorou EA (2022) Learning risk-aware Costmaps for Traversability in challenging environments. IEEE Robot Autom Lett 7(1):279–286
Nikolovski G, Reke M, Elsen I, Schiffer S (2021) Machine learning based 3D object detection for navigation in unstructured environments. In: IEEE Intelligent vehicles symposium workshops (IV Workshops), pp 236–242
Wigness M, Rogers JG (2017) Unsupervised semantic scene labeling for streaming data. In: IEEE Conference on computer vision and pattern recognition (CVPR), pp 5910–5919
Humblot-Renaux G, Marchegiani L, Moeslund TB, Gade R (2022) Navigation-oriented scene understanding for robotic autonomy: learning to segment driveability in egocentric images. IEEE Robot Autom Lett 7(2):2913–2920
Holder CJ, Drive Breckon TP (2021) Learning to end-to-end off-road path prediction. IEEE Intell Transp Syst Mag 13(2):217–221
Baheti B, Innani S, Gajre SS, Talbar SN (2020) Semantic scene segmentation in unstructured environment with modified DeepLabV3+. Pattern Recognit Lett 138:223–229
Bosilj P, Aptoula E, Duckett T, Cielniak G (2020) Transfer learning between crop types for semantic segmentation of crops versus weeds in precision agriculture. J Field Robot 37(1):7–19
Gao B, Hu S, Zhao X, Zhao H (2021) Fine-grained off-road semantic segmentation and mapping via contrastive learning. In: IEEE/RSJ international conference on intelligent robots and systems, IROS, Prague, Czech Republic, pp 5950–5957
Tassis LM, Tozzi de Souza JE, Krohling RA (2021) A deep learning approach combining instance and semantic segmentation to identify diseases and pests of coffee leaves from in-field images. Comput Electron Agric 186:106191
Zurn J, Burgard W, Valada A (2021) Self-supervised visual terrain classification from unsupervised acoustic feature learning. IEEE Trans Robot 37(2):466–481
Bernuy F, Ruiz-del-Solar J (2015) Semantic mapping of large-scale outdoor scenes for autonomous off-road driving. In: IEEE international conference on computer vision workshop, ICCV workshops, Santiago, Chile. IEEE Computer Society, pp 124– 130
Dong W, Roy P, Isler V (2020) Semantic mapping for orchard environments by merging two-sides reconstructions of tree rows. Journal of Field Robotics 37(1):97–121
Maturana D, Chou P, Uenoyama M, Scherer SA (2017) Real-time semantic mapping for autonomous off-road navigation. In: Field and service robotics, results of the 11th international conference, FSR, Zurich, Switzerland, vol 5, pp 335–350
Yang Y, Tang D, Wang D, Song W, Wang J, Fu M (2020) Multi-camera visual SLAM for off-road navigation. Robotics Auton Syst 128:103505
Arbelaez P, FC Maire M (2019) From contours to regions: An empirical evaluation. In: CVPR, pp 2294–2301
Rother C (2000) A new approach for vanishing point detection in architectural environments. In: Mirmehdi M, Thomas B T (eds) Proceedings of the british machine vision conference, BMVC, Bristol, UK. British machine vision association, pp 1–10
García-Faura Á, Martínez FF, Kleinlein R, San-Segundo-Hernández R, Díaz-de-María F (2019) A multi-threshold approach and a realistic error measure for vanishing point detection in natural landscapes. Eng Appl Artif Intell 85:713–726
Xiao J, Hays J, Ehinger K, Oliva A, Torralba ASUN (2010) Database: large-scale scene recognition from Abbey to Zoo. CVPR, pp 485–3492
Metzger KA, Mortimer P, Wuensche H (2020) A fine-grained dataset and its efficient semantic segmentation for unstructured driving scenarios. In: 25th International conference on pattern recognition, ICPR, Virtual Event / Milan, Italy, pp 7892–7899
Holder CJ, Breckon TP (2021) Learning to drive: end-to-end off-road path prediction. IEEE Intell Trans Syst Mag 13(2):217–221
Geiger A, Lenz P, Stiller C, Urtasun R (2013) Vision meets robotics: the KITTI dataset. Int J Robotics Res 32(11):1231–1237
Wigness MB, Eum S, Rogers JG, Han D, Kwon H (2019) A RUGD Dataset for autonomous navigation and visual perception in unstructured outdoor environments. In: IEEE/RSJ International conference on intelligent robots and systems, IROS, Macau, SAR, China, pp 5000–5007
Badrinarayanan V, Kendall A, Cipolla R (2017) SegNet: A deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans Pattern Anal Mach Intell 39(12):2481–2495