Road Surface Reconstruction by Stereo Vision

Hauke Brunken1, Clemens Gühmann1
1Chair of Electronic Measurement and Diagnostic Technology, Technische Universität Berlin, Berlin, Germany

Tóm tắt

This paper covers the problem of road surface reconstruction by stereo vision with cameras placed behind the windshield of a moving vehicle. An algorithm was developed that employs a plane-sweep approach and uses semi-global matching for optimization. Different similarity measures were evaluated for the task of matching pixels, namely mutual information, background subtraction by bilateral filtering, and Census. The chosen sweeping direction is the plane normal of the mean road surface. Since the cameras’ position in relation to the base plane is continuously changing due to the suspension of the vehicle, the search for the base plane was integrated into the stereo algorithm. Experiments were conducted for different types of pavement and different lighting conditions. Results are presented for the target application of road surface reconstruction, and they show high correspondence to laser scan reference measurements. The method handles motion blur well, and elevation maps are reconstructed on a millimeter-scale, while images are captured at driving speed.

Tài liệu tham khảo

Ahmed M, Haas C, Haas R (2011) Toward low-cost 3D automatic pavement distress surveying: the close range photogrammetry approach. Can J Civ Eng 38(12):1301–1313 Ansar A, Castano A, Matthies L (2004) Enhanced real-time stereo using bilateral filtering. In: Proceedings. 2nd international symposium on 3D data processing, visualization and transmission, 2004. 3DPVT 2004, pp 455–462. https://doi.org/10.1109/TDPVT.2004.1335273 Besl PJ, McKay ND (1992) A method for registration of 3-D shapes. IEEE Trans Pattern Anal Mach Intell 14(2):239–256. https://doi.org/10.1109/34.121791 Boykov Y, Veksler O, Zabih R (2001) Fast approximate energy minimization via graph cuts. IEEE Trans Pattern Anal Mach Intell 23(11):1222–1239 Brunken H, Gühmann C (2019) Incorporating plane-sweep in convolutional neural network stereo imaging for road surface reconstruction. In: Proceedings of the 14th international joint conference on computer vision, imaging and computer graphics theory and applications—volume 5: VISAPP, SciTePress, pp 784–791. https://doi.org/10.5220/0007352107840791 Bulatov D (2015) Temporal selection of images for a fast algorithm for depth-map extraction in multi-baseline configurations. In: Proceedings of the 10th international conference on computer vision theory and applications, SciTePress, Berlin, Germany, pp 395–402. https://doi.org/10.5220/0005239503950402. http://www.scitepress.org/DigitalLibrary/Link.aspx?doi=10.5220/0005239503950402 Bundesanstalt für Straßenwesen (2018) Measuring vehicles and systems. https://www.bast.de/durabast/DE/durabast/Messfahrzeuge/fahrzeuge_node.html. Accessed 10 Nov 2018 Chang JR, Chen YS (2018) Pyramid stereo matching network, p 9. arXiv:180308669 Coenen TBJ, Golroo A (2017) A review on automated pavement distress detection methods. Cogent Eng 4(1):1374822. https://doi.org/10.1080/23311916.2017.1374822 Collins RT (1996) A space-sweep approach to true multi-image matching. In: Proceedings CVPR IEEE computer society conference on computer vision and pattern recognition, IEEE, pp 358–363. http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=517097 Devernay F (1994) Computing differential properties of 3-D shapes from stereoscopic images without 3-D models. In: Proceedings of IEEE conference on computer vision and pattern recognition CVPR-94, IEEE Computer Society Press, Seattle, WA, USA, pp 208–213. https://doi.org/10.1109/CVPR.1994.323831. http://ieeexplore.ieee.org/document/323831/ Eisenbach M, Stricker R, Seichter D, Amende K, Debes K, Sesselmann M, Ebersbach D, Stoeckert U, Gross HM (2017) How to get pavement distress detection ready for deep learning? A systematic approach. In: 2017 international joint conference on neural networks (IJCNN), IEEE, Anchorage, AK, USA, pp 2039–2047. https://doi.org/10.1109/IJCNN.2017.7966101. http://ieeexplore.ieee.org/document/7966101/ El Gendy A, Shalaby A, Saleh M, Flintsch GW (2011) Stereo-vision applications to reconstruct the 3D texture of pavement surface. Int J Pavement Eng 12(3):263–273. https://doi.org/10.1080/10298436.2010.546858 Fan R, Ai X, Dahnoun N (2018) Road surface 3D reconstruction based on dense subpixel disparity map estimation. IEEE Trans Image Process 27(6):3025–3035. https://doi.org/10.1109/TIP.2018.2808770 Fischler MA, Bolles RC (1981) Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun ACM 24(6):381–395 Gallup D, Frahm JM, Mordohai P, Yang Q, Pollefeys M (2007) Real-time plane-sweeping stereo with multiple sweeping directions. In: 2007 IEEE conference on computer vision and pattern recognition, IEEE, pp 1–8. http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=4270270 Gang L, Zucker S (2006) Surface geometric constraints for stereo in belief propagation. In: 2006 IEEE computer society conference on computer vision and pattern recognition—volume 2 (CVPR’06), IEEE, New York, NY, USA, pp 2355–2362. https://doi.org/10.1109/CVPR.2006.299. http://ieeexplore.ieee.org/document/1641042/. Accessed 30 April 2020 GPL software (2017) Cloudcompare 2.8. http://www.cloudcompare.org. Accessed 01 Feb 2018 Hirschmüller H (2008) Stereo processing by semiglobal matching and mutual information. IEEE Trans Pattern Anal Mach Intell 30(2):328–341. https://doi.org/10.1109/TPAMI.2007.1166 Hirschmüller H, Scharstein D (2009) Evaluation of stereo matching costs on images with radiometric differences. IEEE Trans Pattern Anal Mach Intell 31(9):1582–1599. https://doi.org/10.1109/TPAMI.2008.221 Irschara A, Rumpler M, Meixner P, Pock T, Bischof H (2012) Efficient and globally optimal multi view dense matching for aerial images. In: ISPRS annals of photogrammetry, remote sensing and spatial information sciences, vol I-3, pp 227–232. https://doi.org/10.5194/isprsannals-I-3-227-2012. http://www.isprs-ann-photogramm-remote-sens-spatial-inf-sci.net/I-3/227/2012/. Accessed 16 Oct 2018 Ivanchenko V, Shen H, Coughlan J (2009) Elevation-based MRF stereo implemented in real-time on a GPU. In: 2009 workshop on applications of computer vision (WACV), IEEE, pp 1–8. Kim J, Kolmogorov V, Zabih R (2003) Visual correspondence using energy minimization and mutual information. In: Proceedings ninth IEEE international conference on computer vision, IEEE, vol 2, pp 1033–1040. http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=1238463. Accessed 18 Aug 2016 Li L, Zhang S, Yu X, Zhang L (2018) PMSC: PatchMatch-based superpixel cut for accurate stereo matching. IEEE Trans Circuits Syst Video Technol 28(3):679–692 Olsson C, Ulén J, Boykov Y (2013) In defense of 3D-label stereo. In: 2013 IEEE conference on computer vision and pattern recognition, pp 1730–1737. https://doi.org/10.1109/CVPR.2013.226 Roth L, Mayer H (2019) Reduction of the fronto-parallel bias for wide-baseline semi-global matching. In: ISPRS annals of photogrammetry, remote sensing and spatial information sciences, vol IV-2/W5, pp 69–76. https://doi.org/10.5194/isprs-annals-IV-2-W5-69-2019. https://www.isprs-ann-photogramm-remote-sens-spatial-inf-sci.net/IV-2-W5/69/2019/. Accessed 23 Dec 2019 Scharstein D, Taniai T, Sinha SN (2017) Semi-global stereo matching with surface orientation priors. In: 2017 international conference on 3D vision (3DV), IEEE, Qingdao, pp 215–224. https://doi.org/10.1109/3DV.2017.00033. https://ieeexplore.ieee.org/document/8374574/. Accessed 16 Oct 2018 Sinha SN, Scharstein D, Szeliski R (2014) Efficient high-resolution stereo matching using local plane sweeps. In: 2014 IEEE conference on computer vision and pattern recognition, pp 1582–1589. https://doi.org/10.1109/CVPR.2014.205 Szeliski R (2011) Computer vision. Texts in computer science. Springer, London. http://link.springer.com/10.1007/978-1-84882-935-0 Szeliski R, Zabih R, Scharstein D, Veksler O, Kolmogorov V, Agarwala A, Tappen M, Rother C (2008) A comparative study of energy minimization methods for Markov random fields with smoothness-based priors. IEEE Trans Pattern Anal Mach Intell 30(6):1068–1080. https://doi.org/10.1109/TPAMI.2007.70844 Wang KC, Gong W (2002) Automated pavement distress survey: a review and a new direction. In: Pavement evaluation conference, pp 21–25 Woodford O, Torr P, Reid I, Fitzgibbon A (2009) Global stereo reconstruction under second-order smoothness priors. IEEE Trans Pattern Anal Mach Intell 31(12):2115–2128. https://doi.org/10.1109/TPAMI.2009.131 Yang R, Welch G, Bishop G (2002) Real-time consensus-based scene reconstruction using commodity graphics hardware. In: 10th Pacific conference on computer graphics and applications, 2002. Proceedings., vol 22, pp 225–234 Zabih R, Woodfill J (1994) Non-parametric local transforms for computing visual correspondence. European conference on computer vision. Springer, Berlin, pp 151–158 Zbontar J, LeCun Y (2016) Stereo matching by training a convolutional neural network to compare image patches. J Mach Learn Res 17(1–32):2 Zhang Z, Ai X, Chan CK, Dahnoun N (2014) An efficient algorithm for pothole detection using stereo vision. In: 2014 IEEE international conference on acoustics, speech and signal processing (ICASSP), IEEE, Florence, Italy, pp 564–568. https://doi.org/10.1109/ICASSP.2014.6853659. http://ieeexplore.ieee.org/document/6853659/. Accessed 16 Oct 2018 Zhao W, Yan L, Zhang Y (2018) Geometric-constrained multi-view image matching method based on semi-global optimization. Geo-spat Inf Sci 21(2):115–126. https://doi.org/10.1080/10095020.2018.1441754