Road Surface Reconstruction by Stereo Vision
PFG – Journal of Photogrammetry, Remote Sensing and Geoinformation Science - Tập 88 - Trang 433-448 - 2020
Tóm tắt
This paper covers the problem of road surface reconstruction by stereo vision with cameras placed behind the windshield of a moving vehicle. An algorithm was developed that employs a plane-sweep approach and uses semi-global matching for optimization. Different similarity measures were evaluated for the task of matching pixels, namely mutual information, background subtraction by bilateral filtering, and Census. The chosen sweeping direction is the plane normal of the mean road surface. Since the cameras’ position in relation to the base plane is continuously changing due to the suspension of the vehicle, the search for the base plane was integrated into the stereo algorithm. Experiments were conducted for different types of pavement and different lighting conditions. Results are presented for the target application of road surface reconstruction, and they show high correspondence to laser scan reference measurements. The method handles motion blur well, and elevation maps are reconstructed on a millimeter-scale, while images are captured at driving speed.
Tài liệu tham khảo
Ahmed M, Haas C, Haas R (2011) Toward low-cost 3D automatic pavement distress surveying: the close range photogrammetry approach. Can J Civ Eng 38(12):1301–1313
Ansar A, Castano A, Matthies L (2004) Enhanced real-time stereo using bilateral filtering. In: Proceedings. 2nd international symposium on 3D data processing, visualization and transmission, 2004. 3DPVT 2004, pp 455–462. https://doi.org/10.1109/TDPVT.2004.1335273
Besl PJ, McKay ND (1992) A method for registration of 3-D shapes. IEEE Trans Pattern Anal Mach Intell 14(2):239–256. https://doi.org/10.1109/34.121791
Boykov Y, Veksler O, Zabih R (2001) Fast approximate energy minimization via graph cuts. IEEE Trans Pattern Anal Mach Intell 23(11):1222–1239
Brunken H, Gühmann C (2019) Incorporating plane-sweep in convolutional neural network stereo imaging for road surface reconstruction. In: Proceedings of the 14th international joint conference on computer vision, imaging and computer graphics theory and applications—volume 5: VISAPP, SciTePress, pp 784–791. https://doi.org/10.5220/0007352107840791
Bulatov D (2015) Temporal selection of images for a fast algorithm for depth-map extraction in multi-baseline configurations. In: Proceedings of the 10th international conference on computer vision theory and applications, SciTePress, Berlin, Germany, pp 395–402. https://doi.org/10.5220/0005239503950402. http://www.scitepress.org/DigitalLibrary/Link.aspx?doi=10.5220/0005239503950402
Bundesanstalt für Straßenwesen (2018) Measuring vehicles and systems. https://www.bast.de/durabast/DE/durabast/Messfahrzeuge/fahrzeuge_node.html. Accessed 10 Nov 2018
Chang JR, Chen YS (2018) Pyramid stereo matching network, p 9. arXiv:180308669
Coenen TBJ, Golroo A (2017) A review on automated pavement distress detection methods. Cogent Eng 4(1):1374822. https://doi.org/10.1080/23311916.2017.1374822
Collins RT (1996) A space-sweep approach to true multi-image matching. In: Proceedings CVPR IEEE computer society conference on computer vision and pattern recognition, IEEE, pp 358–363. http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=517097
Devernay F (1994) Computing differential properties of 3-D shapes from stereoscopic images without 3-D models. In: Proceedings of IEEE conference on computer vision and pattern recognition CVPR-94, IEEE Computer Society Press, Seattle, WA, USA, pp 208–213. https://doi.org/10.1109/CVPR.1994.323831. http://ieeexplore.ieee.org/document/323831/
Eisenbach M, Stricker R, Seichter D, Amende K, Debes K, Sesselmann M, Ebersbach D, Stoeckert U, Gross HM (2017) How to get pavement distress detection ready for deep learning? A systematic approach. In: 2017 international joint conference on neural networks (IJCNN), IEEE, Anchorage, AK, USA, pp 2039–2047. https://doi.org/10.1109/IJCNN.2017.7966101. http://ieeexplore.ieee.org/document/7966101/
El Gendy A, Shalaby A, Saleh M, Flintsch GW (2011) Stereo-vision applications to reconstruct the 3D texture of pavement surface. Int J Pavement Eng 12(3):263–273. https://doi.org/10.1080/10298436.2010.546858
Fan R, Ai X, Dahnoun N (2018) Road surface 3D reconstruction based on dense subpixel disparity map estimation. IEEE Trans Image Process 27(6):3025–3035. https://doi.org/10.1109/TIP.2018.2808770
Fischler MA, Bolles RC (1981) Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun ACM 24(6):381–395
Gallup D, Frahm JM, Mordohai P, Yang Q, Pollefeys M (2007) Real-time plane-sweeping stereo with multiple sweeping directions. In: 2007 IEEE conference on computer vision and pattern recognition, IEEE, pp 1–8. http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=4270270
Gang L, Zucker S (2006) Surface geometric constraints for stereo in belief propagation. In: 2006 IEEE computer society conference on computer vision and pattern recognition—volume 2 (CVPR’06), IEEE, New York, NY, USA, pp 2355–2362. https://doi.org/10.1109/CVPR.2006.299. http://ieeexplore.ieee.org/document/1641042/. Accessed 30 April 2020
GPL software (2017) Cloudcompare 2.8. http://www.cloudcompare.org. Accessed 01 Feb 2018
Hirschmüller H (2008) Stereo processing by semiglobal matching and mutual information. IEEE Trans Pattern Anal Mach Intell 30(2):328–341. https://doi.org/10.1109/TPAMI.2007.1166
Hirschmüller H, Scharstein D (2009) Evaluation of stereo matching costs on images with radiometric differences. IEEE Trans Pattern Anal Mach Intell 31(9):1582–1599. https://doi.org/10.1109/TPAMI.2008.221
Irschara A, Rumpler M, Meixner P, Pock T, Bischof H (2012) Efficient and globally optimal multi view dense matching for aerial images. In: ISPRS annals of photogrammetry, remote sensing and spatial information sciences, vol I-3, pp 227–232. https://doi.org/10.5194/isprsannals-I-3-227-2012. http://www.isprs-ann-photogramm-remote-sens-spatial-inf-sci.net/I-3/227/2012/. Accessed 16 Oct 2018
Ivanchenko V, Shen H, Coughlan J (2009) Elevation-based MRF stereo implemented in real-time on a GPU. In: 2009 workshop on applications of computer vision (WACV), IEEE, pp 1–8.
Kim J, Kolmogorov V, Zabih R (2003) Visual correspondence using energy minimization and mutual information. In: Proceedings ninth IEEE international conference on computer vision, IEEE, vol 2, pp 1033–1040. http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=1238463. Accessed 18 Aug 2016
Li L, Zhang S, Yu X, Zhang L (2018) PMSC: PatchMatch-based superpixel cut for accurate stereo matching. IEEE Trans Circuits Syst Video Technol 28(3):679–692
Olsson C, Ulén J, Boykov Y (2013) In defense of 3D-label stereo. In: 2013 IEEE conference on computer vision and pattern recognition, pp 1730–1737. https://doi.org/10.1109/CVPR.2013.226
Roth L, Mayer H (2019) Reduction of the fronto-parallel bias for wide-baseline semi-global matching. In: ISPRS annals of photogrammetry, remote sensing and spatial information sciences, vol IV-2/W5, pp 69–76. https://doi.org/10.5194/isprs-annals-IV-2-W5-69-2019. https://www.isprs-ann-photogramm-remote-sens-spatial-inf-sci.net/IV-2-W5/69/2019/. Accessed 23 Dec 2019
Scharstein D, Taniai T, Sinha SN (2017) Semi-global stereo matching with surface orientation priors. In: 2017 international conference on 3D vision (3DV), IEEE, Qingdao, pp 215–224. https://doi.org/10.1109/3DV.2017.00033. https://ieeexplore.ieee.org/document/8374574/. Accessed 16 Oct 2018
Sinha SN, Scharstein D, Szeliski R (2014) Efficient high-resolution stereo matching using local plane sweeps. In: 2014 IEEE conference on computer vision and pattern recognition, pp 1582–1589. https://doi.org/10.1109/CVPR.2014.205
Szeliski R (2011) Computer vision. Texts in computer science. Springer, London. http://link.springer.com/10.1007/978-1-84882-935-0
Szeliski R, Zabih R, Scharstein D, Veksler O, Kolmogorov V, Agarwala A, Tappen M, Rother C (2008) A comparative study of energy minimization methods for Markov random fields with smoothness-based priors. IEEE Trans Pattern Anal Mach Intell 30(6):1068–1080. https://doi.org/10.1109/TPAMI.2007.70844
Wang KC, Gong W (2002) Automated pavement distress survey: a review and a new direction. In: Pavement evaluation conference, pp 21–25
Woodford O, Torr P, Reid I, Fitzgibbon A (2009) Global stereo reconstruction under second-order smoothness priors. IEEE Trans Pattern Anal Mach Intell 31(12):2115–2128. https://doi.org/10.1109/TPAMI.2009.131
Yang R, Welch G, Bishop G (2002) Real-time consensus-based scene reconstruction using commodity graphics hardware. In: 10th Pacific conference on computer graphics and applications, 2002. Proceedings., vol 22, pp 225–234
Zabih R, Woodfill J (1994) Non-parametric local transforms for computing visual correspondence. European conference on computer vision. Springer, Berlin, pp 151–158
Zbontar J, LeCun Y (2016) Stereo matching by training a convolutional neural network to compare image patches. J Mach Learn Res 17(1–32):2
Zhang Z, Ai X, Chan CK, Dahnoun N (2014) An efficient algorithm for pothole detection using stereo vision. In: 2014 IEEE international conference on acoustics, speech and signal processing (ICASSP), IEEE, Florence, Italy, pp 564–568. https://doi.org/10.1109/ICASSP.2014.6853659. http://ieeexplore.ieee.org/document/6853659/. Accessed 16 Oct 2018
Zhao W, Yan L, Zhang Y (2018) Geometric-constrained multi-view image matching method based on semi-global optimization. Geo-spat Inf Sci 21(2):115–126. https://doi.org/10.1080/10095020.2018.1441754