Research and implementation of adaptive stereo matching algorithm based on ZYNQ
Tóm tắt
Stereo matching is an important method in computer vision for simulating human binocular vision to acquire spatial distance information. Implementing high-precision and real-time stereo-matching algorithms on hardware platforms with limited resources remains a significant challenge. Although the semi-global stereo-matching algorithm strikes a good balance between obtaining accuracy in the disparity map and computational complexity, it uses a fixed window for matching, resulting in lower matching accuracy in image regions with depth discontinuities and weak textures. To address the shortcomings of existing semi-global stereo-matching algorithms, an adaptive window semi-global stereo-matching algorithm is proposed, along with post-processing disparity optimization through left–right consistency check and median filtering. On test images provided by the Middlebury dataset, the average matching accuracy improved by 5.07% compared to traditional-matching algorithms. This algorithm is implemented on a Zynq UltraScale + chip, utilising 42,072 LUTs, 66,532 registers, and 101 BRAMs for the entire stereo-matching architecture. For images with a resolution of 1280 × 720 and 64 disparity levels, the final-processing speed can reach 54.24 fps.
Từ khóa
Tài liệu tham khảo
Perri, S., et al.: Stereo vision architecture for heterogeneous systems-on-chip. J. Real-Time Image Proc. 17(2), 393–415 (2018)
Lu, K., et al. Binocular stereo vision based on OpenCV; proceedings of the Smart and Sustainable City (ICSSC 2011), IET International Conference on, F, 2011 [C].
Brown, M.Z., Burschka, D., Hager, G.D.: Advances in computational stereo. IEEE Trans. Pattern Anal. Mach. Intell. 25(8), 993–1008 (2003)
Kriegman, D.J., Triendl, E., Binford, T.O.: Stereo vision and navigation in buildings for mobile robots. IEEE Trans. Robot. Autom. 5(6), 792–803 (1989)
Dong, P., et al.: A 4.29 nJ/pixel stereo depth coprocessor with pixel level pipeline and region optimized semi-global matching for IoT application. IEEE Trans. Circuits Syst. I Regul. Pap. 69(1), 334–346 (2022)
Bertozzi, M., et al.: Stereo vision-based vehicle detection. In: Proceedings of the Proceedings of the IEEE Intelligent Vehicles Symposium 2000 (Cat No 00TH8511). IEEE (2000)
Mckinnon, D., Smith, R.N., Upcroft, B.: A semi-local method for iterative depth-map refinement. In: Proceedings of the 2012 IEEE International Conference on Robotics and Automation. IEEE (2012)
De Silva, D., et al.: Improved depth map filtering for 3D-TV systems. In: Proceedings of the 2011 IEEE International Conference on Consumer Electronics (ICCE). IEEE (2011)
Arief Setyawan, R., et al.: Implementation of stereo vision semi-global block matching methods for distance measurement. Indones. J. Electr. Eng. Comput. Sci. 12(2), 585–591 (2018)
Luo, G., Zhu, Y.: Hole filling for view synthesis using depth guided global optimization. IEEE Access 6, 32874–32889 (2018)
Diebel, J., Thrun, S.: An application of markov random fields to range sensing. In: Advances in Neural Information Processing Systems, vol 18 (2005)
Hirschmuller, H.: Stereo processing by semiglobal matching and mutual information. IEEE Trans. Pattern Anal. Mach. Intell. 30(2), 328–341 (2007)
Zabih, R., Woodfill, J.: Non-parametric local transforms for computing visual correspondence. In: Proceedings of the Computer Vision—ECCV'94: Third European Conference on Computer Vision Stockholm, Sweden, May 2–6 1994 Proceedings, Volume II 3. Springer (1994)
Li, J.: On fractional differential operators for image edge detection. Comput. Appl. Softw. 32(12), 5 (2015)
Shan, Y., et al.: Hardware acceleration for an accurate stereo vision system using mini-census adaptive support region. ACM Trans. Embed. Comput. Syst. 13(4s), 1–24 (2014)
Ttofis, C., Kyrkou, C., Theocharides, T.: A low-cost real-time embedded stereo vision system for accurate disparity estimation based on guided image filtering. IEEE Trans. Comput. 65(9), 2678–2693 (2016)
Gehrig, S.K., Eberli, F., Meyer, T. A real-time low-power stereo vision engine using semi-global matching. In: Proceedings of the International Conference on Computer Vision Systems. Springer (2009)
Wang, J., et al.: Low-resource hardware architecture for semi-global stereo matching. In: Proceedings of the 2019 IEEE International Symposium on Circuits and Systems (ISCAS). IEEE (2019)
Li, Y., et al.: High throughput hardware architecture for accurate semi-global matching. Integration 65, 417–427 (2019)
Chai, Y., Cao, X.: Stereo matching algorithm based on joint matching cost and adaptive window. In: Proceedings of the 2018 IEEE 3rd Advanced Information Technology, Electronic and Automation Control Conference (IAEAC). IEEE (2018)
Razak, S., Othman, M.A., Kadmin, A.F.: The effect of adaptive weighted bilateral filter on stereo matching algorithm. IJEAT 8(3), 2249–8958 (2019)
Ke, Z., Jiangbo, L., Lafruit, G.: Cross-based local stereo matching using orthogonal integral images. IEEE Trans. Circuits Syst. Video Technol. 19(7), 1073–1079 (2009)
Li, J., et al.: Bayesian stereo matching method based on edge constraints. Int. J. Adv. Comput .Technol. 4(22), 36–47 (2012)
Kowalczuk, J., Psota, E.T., Perez, L.C.: Real-time stereo matching on CUDA using an iterative refinement method for adaptive support-weight correspondences. IEEE Trans. Circuits Syst. Video Technol. 23(1), 94–104 (2013)
Wang, Y., et al.: Improvement of AD-census algorithm based on stereo vision. Sensors (Basel) 22(18), 6933 (2022)
Peng, Z., Wu, L., Xiao, B.: High-speed dense matching algorithm for high-resolution aerial image based on CPU-FPGA. Vis. Comput. 39, 5263–5278 (2023)
Wang, W., et al.: Real-time high-quality stereo vision system in FPGA. IEEE Trans. Circuits Syst. Video Technol. 25(10), 1696–1708 (2015)
Ma, Y., et al.: Five-direction occlusion filling with five layer parallel two-stage pipeline for stereo matching with sub-pixel disparity map estimation. Sensors (Basel) 22(22), 8605 (2022)
Lu, Z., et al.: A resource-efficient pipelined architecture for real-time semi-global stereo matching. IEEE Trans. Circuits Syst. Video Technol. 32(2), 660–673 (2022)
Sabihuddin, S., Islam, J., Maclean, W.J.: Dynamic programming approach to high frame-rate stereo correspondence: a pipelined architecture implemented on a field programmable gate array. In: Proceedings of the 2008 Canadian Conference on Electrical and Computer Engineering. IEEE (2008)
Jin, M., Maruyama, T.: A real-time stereo vision system using a tree-structured dynamic programming on FPGA. In: Proceedings of the ACM/SIGDA International Symposium on Field Programmable Gate Arrays, 2012
Banz, C., et al.: Real-time stereo vision system using semi-global matching disparity estimation: architecture and FPGA-implementation. In: Proceedings of the 2010 International Conference on Embedded Computer Systems: Architectures, Modeling and Simulation. IEEE (2010)
Zhao, G., Zhang, L., Wu, F.: Application of improved median filtering algorithm to image de-noising. J. Appl. Opt. 32(4), 5 (2011)
Han, J., Wu, Z., Li, L.: FPGA implementation for binocular stereo matching algorithm. J. Harbin Univ. Sci. Technol. 21(004), 25–29 (2016)
Jia, Y., et al.: A miniature stereo vision machine (MSVM-III) for dense disparity mapping. In: Proceedings of the 17th International Conference on Pattern Recognition, 2004 ICPR 2004. IEEE (2004)
Perri, S., et al.: Design of real-time FPGA-based embedded system for stereo vision. In: Proceedings of the 2018 IEEE international symposium on circuits and systems (ISCAS). IEEE (2018)
Pan, Y., et al.: A hardware/software co-design approach for real-time binocular stereo vision based on ZYNQ (short paper). In: Proceedings of the Collaborative Computing: Networking, Applications and Worksharing: 14th EAI International Conference, CollaborateCom 2018, Shanghai, China, December 1–3, 2018, Proceedings 14. Springer (2019)
Wu, G., Yang, J., Yang, H.: Real-time low-power binocular stereo vision based on FPGA. J. Real-Time Image. Proc. 19, 29–39 (2022)