Depth estimation using modified cost function for occlusion handling
Tóm tắt
This paper presents a novel approach to the occlusion handling problem in depth estimation using three views. A solution based on modification of similarity cost function is proposed. During the depth estimation via optimization algorithms, like Graph Cuts, the similarity metric is constantly updated so that only non-occluded fragments in the side views are considered. At each iteration of the algorithm, non-occluded fragments are detected based on side view virtual depth maps synthesized from currently the best estimated depth map of the center view. Then, the similarity metric is updated for correspondence search only in non-occluded regions of the side views. The experimental results, conducted on well-known 3D video test sequences, show that the depth maps estimated with the proposed approach provide about 1.25 dB virtual view quality improvement in comparison with the virtual view synthesized based on depth maps generated with the use of the state-of-the-art MPEG Depth Estimation Reference Software.
Tài liệu tham khảo
Muller, K., Merkle, P., Wiegand, T.: 3-D video representation using depth maps. Proc. IEEE 99(4), 643–656 (2011)
Zhang, L., Tam, W.J.: Stereoscopic image generation based on depth images for 3DTV. IEEE Trans. Broadcast. 51(2), 191–199 (2005)
Annex, I.: Multiview and depth video coding of ISO/IEC 14496-10. International Standard Generic coding of audio-visual objects—Part 10: Advanced Video Coding, 8th ed., 2013, also: ITU-T Rec. H.264, Edition 8.0 (2013)
3D-AVC Draft Text 9, JCT-3V of ITU T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, Doc. JCT3V-G1003, San Jose, USA (2014)
Tech, G., Wegner, K., Chen, Y., Yea, S.: 3D-HEVC draft text 6 joint collaborative team on 3D video coding extension development of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 Doc. JCT3V-J1001, 10th Meeting: Strasbourg, FR, 1824 (2014)
Kim, S.Y., Cho, J.H., Koschan, A.: 3D video generation and service based on a TOF depth sensor in MPEG-4 multimedia framework. IEEE Trans. Consum. Electron. 56(3), 1730–1738 (2010)
Domański, M., Dziembowski, A., Kuehn, A., Kurc, M., Łuczak, A., Mieloch, D., Siast, J., Stankiewicz, O., Wegner, K.: Experiments on acquisition and processing of video for free-viewpoint television. In: 3DTV Conference 2014, Budapest, Hungary (2014)
Hartley, R., Zisserman, A.: Multiple View Geometry in Computer Vision, 2nd edn, pp. 262–278. Cambridge University Press, Cambridge (2003)
Scharstein, D., Szeliski, R.: A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. Int. J. Comput. Vis. 47(1/2/3), 7–42 (2002)
Middlebury Stereo Evaluation—Version 2. Webpage visited 2015-01-24. http://vision.middlebury.edu/stereo/eval
Okutomi, M., Kanade, T.: A multiple baseline stereo. IEEE Trans. PAMI 15(4), 353–363 (1993)
Collins, R.T.: A space-sweep approach to true multi-image matching. In: CVPR96, San Francisco, pp. 358–363 (1996)
Seitz, S.M., Curless, B., Diebel, J., Scharstein, D., Szeliski, R.: A comparison and evaluation of multi-view stereo reconstruction algorithms. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 06), pp. 519–526 (2006)
Stankowski, J., Klimaszewski, K., Stankiewicz, O., Wegner, K., Domański, M.: Preprocessing methods used for Poznan 3D/FTV test sequences. ISO/IEC JTC1/SC29/WG11 MPEG 2010/M17174, Doc. m17174, Kyoto, Japan (2010)
Stankowski, J., Klimaszewski, K.: Application of epipolar rectification algorithm in 3D Television. In: Image Processing and Communications Challenges 2. Advances in Intelligent and Soft Computing, vol. 84, pp. 345-352. Springer, Berlin (2010). ISBN: 978-3-642-16294-7
Tanimoto, M., Fujii, T., Suzuki, K., Fukushima, N., Mori, Y.: Reference softwares for depth estimation and view synthesis. ISO/IEC JTC1/SC29/WG11, Doc. M15377, Archamps, France (2008)
Wildeboer, M., Fukushima, N., Yendo, T., Panahpour, M.T., Fujii, T., Tanimoto, M.: A semi-automatic multi-view depth estimation method. In: Proceedings of the SPIE, vol. 7744 (2010)
Lee, S.-B., Ho, Y.-S.: Multi-view depth map estimation enhancing temporal consistency. In: 23rd International Technical Conference on Circuits/Systems, Computers and Communications
Stankiewicz, O.: Stereoscopic depth map estimation and coding techniques for multiview video systems. Ph.D. Dissertation at Poznan University of Technology, Faculty of Electronics and Telecommunications (2014)
Wegner, K., Stankiewicz, O.: Similiarity measures for depth estimation. In: 3DTV-Conference 2009, Potsdam, Germany (2009)
Birchfield, S., Tomasi, C.: A pixel dissimilarity measure that is insensitive to image sampling. IEEE Trans. Pattern Anal. Mach. Intell. 20(4), 401–406 (1998)
Sun, J., Zheng, N.N., Shum, H.Y.: Stereo matching using belief propagation. IEEE Trans. Pattern Anal. Mach. Intell. 25(7), 787–800 (2003)
Felzenszwalb, P., Huttenlocher, D.: Efficient belief propagation for early vision. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 261–268 (2004)
Veksler, O.: Stereo correspondence by dynamic programming on a tree. In: CVPR (2005)
Boykov, Y., Veksler, O., Zabih, R.: Fast approximate energy minimization via graph cuts. IEEE Trans. Pattern Anal. Mach. Intell. 23(11), 1222–1239 (2001)
Egnal, G., Wildes, R.: Detecting binocular halfocclusions: empirical comparisons of five approaches. IEEE Trans. Pattern Anal. Mach. Intell. 24(8), 1127–1133 (2002)
Bobick, A., Intille, S.: Large occlusion stereo. Int. J. Comput. Vis. 33(3), 181–200 (1999)
Marr, D., Poggio, T.A.: Cooperative computation of stereo disparity. Science 194(4262), 283–287 (1976)
Kolmogorov, V., Zabih, R.: Computing visual correspondence with occlusions using graph cuts. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 508–515 (2001)
Jang, W.-S., Ho, Y.-S.: Efficient depth map generation with occlusion handling for various camera arrays. Signal Image Video Process. 8(2), 287–297 (2014)
Ben-Ari, R., Sochen, N.: Stereo matching with Mumford–Shah regularization and occlusion handling. IEEE Trans. Pattern Anal. Mach. Intell. 32(11), 2071–2084 (2010)
Wildeboer, M., Stankiewicz, O., Wegner, K.: A soft-segmentation matching in depth estimation reference software (DERS) 5.0. ISO/IEC JTC1/SC29/WG11 Doc. M17049, Xian, China (2009)
Domański, M., Stankiewicz, O., Wegner, K., et al.: Pozna multi-view video test sequences and camera parameters. ISO/IEC JTC1/SC29/WG11 Doc. M17050, Xian, China, October (2009)
Feldmann, I., Smolic, A., et al.: HHI test material for 3D video. ISO/IEC JTC1/SC29/WG11, Doc. M15413, Archamps, France (2008)
Jinmi, K., Kidong, C.: High-performance depth map coding for 3D-AVC Signal. Image Video Process. 10(6), 1017–1024 (2016)
Scharstein, D., Szeliski, R.: High-accuracy stereo depth maps using structured light. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2003), vol. 1, pp. 195–202, Madison, WI (2003)