Progressive edge-sensing dynamic scene deblurring
Tóm tắt
Deblurring images of dynamic scenes is a challenging task because blurring occurs due to a combination of many factors. In recent years, the use of multi-scale pyramid methods to recover high-resolution sharp images has been extensively studied. We have made improvements to the lack of detail recovery in the cascade structure through a network using progressive integration of data streams. Our new multi-scale structure and edge feature perception design deals with changes in blurring at different spatial scales and enhances the sensitivity of the network to blurred edges. The coarse-to-fine architecture restores the image structure, first performing global adjustments, and then performing local refinement. In this way, not only is global correlation considered, but also residual information is used to significantly improve image restoration and enhance texture details. Experimental results show quantitative and qualitative improvements over existing methods.
Tài liệu tham khảo
Gupta, A.; Joshi, N.; Lawrence Zitnick, C.; Cohen, M.; Curless, B. Single image deblurring using motion density functions. In: Computer Vision — ECCV 2010. Lecture Notes in Computer Science, Vol. 6311. Daniilidis, K.; Maragos, P.; Paragios, N. Eds. Springer Berlin Heidelberg, 171–184, 2010.
Harmeling, S.; Hirsch, M.; Schölkopf, B. Space-variant single-image blind deconvolution for removing camera shake. In: Proceedings of the Advances in Neural Information Processing Systems 23, 829–837, 2010
Hirsch, M.; Schuler, C. J.; Harmeling, S.; Schölkopf, B. Fast removal of non-uniform camera shake. In: Proceedings of the International Conference on Computer Vision, 463–470, 2011.
Pan, J. S.; Sun, D. Q.; Pfister, H.; Yang, M. H. Blind image deblurring using dark channel prior. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1628–1636, 2016.
Xu, L.; Zheng, S. C.; Jia, J. Y. Unnatural L0 sparse representation for natural image deblurring. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1107–1114, 2013.
Chan, T. F.; Wong, C. K. Total variation blind deconvolution. IEEE Transactions on Image Processing Vol. 7, No. 3, 370–375, 1998.
Cho, S.; Lee, S. Fast motion deblurring. ACM Transactions on Graphics Vol. 28, No. 5, 1–8, 2009.
Pan, J. S.; Hu, Z.; Su, Z. X.; Yang, M. H. Deblurring text images via L0-regularized intensity and gradient prior. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2901–2908, 2014.
Couzinié-Devy, F.; Sun, J.; Alahari, K.; Ponce, J. Learning to estimate and remove non-uniform image blur. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1075–1082, 2013.
Kim, T. H.; Ahn, B.; Lee, K. M. Dynamic scene deblurring. In: Proceedings of the IEEE International Conference on Computer Vision, 3160–3167, 2013.
Li, J. J.; Li, G. H.; Fan, H. Image dehazing using residual-based deep CNN. IEEE Access Vol. 6, 26831–26842, 2018.
Li, J. J.; Feng, X. M.; Hua, Z. Low-light image enhancement via progressive-recursive network. IEEE Transactions on Circuits and Systems for Video Technology Vol. 31, No. 11, 4227–4240, 2021.
Zhang, T. L.; Li, J. J.; Hua, Z. Iterative multi-scale residual network for deblurring. IET Image Processing Vol. 15, No. 8, 1583–1595, 2021.
Schuler, C. J.; Burger, H. C.; Harmeling, S.; Schölkopf, B. A machine learning approach for non-blind image deconvolution. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1067–1074, 2013.
Xu, L.; Ren, J. S. J.; Liu, C.; Jia, J. Y. Deep convolutional neural network for image deconvolution. In: Proceedings of the 27th International Conference on Neural Information Processing Systems, Vol. 1, 1790–1798, 2014.
Schuler, C. J.; Hirsch, M.; Harmeling, S.; Schölkopf, B. Learning to deblur. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 38, No. 7, 1439–1451, 2016.
Tai, Y. W.; Chen, X. G.; Kim, S.; Kim, S. J.; Li, F.; Yang, J.; Yu, J.; Matsushita, Y.; Brown, M. S. Nonlinear camera response functions and image deblurring: Theoretical analysis and practice. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 35, No. 10, 2498–2512, 2013.
Deng, J.; Dong, W.; Socher, R.; Li, L. J.; Kai, L.; Li, F. F. ImageNet: A large-scale hierarchical image database. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 248–255, 2009.
Chakrabarti, A. A neural approach to blind motion deblurring. In: Computer Vision — ECCV 2016. Lecture Notes in Computer Science, Vol. 9907. Leibe, B.; Matas, J.; Sebe, N.; Welling, M. Eds. Springer Cham, 221–235, 2016.
Kupyn, O.; Budzan, V.; Mykhailych, M.; Mishkin, D.; Matas, J. DeblurGAN: Blind motion deblurring using conditional adversarial networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 8183–8192, 2018.
Kupyn, O.; Martyniuk, T.; Wu, J. R.; Wang, Z. Y. DeblurGAN-v2: Deblurring (orders-of-magnitude) faster and better. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, 8877–8886, 2019.
Gao, H. Y.; Tao, X.; Shen, X. Y.; Jia, J. Y. Dynamic scene deblurring with parameter selective sharing and nested skip connections. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 3843–3851, 2019.
Nah, S.; Kim, T. H.; Lee, K. M. Deep multi-scale convolutional neural network for dynamic scene deblurring. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 257–265, 2017.
Tao, X.; Gao, H. Y.; Shen, X. Y.; Wang, J.; Jia, J. Y. Scale-recurrent network for deep image deblurring. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 8174–8182, 2018.
Park, D.; Kang, D. U.; Kim, J.; Chun, S. Y. Multitemporal recurrent neural networks for progressive non-uniform single image deblurring with incremental temporal training. In: Computer Vision — ECCV 2020. Lecture Notes in Computer Science, Vol. 12351. Vedaldi, A.; Bischof, H.; Brox, T.; Frahm, J. M. Eds. Springer Cham, 327–343, 2020.
Fergus, R.; Singh, B.; Hertzmann, A.; Roweis, S. T.; Freeman, W. T. Removing camera shake from a single photograph. In: Proceedings of the ACM SIGGRAPH 2006 Papers, 787–794, 2006.
Zhang, H. G.; Dai, Y. C.; Li, H. D.; Koniusz, P. Deep stacked hierarchical multi-patch network for image deblurring. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 5971–5979, 2019.
Mao, X. J.; Shen, C. H.; Yang, Y. B. Image restoration using very deep convolutional encoderdecoder networks with symmetric skip connections. arXiv preprint arXiv:1603.09056, 2016.
Ronneberger, O.; Fischer, P.; Brox, T. U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention — MICCAI 2015. Lecture Notes in Computer Science, Vol. 9351. Navab, N.; Hornegger, J.; Wells, W.; Frangi, A. Eds. Springer Cham, 234–241, 2015.
Ye, M. Y.; Lyu, D.; Chen, G. S. Scale-iterative upscaling network for image deblurring. IEEE Access Vol. 8, 18316–18325, 2020.
Lim, B.; Son, S.; Kim, H.; Nah, S.; Lee, K. M. Enhanced deep residual networks for single image superresolution. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, 1132–1140, 2017.
Mirza, M.; Osindero, S. Conditional generative adversarial nets. arXiv preprint arXiv:1411.1784, 2014.
Wang, Z.; Bovik, A. C.; Sheikh, H. R.; Simoncelli, E. P. Image quality assessment: From error visibility to structural similarity. IEEE Transactions on Image Processing Vol. 13, No. 4, 600–612, 2004.
Levin, A.; Weiss, Y.; Durand, F.; Freeman, W. T. Understanding and evaluating blind deconvolution algorithms. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1964–1971, 2009.
Sun, L. B.; Cho, S.; Wang, J.; Hays, J. Edge-based blur kernel estimation using patch priors. In: Proceedings of the IEEE International Conference on Computational Photography, 1–8, 2013.
Köhler, R.; Hirsch, M.; Mohler, B.; Schölkopf, B.; Harmeling, S. Recording and playback of camera shake: Benchmarking blind deconvolution with a real-world database. In: Computer Vision — ECCV 2012. Lecture Notes in Computer Science, Vol. 7578. Fitzgibbon, A.; Lazebnik, S.; Perona, P.; Sato, Y.; Schmid, C. Eds. Springer Berlin Heidelberg, 27–40, 2012.
Kingma, D. P.; Ba, J. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
Valberg, A. Light Vision Color. John Wiley and Sons, 2005.
Bai, Y. C.; Cheung, G.; Liu, X. M.; Gao, W. Graph-based blind image deblurring from a single photograph. IEEE Transactions on Image Processing Vol. 28, No. 3, 1404–1418, 2019.
Liu, J.; Yan, M.; Zeng, T. Surface-aware blind image deblurring. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 43, No. 3, 1041–1055, 2021.
Whyte, O.; Sivic, J.; Zisserman, A.; Ponce, J. Nonuniform deblurring for Shaken images. International Journal of Computer Vision Vol. 98, No. 2, 168–186, 2012.