A survey on facial image deblurring

Lu, Y. Z. Out-of-focus blur: Image de-blurring. arXiv preprint arXiv:1710.00620, 2017. Chakrabarti, A. A neural approach to blind motion deblurring. In: Computer Vision–ECCV 2016. Lecture Notes in Computer Science, Vol. 9907. Leibe, B.; Matas, J.; Sebe, N.; Welling, M. Eds. Springer Cham, 221–235, 2016. Boracchi, G.; Foi, A. Modeling the performance of image restoration from motion blur. IEEE Transactions on Image Processing Vol. 21, No. 8, 3502–3517, 2012. Song, Y. B.; Zhang, J. W.; Gong, L. J.; He, S. F.; Bao, L. C.; Pan, J. S.; Yang, Q. X.; Yang, M. H. Joint face hallucination and deblurring via structure generation and detail enhancement. International Journal of Computer Vision Vol. 127, Nos. 6–7, 785–800, 2019. Wang, X. T.; Li, Y.; Zhang, H. L.; Shan, Y. Towards real-world blind face restoration with generative facial prior. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 9164–9174, 2021. Li, C. M. A survey on image deblurring. arXiv preprint arXiv:2202.07456, 2022. Zhang, K. H.; Ren, W. Q.; Luo, W. H.; Lai, W. S.; Stenger, B.; Yang, M. H.; Li, H. D. Deep image deblurring: A survey. International Journal of Computer Vision Vol. 130, No. 9, 2103–2130, 2022. Nishiyama, M.; Hadid, A.; Takeshima, H.; Shotton, J.; Kozakaya, T.; Yamaguchi, O. Facial deblur inference using subspace analysis for recognition of blurred faces. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 33, No. 4, 838–845, 2011. Zhang, H. C.; Yang, J. C.; Zhang, Y. N.; Nasrabadi, N. M.; Huang, T. S. Close the loop: Joint blind image restoration and recognition with sparse representation prior. In: Proceedings of the International Conference on Computer Vision, 770–777, 2011. Tian, L.; Fan, C. X.; Ming, Y.; Hong, X. P. Weighted non-locally self-similarity sparse representation for face deblurring. In: Computer Vision–ACCV 2016 Workshops. Lecture Notes in Computer Science, Vol. 10116. Chen, C. S.; Lu, J.; Ma, K. K. Eds. Springer Cham, 576–589, 2017. Jiang, J. L.; Zhang, L.; Yang, J. Mixed noise removal by weighted encoding with sparse nonlocal regularization. IEEE Transactions on Image Processing Vol. 23, No. 6, 2651–2662, 2014. Anwar, S.; Huynh, C. P.; Porikli, F. Image deblurring with a class-specific prior. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 41, No. 9, 2112–2130, 2019. Zhang, M. N.; Fang, Y. Y.; Ni, G. X.; Zeng, T. Y. Pixel screening based intermediate correction for blind deblurring. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 1–9, 2022. Tian, D. Y.; Tao, D. C. Coupled learning for facial deblur. IEEE Transactions on Image Processing Vol. 25, No. 2, 961–972, 2016. Mittal, A.; Moorthy, A. K.; Bovik, A. C. Making image quality assessment robust. In: Proceedings of the 46th Asilomar Conference on Signals, Systems and Computers, 1718–1722, 2012. Pan, J. S.; Hu, Z.; Su, Z. X.; Yang, M. H. Deblurring face images with exemplars. In: Computer Vision–ECCV 2014. Lecture Notes in Computer Science, Vol. 8695. Fleet, D.; Pajdla, T.; Schiele, B.; Tuytelaars, T. Eds. Springer Cham, 47–62, 2014. Hacohen, Y.; Shechtman, E.; Lischinski, D. Deblurring by example using dense correspondence. In: Proceedings of the IEEE International Conference on Computer Vision, 2384–2391, 2013. Szegedy, C.; Liu, W.; Jia, Y. Q.; Sermanet, P.; Reed, S.; Anguelov, D.; Erhan, D.; Vanhoucke, V.; Rabinovich, A. Going deeper with convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1–9, 2015. Ronneberger, O.; Fischer, P.; Brox, T. U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015. Lecture Notes in Computer Science, Vol. 9351. Navab, N.; Hornegger, J.; Wells, W.; Frangi, A. Eds. Springer Cham, 234–241, 2015. Nah, S.; Kim, T. H.; Lee, K. M. Deep multi-scale convolutional neural network for dynamic scene deblurring. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 257–265, 2017. Goodfellow, I. J.; Pouget-Abadie, J.; Mirza, M.; Xu, B.; Warde-Farley, D.; Ozair, S.; Courville, A.; Bengio, Y. Generative adversarial nets. In: Proceedings of the 27th International Conference on Neural Information Processing Systems, Vol. 2, 2672–2680, 2014. Schuler, C. J.; Hirsch, M.; Harmeling, S.; Schölkopf, B. Learning to deblur. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 38, No. 7, 1439–1451, 2015. Jin, M. G.; Hirsch, M.; Favaro, P. Learning face deblurring fast and wide. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, 858–866, 2018. He, K. M.; Zhang, X. Y.; Ren, S. Q.; Sun, J. Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 770–778, 2016. Lin, S. N.; Zhang, J. W.; Pan, J. S.; Liu, Y. C.; Wang, Y. T.; Chen, J.; Ren, J. Learning to deblur face images via sketch synthesis. Proceedings of the AAAI Conference on Artificial Intelligence Vol. 34, No. 7, 11523–11530, 2020. Xu, L.; Ren, J. S. J.; Liu, C.; Jia, J. Y. Deep convolutional neural network for image deconvolution. In: Proceedings of the 27th International Conference on Neural Information Processing Systems, Vol. 1, 1790–1798, 2014. Chrysos, G. G.; Favaro, P.; Zafeiriou, S. Motion deblurring of faces. International Journal of Computer Vision Vol. 127, Nos. 6–7, 801–823, 2019. Chrysos, G. G.; Zafeiriou, S. Deep face deblurring. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2015–2024, 2017. Wang, L. X.; Li, Y. L.; Wang, S. J. DeepDeblur: Fast one-step blurry face images restoration. arXiv preprint arXiv:1711.09515, 2017. Qi, Q.; Guo, J. C.; Li, C. Y.; Xiao, L. J. Blind face images deblurring with enhancement. Multimedia Tools and Applications Vol. 80, No. 2, 2975–2995, 2021. Shen, Z. Y.; Lai, W. S.; Xu, T. F.; Kautz, J.; Yang, M. H. Deep semantic face deblurring. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 8260–8269, 2018. Yasarla, R.; Perazzi, F.; Patel, V. M. Deblurring face images using uncertainty guided multi-stream semantic networks. IEEE Transactions on Image Processing Vol. 29, 6251–6263, 2020. Shen, Z. Y.; Lai, W. S.; Xu, T. F.; Kautz, J.; Yang, M. H. Exploiting semantics for face image deblurring. International Journal of Computer Vision Vol. 128, No. 7, 1829–1846, 2020. Lee, T. B.; Jung, S. H.; Heo, Y. S. Progressive semantic face deblurring. IEEE Access Vol. 8, 223548–223561, 2020. Ren, W. Q.; Yang, J. L.; Deng, S. Y.; Wipf, D.; Cao, X. C.; Tong, X. Face video deblurring using 3D facial priors. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, 9387–9396, 2019. Zhu, F. D.; Zhu, J. W.; Chu, W. Q.; Zhang, X. Y.; Ji, X. Z.; Wang, C. J.; Tai, Y. Blind face restoration via integrating face shape and generative priors. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 7652–7661, 2022. Jung, S. H.; Lee, T. B.; Heo, Y. S. Deep feature prior guided face deblurring. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 884–893, 2022. Zhang, X.; Zhang, H.; Lv, J. C.; Li, X. J. Face deblurring based on separable normalization and adaptive denormalization. arXiv preprint arXiv:2112.09833, 2021. Lin, T. Y.; Goyal, P.; Girshick, R.; He, K. M.; Dollár, P. Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, 2999–3007, 2017. Hu, X. B.; Ren, W. Q.; Yang, J. L.; Cao, X. C.; Wipf, D.; Menze, B.; Tong, X.; Zha, H. B. Face restoration via plug-and-play 3D facial priors. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 44, No. 12, 8910–8926, 2022. Paysan, P.; Knothe, R.; Amberg, B.; Romdhani, S.; Vetter, T. A 3D face model for pose and illumination invariant face recognition. In: Proceedings of the 6th IEEE International Conference on Advanced Video and Signal Based Surveillance, 296–301, 2009. Xu, X. Y.; Sun, D. Q.; Pan, J. S.; Zhang, Y. J.; Pfister, H.; Yang, M. H. Learning to super-resolve blurry face and text images. In: Proceedings of the IEEE International Conference on Computer Vision, 251–260, 2017. Karras, T.; Laine, S.; Aittala, M.; Hellsten, J.; Lehtinen, J.; Aila, T. M. Analyzing and improving the image quality of StyleGAN. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 8107–8116, 2020. Madam, N. T.; Kumar, S.; Rajagopalan, A. N. Unsupervised class-specific deblurring. In: Computer Vision - ECCV 2018. Lecture Notes in Computer Science, Vol. 11214. Ferrari, V.; Hebert, M.; Sminchisescu, C.; Weiss, Y. Eds. Springer Cham, 358–374, 2018. Lu, B. Y.; Chen, J. C.; Chellappa, R. Unsupervised domain-specific deblurring via disentangled representations. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 10217–10226, 2019. Zhu, J. Y.; Park, T.; Isola, P.; Efros, A. A. Unpaired image-to-image translation using cycle-consistent adversarial networks. In: Proceedings of the IEEE International Conference on Computer Vision, 2242–2251, 2017. Xia, Z. H.; Chakrabarti, A. Training image estimators without image ground-truth. In: Proceedings of the 33rd International Conference on Neural Information Processing Systems, Article No. 219, 2439–2449, 2019. Huber, P. J. Robust regression: Asymptotics, conjectures and Monte Carlo. The Annals of Statistics Vol. 1, No. 5, 799–821, 1973. Lai, W. S.; Huang, J. B.; Hu, Z.; Ahuja, N.; Yang, M. H. A comparative study for single image blind deblurring. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1701–1709, 2016. Le, V.; Brandt, J.; Lin, Z.; Bourdev, L.; Huang, T. S. Interactive facial feature localization. In: Computer Vision–ECCV 2012. Lecture Notes in Computer Science, Vol. 7574. Fitzgibbon, A.; Lazebnik, S.; Perona, P.; Sato, Y.; Schmid, C. Eds. Springer Berlin Heidelberg, 679–692, 2012. Sim, T.; Baker, S.; Bsat, M. The CMU pose, illumination, and expression database. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 25, No. 12, 1615–1618, 2003. Liu, Z. W.; Luo, P.; Wang, X. G.; Tang, X. O. Deep learning face attributes in the wild. In: Proceedings of the IEEE International Conference on Computer Vision, 3730–3738, 2015. Lee, C. H.; Liu, Z. W.; Wu, L. Y.; Luo, P. MaskGAN: Towards diverse and interactive facial image manipulation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 5548–5557, 2020. Karras, T.; Laine, S.; Aila, T. M. A style-based generator architecture for generative adversarial networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 4396–4405, 2019. Ng, H. W.; Winkler, S. A data-driven approach to cleaning large face datasets. In: Proceedings of the IEEE International Conference on Image Processing, 343–347, 2014. Kumar, N.; Berg, A. C.; Belhumeur, P. N.; Nayar, S. K. Attribute and simile classifiers for face verification. In: Proceedings of the IEEE 12th International Conference on Computer Vision, 365–372, 2009. Gross, R.; Matthews, I.; Cohn, J.; Kanade, T.; Baker, S. Multi-PIE. Image and Vision Computing Vol. 28, No. 5, 807–813, 2010. Deng, J. K.; Guo, J.; Xue, N. N.; Zafeiriou, S. ArcFace: Additive angular margin loss for deep face recognition. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 4685–4694, 2019. Wang, Z.; Bovik, A. C.; Sheikh, H. R.; Simoncelli, E. P. Image quality assessment: From error visibility to structural similarity. IEEE Transactions on Image Processing Vol. 13, No. 4, 600–612, 2004. Zhang, R.; Isola, P.; Efros, A. A.; Shechtman, E.; Wang, O. The unreasonable effectiveness of deep features as a perceptual metric. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 586–595, 2018. Mittal, A.; Soundararajan, R.; Bovik, A. C. Making a “completely blind” image quality analyzer. IEEE Signal Processing Letters Vol. 20, No. 3, 209–212, 2013. Ignatov, A.; Timofte, R.; Van Vu, T.; Luu, T. M.; Pham, T. X.; Van Nguyen, C.; Kim, Y.; Choi, J. S.; Kim, M.; Huang, J.; et al. PIRM challenge on perceptual image enhancement on smartphones: Report. In: Computer Vision–ECCV 2018 Workshops. Lecture Notes in Computer Science, Vol. 11133. Leal-Taixé, L.; Roth, S. Eds. Springer Cham, 315–333, 2019. Mittal, A.; Moorthy, A. K.; Bovik, A. C. No-reference image quality assessment in the spatial domain. IEEE Transactions on Image Processing Vol. 21, No. 12, 4695–4708, 2012. Ma, C.; Yang, C. Y.; Yang, X. K.; Yang, M. H. Learning a no-reference quality metric for single-image super-resolution. Computer Vision and Image Understanding Vol. 158, 1–16, 2017. Venkatanath, N.; Praneeth, D.; Bh, M. C.; Channappayya, S. S.; Medasani, S. S. Blind image quality evaluation using perception based features. In: Proceedings of the 21st National Conference on Communications, 1–6, 2015. Heusel, M.; Ramsauer, H.; Unterthiner, T.; Nessler, B.; Hochreiter, S. GANs trained by a two time-scale update rule converge to a local Nash equilibrium. In: Proceedings of the 31st International Conference on Neural Information Processing Systems, 6629–6640, 2017. Parkhi, O. M.; Vedaldi, A.; Zisserman, A. Deep face recognition. In: Proceedings of the British Machine Vision Conference, 2015. Amos, B.; Ludwiczuk, B.; Satyanarayanan, M. OpenFace: A general-purpose face recognition library with mobile applications. School of Computer Science, Carnegie Mellon University, 2016. Howard, A. G.; Zhu, M. L.; Chen, B.; Kalenichenko, D.; Wang, W. J.; Weyand, T.; Andreetto, M.; Adam, H. MobileNets: Efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861, 2017. Huang, G. B.; Mattar, M. A.; Berg, T. L.; Learned-Miller, E. Labeled faces in the wild: A database for studying face recognition in unconstrained environments. In: Proceedings of the Workshop on Faces in “Real-Life” Images: Detection, Alignment, and Recognition, 2008. Shan, Q.; Jia, J. Y.; Agarwala, A. High-quality motion deblurring from a single image. ACM Transactions on Graphics Vol. 27, No. 3, 1–10, 2008. Krishnan, D.; Tay, T.; Fergus, R. Blind deconvolution using a normalized sparsity measure. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 233–240, 2011. Cho, S.; Lee, S. Fast motion deblurring. ACM Transactions on Graphics Vol. 28, No. 5, 1–8, 2009. Xu, L.; Jia, J. Y. Two-phase kernel estimation for robust motion deblurring. In: Computer Vision–ECCV 2010. Lecture Notes in Computer Science, Vol. 6311. Daniilidis, K.; Maragos, P.; Paragios, N. Eds. Springer Berlin Heidelberg, 157–170, 2010. Zhang, J. W.; Pan, J. S.; Ren, J.; Song, Y. B.; Bao, L. C.; Lau, R. W. H.; Yang, M. H. Dynamic scene deblurring using spatially variant recurrent neural networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2521–2529, 2018. Gao, H. Y.; Tao, X.; Shen, X. Y.; Jia, J. Y. Dynamic scene deblurring with parameter selective sharing and nested skip connections. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 3843–3851, 2019. Yuan, Y.; Su, W.; Ma, D. D. Efficient dynamic scene deblurring using spatially variant deconvolution network with optical flow guided training. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 3552–3561, 2020. Zamir, S. W.; Arora, A.; Khan, S.; Hayat, M.; Khan, F. S.; Yang, M. H. Restormer: efficient transformer for high-resolution image restoration. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 5718–5729, 2022. Chen, H. T.; Wang, Y. H.; Guo, T. Y.; Xu, C.; Deng, Y. P.; Liu, Z. H.; Ma, S. W.; Xu, C. J.; Xu, C.; Gao, W. Pre-trained image processing transformer. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 12294–12305, 2021. Lai, W. S.; Shih, Y.; Chu, L. C.; Wu, X. T.; Tsai, S. F.; Krainin, M.; Sun, D. Q.; Liang, C. K. Face deblurring using dual camera fusion on mobile phones. ACM Transactions on Graphics Vol. 41, No. 4, Article No. 148, 2022.