Structural residual learning for single image rain removal

Knowledge-Based Systems - Tập 213 - Trang 106595 - 2021
Hong Wang1, Yichen Wu1, Qi Xie1, Qian Zhao1, Yong Liang2, Zhang Shi-jun3, Deyu Meng2,1
1Xi’an Jiaotong University, Shaanxi, 710049, PR China
2Macau University of Science and Technology, Macau, PR China
3China Mobile Research Institute, Beijing, PR China

Tóm tắt

Từ khóa


Tài liệu tham khảo

M. Li, Q. Xie, Q. Zhao, W. Wei, S. Gu, J. Tao, D. Meng, Video rain streak removal by multiscale convolutional sparse coding, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 6644–6653.

Shehata, 2008, Video-based automatic incident detection for smart roads: The outdoor environmental challenges regarding false alarms, IEEE Trans. Intell. Transp. Syst., 9, 349, 10.1109/TITS.2008.915644

Bahnsen, 2018, Rain removal in traffic surveillance: Does it matter?, IEEE Trans. Intell. Transp. Syst., 20, 2802, 10.1109/TITS.2018.2872502

C.E. Smith, C. Richards, S. Brandt, N. Papanikolopoulos, Visual tracking for intelligent vehicle-highway systems, IEEE Trans. Veh. Technol. 45 (4) 744–759.

Wang, 2019

S. Li, I.B. Araujo, W. Ren, Z. Wang, E.K. Tokuda, R.H. Junior, R. Cesar-Junior, J. Zhang, X. Guo, X. Cao, Single image deraining: A comprehensive benchmark analysis, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2019, pp. 3838–3847.

Yasarla, 2020, Confidence measure guided single image de-raining, IEEE Trans. Image Process., 29, 4544, 10.1109/TIP.2020.2973802

J. Pan, S. Liu, D. Sun, . Zhang, et al. Learning dual convolutional neural networks for low-level vision, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 3070–3079.

Yang, 2020, Single image deraining: From model-based to data-driven and beyond, IEEE Trans. Pattern Anal. Mach. Intell.

Jing, 2012, Removing rain and snow in a single image using guided filter, 304

He, 2010, Guided image filtering, IEEE Trans. Pattern Anal. Mach. Intell., 35, 1397, 10.1109/TPAMI.2012.213

Ding, 2016, Single image rain and snow removal via guided L0 smoothing filter, Multimedia Tools Appl., 75, 2697, 10.1007/s11042-015-2657-7

Y. Li, Rain streak removal using layer priors, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 2736–2744.

L. Yu, X. Yong, J. Hui, Removing rain from a single image via discriminative sparse coding, in: Proceedings of the IEEE International Conference on Computer Vision, 2015, pp. 3397–3405.

S. Gu, D. Meng, W. Zuo, Z. Lei, Joint convolutional analysis and synthesis sparse representation for single image layer separation, in: Proceedings of the IEEE International Conference on Computer Vision, 2017, pp. 1708–1716.

Li, 2018, Non-locally enhanced encoder-decoder network for single image de-raining

S. Li, I.B. Araujo, W. Ren, Z. Wang, E.K. Tokuda, R.H. Junior, R. Cesar-Junior, J. Zhang, X. Guo, X. Cao, Single image deraining: A comprehensive benchmark analysis, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2019, pp. 3838–3847.

Liu, 2019, Knowledge-driven deep unrolling for robust image layer separation, IEEE Trans. Neural Netw. Learn. Syst., 31, 1653, 10.1109/TNNLS.2019.2921597

Wang, 2020, Single image rain streaks removal: a review and an exploration, Int. J. Mach. Learn. Cybern., 11, 853, 10.1007/s13042-020-01061-2

R. Li, L.-F. Cheong, R.T. Tan, Heavy rain image restoration: Integrating physics model and conditional adversarial learning, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2019, pp. 1633–1642.

G. Wang, C. Sun, A. Sowmya, Erl-net: Entangled representation learning for single image de-raining, in: Proceedings of the IEEE International Conference on Computer Vision, 2019, pp. 5644–5652.

Fu, 2017, Clearing the skies: A deep network architecture for single-image rain removal, IEEE Trans. Image Process., 26, 2944, 10.1109/TIP.2017.2691802

X. Fu, J. Huang, D. Zeng, H. Yue, X. Ding, J. Paisley, Removing rain from single images via a deep detail network, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp. 3855–3863.

H. Zhang, V.M. Patel, Density-aware single image de-raining using a multi-stream dense network, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 695–704.

W. Yang, R.T. Tan, J. Feng, J. Liu, Z. Guo, S. Yan, Deep joint rain detection and removal from a single image, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp. 1357–1366.

Yang, 2019, Joint rain detection and removal from a single image with contextualized deep networks, IEEE Trans. Pattern Anal. Mach. Intell., PP, 1

X. Li, J. Wu, Z. Lin, H. Liu, H. Zha, Recurrent squeeze-and-excitation context aggregation net for single image deraining, in: Proceedings of the European Conference on Computer Vision, 2018, pp. 254–269.

D. Ren, W. Zuo, Q. Hu, P. Zhu, D. Meng, Progressive image deraining networks: a better and simpler baseline, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2019, pp. 3937–3946.

X. Hu, C.-W. Fu, L. Zhu, P.-A. Heng, Depth-attentional features for single-image rain removal, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2019, pp. 8022–8031.

T. Wang, X. Yang, K. Xu, S. Chen, Q. Zhang, R.W. Lau, Spatial attentive single-image deraining with a high quality real rain dataset, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2019, pp. 12270–12279.

H. Wang, Q. Xie, Q. Zhao, D. Meng, A model-driven deep neural network for single image rain removal, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 3103–3112.

He, 2017, Convolutional sparse and low-rank coding-based rain streak removal, 1259

Yu, 2015

K. Garg, S.K. Nayar, Detection and removal of rain from videos, in: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Vol. 1, 2004, p. I.

Garg, 2007, Vision and rain, Int. J. Comput. Vis., 75, 3, 10.1007/s11263-006-0028-6

Zhang, 2006, Rain removal in video by combining temporal and chromatic properties, 461

Park, 2008, Rain removal using Kalman filter in video, 494

Barnum, 2010, Analysis of rain and snow in frequency space, Int. J. Comput. Vis., 86, 256, 10.1007/s11263-008-0200-2

Tripathi, 2012, Video post processing: low-latency spatiotemporal approach for detection and removal of rain, IET Image Process., 6, 181, 10.1049/iet-ipr.2010.0547

Y.L. Chen, C.T. Hsu, A generalized low-rank appearance model for spatio-temporally correlated rain streaks, in: Proceedings of the IEEE International Conference on Computer Vision, 2013, pp. 1968–1975.

Jin-Hwan, 2015, Video deraining and desnowing using temporal correlation and low-rank matrix completion, IEEE Trans. Image Process., 24, 2658, 10.1109/TIP.2015.2428933

W. Ren, J. Tian, H. Zhi, A. Chan, Y. Tang, Video desnowing and deraining based on matrix decomposition, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp. 4210–4219.

W. Wei, L. Yi, Q. Xie, Q. Zhao, D. Meng, Z. Xu, Should we encode rain streaks in video as deterministic or stochastic? in: Proceedings of the IEEE International Conference on Computer Vision, 2017, pp. 2516–2525.

Zhang, 2017, Beyond a gaussian denoiser: Residual learning of deep cnn for image denoising, IEEE Trans. Image Process., 26, 3142, 10.1109/TIP.2017.2662206

Zhang, 2018, FFDNet: Toward a fast and flexible solution for CNN-based image denoising, IEEE Trans. Image Process., 27, 4608, 10.1109/TIP.2018.2839891

Dong, 2015, Image super-resolution using deep convolutional networks, IEEE Trans. Pattern Anal. Mach. Intell., 38, 295, 10.1109/TPAMI.2015.2439281

Y. Zhang, Y. Tian, Y. Kong, B. Zhong, Y. Fu, Residual dense network for image super-resolution, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 2472–2481.

Tai, 2017, 2790

X. Tao, H. Gao, X. Shen, J. Wang, J. Jia, Scale-recurrent network for deep image deblurring, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 8174–8182.

O. Kupyn, V. Budzan, M. Mykhailych, D. Mishkin, J. Matas, Deblurgan: Blind motion deblurring using conditional adversarial networks, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 8183–8192.

C. Jie, C.H. Tan, J. Hou, L.P. Chau, L. He, Robust video content alignment and compensation for rain removal in a CNN framework, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 6286–6295.

J. Liu, W. Yang, S. Yang, Z. Guo, Erase or fill? deep joint recurrent rain removal and reconstruction in videos, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 3233–3242.

Liu, 2018, D3R-Net: Dynamic routing residue recurrent network for video rain removal, IEEE Trans. Image Process., 28, 699, 10.1109/TIP.2018.2869722

Yang, 2020, Frame-consistent recurrent video deraining with dual-level flow

Kim, 2014, Single-image deraining using an adaptive nonlocal means filter, 914

Son, 2016, Rain removal via shrinkage of sparse codes and learned rain dictionary, 1

Kang, 2012, Automatic single-image-based rain streaks removal via image decomposition, IEEE Trans. Image Process., 21, 1742, 10.1109/TIP.2011.2179057

Wang, 2017, A hierarchical approach for rain or snow removing in a single color image, IEEE Trans. Image Process., 26, 3936, 10.1109/TIP.2017.2708502

Fu, 2011, Single-frame-based rain removal via image decomposition, 1453

L. Zhu, C.W. Fu, D. Lischinski, P.A. Heng, Joint bi-layer optimization for single-image rain streak removal, in: Proceedings of the IEEE International Conference on Computer Vision, 2017, pp. 2526–2534.

Zhang, 2019, Image de-raining using a conditional generative adversarial network, IEEE Trans. Circuits Syst. Video Technol.

Fu, 2019, Lightweight pyramid networks for image deraining, IEEE Trans. Neural Netw. Learn. Syst.

Zheng, 2019, Residual multiscale based single image deraining

R. Yasarla, V.M. Patel, Uncertainty guided multi-scale residual learning-using a cycle spinning cnn for single image de-raining, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2019, pp. 8405–8414.

K. Jiang, Z. Wang, P. Yi, C. Chen, B. Huang, Y. Luo, J. Ma, J. Jiang, Multi-scale progressive fusion network for single image deraining, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 8346–8355.

S.S. Halder, J.-F. Lalonde, R.d. Charette, Physics-based rendering for improving robustness to rain, in: Proceedings of the IEEE International Conference on Computer Vision, 2019, pp. 10203–10212.

Mu, 2019, Learning bilevel layer priors for single image rain streaks removal, IEEE Signal Process. Lett., 26, 307, 10.1109/LSP.2018.2889277

W. Wei, D. Meng, Q. Zhao, Z. Xu, Y. Wu, Semi-supervised transfer learning for image rain removal, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2019, pp. 3877–3886.

K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recognition, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 770–778.

Huang, 2017, 2261

Paszke, 2017

Zhou, 2004, Image quality assessment: from error visibility to structural similarity, IEEE Trans. Image Process., 13, 600, 10.1109/TIP.2003.819861

Kingma, 2014, Adam: A method for stochastic optimization, Comput. Sci.

Huynh-Thu, 2008, Scope of validity of PSNR in image/video quality assessment, Electron. Lett., 44, 800, 10.1049/el:20080522