RFN-Nest: An end-to-end residual fusion network for infrared and visible images

Information Fusion - Tập 73 - Trang 72-86 - 2021
Hui Li1, Xiao‐Jun Wu1, Josef Kittler2
1Jiangsu Provincial Engineering Laboratory of Pattern Recognition and Computational Intelligence, School of Artificial Intelligence and Computer Science, Jiangnan University, 214122, Wuxi, China
2The Center for Vision, Speech and Signal Processing, University of Surrey, Guildford, GU2 7XH, UK

Tóm tắt

Từ khóa


Tài liệu tham khảo

Li, 2019, RGB-T object tracking: benchmark and baseline, Pattern Recognit., 96, 10.1016/j.patcog.2019.106977

Li, 2018, Learning local-global multi-graph descriptors for RGB-T object tracking, IEEE Trans. Circuits Syst. Video Technol.

Luo, 2019, Thermal infrared and visible sequences fusion tracking based on a hybrid tracking framework with adaptive weighting scheme, Infrared Phys. Technol., 99, 265, 10.1016/j.infrared.2019.04.017

Shrinidhi, 2018, IR and visible video fusion for surveillance, 1

Ma, 2019, Infrared and visible image fusion methods and applications: A survey, Inf. Fusion, 45, 153, 10.1016/j.inffus.2018.02.004

Li, 2017, Pixel-level image fusion: A survey of the state of the art, Inf. Fusion, 33, 100, 10.1016/j.inffus.2016.05.004

Liu, 2018, Deep learning for pixel-level image fusion: Recent advances and future prospects, Inf. Fusion, 42, 158, 10.1016/j.inffus.2017.10.007

Pajares, 2004, A wavelet-based image fusion tutorial, Pattern Recognit., 37, 1855, 10.1016/j.patcog.2004.03.010

Ben Hamza, 2005, A multiscale approach to pixel-level image fusion, Integr. Comput.-Aided Eng., 12, 135, 10.3233/ICA-2005-12201

Yang, 2010, Image fusion based on a new contourlet packet, Inf. Fusion, 11, 78, 10.1016/j.inffus.2009.05.001

Li, 2013, Image fusion with guided filtering, IEEE Trans. Image Process., 22, 2864, 10.1109/TIP.2013.2244222

Wright, 2008, Robust face recognition via sparse representation, IEEE Trans. Pattern Anal. Mach. Intell., 31, 210, 10.1109/TPAMI.2008.79

Liu, 2010, Robust subspace segmentation by low-rank representation, 8

Liu, 2012, Robust recovery of subspace structures by low-rank representation, IEEE Trans. Pattern Anal. Mach. Intell., 35, 171, 10.1109/TPAMI.2012.88

Zhang, 2013, Dictionary learning method for joint sparse representation-based image fusion, Opt. Eng., 52, 10.1117/1.OE.52.5.057006

Liu, 2017, Infrared and visible image fusion method based on saliency detection in sparse domain, Infrared Phys. Technol., 83, 94, 10.1016/j.infrared.2017.04.018

Gao, 2017, Image fusion with cosparse analysis operator, IEEE Signal Process. Lett., 24, 943, 10.1109/LSP.2017.2696055

Li, 2017, Multi-focus image fusion using dictionary learning and low-rank representation, 675

Lu, 2014, The infrared and visible image fusion algorithm based on target separation and sparse representation, Infrared Phys. Technol., 67, 397, 10.1016/j.infrared.2014.09.007

Yin, 2017, A novel infrared and visible image fusion algorithm based on shift-invariant dual-tree complex shearlet transform and sparse representation, Neurocomputing, 226, 182, 10.1016/j.neucom.2016.11.051

Liu, 2017, Infrared and visible image fusion method based on saliency detection in sparse domain, Infrared Phys. Technol., 83, 94, 10.1016/j.infrared.2017.04.018

Li, 2018, Infrared and visible image fusion using a deep learning framework, 2705

Li, 2019, Infrared and visible image fusion with resnet and zero-phase component analysis, Infrared Phys. Technol., 102, 10.1016/j.infrared.2019.103039

Song, 2018, Multi-focus image fusion with PCA filters of PCANet, 1

Li, 2019, Densefuse: A fusion approach to infrared and visible images, IEEE Trans. Image Process., 28, 2614, 10.1109/TIP.2018.2887342

Li, 2020, MDLatLRR: A novel decomposition method for infrared and visible image fusion, IEEE Trans. Image Process.

Liu, 2016, Image fusion with convolutional sparse representation, IEEE Signal Process. Lett., 23, 1882, 10.1109/LSP.2016.2618776

Liu, 2017, Multi-focus image fusion with a deep convolutional neural network, Inf. Fusion, 36, 191, 10.1016/j.inffus.2016.12.001

Ma, 2019, FusionGAN: A generative adversarial network for infrared and visible image fusion, Inf. Fusion, 48, 11, 10.1016/j.inffus.2018.09.004

Ma, 2020, Infrared and visible image fusion via detail preserving adversarial learning, Inf. Fusion, 54, 85, 10.1016/j.inffus.2019.07.005

Ma, 2020, DDcGAN: A dual-discriminator conditional generative adversarial network for multi-resolution image fusion, IEEE Trans. Image Process., 29, 4980, 10.1109/TIP.2020.2977573

Li, 2020, Nestfuse: An infrared and visible image fusion architecture based on nest connection and spatial/channel attention models, IEEE Trans. Instrum. Meas., 10.1109/TIM.2020.3005230

Zhou, 2018, Unet++: A nested u-net architecture for medical image segmentation, 3

K. Ram Prabhakar, V. Sai Srikar, R. Venkatesh Babu, Deepfuse: A deep unsupervised approach for exposure fusion with extreme exposure image pairs, in: Proceedings of the IEEE International Conference on Computer Vision, 2017, pp. 4714–4722.

Simonyan, 2014

K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recognition, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 770–778.

G. Huang, Z. Liu, L. Van Der Maaten, K.Q. Weinberger, Densely connected convolutional networks, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp. 4700–4708.

Zhang, 2020, IFCNN: A general image fusion framework based on convolutional neural network, Inf. Fusion, 54, 99, 10.1016/j.inffus.2019.07.011

H. Zhang, H. Xu, Y. Xiao, X. Guo, J. Ma, Rethinking the image fusion: A fast unified image fusion network based on proportional maintenance of gradient and intensity, in: Proceedings of the AAAI Conference on Artificial Intelligence, 2020, pp. 12797–12804.

Xu, 2020, U2fusion: A unified unsupervised image fusion network, IEEE Trans. Pattern Anal. Mach. Intell.

Wang, 2004, Image quality assessment: from error visibility to structural similarity, IEEE Trans. Image Process., 13, 600, 10.1109/TIP.2003.819861

Lin, 2014, Microsoft coco: Common objects in context, 740

S. Hwang, J. Park, N. Kim, Y. Choi, I. So Kweon, Multispectral pedestrian detection: Benchmark dataset and baseline, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015, pp. 1037–1045.

Toet, 2014

M. Kristan, J. Matas, A. Leonardis, M. Felsberg, et al. The eighth visual object tracking VOT2020 challenge results, in: Proc. 16th Eur. Conf. Comput. Vis. Workshop, 2020.

Li, 2020

Roberts, 2008, Assessment of image fusion procedures using entropy, image quality, and multispectral classification, J. Appl. Remote Sens., 2

Rao, 1997, In-fibre Bragg grating sensors, Meas. Sci. Technol., 8, 355, 10.1088/0957-0233/8/4/002

Qu, 2002, Information measure for performance of image fusion, Electron. Lett., 38, 313, 10.1049/el:20020212

Kumar, 2013, Multifocus and multispectral image fusion based on pixel significance using discrete cosine harmonic wavelet transform, Signal Image Video Process., 7, 1125, 10.1007/s11760-012-0361-x

Aslantas, 2015, A new image quality metric for image fusion: The sum of the correlations of differences, AEU-Int. J. Electron. Commun., 69, 1890, 10.1016/j.aeue.2015.09.004

Ma, 2015, Perceptual quality assessment for multi-exposure image fusion, IEEE Trans. Image Process., 24, 3345, 10.1109/TIP.2015.2442920

Ma, 2016, Infrared and visible image fusion via gradient transfer and total variation minimization, Inf. Fusion, 31, 100, 10.1016/j.inffus.2016.02.001

M. Kristan, J. Matas, A. Leonardis, M. Felsberg, R. Pflugfelder, J.-K. Kamarainen, L. Cehovin Zajc, O. Drbohlav, A. Lukezic, A. Berg, et al. The seventh visual object tracking vot2019 challenge results, in: Proceedings of the IEEE International Conference on Computer Vision Workshops, 2019, pp. 1–36.

Xu, 2020

Li, 2016, Learning collaborative sparse representation for grayscale-thermal tracking, IEEE Trans. Image Process., 25, 5743, 10.1109/TIP.2016.2614135

Tang, 2019, RGBT salient object detection: Benchmark and A novel cooperative ranking approach, IEEE Trans. Circuits Syst. Video Technol.

Tu, 2019, RGB-T image saliency detection via collaborative graph learning, IEEE Trans. Multimed., 22, 160, 10.1109/TMM.2019.2924578

M. Kristan, J. Matas, A. Leonardis, M. Felsberg, L. Cehovin, G. Fernandez, T. Vojir, G. Hager, G. Nebehay, R. Pflugfelder, The visual object tracking vot2015 challenge results, in: Proceedings of the IEEE International Conference on Computer Vision Workshops, 2015, pp. 1–23.