High-to-low-level feature matching and complementary information fusion for reference-based image super-resolution
Tóm tắt
The aim of the reference-based image super-resolution (RefSR) is to reconstruct high-resolution (HR) when a reference (Ref) image with similar content as that of the low-resolution (LR) input is given. In the task, the quality of existing approaches degrades severely when there are several similar objects but different contents. Besides, not all similar information in the reference image is useful for the input image. Therefore, we propose high-to-low-level feature matching and complementary information fusion (HMCF) network for RefSR. The matching strategy adopts high-level to low-level feature matching to distinguish similar objects but different contents according to high-level semantics. The complementary information fusion module utilizes the channel and spatial attention to select the complement information for LR image and keeps the pixel consistency of input and Ref image. We perform extensive experiments to demonstrate that our proposed HMCF obtains the SOTA performance on the RefSR benchmarks and presents a high visual quality.
Tài liệu tham khảo
Mahapatra, D., Bozorgtabar, B., Garnavi, R.: Image super-resolution using progressive generative adversarial networks for medical image analysis. Comput. Med. Imaging Graph. 71, 30–39 (2019)
Qiu, D., Zheng, L., Zhu, J., Huang, D.: Multiple improved residual networks for medical image super-resolution. Futur. Gener. Comput. Syst. 116, 200–208 (2021)
Demirel, H., Anbarjafari, G.: Discrete wavelet transform-based satellite image resolution enhancement. IEEE Trans. Geosci. Remote Sens. 49(6), 1997–2004 (2011)
Xia, R., Chen, Y., Ren, B.: Improved anti-occlusion object tracking algorithm using unscented Rauch-Tung-Striebel smoother and Kernel correlation filter, J. King Saud Univ-Comput. Inf. Sci.
Zhang, J., Feng, W., Yuan, T., Wang, J., Sangaiah, A.K.: Scstcf: spatial-channel selection and temporal regularized correlation filters for visual tracking. Appl. Soft Comput. 118, 108485 (2022)
Chen, Y., Liu, L., Phonevilay, V., Gu, K., Xia, R., Xie, J., Zhang, Q., Yang, K.: Image super-resolution reconstruction based on feature map attention mechanism. Appl. Intell. 51(7), 4367–4380 (2021)
Zheng, H., Ji, M., Wang, H., Liu, Y., Fang, L.: Crossnet: An end-to-end reference-based super resolution network using cross-scale warping. In: Proceedings of the European conference on computer vision (ECCV) pp. 88–104 (2018)
Zhang, Z., Wang, Z., Lin, Z., Qi, H.: Image super-resolution by neural texture transfer. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 7982–7991 (2019)
Yang, F., Yang, H., Fu, J., Lu, H., Guo, B.: Learning texture transformer network for image super-resolution. In: IEEE/CVF conference on computer vision and pattern recognition (CVPR) pp. 5791–5800 (2020)
Lu, L., Li, W., Tao, X., Lu, J., Jia, J.: Masa-sr: Matching acceleration and spatial adaptation for reference-based image super-resolution. In: IEEE/CVF conference on computer vision and pattern recognition (CVPR) pp. 6368–6377 (2021)
Jiang, Y., Chan, K., Wang, X., Loy, C. C., Liu, Z.: Robust reference-based super-resolution via c2-matching. In: IEEE/CVF conference on computer vision and pattern recognition (CVPR) pp. 2103–2112 (2021)
Xia, B., Tian, Y., Hang, Y., Yang, W., Zhou, J.: Coarse-to-fine embedded patchmatch and multi-scale dynamic aggregation for reference-based super-resolution. arXiv preprint arXiv:2201.04358
Dong, C., Loy, C.C., He, K., Tang, X.: Image super-resolution using deep convolutional networks. IEEE Trans. Pattern Anal. Mach. Intell. 38(2), 295–307 (2015)
Chen, H., Gu, J., Zhang, Z.: Attention in attention network for image super-resolution. arXiv preprint arXiv:2104.09497
Xia, B., Hang, Y., Tian, Y., Yang, W., Liao, Q., Zhou, J.: Efficient non-local contrastive attention for image super-resolution. arXiv preprint arXiv:2201.03794
Kim, J., Kwon Lee, J., Mu Lee, K.: Accurate image super-resolution using very deep convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1646–1654 (2016)
Kim, J., Lee, J. K., Lee, K. M.: Deeply-recursive convolutional network for image super-resolution. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1637–1645 (2016)
Lim,B., Son, S., Kim, H., Nah, S., Mu Lee, K.: Enhanced deep residual networks for single image super-resolution. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp. 136–144 (2017)
Zhang, Y., Li, K., Li, K., Wang, L., Zhong, B., Fu, Y.: Image super-resolution using very deep residual channel attention networks. In: Proceedings of the European conference on computer vision (ECCV) pp. 286–301 (2018)
Hu, Y., Li, J., Huang, Y., Gao, X.: Channel-wise and spatial feature modulation network for single image super-resolution. IEEE Trans. Circuits Syst. Video Technol. 30(11), 3911–3927 (2019)
Kim, J.-H., Choi, J.-H., Cheon, M., Lee, J.-S.: Ram: Residual attention module for single image super-resolution. arXiv preprint arXiv:1811.12043 (2018)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556
Lim, B., Son, S., Kim, H., Nah, S., Mu Lee, K.: Enhanced deep residual networks for single image super-resolution. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops pp. 136–144 (2017)
Huang, J.-B., Singh, A., Ahuja, N.: Single image super-resolution from transformed self-exemplars. In: Proceedings of the IEEE conference on computer vision and pattern recognition pp. 5197–5206 (2015)
Kingma Diederik, P., Adam, J. B.: A method for stochastic optimization. In: International conference on learning representations. ICLR