Pixel-level image fusion: A survey of the state of the art

Web of science, (http://www.webofknowledge.com). Olshausen, 1996, Emergence of simple-cell receptive field properties by learning a sparse code for natural images, Nature, 381, 607, 10.1038/381607a0 Rubinstein, 2010, Dictionaries for sparse representation modeling, Proc. IEEE, 98, 1045, 10.1109/JPROC.2010.2040551 Mertens, 2007, Exposure fusion, 382 Zhang, 1999, A categorization of multiscale-decomposition-based image fusion schemes with a performance study for a digital camera application, Proc. IEEE, 87, 1315, 10.1109/5.775414 Goshtasby, 2007, Image fusion: Advances in the state of the art, Inf. Fus., 8, 114, 10.1016/j.inffus.2006.04.001 Thomas, 2008, Synthesis of multispectral images to high spatial resolution: A critical review of fusion methods based on remote sensing physics, IEEE Transactions on Geoscience and Remote Sensing, 46, 1301, 10.1109/TGRS.2007.912448 Vivone, 2015, A critical comparison among pansharpening algorithms, IEEE Trans. Geosci. Remote Sensing, 53, 2565, 10.1109/TGRS.2014.2361734 Zhang, 2010, Multi-source remote sensing data fusion: status and trends, Int. J. Image Data Fus., 1, 5, 10.1080/19479830903561035 James, 2014, Medical image fusion: A survey of the state of the art, Inf. Fus., 19, 4, 10.1016/j.inffus.2013.12.002 Pajares, 2004, A wavelet-based image fusion tutorial, Pattern Recognit., 37, 1855, 10.1016/j.patcog.2004.03.010 Li, 2002, Using the discrete wavelet frame transform to merge landsat TM and SPOT panchromatic images, Inf. Fus., 3, 17, 10.1016/S1566-2535(01)00037-9 Lewis, 2007, Pixel- and region-based image fusion with complex wavelets, Inf. Fus., 8, 119, 10.1016/j.inffus.2005.09.006 Cands, 2001, Curvelets and curvilinear integrals, J. Approximation Theor., 113, 59, 10.1006/jath.2001.3624 Nencini, 2007, Remote sensing image fusion using the curvelet transform, Inf. Fus., 8, 143, 10.1016/j.inffus.2006.02.001 Do, 2002, Contourlets: a directional multiresolution image representation, vol. 1, I Li, 2011, Biological image fusion using a NSCT based variable-weight method, Inf. Fus., 12, 85, 10.1016/j.inffus.2010.03.007 Yang, 2010, Image fusion based on a new contourlet packet, Inf. Fus., 11, 78, 10.1016/j.inffus.2009.05.001 Yang, 2008, Multimodality medical image fusion based on multiscale geometric analysis of contourlet transform, Neurocomputing, 72, 203, 10.1016/j.neucom.2008.02.025 Zhang, 2016, An adaptive fusion approach for infrared and visible images based on NSCT and compressed sensing, Infrared Phys. Technol., 74, 11, 10.1016/j.infrared.2015.11.003 Zhao, 2015, A fast fusion scheme for infrared and visible light images in NSCT domain, Infrared Phys. Technol., 72, 266, 10.1016/j.infrared.2015.07.026 Saeedi, 2011, A new pan-sharpening method using multiobjective particle swarm optimization and the shiftable contourlet transform, ISPRS J. Photogramm. Remote Sensing, 66, 365, 10.1016/j.isprsjprs.2011.01.006 Upla, 2015, An edge preserving multiresolution fusion: use of contourlet transform and MRF prior, IEEE Trans. Geosci. Remote Sensing, 53, 3210, 10.1109/TGRS.2014.2371812 Easley, 2008, Sparse directional image representations using the discrete shearlet transform, Appl. Comput. Harmonic Anal., 25, 25, 10.1016/j.acha.2007.09.003 Wang, 2014, Multi-modal medical image fusion using the inter-scale and intra-scale dependencies between image shift-invariant shearlet coefficients, Inf. Fus., 19, 20, 10.1016/j.inffus.2012.03.002 Wang, 2014, EGGDD: an explicit dependency model for multi-modal medical image fusion in shift-invariant shearlet transform domain, Inf. Fus., 19, 29, 10.1016/j.inffus.2013.04.005 Farbman, 2008, Edge-preserving decompositions for multi-scale tone and detail manipulation, ACM Trans. Graph., 27, 67:1, 10.1145/1360612.1360666 Hu, 2012, The multiscale directional bilateral filter and its application to multisensor image fusion, Inf. Fus., 13, 196, 10.1016/j.inffus.2011.01.002 Zhou, 2016, Perceptual fusion of infrared and visible images through a hybrid multi-scale decomposition with gaussian and bilateral filters, Inf. Fus., 30, 15, 10.1016/j.inffus.2015.11.003 Wang, 2015, Robust multi-modal medical image fusion via anisotropic heat diffusion guided low-rank structural analysis, Inf. Fus., 26, 103, 10.1016/j.inffus.2015.01.001 Redondo, 2009, Multifocus image fusion using the log-gabor transform and a multisize windows technique, Inf. Fus., 10, 163, 10.1016/j.inffus.2008.08.006 Yang, 2012, Fusion of multispectral and panchromatic images based on support value transform and adaptive principal component analysis, Inf. Fus., 13, 177, 10.1016/j.inffus.2010.09.003 Zheng, 2007, Multisource image fusion method using support value transform, IEEE Trans. Image Process., 16, 1831, 10.1109/TIP.2007.896687 Li, 2011, Performance comparison of different multi-resolution transforms for image fusion, Inf. Fus., 12, 74, 10.1016/j.inffus.2010.03.002 Pradhan, 2006, Estimation of the number of decomposition levels for a wavelet-based multiresolution multisensor image fusion, IEEE Trans. Geosci. Remote Sensing, 44, 3674, 10.1109/TGRS.2006.881758 Ben Hamza, 2005, A multiscale approach to pixel-level image fusion, Integrated Computer-Aided Engineering, 12, 135, 10.3233/ICA-2005-12201 Zheng, 2007, A new metric based on extended spatial frequency and its application to DWT based fusion algorithms, Inf. Fus., 8, 177, 10.1016/j.inffus.2005.04.003 Jang, 2012, Contrast-enhanced fusion of multisensor images using subband-decomposed multiscale retinex, IEEE Trans. Image Process., 21, 3479, 10.1109/TIP.2012.2197014 Piella, 2003, A general framework for multiresolution image fusion: from pixels to regions, Inf. Fus., 4, 259, 10.1016/S1566-2535(03)00046-0 Shen, 2013, Cross-scale coefficient selection for volumetric medical image fusion, IEEE Trans. Biomed. Eng., 60, 1069, 10.1109/TBME.2012.2211017 Li, 2013, Image fusion with guided filtering, IEEE Trans. Image Process., 22, 2864, 10.1109/TIP.2013.2244222 Yang, 2010, Multifocus image fusion and restoration with sparse representation, IEEE Trans. Instrum. Meas., 59, 884, 10.1109/TIM.2009.2026612 Pati, 1993, Orthogonal matching pursuit: recursive function approximation with applications to wavelet decomposition, vol. 1, 40 Li, 2012, Group-sparse representation with dictionary learning for medical image denoising and fusion, IEEE Trans. Biomed. Eng., 59, 3450, 10.1109/TBME.2012.2217493 Chen, 2014, Image fusion with local spectral consistency and dynamic gradient sparsity, 2760 Yang, 2012, Pixel-level image fusion with simultaneous orthogonal matching pursuit, Inf. Fus., 13, 10, 10.1016/j.inffus.2010.04.001 Yin, 2011, Multimodal image fusion with joint sparsity model, Opt. Eng., 50, 067007.1, 10.1117/1.3584840 Yu, 2011, Image features extraction and fusion based on joint sparse representation, IEEE J. Selected Topics Signal Process., 5, 1074, 10.1109/JSTSP.2011.2112332 Yang, 2012, Color image fusion with extend joint sparse model, 376 Zhang, 2013, Dictionary learning method for joint sparse representation-based image fusion, Opt. Eng., 52, 057006.1, 10.1117/1.OE.52.5.057006 Yin, 2013, Simultaneous image fusion and super-resolution using sparse representation, Inf. Fus., 14, 229, 10.1016/j.inffus.2012.01.008 Li, 2013, Remote sensing image fusion via sparse representations over learned dictionaries, IEEE Transactions on Geoscience and Remote Sensing, 51, 4779, 10.1109/TGRS.2012.2230332 Nejati, 2015, Multi-focus image fusion using dictionary-based sparse representation, Inf. Fus., 25, 72, 10.1016/j.inffus.2014.10.004 Kim, 2016, Joint patch clustering-based dictionary learning for multimodal image fusion, Inf. Fus., 27, 198, 10.1016/j.inffus.2015.03.003 Wang, 2014, Fusion of multispectral and panchromatic images via sparse representation and local autoregressive model, Inf. Fus., 20, 73, 10.1016/j.inffus.2013.11.004 Zhu, 2013, A sparse image fusion algorithm with application to pan-sharpening, IEEE Trans. Geosci. Remote Sensing, 51, 2827, 10.1109/TGRS.2012.2213604 Zhang, 2016, Robust multi-focus image fusion using multi-task sparse representation and spatial context, IEEE Trans. Image Process. Gangapure, 2015, Steerable local frequency based multispectral multifocus image fusion, Inf. Fus., 23, 99, 10.1016/j.inffus.2014.07.003 Li, 2002, Multifocus image fusion using artificial neural networks, Pattern Recognit. Lett., 23, 985, 10.1016/S0167-8655(02)00029-6 Li, 2004, Fusing images with different focuses using support vector machines, IEEE Trans. Neural Netw., 15, 1555, 10.1109/TNN.2004.837780 Li, 2001, Combination of images with diverse focuses using the spatial frequency, Inf. Fus., 2, 169, 10.1016/S1566-2535(01)00038-0 De, 2013, Multi-focus image fusion using a morphology-based focus measure in a quad-tree structure, Inf. Fus., 14, 136, 10.1016/j.inffus.2012.01.007 Bai, 2015, Quadtree-based multi-focus image fusion using a weighted focus-measure, Inf. Fus., 22, 105, 10.1016/j.inffus.2014.05.003 Li, 2008, Multifocus image fusion using region segmentation and spatial frequency, Image Vis. Comput., 26, 971, 10.1016/j.imavis.2007.10.012 Li, 2008, Region-based multi-focus image fusion, 343 Li, 2013, Image matting for fusion of multi-focus images in dynamic scenes, Inf. Fus., 14, 147, 10.1016/j.inffus.2011.07.001 Zhang, 2014, Multi-modal image fusion with KNN matting, vol. 484, 89 Liu, 2015, Multi-focus image fusion with dense SIFT, Inf. Fus., 23, 139, 10.1016/j.inffus.2014.05.004 Li, 2012, Fast multi-exposure image fusion with median filter and recursive filter, IEEE Trans. Consum. Electron., 58, 626, 10.1109/TCE.2012.6227469 Shen, 2011, Generalized random walks for fusion of multi-exposure images, IEEE Trans. Image Process., 20, 3634, 10.1109/TIP.2011.2150235 Shen, 2013, QoE-based multi-exposure fusion in hierarchical multivariate gaussian CRF, IEEE Trans. Image Process., 22, 2469, 10.1109/TIP.2012.2236346 Zhang, 2014, Multi-focus image fusion based on robust principal component analysis and pulse-coupled neural network, Optik - Int. J. Light Electron Opt., 125, 5002, 10.1016/j.ijleo.2014.04.002 Kumar, 2009, A total variation-based algorithm for pixel-level image fusion, IEEE Trans. Image Process., 18, 2137, 10.1109/TIP.2009.2025006 Tu, 2001, A new look at IHS-like image fusion methods, Inf. Fus., 2, 177, 10.1016/S1566-2535(01)00036-7 Tu, 2004, A fast intensity-hue-saturation fusion technique with spectral adjustment for IKONOS imagery, IEEE Geosci. Remote Sensing Lett., 1, 309, 10.1109/LGRS.2004.834804 Rahmani, 2010, An adaptive IHS pan-sharpening method, IEEE Geosci. Remote Sensing Lett., 7, 746, 10.1109/LGRS.2010.2046715 Choi, 2011, A new adaptive component-substitution-based satellite image fusion by using partial replacement, IEEE Trans. Geosci. Remote Sensing, 49, 295, 10.1109/TGRS.2010.2051674 Shahdoosti, 2016, Combining the spectral PCA and spatial PCA fusion methods by an optimal filter, Inf. Fus., 27, 150, 10.1016/j.inffus.2015.06.006 C.A. Laben, B.V. Brower, Process for enhancing the spatial resolution of multispectral imagery using pan-sharpening, 2000, US Patent 6011875. Kang, 2014, Pansharpening with matting model, IEEE Transactions on Geoscience and Remote Sensing, 52, 5088, 10.1109/TGRS.2013.2286827 Mitianoudis, 2007, Pixel-based and region-based image fusion schemes using ICA bases, Inf. Fus., 8, 131, 10.1016/j.inffus.2005.09.001 Sun, 2013, Poisson image fusion based on markov random field fusion model, Inf. Fus., 14, 241, 10.1016/j.inffus.2012.07.003 Balasubramaniam, 2014, Image fusion using intuitionistic fuzzy sets, Inf. Fus., 20, 21, 10.1016/j.inffus.2013.10.011 Li, 2010, Hybrid multiresolution method for multisensor multimodal image fusion, IEEE Sens. J., 10, 1519, 10.1109/JSEN.2010.2041924 Liu, 2015, A general framework for image fusion based on multi-scale transform and sparse representation, Inf. Fus., 24, 147, 10.1016/j.inffus.2014.09.004 Jiang, 2014, Image fusion with morphological component analysis, Inf. Fus., 18, 107, 10.1016/j.inffus.2013.06.001 Wang, 2013, Image fusion with nonsubsampled contourlet transform and sparse representation, J. Electron. Imaging, 22, 10.1117/1.JEI.22.4.043019 Daneshvar, 2010, MRI and PET image fusion by combining IHS and retina-inspired models, Inf. Fus., 11, 114, 10.1016/j.inffus.2009.05.003 Zhang, 2005, An IHS and wavelet integrated approach to improve pan-sharpening visual quality of natural colour IKONOS and QuickBird images, Inf. Fus., 6, 225, 10.1016/j.inffus.2004.06.009 Palsson, 2016, Quantitative quality evaluation of pansharpened imagery: Consistency versus synthesis, IEEE Trans. Geosci. Remote Sensing, 54, 1247, 10.1109/TGRS.2015.2476513 Wang, 2009, Mean squared error: Love it or leave it? a new look at signal fidelity measures, IEEE Signal Process. Mag., 26, 98, 10.1109/MSP.2008.930649 Garzelli, 2009, Hypercomplex quality assessment of multi/hyperspectral images, IEEE Geosci. Remote Sensing Lett., 6, 662, 10.1109/LGRS.2009.2022650 LilloSaavedra, 2006, Spectral or spatial quality for fused satellite imagery? a tradeoff solution using the wavelet á trous algorithm, Int. J. Remote Sensing, 27, 1453, 10.1080/01431160500462188 Wang, 2004, Image quality assessment: from error visibility to structural similarity, IEEE Trans. Image Process., 13, 600, 10.1109/TIP.2003.819861 Zhang, 2011, FSIM: A feature similarity index for image quality assessment, IEEE Trans. Image Process., 20, 2378, 10.1109/TIP.2011.2109730 Xue, 2014, Gradient magnitude similarity deviation: A highly efficient perceptual image quality index, IEEE Trans. Image Process., 23, 684, 10.1109/TIP.2013.2293423 Capodiferro, 2012, Two-dimensional approach to full-reference image quality assessment based on positional structural information, IEEE Trans. Image Process., 21, 505, 10.1109/TIP.2011.2165293 Toet, 2010, Towards cognitive image fusion, Inf. Fus., 11, 95, 10.1016/j.inffus.2009.06.008 Qu, 2002, Information measure for performance of image fusion, Elec. Lett., 38, 313, 10.1049/el:20020212 Hossny, 2008, Comments on “Information measure for performance of image fusion”, Elec. Lett., 44, 1066, 10.1049/el:20081754 Cvejic, 2006, Image fusion metric based on mutual information and tsallis entropy, Elec. Lett., 42, 626, 10.1049/el:20060693 Hossny, 2010, Image fusion performance metric based on mutual information and entropy driven quadtree decomposition, Elec. Lett., 46, 1266, 10.1049/el.2010.1778 Wang, 2008, Performance evaluation of image fusion techniques, 469 Xydeas, 2000, Objective image fusion performance measure, Elec. Lett., 36, 308, 10.1049/el:20000267 Liu, 2008, A feature-based metric for the quantitative evaluation of pixel-level image fusion, Comput. Vis. Image Understand., 109, 56, 10.1016/j.cviu.2007.04.003 Yang, 2008, A novel similarity based quality metric for image fusion, Inf. Fus., 9, 156, 10.1016/j.inffus.2006.09.001 Petrović, 2015, Focused pooling for image fusion evaluation, Inf. Fus., 22, 119, 10.1016/j.inffus.2014.05.002 Hassen, 2015, Objective quality assessment for multiexposure multifocus image fusion, IEEE Trans. Image Process., 24, 2712, 10.1109/TIP.2015.2428051 Alparone, 2008, Multispectral and panchromatic data fusion assessment without reference, Photogramm. Eng. Remote Sensing, 74, 193, 10.14358/PERS.74.2.193 Han, 2015, Multimodal gray image fusion metric based on complex wavelet structural similarity, Optik-Int. J. Light Electron Opt., 126, 5842, 10.1016/j.ijleo.2015.08.250 Chen, 2007, A human perception inspired quality metric for image fusion based on regional information, Inf. Fus., 8, 193, 10.1016/j.inffus.2005.10.001 Han, 2013, A new image fusion performance metric based on visual information fidelity, Inf. Fus., 14, 127, 10.1016/j.inffus.2011.08.002 Liu, 2012, Objective assessment of multiresolution image fusion algorithms for context enhancement in night vision: A comparative study, IEEE Trans. Pattern Anal. Mach. Intell., 34, 94, 10.1109/TPAMI.2011.109 Wei, 2010, Theoretical analysis of correlation-based quality measures for weighted averaging image fusion, Inf. Fus., 11, 301, 10.1016/j.inffus.2009.10.006 Ma, 2015, Perceptual quality assessment for multi-exposure image fusion, IEEE Trans. Image Process., 24, 3345, 10.1109/TIP.2015.2442920 Simone, 2002, Image fusion techniques for remote sensing applications, Inf. Fus., 3, 3, 10.1016/S1566-2535(01)00056-2 Bovolo, 2010, Analysis of the effects of pansharpening in change detection on vhr images, IEEE Geosci. Remote Sensing Lett., 7, 53, 10.1109/LGRS.2009.2029248 Palsson, 2012, Classification of pansharpened urban satellite images, IEEE J. Selected Topics Appl. Earth Observ. Remote Sensing, 5, 281, 10.1109/JSTARS.2011.2176467 Fauvel, 2013, Advances in spectral-spatial classification of hyperspectral images, Proc. IEEE, 101, 652, 10.1109/JPROC.2012.2197589 Bioucas-Dias, 2012, Hyperspectral unmixing overview: Geometrical, statistical, and sparse regression-based approaches, IEEE J. Selected Topics Appl. Earth Observ. Remote Sensing, 5, 354, 10.1109/JSTARS.2012.2194696 Song, 2014, Spatio-spectral fusion of satellite images based on dictionary-pair learning, Inf. Fus., 18, 148, 10.1016/j.inffus.2013.08.005 Alparone, 2004, Landsat ETM+ and SAR image fusion based on generalized intensity modulation, IEEE Trans. Geosci. Remote Sensing, 42, 2832, 10.1109/TGRS.2004.838344 Byun, 2013, An area-based image fusion scheme for the integration of SAR and optical satellite imagery, IEEE J. Selected Topics Appl. Earth Observ. Remote Sensing, 6, 2212, 10.1109/JSTARS.2013.2272773 Brell, 2016, Improving sensor fusion: A parametric method for the geometric coalignment of airborne hyperspectral and LiDAR data, IEEE Trans. Geosci. Remote Sensing, 1 Wei, 2015, Hyperspectral and multispectral image fusion based on a sparse representation, IEEE Trans. Geosci. Remote Sensing, 53, 3658, 10.1109/TGRS.2014.2381272 Global land cover facility, (http://www.glcf.umiacs.umd.edu/data/). Digitalglobe, (https://www.digitalglobe.com/). Iwasaki, 2011, Hyperspectral imager suite (HISUI) -japanese hyper-multi spectral radiometer, 1025 Debes, 2014, Hyperspectral and LiDAR data fusion: Outcome of the 2013 GRSS data fusion contest, IEEE J. Selected Topics Appl. Earth Observ. Remote Sensing, 7, 2405, 10.1109/JSTARS.2014.2305441 Moser, 2015, 2015 IEEE GRSS data fusion contest: Extremely high resolution lidar and optical data [technical committees], IEEE Geosci. Remote Sensing Mag., 3, 40, 10.1109/MGRS.2015.2397448 Bhatnagar, 2013, Directive contrast based multimodal medical image fusion in NSCT domain, IEEE Trans. Multim., 15, 1014, 10.1109/TMM.2013.2244870 GholamHosseini, 2006, Fusion of vibro-acoustography images and X-ray mammography, 2803 Whole brain web site of the harvard medical school, (http://www.med.harvard.edu/AANLIB/home.html). Mcconnell brain imaging centre of the montreal neurological institute, (http://www.mouldy.bic.mni.mcgill.ca/brainweb). Hogervorst, 2010, Fast natural color mapping for night-time imagery, Inf. Fus., 11, 69, 10.1016/j.inffus.2009.06.005 Yamasaki, 2008, Denighting: Enhancement of nighttime images for a surveillance camera, 1 Schaul, 2009, Color image dehazing using the near-infrared, 1629 Gundimada, 2010, Face recognition in multi-sensor images based on a novel modular feature selection technique, Inf. Fus., 11, 124, 10.1016/j.inffus.2009.05.002 Wong, 2013, Eyeglasses removal of thermal image based on visible information, Inf. Fus., 14, 163, 10.1016/j.inffus.2011.09.002 Singh, 2008, Hierarchical fusion of multi-spectral face images for improved recognition performance, Inf. Fus., 9, 200, 10.1016/j.inffus.2006.06.002 Muller, 2009, Cognitively-engineered multisensor image fusion for military applications, Inf. Fus., 10, 137, 10.1016/j.inffus.2008.08.008 The EQUINOX face database, (http://www.face-rec.org/databases/). Raskar, 2005, Image fusion for context enhancement and video surrealism, 1 Petschnigg, 2004, Digital photography with flash and no-flash image pairs, ACM Trans. Graph., 23, 664, 10.1145/1015706.1015777 Petrović, 2007, Subjective tests for image fusion evaluation and objective metric validation, Inf. Fus., 8, 208, 10.1016/j.inffus.2005.05.001