Performance evaluation of state-of-the-art multimodal remote sensing image matching methods in the presence of noise

Alcantarilla PF, Bartoli A, Davison AJ (2012) KAZE features. Computer Vision–ECCV 2012. In: 12th European Conference on Computer Vision, Florence, Italy October 7-13, 2012, Proceedings, Part VI 12 Balntas V, Riba E, Ponsa D, Mikolajczyk K (2016) Learning local feature descriptors with triplets and shallow convolutional neural networks. Bmvc Barroso-Laguna A, Riba E, Ponsa D, Mikolajczyk K (2019) Key. net: keypoint detection by handcrafted and learned cnn filters. In: Proceedings of the IEEE/CVF international conference on computer vision Belongie S, Malik J, Puzicha J (2000) Shape context: a new descriptor for shape matching and object recognition. Adv Neural Inform Process Syst 13 Bhowmik A, Gumhold S, Rother C, Brachmann E (2020) Reinforced feature points: optimizing feature detection and description for a high-level task. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 4948–4957 Chang H-H, Wu G-L, Chiang M-H (2019) Remote sensing image registration based on modified SIFT and feature slope grouping. IEEE Geosci Remote Sens Lett 16(9):1363–1367 Chen J, Chen X, Chen S, Liu Y, Rao Y, Yang Y, Wang H, Wu D (2023) Shape-former: bridging CNN and transformer via ShapeConv for multimodal image matching. Inform Fusion 91:445–457 Chen J, Tian J, Lee N, Zheng J, Smith RT, Laine AF (2010) A partial intensity invariant feature descriptor for multimodal retinal image registration. IEEE Trans Biomed Eng 57(7):1707–1718 Cui S, Xu M, Ma A, Zhong Y (2020) Modality-free feature detector and descriptor for multimodal remote sensing image registration. Remote Sens 12(18):2937 de Oliveira Jr EM, dos Santos DR (2023) Closed-form solution to point-and plane-based co-registration of terrestrial LiDAR point clouds. Appl Geomatics:1–19 Dusmanu, M., Rocco, I., Pajdla, T., Pollefeys, M., Sivic, J., Torii, A., & Sattler, T. (2019). D2-net: a trainable cnn for joint detection and description of local features. arXiv preprint arXiv:1905.03561. Elboher E, Werman M (2013) Asymmetric correlation: a noise robust similarity measure for template matching. IEEE Trans Image Process 22(8):3062–3073 Fan J, Wu Y, Li M, Liang W, Cao Y (2018) SAR and optical image registration using nonlinear diffusion and phase congruency structural descriptor. IEEE Trans Geosci Remote Sens 56(9):5368–5379 Fan J, Ye Y, Liu G, Li J, Li Y (2022) Phase congruency order-based local structural feature for SAR and optical image matching. IEEE Geosci Remote Sens Lett 19:1–5 Förstner W (1986) A feature based correspondence algorithm for image matching. ISPRS ComIII, Rovaniemi, pp 150–166 Geremew T, Zewdie W, Pellikka P (2023) Ecosystem extent mapping by integrating Landsat 8, PALSAR-2, and GEDI lidar. Appl Geomatics 15(1):61–76 Ghannadi MA, SaadatSeresht M, Izadi M, Alebooye S (2020) Optimal texture image reconstruction method for improvement of SAR image matching. IET Radar, Sonar Navig 14(8):1229–1235 Hake F, Lippmann P, Alkhatib H, Oettel V, Neumann I (2023) Automated damage detection for port structures using machine learning algorithms in heightfields. Appl Geomatics 15:349–357 Hanif MS (2019) Patch match networks: Improved two-channel and Siamese networks for image patch matching. Pattern Recognit Lett 120:54–61 Harris C, Stephens M (1988) A combined corner and edge detector. In: Alvey vision conference. Citeseer Hosseiny B, Rastiveis H, Homayouni S (2020) An automated framework for plant detection based on deep simulated learning from drone imagery. Remote Sens 12(21):3521 Irshad A, Hafiz R, Ali M, Faisal M, Cho Y, Seo J (2019) Twin-net descriptor: twin negative mining with quad loss for patch-based matching. IEEE Access 7:136062–136072 Jiang X, Ma J, Xiao G, Shao Z, Guo X (2021) A review of multimodal image matching: methods and applications. Inform Fusion 73:22–71 Joshi K, Patel MI (2020) Recent advances in local feature detector and descriptor: a literature survey. Int J Multimed Inform Retriev 9(4):231–247 Jovhari N, Farhadi N, Sedaghat A, Mohammadi N (2023) Performance evaluation of learning-based methods for multispectral satellite image matching. ISPRS Ann Photogramm Remote Sens Spatial Inform Sci 10:335–341 Kovesi P (2003) Phase congruency detects corners and edges. The Australian pattern recognition society conference: DICTA Li J, Hu Q, Ai M (2019) RIFT: multi-modal image matching based on radiation-variation insensitive feature transform. IEEE Trans Image Process 29:3296–3310 Li J, Xu W, Shi P, Zhang Y, Hu Q (2022) LNIFT: locally normalized image for rotation invariant multimodal feature matching. IEEE Trans Geosci Remote Sens 60:1–14 Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vision 60:91–110 Luo Z, Shen T, Zhou L, Zhang J, Yao Y, Li S, Fang T, Quan L (2019) Contextdesc: local descriptor augmentation with cross-modality context. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 2527–2536 Ma J, Jiang X, Fan A, Jiang J, Yan J (2021) Image matching from handcrafted to deep features: a survey. Int J Comput Vision 129:23–79 Ma W, Wen Z, Wu Y, Jiao L, Gong M, Zheng Y, Liu L (2016) Remote sensing image registration with modified SIFT and enhanced feature matching. IEEE Geosci Remote Sens Lett 14(1):3–7 Ma W, Wu Y, Liu S, Su Q, Zhong Y (2018) Remote sensing image registration based on phase congruency feature detection and spatial constraint matching. IEEE Access 6:77554–77567 Mikolajczyk K, Schmid C (2005) A performance evaluation of local descriptors. IEEE Trans Pattern Anal Machine Intell 27(10):1615–1630 Mishchuk A, Mishkin D, Radenovic F, Matas J (2017) Working hard to know your neighbor’s margins: local descriptor learning loss. Adv Neural Inform Process Syst 30 Mohammadi N, Sedaghat A, Jodeiri Rad M (2022) Rotation-invariant self-similarity descriptor for multi-temporal remote sensing image registration. The Photogramm Record 37(177):6–34 Moravec HP (1980) Obstacle avoidance and navigation in the real world by a seeing robot rover. Stanford University Motayyeb S, Samadzedegan F, Javan FD, Hosseinpour H (2023) Fusion of UAV-based infrared and visible images for thermal leakage map generation of building facades. Heliyon 9(3) Mousavi V, Varshosaz M, Remondino F, Pirasteh S, Li J (2022) A two-step descriptor-based keypoint filtering algorithm for robust image matching. IEEE Trans Geosci Remote Sens 60:1–21 Mukundan A, Tolias G, Bursuc A, Jégou H, Chum O (2019) Understanding and improving kernel local descriptors. Int J Comput Vision 127(11-12):1723–1737 Nunes CF, Pádua FL (2017) A local feature descriptor based on log-Gabor filters for keypoint matching in multispectral images. IEEE Geosci Remote Sens Lett 14(10):1850–1854 Ojala T, Pietikainen M, Harwood D (1994) Performance evaluation of texture measures with classification based on Kullback discrimination of distributions. In: Proceedings of 12th international conference on pattern recognition. IEEE, pp 582–585 Paul S, Pati UC (2021) A comprehensive review on remote sensing image registration. Int J Remote Sens 42(14):5396–5432 Rosten E, Porter R, Drummond T (2008) Faster and better: a machine learning approach to corner detection. IEEE Trans Pattern Anal Machine Intell 32(1):105–119 Sedaghat A, Ebadi H (2015a) Distinctive order based self-similarity descriptor for multi-sensor remote sensing image matching. ISPRS J Photogramm Remote Sens 108:62–71 Sedaghat A, Ebadi H (2015b) Remote sensing image matching based on adaptive binning SIFT descriptor. IEEE Trans Geosci Remote Sens 53(10):5283–5293 Sedaghat A, Mohammadi N (2018) Uniform competency-based local feature extraction for remote sensing images. ISPRS J Photogramm Remote Sens 135:142–157 Sedaghat A, Mohammadi N (2019a) High-resolution image registration based on improved SURF detector and localized GTM. Int J Remote Sens 40(7):2576–2601 Sedaghat A, Mohammadi N (2019b) Illumination-robust remote sensing image matching based on oriented self-similarity. ISPRS J Photogramm Remote Sens 153:21–35 Sedaghat A, Mokhtarzade M, Ebadi H (2011) Uniform robust scale-invariant feature matching for optical remote sensing images. IEEE Trans Geosci Remote Sens 49(11):4516–4527 Serati G, Sedaghat A, Mohammadi N, Li J (2022) Digital surface model generation from high-resolution satellite stereo imagery based on structural similarity. Geocarto Int 37(26):11390–11419 Shechtman E, Irani M (2007) Matching local self-similarities across images and videos. In: 2007 IEEE Conference on Computer Vision and Pattern Recognition. IEEE Shi J (1994) Good features to track. In: 1994 Proceedings of IEEE conference on computer vision and pattern recognition. IEEE, pp 593–600 Shokri D, Rastiveis H, Sarasua WA, Homayouni S, Hosseiny B, Shams A (2023) An enhanced descriptor extraction algorithm for power line detection from point clouds. Geograph Res 61(4):480–502 Shokri D, Rastiveis H, Shams A, Sarasua W (2019) Utility poles extraction from mobile lidar data in urban area based on density information. Int Arch Photogramm Remote Sens Spat Inf Sci 42:1001–1007 Sun J, Shen Z, Wang Y, Bao H, Zhou X (2021) LoFTR: detector-free local feature matching with transformers. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. CVF, pp 8922–8931 Tian Y, Barroso Laguna A, Ng T, Balntas V, Mikolajczyk K (2020) Hynet: learning local descriptor with hybrid similarity measure and triplet loss. Adv Neural Inform Process Syst 33:7401–7412 Tian Y, Yu X, Fan B, Wu F, Heijnen H, Balntas V (2019) Sosnet: second order similarity regularization for local descriptor learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. CVF, pp 11016–11025 Tola E, Lepetit V, Fua P (2009) Daisy: an efficient dense descriptor applied to wide-baseline stereo. IEEE Trans Pattern Anal Machine Intell 32(5):815–830 Tuytelaars T, Mikolajczyk K (2008) Local invariant feature detectors: a survey. Found Trends Comput Graph Vis 3(3):177–280 Uchida, Y. (2016). Local feature detectors, descriptors, and image representations: a survey. https://doi.org/10.48550/arXiv.1607.08368 . Wang L, Sun M, Liu J, Cao L, Ma G (2020) A robust algorithm based on phase congruency for optical and SAR image registration in suburban areas. Remote Sens 12(20):3339 Wei X, Zhang Y, Gong Y, Zheng N (2018) Kernelized subspace pooling for deep local descriptors. In: Proceedings of the IEEE conference on computer vision and pattern recognition. CVF, pp 1867–1875 Xiang Y, Wang F, You H (2018) OS-SIFT: a robust SIFT-like algorithm for high-resolution optical-to-SAR image registration in suburban areas. IEEE Trans Geosci Remote Sens 56(6):3078–3090 Xiong X, Jin G, Xu Q, Zhang H (2021) Self-similarity features for multimodal remote sensing image matching. IEEE J Select Topics Appl Earth Observ Remote Sens 14:12440–12454 Xiong X, Xu Q, Jin G, Zhang H, Gao X (2019) Rank-based local self-similarity descriptor for optical-to-SAR image matching. IEEE Geosci Remote Sens Lett 17(10):1742–1746 Xue L, Sheng Y, Zhang K (2023) Robust descriptor algorithm considering the changing gray value trends inside ground objects for heterogeneous optical image matching. IEEE J Select Topics Appl Earth Observ Remote Sens. https://doi.org/10.1109/JSTARS.2023.3320552 Yan H, Yang S, Xue Q, Zhang N (2022) HR optical and SAR image registration using uniform optimized feature and extend phase congruency. Int J Remote Sens 43(1):52–74 Yao Y, Zhang Y, Wan Y, Liu X, Yan X, Li J (2022) Multi-modal remote sensing image matching considering co-occurrence filter. IEEE Trans Image Process 31:2584–2597 Ye Y, Bruzzone L, Shan J, Bovolo F, Zhu Q (2019) Fast and robust matching for multimodal remote sensing image registration. IEEE Trans Geosci Remote Sens 57(11):9059–9070 Ye Y, Shan J, Bruzzone L, Shen L (2017a) Robust registration of multimodal remote sensing images based on structural similarity. IEEE Trans Geosci Remote Sens 55(5):2941–2958 Ye Y, Shan J, Hao S, Bruzzone L, Qin Y (2018) A local phase based invariant feature for remote sensing image matching. ISPRS J Photogramm Remote Sens 142:205–221 Ye Y, Shen L, Hao M, Wang J, Xu Z (2017b) Robust optical-to-SAR image matching based on shape properties. IEEE Geosci Remote Sens Lett 14(4):564–568 Ye Y, Zhu B, Tang T, Yang C, Xu Q, Zhang G (2022) A robust multimodal remote sensing image registration method and system using steerable filters with first-and second-order gradients. ISPRS J Photogramm Remote Sens 188:331–350 Yu D, Li L, Zheng Y, Lau M, Song Y-Z, Tai C-L, Fu H (2020) SketchDesc: learning local sketch descriptors for multi-view correspondence. IEEE Trans Circuits Syst Video Technol 31(5):1738–1750 Yu Q, Ni D, Jiang Y, Yan Y, An J, Sun T (2021) Universal SAR and optical image registration via a novel SIFT framework based on nonlinear diffusion and a polar spatial-frequency descriptor. ISPRS J Photogramm Remote Sens 171:1–17 Zhang H, Ni W, Yan W, Xiang D, Wu J, Yang X, Bian H (2019) Registration of multimodal remote sensing image based on deep fully convolutional neural network. IEEE J Select Topics Appl Earth Observ Remote Sens 12(8):3028–3042 Zhang X, Leng C, Hong Y, Pei Z, Cheng I, Basu A (2021a) Multimodal remote sensing image registration methods and advancements: a survey. Remote Sens 13(24):5128 Zhang X, Wang Y, Liu H (2021b) Robust optical and SAR image registration based on OS-SIFT and cascaded sample consensus. IEEE Geosci Remote Sens Lett 19:1–5