Domain adaption based on source dictionary regularized RKHS subspace learning
Tóm tắt
Domain adaption is to transform the source and target domain data into a certain space through a certain transformation, so that the probability distribution of the transformed data is as close as possible. The domain adaption algorithm based on Maximum Mean Difference (MMD) Maximization and Reproducing Kernel Hilbert Space (RKHS) subspace transformation is the current main algorithm for domain adaption, in which the RKHS subspace transformation is determined by MMD of the transformed source and target domain data. However, MMD has inherent defects in theory. The probability distributions of two different random variables will not change after subtracting their respective mean values, but their MMD becomes zero. A reasonable method should be that the MMD of the source and target domain data with the same label should be as small as possible after RKHS subspace transformation. However, the labels of target domain data are unknown and there is no way to model according to this criterion. In this paper, a domain adaption algorithm based on source dictionary regularized RKHS subspace learning is proposed, in which the source domain data are used as a dictionary, and the target domain data are approximated by the sparse coding of the dictionary. That is to say, in the process of RKHS subspace transformation, the target domain data are distributed around the mostly relevant source domain data. In this way, the proposed algorithm indirectly achieves the MMD of the source and target domain data with the same label after RKHS subspace transformation. So far there has been no similar work reported in the published academic papers. The experimental results presented in this paper show that the proposed algorithm outperforms 5 other state-of-the-art domain adaption algorithms on 5 commonly used datasets.
Tài liệu tham khảo
Pan SJ, Yang Q (2010) A survey on transfer learning. IEEE Trans Knowl Data Eng 22(10):1345–1359
Jiang M, Huang Z, Qiu L et al (2018) Transfer learning-based dynamic multiobjective optimization algorithms. IEEE Trans Evol Comput 22(4):501–514
Xu Y, Fang X, Wu J et al (2016) Discriminative transfer subspace learning via low-rank and sparse representation. IEEE Trans Image Process 25(2):850–863
Saenko K, Kulis B, Frita M, Darrell T (2010) Adapting visual categorymodels to new domains, in Proc. Eur. Conf. Comput. Vis., pp. 213–226
Pan SJ, Tsang IW, Kwok JT, Yang Q (2011) Domain adaptation via transfer component analysis. IEEE Trans Neural Netw 22(2):199–210
Duan L, Tsang IW, Xu D, Maybank SJ (2009) Domain transfer SVM for video concept detection, In: CVPR, pp. 1375–1381
Pan SJ, Kwok JT, Yang Q (2008) Transfer learning via dimensionality reduction. AAAI 8:677–682
Duan L, Xu D, Tsang IW, Learning with augmented features for heterogeneous domain adaptation, [Online]. Available: arXiv:abs/1206.4660
Yao T, Pan Y, Ngo C-W, Li H, Mei T (2015) Semi-supervised domain adaptation with subspace learning for visual recognition. in ICCV, pp. 2142–2150
Li Y, Liu J, Lu H, Ma S (2014) Learning robust face representation with classwise block-diagonal structure. IEEE Trans Inf Forens Secur 9(12):2051–2062
Li L, Zhang Z (2019) Semi-supervised domain adaptation by covariance matching. IEEE Trans Neural Netw Learn Syst 41(11):2724–2739
Long M, Wang J, Ding G, Sun J, Yu PS (2014) Transfer joint matching for unsupervised domain adaptation. In: CVPR, pp. 1410–1417
Baktashmotlagh M, Harandi MT, Lovell BC, Salzmann M (2014) Domain adaptation on the statistical manifold. In: CVPR, pp. 2481-2488
Long M, Zhu H, Wang J, Jordan MI (2016) Unsupervised domain adaptation with residual transfer networks. In: NIPS, pp. 136-144
Pan SJ, Tsang IW, Kwok JT et al (2011) Domain adaptation via transfer component analysis. IEEE Trans Neural Netw 22(2):199–210
Jiang M, Huang W, Huang Z et al (2015) Integration of global and local metrics for domain adaptation learning via dimensionality reduction. IEEE T Cybern 24:1–14
Deng W, Lendasse A, Ong Y, Tsang IW, Chen L, Zheng Q (2019) Domain adaption via feature selection on explicit feature map. IEEE Trans Neural Netw Learn Syst 30(4):1180–1190
Cai R, Li J, Zhang Z, Yang X, Hao Z (2020) DACH: domain adaptation without domain information. IEEE Trans Neural Netw Learn Syst 31:5055–5067
Long M, Wang J, Jordan MI (2017) Deep transfer learning with joint adaptation networks. In: Proc. 34th Int. Conf. Mach. Learn., pp. 2208-2217
Bousmalis K, Silberman N, Dohan D, Erhan D, Krishnan D (2017) Unsupervised Pixel-Level Domain Adaptation with Generative Adversarial Networks. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 95-104
Borgwardt KM, Gretton A, Rasch MJ et al (2006) Integrating structured biological data by kernel maximum mean discrepancy. Bioinformatics 22(14):e49–e57
Kreyszig E (1978) Introductory functional analysis with applications, In. Wiley, New York
Shawe-Taylor J, Cristianini N (2004) Kernel methods for pattern analysis. Cambridge University Press, Cambridge
Cui Z, Li W, Xu D et al (2014) Flowing on Riemannian manifold: domain adaptation by shifting covariance. IEEE T Cybern 44(12):2264–2273
Wang S, Zhang L et al (2020) Class-specific reconstruction transfer learning for visual recognition across domains. IEEE Trans Image Process 29:2424–2438
Li F T, Pan S J, Jin O, et al (2012) Cross-domain co-extraction of sentiment and topic lexicons. In: Meeting of the Association for Computational Linguistics: Long Papers. Association for Computational Linguistics
Long M, Cao Y, Wang J, et al (2015) Learning transferable features with deep adaptation networks. In: ICML, pp. 97–105
Long M, Cao Y, Cao Z et al (2018) Transferable representation learning with deep adaptation networks. IEEE Trans Pattern Anal Mach Intell 41:3071–3085
Zhang J, Ding Z, Li W, et al (2018) Importance weighted adversarial nets for partial domain adaptation. In: CVPR, pp. 8156–8164
Gong B, Shi Y, Sha F, et al (2012) Geodesic flow kernel for unsupervised domain adaptation. In: CVPR,
Fernando B, Habrard A, Sebban M, et al (2013) Unsupervised visual domain adaptation using subspace alignment. In: ICCV, pp. 2960–2967
Pan SJ, Kwok JT, Yang Q (2008) Transfer learning via dimensionality reduction. AAAI 2:677–682
Liu G, Lin Z, Yu Y, et al (2010) Robust subspace segmentation by low-rank representation. In: ICML, pp. 663–670
Liu G, Lin Z, Yan S et al (2012) Robust recovery of subspace structures by low-rank representation. IEEE Trans Pattern Anal Mach Intell 35(1):171–184
Jhuo I H, Liu D, Lee D T, et al (2012) Robust visual domain adaptation with low-rank reconstruction. In: CVPR, pp. 2168–2175
Shekhar S, Patel VM, Nguyen HV et al (2013) Generalized domain-adaptive dictionaries. CVPR 5:361–368
Shekhar S, Patel VM, Nguyen HV et al (2015) Coupled projections for adaptation of dictionaries. IEEE Trans Image Process 24(10):2941–2954
Zhu F, Shao L (2014) Weakly-supervised cross-domain dictionary learning for visual recognition. IJCV 109(1–2):42–59
Li S, Shao M, Fu Y (2015) Cross-view projective dictionary learning for person re-identification. In: IJCAI
Li S, Song S, Huang G et al (2018) Domain invariant and class discriminative feature learning for visual domain adaptation. IEEE Trans Image Process 27(9):4260–4273
Zhang L, Wang S et al (2019) Manifold criterion guided transfer learning via intermediate domain generation. IEEE Trans Neural Netw Learn Syst 30(12):3759–3773
Ramirez I, Lecumberry F, G. Sapiro, et al (2009) Universal priors for sparse modeling. In: IEEE International Workshop on Computational Advances in Multi-sensor Adaptive Processing, pp. 197–200
Castrodad A, Xing Z, Greer J, Bosch E, Carin L et al (2011) Learning discriminative sparse models for source separation and mapping of hyperspectral imagery. IEEE Trans Geosci Remote Sens 49(11):4263–4281
Zhou N, Shen Y, Peng J, et al (2012) Learning inter-related visual dictionary for object recognition. In: CVPR, pp. 3490–3497
Perronnin F (2008) Universal and adapted vocabularies for generic visual categorization. IEEE Trans Pattern Anal Mach Intell 30(7):1243–1256
Gao S, Tsang WH, Ma Y (2014) Learning category-specific dictionary and shared dictionary for fine-grained image categorization. IEEE Trans Image Process 23(2):623–634
Sivalingam R, Boley D, Morellas V et al (2015) Tensor dictionary learning for positive definite matrices. IEEE Trans Image Process 24(11):4592–4601
Harandi M, Salzmann M (2015) Riemannian coding and dictionary learning: Kernels to the rescue. In: CVPR, pp. 3926–3935
[Online]. Available: http://spams-devel.gforge.inria.fr/
[Online]. Available: http://cvxr.com/cvx/
Li J, Lu K, Zhu L (2019) Transfer independently together: a generalized framework for domain adaptation. IEEE T Cybern 49(6):2144–2155
Zhang L, Fu J, Wang S et al (2020) Guide subspace learning for unsupervised domain adaptation. IEEE Trans Neural Netw Learn Syst 31:3374–3388