A general robust low–rank multinomial logistic regression for corrupted matrix data classification
Tóm tắt
Multi-classification of corrupted matrix data is a significant problem in machine learning and pattern recognition. However, most of the existing methods can only handle the clean data or the corrupted data with the know statistical information of noises. Besides, they usually reshape the matrix data into a vector as the input, very likely to destroy the structure of the raw data, thereby reducing the model performance. In order to address the above issues, a general robust low–rank multinomial logistic regression is proposed for corrupted matrix data. The proposed approach has three outstanding merits as follows: (1) by using the multinomial logistic regression combined with three regularization terms corresponding to matrix structure, low–rank and sparsity, the clean data recovery and the classification are simultaneously fulfilled; (2) the proposed method can adapt to more general corrupted matrix data since it does not require strong statistical assumptions about noise, and (3) the theoretical analysis is provided to show the convergence of the proposed multi-block ADMM algorithm, and such a convergence can be rigidly guaranteed by introducing two auxiliary variables such that the coefficients of the equality constraints are orthogonal. Finally, extensive experimental results have demonstrated the effectiveness and robustness of the proposed method.
Tài liệu tham khảo
Tomioka R, Aihara K, Müller KR (2006) Logistic regression for single trial EEG classification[J]. Adv Neural Inf Process Syst, vol 19
Yan S, Xu D, Yang Q et al (2006) Multilinear discriminant analysis for face recognition[J]. IEEE Trans Image Process 16(1):212–220
Koo JJ, Evans AC, Gross WJ (2009) 3-D brain MRI tissue classification on FPGAs[J]. IEEE Trans Image Process 18(12):2735–2746
Shakhnarovich G, Darrell T, Indyk P (2008) Nearest-neighbor methods in learning and vision[J]. IEEE Trans Neural Netw 19(2):377
Kwak C, Clayton-Matthews A (2002) Multinomial logistic regression[J]. Nurs Res 51(6):404–410
Böhning D (1992) Multinomial logistic regression algorithm[J]. Ann Inst Stat Math 44(1):197–200
Erb RJ (1993) Introduction to backpropagation neural network computation[J]. Pharm Res 10 (2):165–170
Neelapu R, Devi GL, Rao KS (2018) Deep learning based conventional neural network architecture for medical image classification[J]. Traitement du Signal 35(2):169
Song K, Nie F, Han J et al (2017) Parameter free large margin nearest neighbor for distance metric learning. In: Proceedings of the AAAI conference on artificial intelligence, vol 31
Cai D, He X, Hu Y et al (2007) Learning a spatially smooth subspace for face recognition. In: 2007 IEEE conference on computer vision and pattern recognition. IEEE, pp 1–7
Gabriel KR (1998) Generalised bilinear regression[J]. Biometrika 85(3):689–700
Hou C, Nie F, Yi D et al (2012) Efficient image classification via multiple rank regression[J]. IEEE Trans Image Process 22(1):340–352
Hou C, Jiao Y, Nie F et al (2017) 2D feature selection by sparse matrix regression[J]. IEEE Trans Image Process 26(9):4255–4268
Yuan H, Li J, Lai LL et al (2020) Low–rank matrix regression for image feature extraction and feature selection[J]. Inf Sci 522:214–226
Fan Y, Qin G, Zhu ZY (2014) Robust variable selection in linear mixed models[J]. Commun Statistics-Theory Methods 43(21):4566–4581
Fan Y, Qin G, Zhu Z (2012) Variable selection in robust regression models for longitudinal data[J]. Multivar Anal 109:156–167
Copas JB (1988) Binary regression models for contaminated data[J]. J Roy Stat Soc: Ser B (Methodol) 50(2):225–253
Xu G, Hu B G, Principe JC (2016) Robust bounded logistic regression in the class imbalance problem. In: 2016 international joint conference on neural networks (IJCNN). IEEE, pp 1434–1441
Gillard JW (2006) An historical overview of linear regression with errors in both variables[J]. Cardiff University School of Mathematics Technical Report
Kummell CH (1879) Reduction of observation equations which contain more than one observed quantity[J]. The Analyst 6(4):97–105
Wald A (1940) The fitting of straight lines if both variables are subject to error[J]. Ann Math Stat 11(3):284–300
Gillard JW, Iles TC (2005) Method of moments estimation in linear regression with errors in both variables[J]. Cardiff University School of Mathematics Technical Paper:3–43
Lindley DV (1947) Regression lines and the linear functional relationship[J]. Suppl J R Stat Soc 9(2):218–244
Yin M, Zeng D, Gao J et al (2018) Robust multinomial logistic regression based on RPCA[J]. IEEE J Sel Top Signal Process 12(6):1144–1154
Chen L, Sun D, Toh KC (2017) A note on the convergence of ADMM for linearly constrained convex optimization problems[J]. Comput Optim Appl 66(2):327–343
Liu Q, Shen X, Gu Y (2019) Linearized ADMM for nonconvex nonsmooth optimization with convergence analysis[J]. IEEE Access 7:76131–76144
Candès EJ, Li X, Ma Y et al (2011) Robust principal component analysis[J]. Journal of the ACM (JACM) 58(3):1–37
Wright J, Ganesh A, Rao S et al (2009) Robust principal component analysis: exact recovery of corrupted low–rank matrices via convex optimization[J]. Adv Neural Inf Process Syst, vol 22
Cai JF, Candès EJ, Shen Z (2010) A singular value thresholding algorithm for matrix completion[J]. SIAM J Optim 20(4):1956–1982
Boyd S, Parikh N, Chu E et al (2011) Distributed optimization and statistical learning via the alternating direction method of multipliers[J]. Foundations and Trends® in Machine Learning 3(1):1–122
Chen C, He B, Ye Y et al (2016) The direct extension of ADMM for multi-block convex minimization problems is not necessarily convergent[J]. Math Program 155(1):57–79
Wang Z (2020) A new clustering method based on morphological operations[J]. Expert Syst Appl 145:113102
Chen L, Jiang X, Liu X et al (2020) Robust low–rank tensor recovery via nonconvex singular value minimization[J]. IEEE Trans Image Process 29:9044–9059
Wang S, Chen Y, Cen Y et al (2022) Nonconvex low–rank and sparse tensor representation for multi-view subspace clustering[J]. Appl Intell:1–14
Qin W, Wang H, Zhang F et al (2022) Low–rank high-order tensor completion with applications in visual data[J]. IEEE Trans Image Process 31:2433–2448
Halder B, Ahmed MM, Amagasa T et al (2022) Missing information in imbalanced data stream: fuzzy adaptive imputation approach[J]. Appl Intell 52(5):5561–5583