Robust semi-supervised data representation and imputation by correntropy based constraint nonnegative matrix factorization

Springer Science and Business Media LLC - Tập 53 - Trang 11599-11617 - 2022
Nan Zhou1,2,3, Yuanhua Du2, Jun Liu2,3, Xiuyu Huang3, Xiao Shen4, Kup-Sze Choi3
1Chengdu University, Chengdu, China
2Chengdu University of Information Technology, Chengdu, China
3Centre for Smart Health, The Hong Kong Polytechnic University, Kowloon, Hong Kong
4School of Computer Science and Cyberspace Security, Hainan University, Danzhou, China

Tóm tắt

Many methods have been proposed recently for high-dimensional data representation to reduce the dimensionality of the data. Matrix Factorization (MF) as an efficient dimension-reduction method is increasingly used in a wide range of applications. However, these methods are often unable to handle data with missing entries. In a Semi-Supervised Learning (SSL) scenario, many commonly used missing value imputation methods, e.g., KNN imputation, cannot utilize the existing information on the labels, which is one of the most discriminative information in the data. Considering the outliers in the observed entries, in this paper, we propose an algorithm called Correntropy based Constraint Nonnegative Matrix Factorization Completion (CCNMF) for simultaneous construction of robust representation and imputation of high-dimensional data in an SSL scenario. Specifically, the Maximum Correntropy Criterion (MCC) is used to construct the model of the CCNMF method to alleviate the negative effects of non-Gaussian noise and outliers in the data. To solve the optimization problem, an iterative algorithm based on a Fenchel Conjugate (FC) and Block Coordinate Update (BCU) framework is proposed. We show that the proposed algorithm can satisfy not only objective sequential convergence but also iterate sequence convergence. The experiments are conducted on the real-world image dataset and community health dataset. In many cases, it is shown that the proposed method outperforms several state-of-the-art methods for both representation and imputation.

Tài liệu tham khảo

Kriegel H-P, Kröger P, Zimek A (2009) Clustering high-dimensional data: a survey on subspace clustering, pattern-based clustering, and correlation clustering. ACM Trans Knowl Discov Data (tkdd) 3(1):1–58 Roweis ST, Saul LK (2000) Nonlinear dimensionality reduction by locally linear embedding. Science 290(5500):2323–2326 Ding C, He X (2004) K-means clustering and principal component analysis. In: International conf. machine learning Wang C, Zhang J, Wu T, Zhang M, Shi G (2022) Semi-supervised nonnegative matrix factorization with positive and negative label propagations. Appl Intell:1–12 Cai D, He X, Han J, Huang TS (2010) Graph regularized nonnegative matrix factorization for data representation. IEEE Trans Pattern Anal Mach Intell 33(8):1548–1560 Meng Y, Shang R, Jiao L, Zhang W, Yang S (2018) Dual-graph regularized non-negative matrix factorization with sparse and orthogonal constraints. Eng Appl Artif Intell 69:24–35 Peng S, Ser W, Chen B, Lin Z (2021) Robust semi-supervised nonnegative matrix factorization for image clustering. Pattern Recognition 111:107683 Cai D, Zhang C, He X (2010) Unsupervised feature selection for multi-cluster data. In: Proceedings of the 16th ACM SIGKDD international conference on knowledge discovery and data mining, pp 333–342 Gu Q, Li Z, Han J (2011) Joint feature selection and subspace learning Wang S, Pedrycz W, Zhu Q, Zhu W (2015) Subspace learning for unsupervised feature selection via matrix factorization. Pattern Recogn 48(1):10–19 Zhou N, Xu Y, Cheng H, Fang J, Pedrycz W (2016) Global and local structure preserving sparse subspace learning: an iterative approach to unsupervised feature selection. Pattern Recogn 53:87–101 Zhang Y, Zhang Q, Chen Z, Shang J, Wei H (2019) Feature assessment and ranking for classification with nonlinear sparse representation and approximate dependence analysis. Decis Support Syst 122:113064 Little RJ, Rubin DB (2019) Statistical analysis with missing data. Wiley, Vol 793 García-Laencina PJ, Sancho-Gómez J-L , Figueiras-Vidal AR (2010) Pattern classification with missing data: a review. Neural Comput Applic 19(2):263–282 Luo X, Zhou M, Li S, Hu L, Shang M (2019) Non-negativity constrained missing data estimation for high-dimensional and sparse matrices from industrial applications. IEEE Trans Cybernetics 50 (5):1844–1855 Schafer JL (1997) Analysis of incomplete multivariate data. CRC Press Schneider T (2001) Analysis of incomplete climate data: estimation of mean values and covariance matrices and imputation of missing values. J Climate 14(5):853–871 Gold MS, Bentler PM (2000) Treatments of missing data: a Monte Carlo comparison of rbhdi, iterative stochastic regression imputation, and expectation-maximization. Struct Equ Modeling 7(3):319–355 Troyanskaya O, Cantor M, Sherlock G, Brown P, Hastie T, Tibshirani R, Botstein D, Altman RB (2001) Missing value estimation methods for dna microarrays. Bioinformatics 17(6):520–525 Aydilek IB, Arslan A (2012) A novel hybrid approach to estimating missing values in databases using k-nearest neighbors and neural networks, International Journal of Innovative Computing. Inf Control 7(8):4705–4717 Silva-Ramírez E-L, Pino-Mejías R, López-Coello M, Cubiles-de-la Vega M-D (2011) Missing value imputation on missing completely at random data using multilayer perceptrons. Neural Netw 24 (1):121–129 Fessant F, Midenet S (2002) Self-organising map for data imputation and correction in surveys. Neural Comput Appl 10(4):300–310 Rahman MG, Islam MZ (2013) Missing value imputation using decision trees and decision forests by splitting and merging records: Two novel techniques. Knowl-Based Syst 53:51–65 Wang G, Deng Z, Choi K-S (2018) Tackling missing data in community health studies using additive ls-svm classifier. IEEE J Biomed Health Inform 22(2):579–587 Batista GE, Monard MC, et al. (2002) A study of k-nearest neighbour as an imputation method. His 87(48):251– 260 Candès EJ, Recht B (2009) Exact matrix completion via convex optimization. Found Comput Math 9(6):717 Xu Y, Yin W, Wen Z, Zhang Y (2012) An alternating direction algorithm for matrix completion with nonnegative factors. Front Math China 7(2):365–384 Chen B, Wang J, Zhao H, Zheng N, Príncipe JC (2015) Convergence of a fixed-point algorithm under maximum correntropy criterion. IEEE Signal Process Lett 22(10):1723–1727 Chen B, Xing L, Zhao H, Zheng N, Príncipe JC (2016) Generalized correntropy for robust adaptive filtering. Trans Signal Process 64(13):3376–3387 He Y, Wang F, Li Y, Qin J, Chen B (2019) Robust matrix completion via maximum correntropy criterion and half-quadratic optimization. IEEE Trans Signal Process 68:181–195 Lee DD, Seung HS (1999) Learning the parts of objects by non-negative matrix factorization. Nature 401(6755):788 Kim H, Park H (2007) Sparse non-negative matrix factorizations via alternating non-negativity-constrained least squares for microarray data analysis. Bioinformatics 23(12):1495–1502 Carmona-Saez P, Pascual-Marqui RD, Tirado F, Carazo JM, Pascual-Montano A (2006) Biclustering of gene expression data by non-smooth non-negative matrix factorization. BMC Bioinformatics 7(1):78 Xu W, Liu X, Gong Y (2003) Document clustering based on non-negative matrix factorization. In: Proceedings of the 26th annual international ACM SIGIR conference on research and development in informaion retrieval, pp 267–273 Liu H, Wu Z, Li X, Cai D, Huang TS (2011) Constrained nonnegative matrix factorization for image representation. IEEE Trans Pattern Anal Mach Intell 34(7):1299–1311 Xu Y, Yin W (2013) A block coordinate descent method for regularized multiconvex optimization with applications to nonnegative tensor factorization and completion. SIAM J Imaging Sci 6(3):1758–1789 Rockafellar RT (2015) Convex analysis Princeton University Press Cai D, He X, Wu X, Han J (2008) Non-negative matrix factorization on manifold. In: Eighth IEEE international conference on data mining, pp 63–72 Cai D, He X, Han J, Huang TS (2011) Graph regularized nonnegative matrix factorization for data representation. IEEE Trans Pattern Anal Mach Intell 33(8):1548–1560 Liu H, Yang G, Wu Z, Cai D (2014) Constrained concept factorization for image representation. IEEE Trans Cybern 44(7):1214 Guo Y, Ding G, Zhou J, Liu Q (2015) Robust and discriminative concept factorization for image representation:115–122 Zhang Z, Zhao K (2012) Low-rank matrix approximation with manifold regularization. IEEE Trans Pattern Anal Mach Intell 35(7):1717–1729 Cai D, He X, Han J (2011) Locally consistent concept factorization for document clustering. IEEE Trans Knowl Data Eng 23(6):902–913 He R, Hu B-G, Zheng W-S, Kong X-W (2011) Robust principal component analysis based on maximum correntropy criterion. Trans Image Process 20(6):1485–1494 Folstein MF, Folstein SE, McHugh PR (1975) “mini-mental state”: a practical method for grading the cognitive state of patients for the clinician. J Psychiatry Res 12(3):189–198 Cleeland C, Ryan K (1994) Pain assessment: global use of the brief pain inventory. Ann Acad Med Singapore Yesavage JA (1988) Geriatric depression scale. Psychopharmacol Bull 24(4):709–711 Smith R (1994) Validation and reliability of the elderly mobility scale. Physiotherapy 80 (11):744–747 Guigoz Y, Vellas B, Garry P (1997) Mini nutritional assessment: a practical assessment tool for grading the nutritional state of elderly patients. Facts Res Intervention Geriatr:15–60 Chan A, Lam K, Hui W, Hu W, Li J, Lai K, Chan C, Yuen M, Lam S, Wong B (2005) Validated questionnaire on diagnosis and symptom severity for functional constipation in the chinese population. Aliment Pharmacol Ther 22(5):483–488 Roper N, Logan WW, Tierney AJ (2000) The Roper-Logan-Tierney model of nursing: based on activities of living. Elsevier Health Sciences Shen X, Wang G, Kwan R. Y -C, Choi K. -S. (2020) Using dual neural network architecture to detect the risk of dementia with community health data: algorithm development and validation study. JMIR Medical Informatics 8(8):e19870 Suykens JA, Vandewalle J (1999) Least squares support vector machine classifiers. Neural Process Lett 9(3):293–300 Fawcett T (2006) An introduction to roc analysis. Pattern Recognit Lett 27(8):861–874 Jeni LA, Cohn JF, De La Torre F (2013) Facing imbalanced data–recommendations for the use of performance metrics. In: 2013 humaine association conference on affective computing and intelligent interaction. IEEE, pp 245–251