Enforcement of the principal component analysis–extreme learning machine algorithm by linear discriminant analysis
Tóm tắt
In the majority of traditional extreme learning machine (ELM) approaches, the parameters of the basis functions are randomly generated and do not need to be tuned, while the weights connecting the hidden layer to the output layer are analytically estimated. The determination of the optimal number of basis functions to be included in the hidden layer is still an open problem. Cross-validation and heuristic approaches (constructive and destructive) are some of the methodologies used to perform this task. Recently, a deterministic algorithm based on the principal component analysis (PCA) and ELM has been proposed to assess the number of basis functions according to the number of principal components necessary to explain the 90 % of the variance in the data.
In this work, the PCA part of the PCA–ELM algorithm is joined to the linear discriminant analysis (LDA) as a hybrid means to perform the pruning of the hidden nodes. This is justified by the fact that the LDA approach is outperforming the PCA one on a set of problems. Hence, the idea of combining the two approaches in a LDA–PCA–ELM algorithm is shown to be in average better than its PCA–ELM and LDA–ELM counterparts. Moreover, the performance in classification and the number of basis functions selected by the algorithm, on a set of benchmark problems, have been compared and validated in the experimental section using nonparametric tests against a set of existing ELM techniques.
Tài liệu tham khảo
Asuncion A, Newman D (2007) UCI machine learning repository. http://www.ics.uci.edu/~mlearn/MLRepository.html
Cai D, He X, Han J (2008) Training linear discriminant analysis in linear time. In: IEEE 24th international conference on data engineering (ICDE 2008), pp 209–217
Cao J, Lin Z, Huang G (2011) Composite function wavelet neural networks with differential evolution and extreme learning machine. Neural Process Lett 33(3):251–265
Castaño A, Fernández-Navarro F, Hervás-Martínez C (2013) PCA-ELM: a robust and pruned extreme learning machine approach based on principal component analysis. Neural Process Lett 37(3):377–392
Chen L, Zhou L, Pung HK (2008) Universal approximation and QoS violation application of extreme learning machine. Neural Process Lett 28(2):81–95
Demšar J (2006) Statistical comparisons of classifiers over multiple data sets. J Mach Learn Res 7:1–30
Feng G, Huang GB, Lin Q, Gay R (2009) Error minimized extreme learning machine with growth of hidden nodes and incremental learning. IEEE Trans Neural Netw 20(8):1352–1357
Fernández-Navarro F, Hervás-Martínez C, Sánchez-Monedero J, Gutierrez PA (2011) MELM-GRBF: a modified version of the extreme learning machine for generalized radial basis function neural networks. Neurocomputing 74(16):2502–2510
Fisher R (1938) The statistical utilization of multiple measurements. Ann Eugen 8:376–386
Friedman M (1940) A comparison of alternative tests of significance for the problem of m rankings. Ann Math Stat 11(1):86–92
Hassibi B, Stork DG (1993) Second order derivatives for network pruning: optimal brain surgeon. In: Advances in neural information processing systems 5 (NIPS 1992). Morgan Kaufmann, San Mateo, CA, pp 164-171. ISBN: 1558602747
Hochberg Y, Tamhane A (1987) Multiple comparison procedures. Wiley, Hoboken
Huang GB, Chen L (2007) Convex incremental extreme learning machine. Neurocomputing 70(16–18):3056–3062
Huang GB, Chen L (2008) Enhanced random search based incremental extreme learning machine. Neurocomputing 71(16–18):3460–3468
Huang GB, Chen L, Siew CK (2006) Universal approximation using incremental constructive feedforward networks with random hidden nodes. IEEE Trans Neural Netw 17(4):879–892
Huang GB, Zhou H, Ding X, Zhang R (2012) Extreme learning machine for regression and multiclass classification. IEEE Trans Syst Man Cybern Part B 42(2):513–529
Huang GB, Zhu Q, Siew C (2006) Extreme learning machine: theory and applications. Neurocomputing 70(1–3):489–501
Huang GB, Zhu QY, Siew CK (2004) Extreme learning machine: a new learning scheme of feedforward neural networks. In: IEEE international conference on neural networks—conference proceedings, vol 2, pp 985–990
Lan Y, Soh YC, Huang GB (2010) Constructive hidden nodes selection of extreme learning machine for regression. Neurocomputing 73(16–18):3191–3199
Martínez AM, Kak AC (2001) PCA versus lDA. IEEE Trans Pattern Anal Mach Intell 23(2):228–233
Miche Y, Sorjamaa A, Bas P, Simula O, Jutten C, Lendasse A (2010) OP-ELM: optimally pruned extreme learning machine. IEEE Trans Neural Netw 21(1):158–162
Miche Y, Sorjamaa A, Lendasse A (2008) OP-ELM: theory, experiments and a toolbox. In: Artificial neural networks—ICANN 2008, lecture notes in computer science, vol 5163, Springer, Berlin, pp 145–154
Rong HJ, Ong YS, Tan AH, Zhu Z (2008) A fast pruned-extreme learning machine for classification problem. Neurocomputing 72(1–3):359–366
Sánchez-Monedero J, Gutiérrez PA, Fernández-Navarro F, Hervás-Martínez C (2011) Weighting efficient accuracy and minimum sensitivity for evolving multi-class classifiers. Neural Process Lett 34(2):101–116
Shrivastava NA, Panigrahi BK, Lim M-H (2014) Electricity price classification using extreme learning machines. Neural Comput Appl 1–10. doi:10.1007/s00521-013-1537-1
Storn R, Price K (1997) Differential evolution. A fast and efficient heuristic for global optimization over continuous spaces. J Global Optim 11:341–359
Thodberg HH (1991) Improving generalization of neural networks through pruning. Int J Neural Syst 1(04):317–326
Uçar A, Demir Y, Güzeliş C (2014) A new facial expression recognition based on curvelet transform and online sequential extreme learning machine initialized with spherical clustering. Neural Comput Appl 1–12. doi:10.1007/s00521-014-1569-1
Wong PK, Wong HC, Vong CM, Xie Z, Huang S (2014) Model predictive engine air-ratio control using online sequential extreme learning machine. Neural Comput Appl 1–14. doi:10.1007/s00521-014-1555-7
Zhu QY, Qin A, Suganthan P, Huang GB (2005) Evolutionary extreme learning machine. Pattern Recogn 38(10):1759–1763