Selection methods for extended least squares support vector machines

Emerald - 2008
JózsefValyon1, GáborHorváth1
1Department of Measurement and Information Systems, Budapest University of Technology and Economics, Budapest, Hungary

Tóm tắt

PurposeThe purpose of this paper is to present extended least squares support vector machines (LS‐SVM) where data selection methods are used to get sparse LS‐SVM solution, and to overview and compare the most important data selection approaches.Design/methodology/approachThe selection methods are compared based on their theoretical background and using extensive simulations.FindingsThe paper shows that partial reduction is an efficient way of getting a reduced complexity sparse LS‐SVM solution, while partial reduction exploits full knowledge contained in the whole training data set. It also shows that the reduction technique based on reduced row echelon form (RREF) of the kernel matrix is superior when compared to other data selection approaches.Research limitations/implicationsData selection for getting a sparse LS‐SVM solution can be done in the different representations of the training data: in the input space, in the intermediate feature space, and in the kernel space. Selection in the kernel space can be obtained by finding an approximate basis of the kernel matrix.Practical implicationsThe RREF‐based method is a data selection approach with a favorable property: there is a trade‐off tolerance parameter that can be used for balancing complexity and accuracy.Originality/valueThe paper gives contributions to the construction of high‐performance and moderate complexity LS‐SVMs.

Từ khóa


Tài liệu tham khảo

Baudat, G. and Anouar, F. (2001), “Kernel‐based methods and function approximation”, Proc. IJCNN, Washington, DC, July, pp. 1244‐9.

Cawley, G.C. and Talbot, N.L.C. (2002a), “Efficient formation of a basis in a kernel induced feature space”, Proceedings of the European Symposium on Artificial Neural Networks (ESANN‐2002), Bruges, Belgium, April 24‐26, pp. 1‐6.

Cawley, G.C. and Talbot, N.L.C. (2002b), “Reduced rank kernel ridge regression”, Neural Processing Letters, Vol. 16 No. 3, pp. 293‐302.

Cizek, P. (2004), “Asymptotics of least trimmed squares regression”, Discussion Paper 72, Center for Economic Research, Tilburg University, Tilburg.

Hoerl, A.E. and Kennard, R.W. (1970), “Ridge regression: biased estimation for nonorthogonal problems”, Technometrics, Vol. 12 No. 3, pp. 55‐67.

Holland, P.W. and Welsch, R.E. (1977), “Robust regression using iteratively reweighted least‐squares”, Communications in Statistics: Theory and Methods, Vol. A6, pp. 813‐27.

Huber, P.J. (1981), Robust Statistics, Wiley, New York, NY.

Lee, Y.J. and Mangasarian, O.L. (2001), “RSVM: reduced support vector machines”, Proceedings of the First SIAM International Conference on Data Mining, Chicago.

Osuna, E., Freund, R. and Girosi, F. (1997), “Improved training algorithm for support vector machines”, Proc. IEEE NNSP '97.

Press, W.H., Teukolsky, S.A., Vetterling, W.T. and Flannery, B.P. (1992), Numerical Recipes in C: The Art of Scientific Computing, 2nd ed., Cambridge University Press, Cambridge.

Schölkopf, B., Mika, S., Burges, C.J.C., Knirsch, P., Müller, K‐R., Rätsch, G. and Smola, A. (1999), “Input space vs. feature space in kernel‐based methods”, IEEE Transactions on Neural Networks, Vol. 10 No. 5, pp. 1000‐17.

Suykens, J.A.K., Lukas, L. and Vandewalle, J. (2000a), “Sparse least squares support vector machine classifiers”, ESANN'2000 European Symposium on Artificial Neural Networks, pp. 37‐42.

Suykens, J.A.K., Lukas, L. and Vandewalle, J. (2000b), “Sparse approximation using least squares support vector machines”, paper presented at IEEE International Symposium on Circuits and Systems ISCAS'2000.

Suykens, J.A.K., Gestel, V.T., de Brabanter, J., de Moor, B. and Vandewalle, J. (2002), Least Squares Support Vector Machines, World Scientific, Singapore.

Valyon, J. and Horváth, G. (2004), “A sparse least squares support vector machine classifier”, Proceedings of the International Joint Conference on Neural Networks IJCNN 2004, Budapest, pp. 543‐8.

Valyon, J. and Horváth, G. (2006), “Extended least squares LS‐SVM”, International Journal of Computational Intelligence, Vol. 3 No. 3, pp. 234‐42.

Vapnik, V. (1995), The Nature of Statistical Learning Theory, Springer, New York, NY.

Vapnik, V. (1998), Statistical Learning Theory, Wiley, New York, NY.

Golub, G.H. and van Loan, C.F. (1989), Matrix Computations, Gene Johns Hopkins University Press, Baltimore, MD.

Thông tin
Thông tin xuất bản

Emerald

Thông tin tác giả