Dimension independent data sets approximation and applications to classification
Tóm tắt
We revisit the classical kernel method of approximation/interpolation theory in a very specific context from the particular point of view of partial differential equations. The goal is to highlight the role of regularization by casting it in terms of actual smoothness of the interpolant obtained by the procedure. The latter will be merely continuous on the data set but smooth otherwise. While the method obtained fits into the category of RKHS methods and hence shares their main features, it explicitly uses smoothness, via a dimension dependent (pseudo-)differential operator, to obtain a flexible and robust interpolant, which can adapt to the shape of the data while quickly transitioning away from it and maintaining continuous dependence on them. The latter means that a perturbation or pollution of the data set, small in size, leads to comparable results in classification applications. The method is applied to both low dimensional examples and a standard high dimensioal benchmark problem (MNIST digit classification).
Tài liệu tham khảo
Wendland H. Scattered data approximations. Cambridge monographs on applied and computational mathematics. Cambridge: Cambridge University Press; 2004.
Vapnik VN, Chervonenkis AY. On a class of algorithms of learning pattern recognition. Avtomatika i Telemekhanika. 1964;25(6):In Russian.
Aizerman MA, Braverman EM, Rozonoer LI. Theoretical foundations of the potential function method in pattern recognition learning. Autom Remote Control. 1964;25:821–37.
Boser BE, Guyon IM, Vapnik VN. A training algorithm for optimal margin classifiers. In Proceedings of the fifth annual Acm workshop on Computational learning theory, pages 144. Assn for Computing Machinery. 1992.
Poggio T, Girosi F. Regularization algorithms for learning that are equivalent to multilayer networks. Science. 1990;247(4945):978–82.
Bousquet O, Elisseeff A, Stability and generalization. J Mach Learn Res. 2002;2.
Paulsen VI, Raghupathi M. An introduction to the theory of reproducing kernel Hilbert spaces. Cambridge Studies in Advanced Mathematics 152. Cambridge University Press, Cambridge. 2016.
Hastie T, Tibshirani R, Friedman J. The elements of statistical learning. Data mining, inference, and prediction. Springer Series in Statistics. Springer, New York. 2009.
Gu C, Wang Y. Grace Wahba and the Wisconsin spline school. Notices of the American Mathematical Society. 2022;69(3).