Dynamics of learning near singularities in radial basis function networks

Amari, 1967, Theory of adaptive pattern classifiers, IEEE Transactions on Electronic Computers, EC-16, 299, 10.1109/PGEC.1967.264666 Amari, 2006, Singularities affect dynamics of learning in neuromanifolds, Neural Computation, 18, 1007, 10.1162/neco.2006.18.5.1007 Biehl, 1995, Learning by online gradient descent, Journal of Physics A-Mathematical and General, 28, 643, 10.1088/0305-4470/28/3/018 Cousseau, F., Ozeki, T., & Amari, S. (2007). Dynamics of learning in multilayer perceptrons near the singularity. IEEE Transactions on Neural Networks (in press) Freeman, 1997, Dynamics of on-line learning in radial basis function networks, Physical Review E, 56, 907, 10.1103/PhysRevE.56.907 Freeman, 1997, Online learning in radial basis function networks, Neural Computation, 9, 1601, 10.1162/neco.1997.9.7.1601 Fukumizu, 1996, A regularity condition of information matrix of a multilayer perceptron network, Neural Networks, 9, 871, 10.1016/0893-6080(95)00119-0 Fukumizu, 2000, Local minima and plateaus in hierarchical structure of multilayer perceptrons, Neural Networks, 13, 317, 10.1016/S0893-6080(00)00009-5 Geman, 1980, Some averaging and stability results for random differential equations, SIAM Journal of Applied Mathematics, 36, 86, 10.1137/0136009 Horn, 1985 Ostrowski, 1959, A quantitative formulation of Sylvester’s law of inertia, Proceedings of the National Academy of Sciences, 45, 740, 10.1073/pnas.45.5.740 Park, 2005, Slow dynamics due to singularities of hierarchical learning machines, Progress of Theoretical Physics Supplement, 157, 275, 10.1143/PTPS.157.275 Saad, 1995, On-line learning in soft committee machines, Physical Review E, 52, 4225, 10.1103/PhysRevE.52.4225 Wei, 2008, Dynamics of learning near singularities in layered networks, Neural Computation, 20, 813, 10.1162/neco.2007.12-06-414