Nonlinear neural networks: Principles, mechanisms, and architectures

Ackley, 1985, A learning algorithm for Boltzmann machines, Cognitive Science, 9, 147, 10.1207/s15516709cog0901_7 Albus, 1971, A theory of cerebellar function, Mathematical Biosciences, 10, 25, 10.1016/0025-5564(71)90051-4 1982 Amari, 1978, Mathematical theory on formation of category detecting nerve cells, Biological Cybernetics, 29, 127, 10.1007/BF00337348 Amit, 1987, Information storage in neural networks with low levels of activity, Physical Review A, 35, 2293, 10.1103/PhysRevA.35.2293 Anderson, 1968, A memory model utilizing spatial correlation functions, Kybernetik, 5, 113, 10.1007/BF00288902 Anderson, 1983, Cognitive and psychological computation with neural models, IEEE Transactions, SMC-13, 799 Anderson, 1977, Distinctive features, categorical perception, and probability learning: Some applications of a neural model, Psychological Review, 84, 413, 10.1037/0033-295X.84.5.413 Arend, 1971, Difference information in brightness perception, Perception and Psychophysics, 9, 367, 10.3758/BF03212669 Banquet, J.-P., & Grossberg, S. (in press). Probing cognitive processes through the structure of event-related potentials during learning: An experimental and theoretical analysis. Applied Optics. Beck, 1972 Bienenstock, 1982, Theory for the development of neuron selectivity: Orientation specificity and binocular interaction in visual cortex, Journal of Neuroscience, 2, 32, 10.1523/JNEUROSCI.02-01-00032.1982 Boring, 1950 Brindley, 1964, The use made by the cerebellum of the information that it receives from sense organs, International Brain Research Organization Bulletin, 3, 80 Bullock, D., & Grossberg, S. (in press-a). Neural dynamics of planned arm movements: Emergent invariants and speed-accuracy properties during trajectory formation. Psychological Review. Bullock, D., & Grossberg, S. (in press-b). Neuromuscular realization of planned trajectories: Adaptive and automatic mechanisms. Caianiello, 1961, Outline of a theory of thought and thinking machines, Journal of Theoretical Biology, 1, 204, 10.1016/0022-5193(61)90046-7 Campbell, 1882 Carpenter, 1977, A geometric approach to singular perturbation problems with applications to nerve impulse equations, Journal of Differential Equations, 23, 335, 10.1016/0022-0396(77)90116-4 Carpenter, 1977, Periodic solutions of nerve impulse equations, Journal of Mathematical Analysis and Applications, 58, 152, 10.1016/0022-247X(77)90235-9 Carpenter, 1979, Bursting phenomena in excitable membranes, SIAM Journal on Applied Mathematics, 36, 334, 10.1137/0136027 Carpenter, 1981, Normal and abnormal signal patterns in nerve cells, 49 Carpenter, 1985, Category learning and adaptive pattern recognition: A neural network model, 37 Carpenter, 1987, A massively parallel architecture for a self-organizing neural pattern recognition machine, Computer Vision, Graphics, and Image Processing, 37, 54, 10.1016/S0734-189X(87)80014-2 Carpenter, 1987, Associative learning, adaptive pattern recognition, and cooperative-competitive decision making by neural networks, 218 Carpenter, G. A., & Grossberg, S. (in press). ART 2: Self-organization of stable category recognition codes for analog input patterns. Applied Optics. Cohen, 1983, Absolute stability of global pattern formation and parallel memory storage by competitive neural networks, IEEE Transactions, SMC-13, 815 Cohen, 1984, Neural dynamics of brightness perception: Features, boundaries, diffusion, and resonance, Perception and Psychophysics, 36, 428, 10.3758/BF03207497 Cohen, 1986, Neural dynamics of speech and language coding: Developmental programs, perceptual grouping, and competition for short term memory, Human Neurobiology, 5, 1 Cohen, 1987, Masking fields: A massively parallel neural architecture for learning, recognizing, and predicting multiple groupings of patterned data, Applied Optics, 26, 1866, 10.1364/AO.26.001866 Cornsweet, 1970 Cubeddu, 1983, Frequency-dependent effects of neuronal uptake inhibitors on the autoreceptor-mediated modulation of dopamine and acetylcholine release from the rabbit striatum, Journal of Pharmacology and Experimental Therapeutics, 226, 88 1986 Dubocovich, 1982, Modulation of the stimulation-evoked release of 3H-dopamine through activation of dopamine autoreceptors of the D-2 subtype in the isolated rabbit retina, 10.1016/B978-0-08-027391-4.50032-0 Duda, 1973 Eigen, 1978, The hypercycle: A principle of natural self-organization, B: The abstract hypercycle, Naturwissenschaften, 65, 7, 10.1007/BF00420631 Ellias, 1975, Pattern formation, contrast control, and oscillations in the short term memory of shunting on-center off-surround networks, Biological Cybernetics, 20, 69, 10.1007/BF00327046 Elman, 1977, Perceptual switching in bilinguals, Journal of the Acoustical Society of America, 62, 971, 10.1121/1.381591 Ermentrout, 1979, Temporal oscillations in neuronal nets, Journal of Mathematical Biology, 7, 265, 10.1007/BF00275728 Ermentrout, 1980, Large scale spatially organized activity in neural nets, SIAM Journal on Applied Mathematics, 38, 1, 10.1137/0138001 Feldman, 1982, Connectionist models and their properties, Cognitive Science, 6, 205, 10.1207/s15516709cog0603_1 Freeman, 1975 Freeman, 1979, EEG analysis gives model of neuronal template-matching mechanism for sensory search with olfactory bulb, Biological Cybernetics, 35, 221, 10.1007/BF00344205 Fujita, 1982, Simulation of adaptive modification of the vestibulo-ocular reflex with an adaptive filter model of the cerebellum, Biological Cybernetics, 45, 207, 10.1007/BF00336193 Fujita, 1982, Adaptive filter model of the cerebellum, Biological Cybernetics, 45, 195, 10.1007/BF00336192 Geman, 1983, Stochastic relaxation methods for image restoration and expert systems Geman, 1984, Stochastic relaxation methods for image restoration and expert systems Geman, 1984, Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images, IEEE Transactions on Pattern Analysis and Machine Intelligence, 6, 721, 10.1109/TPAMI.1984.4767596 Gilpin, 1973, Global models of growth and competition, 70, 3590 Glazebrook, 1905 Goh, 1977, Stability in Gilpin and Ayala's models of competition, Journal of Mathematical Biology, 4, 275, 10.1007/BF00280977 Golden, 1986, The “Brain-state-in-a-box” neural model is a gradient descent algorithm, Journal of Mathematical Psychology, 30, 73, 10.1016/0022-2496(86)90043-X Gray, 1984, Vector quantization, IEEE ASSP Magazine, 4, 10.1109/MASSP.1984.1162229 Grossberg, 1964 Grossberg, 1967, Nonlinear difference-differential equations in prediction and learning theory, 58, 1329 Grossberg, 1968, Some physiological and biochemical consequences of psychological postulates, 60, 758 Grossberg, 1968, Some nonlinear networks capable of learning a spatial pattern of arbitrary complexity, 59, 368 Grossberg, 1969, On learning and energy-entropy dependence in recurrent and nonrecurrent signed networks, Journal of Statistical Physics, 1, 319, 10.1007/BF01007484 Grossberg, 1969, Some networks that can learn, remember, and reproduce any number of complicated space-time patterns, I, Journal of Mathematics and Mechanics, 19, 53 Grossberg, 1969, On the serial learning of lists, Mathematical Biosciences, 4, 201, 10.1016/0025-5564(69)90014-5 Grossberg, 1969, On learning of spatiotemporal patterns by networks with ordered sensory and motor components, I: Excitatory components of the cerebellum, Studies in Applied Mathematics, 48, 105, 10.1002/sapm1969482105 Grossberg, 1970, Some networks that can learn, remember, and reproduce any number of complicated space-time patterns, II, Studies in Applied Mathematics, 49, 135, 10.1002/sapm1970492135 Grossberg, 1970, Neural pattern discrimination, Journal of Theoretical Biology, 27, 291, 10.1016/0022-5193(70)90143-8 Grossberg, 1971, On the dynamics of operant conditioning, Journal of Theoretical Biology, 33, 225, 10.1016/0022-5193(71)90064-6 Grossberg, 1971, Pavlovian pattern learning by nonlinear neural networks, 68, 828 Grossberg, 1972, A neural theory of punishment and avoidance, II: Quantitative theory, Mathematical Biosciences, 15, 253, 10.1016/0025-5564(72)90038-7 Grossberg, 1972, Neural expectation: Cerebellar and retinal analogs of cells fired by learnable or unlearned pattern classes, Kybernetik, 10, 49, 10.1007/BF00288784 Grossberg, 1972, Pattern learning by functional-differential neural networks with arbitrary path weights, 121 Grossberg, 1973, Contour enhancement, short term memory, and constancies in reverberating neural networks, Studies in Applied Mathematics, 52, 217, 10.1002/sapm1973523213 Grossberg, 1974, Classical and instrumental learning by neural networks Grossberg, 1976, Adaptive pattern classification and universal recoding, I: Parallel development and coding of neural feature detectors, Biological Cybernetics, 23, 121, 10.1007/BF00344744 Grossberg, 1976, Adaptive pattern classification and universal recoding, II: Feedback, expectation, olfaction, and illusions, Biological Cybernetics, 23, 187 Grossberg, 1977, Pattern formation by the global limits of a non-linear competitive interaction in n dimensions, Journal of Mathematical Biology, 4, 237, 10.1007/BF00280974 Grossberg, 1978, A theory of human memory: Self-organization and performance of sensory-motor codes, maps, and plans, Vol. 5, 233 Grossberg, 1978, Do all neural models really look alike?, Psychological Review, 85, 592, 10.1037/0033-295X.85.6.592 Grossberg, 1978, Competition, decision, and consensus, Journal of Mathematical Analysis and Applications, 66, 470, 10.1016/0022-247X(78)90249-4 Grossberg, 1978, Decisions, patterns, and oscillations in non-linear competitive systems with applications to Volterra-Lotka systems, Journal of Theoretical Biology, 73, 101, 10.1016/0022-5193(78)90182-0 Grossberg, 1980, Biological competition: Decision rules, pattern formation, and oscillations, 77, 2338 Grossberg, 1980, How does a brain build a cognitive code?, Psychological Review, 87, 1, 10.1037/0033-295X.87.1.1 1981, 107 Grossberg, 1982 Grossberg, 1984, Unitization, automaticity, temporal order, and word recognition, Cognition and Brain Theory, 7, 263 Grossberg, 1986, The adaptive self-organization of serial order in behavior: Speech, language, and motor control, 187 Grossberg, 1987, Cortical dynamics of three-dimensional form, color, and brightness perception, I: Monocular theory, Perception and Psychophysics, 41, 87, 10.3758/BF03204874 Grossberg, 1987, Cortical dynamics of three-dimensional form, color, and brightness perception, II: Binocular theory, Perception and Psychophysics, 41, 117, 10.3758/BF03204875 1987 1987 Grossberg, 1987, Competitive learning: From interactive activation to adaptive resonance, Cognitive Science, 11, 23, 10.1111/j.1551-6708.1987.tb00862.x 1988 Grossberg, 1986 Grossberg, 1975, Some developmental and attentional biases in the contrast enhancement and short term memory of recurrent neural networks, Journal of Theoretical Biology, 53, 341, 10.1016/S0022-5193(75)80009-9 Grossberg, 1985, Neural dynamics of form perception: Boundary completion, illusory figures, and neon color spreading, Psychological Review, 92, 173, 10.1037/0033-295X.92.2.173 Grossberg, 1985, Neural dynamics of perceptual grouping: Textures, boundaries, and emergent segmentations, Perception and Psychophysics, 38, 141, 10.3758/BF03198851 Grossberg, 1987, Neural dynamics of surface perception: Boundary webs, illuminants, and shape-from-shading, Computer Vision, Graphics, and Image Processing, 37, 116, 10.1016/S0734-189X(87)80015-4 Grossberg, 1971, Spiking threshold and overarousal effects in serial learning, Journal of Statistical Physics, 3, 95, 10.1007/BF01019845 Grossberg, 1987, Neural dynamics of Pavlovian conditioning: Conditioned reinforcement, habituation, and opponent processing, Psychobiology, 15, 195 Grossberg, S., & Todorović, D. (in press). Neural dynamics of 1-D and 2-D brightness perception: A unified model of classical and recent phenomena. Perception and Psychophysics. Groves, 1981, Changes in dopaminergic terminal excitability induced by amphetamine and hatoperidol, Brain Research, 221, 425, 10.1016/0006-8993(81)90795-2 Groves, 1983, Neuronal mechanisms of action of amphetamine, 81 Halgren, 1980, Endogenous potentials generated in the human hippocampal formation and amygdala by infrequent events, Science, 210, 803, 10.1126/science.7434000 Hary, 1982, Categorical results do not imply categorical perception, Perception and Psychophysics, 32, 409, 10.3758/BF03202770 Hastings, 1976, The existence of periodic solutions to Nagumo's equation, Quarterly Journal of Mathematics, 27, 123, 10.1093/qmath/27.1.123 Hastings, 1982, Single and multiple pulse waves for the FitzHugh-Nagumo equations, SIAM Journal of Applied Mathematics, 42, 247, 10.1137/0142018 Hebb, 1949 Hecht-Nielsen, 1986, Performance limits of optical, electro-optical, and electronic neurocomputers, 277 Helmholtz, 1962 Hestenes, 1987, How the brain works: The next great scientific revolution Hilgard, 1975 1981 Hirsch, 1982, Systems of differential equations that are competitive or cooperative, I: Limit sets, SIAM Journal of Mathematical Analysis, 13, 167, 10.1137/0513013 Hirsch, 1985, Systems of differential equations that are competitive or cooperative, II: Convergence almost everywhere, SIAM Journal of Mathematical Analysis, 16, 423, 10.1137/0516030 Hodgkin, 1964 Hodgkin, 1952, A quantitative description of membrane current and its application to conduction and excitation in nerve, Journal of Physiology, 117, 500, 10.1113/jphysiol.1952.sp004764 1983 Holst, 1950, The reafference principle: Interaction between the central nervous system and the periphery, Naturwissenschaften, 37, 464, 10.1007/BF00622503 Hopfield, 1982, Neuronal networks and physical systems with emergent collective computational abilities, 79, 2554 Hopfield, 1984, Neurons with graded response have collective computational properties like those of two-state neurons, 81, 3058 Hopfield, 1985, “Neural” computation of decisions in optimization problems, Biological Cybernetics, 52, 141 Hopfield, 1986, Computing with neural circuits: A model, Science, 233, 625, 10.1126/science.3755256 Hurvich, 1981 Ito, 1974, The control mechanisms of cerebellar motor systems, 293 Ito, 1982, Cerebellar control of the vestibulo-ocular reflex—Around the flocculus hypothesis, Annual Review of Neuroscience, 5, 275, 10.1146/annurev.ne.05.030182.001423 Ito, 1984 Iverson, 1981, Invariant properties of masking phenomena in psychoacoustics and their theoretical consequences Kaczmarek, 1977, Spatiotemporal patterns in epileptic seizures, Biological Cybernetics, 26, 199, 10.1007/BF00366591 Kandel, 1981 Kasamatsu, 1976, Depletion of brain catecholamines: Failure of ocular dominance shift after monocular occlusion in kittens, Science, 194, 206, 10.1126/science.959850 Katz, 1966 Kilmer, 1972, On some realistic constraints in prey-predator mathematics, Journal of Theoretical Biology, 36, 9, 10.1016/0022-5193(72)90173-7 Kirkpatrick, 1982 Kirkpatrick, 1983, Optimization by simulated annealing, Science, 220, 671, 10.1126/science.220.4598.671 Koenigsberger, 1906 Kohonen, 1971, A class of randomly organized associative memories, Acta Polytechnica Scandinavica Kohonen, 1977 Kohonen, 1984 Kosko, 1987 Kosko, B., & Guest, C. (in press). Optical bidrectional associative memories. Society for Photo-optical and Instrumentation Engineering (SPIE) Proceedings: Image Understanding. 758. Land, 1977, The retinex theory of color vision, Scientific American, 237, 108, 10.1038/scientificamerican1277-108 Levine, 1979, Existence of a limiting pattern for a system of nonlinear equations describing inter-population competition, Bulletin of Mathematical Biology, 41, 617, 10.1007/BF02462418 Levine, 1983, Neural population modeling and psychology: A review, Mathematical Biosciences, 66, 1, 10.1016/0025-5564(83)90077-9 Levine, 1976, Visual illusions in neural networks: Line neutralization, tilt aftereffects, and angle expansion, Journal of Theoretical Biology, 61, 477, 10.1016/0022-5193(76)90031-X Levy, 1985, Associative changes at the synapse: LTP in the hippocampus, 5 Levy, 1983, Partial quantification of the associative synaptic learning rule of the dentate gyrus, Neuroscience, 8, 799, 10.1016/0306-4522(83)90011-8 Levy, 1985, The rules of elemental synaptic plasticity, 105 Lotka, 1956 MacArthur, 1970, Species packing and competitive equilibrium for many species, Theoretical Population Biology, 1, 1, 10.1016/0040-5809(70)90039-0 Mach, 1914 Malsburg, 1973, Self-organization of orientation sensitive cells in the striate cortex, Kybernetik, 14, 85, 10.1007/BF00288907 Marr, 1969, A theory of cerebellar cortex, Journal of Physiology (London), 202, 437, 10.1113/jphysiol.1969.sp008820 May, 1975, Nonlinear aspects of competition between three species, SIAM Journal on Applied Mathematics, 29, 243, 10.1137/0129022 McClelland, 1987 McClelland, 1981, An interactive activation model of context effects in letter perception, Part I: An account of basic findings, Psychological Review, 88, 375, 10.1037/0033-295X.88.5.375 1986, Vol. II McCormick, 1984, Cerebellum: Essential involvement in the classically conditioned eyelid response, Science, 223, 296, 10.1126/science.6701513 McCulloch, 1943, A logical calculus of the ideas immanent in nervous activity, Bulletin of Mathematical Biophysics, 5, 115, 10.1007/BF02478259 McEliece, 1986, The capacity of the Hopfield associative memory, IEEE Transactions in Information Theory Miller, 1979, Some effects of later-occurring information on the perception of stop consonant and semivowel, Perception and Psychophysics, 25, 457, 10.3758/BF03213823 1983 Mueller, 1962, General principles of operations in neuron nets with application to acoustical pattern recognition, Vol. 1, 192 1960 Niedzwiecki, 1984, Greater potency of mesoridazine and sulforidazine compared with the parent compound, thioridazine, on striatal dopamine autoreceptors, Journal of Pharmacology and Experimental Therapeutics, 228, 636 Optican, 1980, Cerebellar-dependent adaptive control of primate saccadic system, Journal of Neurophysiology, 44, 1058, 10.1152/jn.1980.44.6.1058 Parker, 1982, Learning-logic Parker, 1985 Parker, 1986, A comparison of algorithms for neuron-like cells, 327 Pastore, 1981, Possible psychoacoustic factors in speech perception, 165 Pettigrew, 1978, Local perfusion of noradrenaline maintains visual cortical plasticity, Nature, 271, 761, 10.1038/271761a0 Platt, 1986, Analog decoding using neural networks, 364 Plonsey, 1969 Poggio, 1987, Synapses that compute motion, Scientific American, 256, 46, 10.1038/scientificamerican0587-46 Psaltis, 1986, Nonlinear discriminative functions and associative memories Ratliff, 1965 Ratliff, 1963, Spatial and temporal aspects of retinal inhibitory interactions, Journal of the Optical Society of America, 53, 110, 10.1364/JOSA.53.000110 Rauschecker, 1979, Changes in the circuitry of the kitten's visual cortex are gated by postsynaptic activity, Nature, 280, 58, 10.1038/280058a0 Rescorla, 1972, A theory of Pavlovian conditioning: Variations in the effectiveness of reinforcement and nonreinforcement 1982 Ron, 1973, Eye movements evoked by cerebellar stimulation in the alert monkey, Journal of Neurophysiology, 36, 1004, 10.1152/jn.1973.36.6.1004 Rosenblatt, 1962 Rumelhart, 1986, Learning internal representations by error propagation Rumelhart, 1982, An interactive activation model of context effects in letter perception. Part 2: The contextual enhancement effect and some tests and extensions of the model, Psychological Review, 89, 60, 10.1037/0033-295X.89.1.60 1986, Vol. 1 Rumelhart, 1985, Feature discovery by competitive learning, Cognitive Science, 9, 75, 10.1207/s15516709cog0901_5 Samuel, 1982, Length effects in word perception: We is better than I but worse than you or them, Journal of Experimental Psychology: Human Perception and Performance, 8, 91, 10.1037/0096-1523.8.1.91 Samuel, 1983, Reply to Matthei: We really is worse than you or them, and so are ma and pa, Journal of Experimental Psychology: Human Perception and Performance, 9, 321, 10.1037/0096-1523.9.2.321 Sawusch, 1979, Contextual effects in vowel perception, I: Anchor-induced contrast effects, Perception and Psychophysics, 25, 292, 10.3758/BF03198808 Sawusch, 1980, Contextual effects in vowel perception, II: Evidence for two processing mechanisms, Perception and Psychophysics, 27, 421, 10.3758/BF03204460 Schwab, 1981, The role of second formant transitions in the stop-semivowel distinction, Perception and Psychophysics, 29, 121, 10.3758/BF03207275 Scott, 1977 Sejnowski, 1986 Siever, 1984, Regulations of amine neurotransmitter systems: Implication for the major psychiatric syndromes and their treatment, Psychopharmacology Bulletin, 20, 500 Singer, 1982, The role of attention in developmental plasticity, Human Neurobiology, 1, 41 Singer, 1983, Neuronal activity as a shaping factor in the self-organization of neuron assemblies, 89 Sperling, 1981, Mathematical models of binocular vision, 281 Sperling, 1968, Model for visual luminance discrimination and flicker detection, Journal of the Optical Society of America, 58, 1133, 10.1364/JOSA.58.001133 Studdert-Kennedy, 1980, Speech perception, Language and Speech, 23, 45, 10.1177/002383098002300106 Sutton, 1981, Toward a modern theory of adaptive networks: Expectation and prediction, Psychological Review, 88, 135, 10.1037/0033-295X.88.2.135 Szu, 1986, Three layers of vector outer product neural networks for optical pattern recognition, 312 Takeuchi, 1978, The stability of generalized Volterra equations, Journal of Mathematical Analysis and Applications, 62, 453, 10.1016/0022-247X(78)90139-7 Tepper, 1984, Autoreceptor-mediated changes in dopaminergic terminal excitability: Effects of increases in impulse flow, Brain Research, 309, 309, 10.1016/0006-8993(84)90598-5 Venkatesh, 1986, Epsilon capacity of neural networks Vilis, 1986, A comparison of disorders in saccades and in fast and accurate elbow flexions during cerebellar dysfunction Vilis, 1983, Cerebellar saccadic dysmetria is not equal in the two eyes, Experimental Brain Research, 51, 343, 10.1007/BF00237871 Werblin, 1971, Adaptation in a vertebrate retina: Intracellular recordings in Necturus, Journal of Neurophysiology, 34, 228, 10.1152/jn.1971.34.2.228 Werbos, 1974, Beyond regression: New tools for prediction and analysis in the behavioral sciences Werbos, 1982, Applications of advances in nonlinear sensitivity analysis, 10.1007/BFb0006203 Widrow, 1962, Generalization and information storage in networks of Adaline neurons Wilson, 1972, Excitatory and inhibitory interactions in localized populations of model neurons, Biophysical Journal, 12, 1, 10.1016/S0006-3495(72)86068-5