Simplified spiking neural network architecture and STDP learning algorithm applied to image classification
Tóm tắt
Spiking neural networks (SNN) have gained popularity in embedded applications such as robotics and computer vision. The main advantages of SNN are the temporal plasticity, ease of use in neural interface circuits and reduced computation complexity. SNN have been successfully used for image classification. They provide a model for the mammalian visual cortex, image segmentation and pattern recognition. Different spiking neuron mathematical models exist, but their computational complexity makes them ill-suited for hardware implementation. In this paper, a novel, simplified and computationally efficient model of spike response model (SRM) neuron with spike-time dependent plasticity (STDP) learning is presented. Frequency spike coding based on receptive fields is used for data representation; images are encoded by the network and processed in a similar manner as the primary layers in visual cortex. The network output can be used as a primary feature extractor for further refined recognition or as a simple object classifier. Results show that the model can successfully learn and classify black and white images with added noise or partially obscured samples with up to ×20 computing speed-up at an equivalent classification ratio when compared to classic SRM neuron membrane models. The proposed solution combines spike encoding, network topology, neuron membrane model and STDP learning.
Tài liệu tham khảo
Lovelace JJ, Rickard JT, Cios KJ. A spiking neural network alternative for the analog to digital converter. In: Neural Networks (IJCNN), The 2010 International Joint Conference On. New Jersey, USA: Institute of Electrical and Electronics Engineers-IEEE: 2010. p. 1–8.
Ambard M, Guo B, Martinez D, Bermak A. A spiking neural network for gas discrimination using a tin oxide sensor array. In: Electronic Design, Test and Applications, 2008. DELTA 2008. 4th IEEE International Symposium On. New Jersey, USA: Institute of Electrical and Electronics Engineers-IEEE: 2008. p. 394–397.
Bouganis A, Shanahan M. Training a spiking neural network to control a 4-dof robotic arm based on spike timing-dependent plasticity. In: Neural Networks (IJCNN), The 2010 International Joint Conference On. New Jersey, USA: Institute of Electrical and Electronics Engineers-IEEE: 2010. p. 1–8.
Alnajjar F, Murase K. Sensor-fusion in spiking neural network that generates autonomous behavior in real mobile robot. In: Neural Networks, 2008. IJCNN 2008. (IEEE World Congress on Computational Intelligence). IEEE International Joint Conference On. New Jersey, USA: Institute of Electrical and Electronics Engineers-IEEE: 2008. p. 2200–2206.
Perez-Carrasco JA, Acha B, Serrano C, Camunas-Mesa L, Serrano-Gotarredona T, Linares-Barranco B. Fast vision through frameless event-based sensing and convolutional processing: Application to texture recognition. Neural Networks IEEE Trans. 2010; 21(4):609–620.
Botzheim J, Obo T, Kubota N. Human gesture recognition for robot partners by spiking neural network and classification learning. In: Soft Computing and Intelligent Systems (SCIS) and 13th International Symposium on Advanced Intelligent Systems (ISIS), 2012 Joint 6th International Conference On. New Jersey, USA: Institute of Electrical and Electronics Engineers-IEEE: 2012. p. 1954–1958.
Ratnasingam S, McGinnity TM. A spiking neural network for tactile form based object recognition. In: The 2011 International Joint Conference on Neural Networks (IJCNN). New Jersey, USA: Institute of Electrical and Electronics Engineers-IEEE: 2011. p. 880–885.
Fang H, Wang Y, He J. Spiking neural networks for cortical neuronal spike train decoding. Neural Comput. 2009; 22(4):1060–1085.
Gerstner W, Kistler WM. Spiking Neuron Models: Single Neurons, Populations, Plasticity. Cambridge, United Kingdom: Cambridge University Press; 2002, p. 494.
Arguello E, Silva R, Castillo C, Huerta M. The neuroid: A novel and simplified neuron-model. In: 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). New Jersey, USA: Institute of Electrical and Electronics Engineers-IEEE: 2012. p. 1234–1237.
Ishikawa Y, Fukai S. A neuron mos variable logic circuit with the simplified circuit structure. In: Proceedings of 2004 IEEE Asia-Pacific Conference on Advanced System Integrated Circuits 2004. New Jersey, USA: Institute of Electrical and Electronics Engineers-IEEE: 2004. p. 436–437.
Lorenzo R, Riccardo R, Antonio C. A new unsupervised neural network for pattern recognition with spiking neurons. In: International Joint Conference on Neural Networks, 2006. IJCNN 06. New Jersey, USA: Institute of Electrical and Electronics Engineers-IEEE: 2006. p. 3903–3910.
Painkras E, Plana LA, Garside J, Temple S, Galluppi F, Patterson C, Lester DR, Brown AD, Furber SB. Spinnaker: A 1-w 18-core system-on-chip for massively-parallel neural network simulation. IEEE J. Solid-State Circuits. 2013; 48(8):1943–1953.
Schemmel J, Grubl A, Hartmann S, Kononov A, Mayr C, Meier K, Millner S, Partzsch J, Schiefer S, Scholze S, et al.Live demonstration: A scaled-down version of the brainscales wafer-scale neuromorphic system. In: 2012 IEEE International Symposium on Circuits and Systems (ISCAS). New Jersey, USA: Institute of Electrical and Electronics Engineers-IEEE: 2012. p. 702–702.
Hylton T. 2008. Systems of neuromorphic adaptive plastic scalable electronics. http://www.scribd.com/doc/76634068/Darpa-Baa-Synapse.
Schoenauer T, Atasoy S, Mehrtash N, Klar H. Neuropipe-chip: a digital neuro-processor for spiking neural networks. Neural Networks, IEEE Trans. 2002; 13(1):205–213.
Schrauwen B, Campenhout JV. Parallel hardware implementation of a broad class of spiking neurons using serial arithmetic. In: Proceedings of the 14th European Symposium on Artificial Neural Networks. Evere, Belgium: D-side conference services: 2006. p. 623–628.
Rice KL, Bhuiyan MA, Taha TM, Vutsinas CN, Smith MC. Fpga implementation of izhikevich spiking neural networks for character recognition. In: International Conference on Reconfigurable Computing and FPGAs, 2009. ReConFig 09. New Jersey, USA: Institute of Electrical and Electronics Engineers-IEEE: 2009. p. 451–456.
Xilinx. Spartan-6 family overview. Technical Report DS160, Xilinx, Inc. October 2011. http://www.xilinx.com/support/documentation/data_sheets/ds160.pdf.
Maass W. Networks of spiking neurons: The third generation of neural network models. Neural Networks. 1997; 10(9):1659–1671.
MCWV Rossum, Bi GQ, Turrigiano GG. Stable hebbian learning from spike timing-dependent plasticity. J. Neurosci. 2000; 20(23):8812–8821.
Pham DT, Packianather MS, Charles EYA. A self-organising spiking neural network trained using delay adaptation. In: Industrial Electronics, 2007. ISIE 2007. IEEE International Symposium On. New Jersey, USA: Institute of Electrical and Electronics Engineers-IEEE: 2007. p. 3441–3446.
Paugam-Moisy H, SM Bohte. Computing with Spiking Neuron Networks In: G Rozenberg, JK T Back, editors. Handbook of Natural Computing. Heidelberg, Germany: Springer: 2009.
Booij O. Temporal pattern classification using spiking neural networks. (August 2004). Available from http://obooij.home.xs4all.nl/study/download/booij04Temporal.pdf.
Bi G-Q, Poo M-M. Synaptic modifications in cultured hippocampal neurons: dependence on spike timing, synaptic strength, and postsynaptic cell type. J. Neurosci. 1998; 18(24):10464.
Martinez LM, Alonso J-M. Complex receptive fields in primary visual cortex. Neuroscientist: Rev. J Bringing Neurobiology, Neurology Psychiatry. 2003; 9(5):317–331. PMID: 14580117.
Foldiak P, Young MP. The Handbook of Brain Theory and Neural Networks. Cambridge, MA, USA: MIT Press; 1998, pp. 895–898. http://dl.acm.org/citation.cfm?id=303568.303958.
Repository UML. Semeion Handwritten Digit Dataset. 2014. http://archive.ics.uci.edu/ml/datasets/Semeion+Handwritten+Digit Accessed 2014-10-30.
Perez-Carrasco JA, Zhao B, Serrano C, Acha B, Serrano-Gotarredona T, Chen S, Linares-Barranco B. Mapping from frame-driven to frame-free event-driven vision systems by low-rate rate coding and coincidence processing–application to feedforward convnets. Pattern Anal. Machine Intelligence, IEEE Trans. 2013; 35(11):2706–2719.