Research ArticleNo Access

Design and Implementation of a Spiking Neural Network with Integrate-and-Fire Neuron Model for Pattern Recognition

Digital Signal Processing Research Lab, Department of Electrical and Computer Engineering, Isfahan University of Technology, Isfahan 84156-83111, Iran

E-mail Address: [email protected]

Search for more papers by this author

Mohammad Reza Ahmadzadeh

Digital Signal Processing Research Lab, Department of Electrical and Computer Engineering, Isfahan University of Technology, Isfahan 84156-83111, Iran

E-mail Address: [email protected]

Corresponding author.

Search for more papers by this author

, and

Farzaneh Shayegh

Digital Signal Processing Research Lab, Department of Electrical and Computer Engineering, Isfahan University of Technology, Isfahan 84156-83111, Iran

E-mail Address: [email protected]

Search for more papers by this author

https://doi.org/10.1142/S0129065720500732Cited by:10 (Source: Crossref)

Abstract

In contrast to the previous artificial neural networks (ANNs), spiking neural networks (SNNs) work based on temporal coding approaches. In the proposed SNN, the number of neurons, neuron models, encoding method, and learning algorithm design are described in a correct and pellucid fashion. It is also discussed that optimizing the SNN parameters based on physiology, and maximizing the information they pass leads to a more robust network. In this paper, inspired by the “center-surround” structure of the receptive fields in the retina, and the amount of overlap that they have, a robust SNN is implemented. It is based on the Integrate-and-Fire (IF) neuron model and uses the time-to-first-spike coding to train the network by a newly proposed method. The Iris and MNIST datasets were employed to evaluate the performance of the proposed network whose accuracy, with 60 input neurons, was 96.33% on the Iris dataset. The network was trained in only 45 iterations indicating its reasonable convergence rate. For the MNIST dataset, when the gray level of each pixel was considered as input to the network, 600 input neurons were required, and the accuracy of the network was 90.5%. Next, 14 structural features were used as input. Therefore, the number of input neurons decreased to 210, and accuracy increased up to 95%, meaning that an SNN with fewer input neurons and good skill was implemented. Also, the ABIDE1 dataset is applied to the proposed SNN. Of the 184 data, 79 are used for healthy people and 105 for people with autism. One of the characteristics that can differentiate between these two classes is the entropy of the existing data. Therefore, Shannon entropy is used for feature extraction. Applying these values to the proposed SNN, an accuracy of 84.42% was achieved by only 120 iterations, which is a good result compared to the recent results.

Keywords:

References

1. A. Taherkhani, A. Belatreche, Y. Li, G. Cosma, L. P. Maguire and T. M. McGinnity , A review of learning in biologically plausible spiking neural networks, Neural Netw. 122 (2020) 253–272. Crossref, Medline, Web of Science, Google Scholar
2. H. Pang, J. Wang, M. J. Pérez-Jiménez and A. Riscos-Núñez , Dynamic threshold neural P systems, Knowl.-Based Syst. 163 (2019) 875–884. Crossref, Web of Science, Google Scholar
3. S. A. Mohamed, M. Othman and M. H. Afifi , A review on data clustering using spiking neural network (SNN) models, Indonesian J. Electr. Eng. Comput. Sci. 15(3) (2019) 1392–1400. Crossref, Google Scholar
4. Y. Todo, Z. Tang, H. Todo, J. Ji and K. Yamashita , Neurons with multiplicative interactions of nonlinear synapses, Int. J. Neural Syst. 29(8) (2019) 1950012. Link, Web of Science, Google Scholar
5. A. Geminiani, C. Casellato, A. Antonietti, E. D’Angelo and A. Pedrocchi , A multiple-plasticity spiking neural network embedded in a closed-loop control system to model cerebellar pathologies, Int. J. Neural Syst. 28(5) (2018) 1750017. Link, Web of Science, Google Scholar
6. S. Ghosh-Dastidar and H. Adeli , Spiking neural networks, Int. J. Neural Syst. 19(4) (2009) 295–308. Link, Web of Science, Google Scholar
7. S. Ghosh-Dastidar and H. Adeli , Third generation neural networks: Spiking neural networks, in Advances in Computational Intelligence (Springer, Germany, 2009), pp. 167–178. Crossref, Google Scholar
8. F. Galán-Prado, A. Morán, J. Font, M. Roca and Rosselló , Compact hardware synthesis of stochastic spiking neural networks, Int. J. Neural Syst. 29(8) (2019) 1950004. Link, Web of Science, Google Scholar
9. Q. Yu, H. Tang, K. C. Tan and H. Yu , A brain-inspired spiking neural network model with temporal encoding and learning, Neurocomputing 138 (2014) 3–13. Crossref, Web of Science, Google Scholar
10. G. López-Vázquez, M. Ornelas-Rodriguez, A. Espinal, J. A. Soria-Alcaraz, A. Rojas-Domínguez, H. J. Puga-Soberanes, J. M. Carpio and H. Rostro-Gonzalez , Evolutionary spiking neural networks for solving supervised classification problems, Comput. Intell. Neurosci. 2019 (2019) 4182639. Crossref, Medline, Web of Science, Google Scholar
11. S. Ghosh-Dastidar and H. Adeli , A new supervised learning algorithm for multiple spiking neural networks with application in epilepsy and seizure detection, Neural Netw. 22(10) (2009) 1419–1431. Crossref, Medline, Web of Science, Google Scholar
12. K. Whalley , Neural coding: Timing is key in the olfactory system, Nat. Rev. Neurosci. 14(7) (2013) 458. Crossref, Medline, Web of Science, Google Scholar
13. X. Lin, X. Wang and Z. Hao , Supervised learning in multilayer spiking neural networks with inner products of spike trains, Neurocomputing 237 (2017) 59–70. Crossref, Web of Science, Google Scholar
14. H. Peng and J. Wang , Coupled neural P systems, IEEE Trans. Neural Netw. Learn. Syst. 30(6) (2018) 1672–1682. Crossref, Medline, Web of Science, Google Scholar
15. J. M. Antelis and L. E. Falcón , Spiking neural networks applied to the classification of motor tasks in EEG signals, Neural Netw. 122 (2020) 130–143. Crossref, Medline, Web of Science, Google Scholar
16. H. Adeli and S. Ghosh-Dastidar , Automated EEG-Based Diagnosis of Neurological Disorders: Inventing the Future of Neurology (CRC Press, Boca Raton, 2010). Crossref, Google Scholar
17. M. Zhang, H. Qu, A. Belatreche and X. Xie , EMPD: An efficient membrane potential driven supervised learning algorithm for spiking neurons, IEEE Trans. Cogn. Dev. Syst. 10(2) (2017) 151–162. Crossref, Web of Science, Google Scholar
18. T. Wu, F. D. Bîlbîe, A. Pǎun, L. Pan and F. Neri , Simplified and yet Turing universal spiking neural P systems with communication on request, Int. J. Neural Syst. 28(8) (2018) 1850013. Link, Web of Science, Google Scholar
19. A. Pregowska, E. Kaplan and J. Szczepanski , How far can neural correlations reduce uncertainty? Comparison of information transmission rates for Markov and Bernoulli processes, Int. J. Neural Syst. 29(8) (2019) 1950003. Link, Web of Science, Google Scholar
20. X. Zhang, G. Foderaro, C. Henriquez and S. Ferrari , A scalable weight-free learning algorithm for regulatory control of cell activity in spiking neuronal networks, Int. J. Neural Syst. 28(2) (2018) 1750015. Link, Google Scholar
21. R. Hu, Q. Huang, H. Wang, J. He and S. Chang , Monitor-based spiking recurrent network for the representation of complex dynamic patterns, Int. J. Neural Syst. 29(8) (2019) 1950006. Link, Web of Science, Google Scholar
22. M. Bernert and B. Yvert , An attention-based spiking neural network for unsupervised spike-sorting, Int. J. Neural Syst. 29(8) (2019) 1850059. Link, Web of Science, Google Scholar
23. G. Fang and L. Long , A review of biologically plausible neuron models for spiking neural networks, in AIAA Infotech@Aerospace 2010 (American Institute of Aeronautics and Astronautics Inc., 2010), p. 3540. Google Scholar
24. S. Panzeri, N. Brunel, N. K. Logothetis and C. Kayser , Sensory neural codes using multiplexed temporal scales, Trends Neurosci. 33(3) (2010) 111–120. Crossref, Medline, Web of Science, Google Scholar
25. W. Singer , Synchronization of cortical activity and its putative role in information processing and learning, Ann. Rev. Physiol. 55(1) (1993) 349–374. Crossref, Medline, Web of Science, Google Scholar
26. C. von der Malsburg and E. Bienenstock , A neural network for the retrieval of superimposed connection patterns, EPL (Europhys. Lett.) 3(11) (1987) 1243. Crossref, Google Scholar
27. F. Ponulak and A. Kasinski , Introduction to spiking neural networks: Information processing, learning and applications, Acta Neurobiol. Exper. 71(4) (2011) 409–433. Crossref, Medline, Web of Science, Google Scholar
28. http://www.saedsayad.com/datasets/Iris. Google Scholar
29. V. Balasubramanian and P. Sterling , Receptive fields and functional architecture in the retina, J. Physiol. 587(12) (2009) 2753–2767. Crossref, Medline, Web of Science, Google Scholar
30. P. Martínez-Cañada, C. Morillas and F. Pelayo , A neuronal network model of the primate visual system: Color mechanisms in the retina, LGN and V1, Int. J. Neural Syst. 29(2) (2019) 1850036. Link, Web of Science, Google Scholar
31. S. M. Bohte, J. N. Kok and H. La Poutre , Error-backpropagation in temporally encoded networks of spiking neurons, Neurocomputing 48(1–4) (2002) 17–37. Crossref, Web of Science, Google Scholar
32. S. Ghosh-Dastidar and H. Adeli , Improved spiking neural networks for EEG classification and epilepsy and seizure detection, Integr. Comput.-Aided Eng. 14(3) (2007) 187–212. Crossref, Web of Science, Google Scholar
33. Z. Lin, D. Ma, J. Meng and L. Chen , Relative ordering learning in spiking neural network for pattern recognition, Neurocomputing 275 (2018) 94–106. Crossref, Web of Science, Google Scholar
34. http://www.yann.lecun.com/exdb/mnist. Google Scholar
35. M. Zhang, H. Qu, X. Xie and J. Kurths , Supervised learning in spiking neural networks with noise-threshold, Neurocomputing 219 (2017) 333–349. Crossref, Web of Science, Google Scholar
36. M. D. Tissera and M. D. McDonnell , Deep extreme learning machines: Supervised autoencoding architecture for classification, Neurocomputing 174 (2016) 42–49. Crossref, Web of Science, Google Scholar
37. P. U. Diehl and M. Cook , Unsupervised learning of digit recognition using spike-timing-dependent plasticity, Front. Comput. Neurosci. 9 (2015) 99. Crossref, Medline, Web of Science, Google Scholar
38. U. R. Babu, A. K. Chintha and Y. Venkateswarlu , Handwritten digit recognition using structural, statistical features and $k$ -nearest neighbor classifier, Int. J. Inf. Eng. Electron. Bus. 1 (2014) 62–68. Google Scholar
39. Y. Xie and Y. Liu , A handwritten numeral recognition method based on STDP based with unsupervised learning, in 2017 Int. Conf. Wireless Communications, Signal Processing and Networking (WiSPNET), (IEEE, 2017), pp. 839–842. Crossref, Google Scholar
40. J. M. Brader, W. Senn and S. Fusi , Learning real-world stimuli in a neural network with spike-driven synaptic dynamics, Neural Comput. 19(11) (2007) 2881–2912. Crossref, Medline, Web of Science, Google Scholar
41. S. Nazari , Spiking pattern recognition using informative signal of image and unsupervised biologically plausible learning, Neurocomputing 330 (2019) 196–211. Crossref, Web of Science, Google Scholar
42. http://fcon_1000.projects.nitrc.org/indi/abide/abide_I.html. Google Scholar
43. Z. Shakeri, F. Shayegh and M. Kamali , Investigating the extent of changes in the functional connections of the brain network over time for diagnosis Autism from fMRI images, 28th Iranian Conference on Electrical Engineering (ICEE) (2020). Google Scholar