Special Issue — Neuroengineering: from Neurosciences to Computations; Guest Editors: Diego Andina, Kunihiko Fukushima, Javier Ropero Peláez and Duc Truong Pham; Research ArticlesNo Access

Structured Pyramidal Neural Networks

Alessandra M. Soares

ECOMP, Polytechnic School of Pernambuco, University of Pernambuco, Recife, Pernambuco 50720-001, Brazil

E-mail Address: [email protected]

Search for more papers by this author

Bruno J. T. Fernandes

ECOMP, Polytechnic School of Pernambuco, University of Pernambuco, Recife, Pernambuco 50720-001, Brazil

E-mail Address: [email protected]

Corresponding author.

Search for more papers by this author

, and

Carmelo J. A. Bastos-Filho

ECOMP, Polytechnic School of Pernambuco, University of Pernambuco, Recife, Pernambuco 50720-001, Brazil

E-mail Address: [email protected]

Search for more papers by this author

https://doi.org/10.1142/S0129065717500216Cited by:4 (Source: Crossref)

Abstract

The Pyramidal Neural Networks (PNN) are an example of a successful recently proposed model inspired by the human visual system and deep learning theory. PNNs are applied to computer vision and based on the concept of receptive fields. This paper proposes a variation of PNN, named here as Structured Pyramidal Neural Network (SPNN). SPNN has self-adaptive variable receptive fields, while the original PNNs rely on the same size for the fields of all neurons, which limits the model since it is not possible to put more computing resources in a particular region of the image. Another limitation of the original approach is the need to define values for a reasonable number of parameters, which can turn difficult the application of PNNs in contexts in which the user does not have experience. On the other hand, SPNN has a fewer number of parameters. Its structure is determined using a novel method with Delaunay Triangulation and k-means clustering. SPNN achieved better results than PNNs and similar performance when compared to Convolutional Neural Network (CNN) and Support Vector Machine (SVM), but using lower memory capacity and processing time.

Keywords:

References

1. A. M. Williams, K. Davids and J. G. P. Williams, Indirect Theories of Perception and Action, Visual Perception and Action in Sport, Chap. 1 eds. A. M. Williams and J. G. P. Williams (Routledge, New York, 2005), pp. 3–25. Google Scholar
2. H. Rowley, S. Baluja and T. Kanade, Neural network-based face detection, IEEE Trans. Pattern Anal. Mach. Intell. 20(1) (1998) 23–38. Crossref, Web of Science, Google Scholar
3. M. D. Ganis, C. L. Wilson and J. L. Blue, Neural network-based systems for handprint OCR applications, IEEE Trans. Image Process. 7 (1998) 1097–1112. Crossref, Medline, Web of Science, Google Scholar
4. S. Ghosh-Dastidar and H. Adeli, Improved spiking neural networks for EEG classification and epilepsy and seizure detection, Integr. Comput.-Aided Eng. 14(3) (2007) 187–212. Crossref, Web of Science, Google Scholar
5. S. Ghosh-Dastidar and H. Adeli, A new supervised learning algorithm for multiple spiking neural networks with application in epilepsy and seizure detection, Neural Netw. 22(10) (2009) 1419–1431. Crossref, Medline, Web of Science, Google Scholar
6. A. Martínez-Álvarez, R. Crespo-Cano, A. Díaz-Tahoces, S. Cuenca-Asensi, J. M. Ferrández Vicente and E. Fernández, Automatic tuning of a retina model for a cortical visual neuroprosthesis using a multiobjective optimization genetic algorithm, Int. J. Neural Syst. 26(7) (2016) 1650021. Link, Web of Science, Google Scholar
7. Y. Lecun, L. Bottou, Y. Bengio and P. Haffner, Gradient-based learning applied to document recognition, Proc. IEEE 86(11) (1998) 2278–2324. Crossref, Web of Science, Google Scholar
8. L. Barghout and L. Lee, Perceptual information processing system, U.S. Patent Application 10/618, 543, Filed July 11, Google Patents (2004). Google Scholar
9. N. R. Carlson and D. Heth, Psychology: The Science of Behaviour (Pearson Education Canada, 2010). Google Scholar
10. N. R. Luque, J. A. Garrido, J. Ralli, J. J. Laredo and E. Ros, From sensors to spikes: Evolving receptive fields to enhance sensorimotor information in a robot-arm, Int. J. Neural Syst. 22(4) (2012) 1250013. Link, Web of Science, Google Scholar
11. A. R. Garg, K. Obermayer and B. Bhaumik, Development of feedforward receptive field structure of a simple cell and its contribution to the orientation selectivity: A modeling study, Int. J. Neural Syst. 15(1–2) (2005) 55–70. Link, Web of Science, Google Scholar
12. H. Adeli and S. L. Hung, Machine Learning — Neural Networks, Genetic Algorithms, and Fuzzy Systems (John Wiley & Sons, New York, NY, USA, 1995). Google Scholar
13. S. Ghosh-Dastidar and H. Adeli, Spiking neural networks, Int. J. Neural Syst. 19(4) (2009) 295–308. Link, Web of Science, Google Scholar
14. K. Fukushima, S. Miyake and T. Ito, Neocognitron: A neural network model for a mechanism of visual pattern recognition, IEEE Trans. Syst. Man Cyber. SMC-13 (1983) 826–834. Crossref, Web of Science, Google Scholar
15. G. E. Hinton and R. R. Salakhutdinov, Reducing the dimensionality of data with neural networks, Science 313 (2006) 504–507. Crossref, Medline, Web of Science, Google Scholar
16. Y. Bengio and Y. LeCun, Scaling learning algorithms towards AI, in Large Scale Kernel Machines, eds. L. Bottou, O. Chapelle, D. DeCoste and J. Weston (MIT Press, 2007), pp. 321–360. Crossref, Google Scholar
17. D. H. Hubel, The visual cortex of the brain, Sci. Am. 209(5) (1963) 54–62. Crossref, Medline, Google Scholar
18. S. L. Phung and A. Bouzerdoum, A pyramidal neural network for visual pattern recognition, IEEE Trans. Neural Netw. 18(2) (2007) 329–343. Crossref, Medline, Web of Science, Google Scholar
19. H. K. Hartline, The response of single optic nerve fibers of the vertebrate eye to illumination of the retina, Am. J. Phisiol. 121 (1938) 400–445. Crossref, Google Scholar
20. M. Levine and J. Shefner, Fundamentals of Sensation and Perception (Oxford University Press, 2000). Google Scholar
21. K. Fukushima, Artificial vision by multi-layered neural networks: Neocognitron and its advances, Neural Netw. 37 (2013) 103–119. Crossref, Medline, Web of Science, Google Scholar
22. B. J. T. Fernandes, G. D. C. Cavalcanti and T. I. Ren, Lateral inhibition pyramidal neural network for image classification, IEEE Trans. Cybern. 43 (2013) 2082–2092. Crossref, Medline, Web of Science, Google Scholar
23. C. D. Gilbert and T. N. Wiesel, Receptive field dynamics in adult primary visual cortex, Nature 356 (1992) 150–152. Crossref, Medline, Web of Science, Google Scholar
24. M. W. Pettet and C. D. Gilbert, Dynamic changes in receptive-field size in cat primary visual cortex, Proc. Nat. Acad. Sci. U. S. A. 89 (1992) 8366–8370. Crossref, Medline, Web of Science, Google Scholar
25. Y. LeCun, B. Boser, J. S. Denker, D. Henderson, R. E. Howard, W. Hubbard and L. D. Jackel, Backpropagation applied to handwritten zip code recognition, Neural Comput. 1 (1989) 541–551. Crossref, Web of Science, Google Scholar
26. F. Scarselli, M. Gori, A. C. Tsoi, M. Hagenbuchner and G. Monfardini, The graph neural network model, IEEE Trans. Neural Netw. 20 (2009) 61–80. Crossref, Medline, Web of Science, Google Scholar
27. A. Quek, Z. Wang, J. Zhang and D. Feng, Structural image classification with graph neural networks, 2011 Int. Conf. Digital Image Computing: Techniques and Applications (IEEE, 2011), pp. 416–421. Google Scholar
28. B. Delaunay, Sur la sphére vide. A la mémoire de Georges Voronoï, Bulletin de l’Académie des Sciences de l’URSS, Classe des sciences mathématiques et naturelles 7(6) (1934) 793–800 (in French). Google Scholar
29. T. Kohonen, Self-Organizing Maps, Springer Series in Information Sciences, Vol. 30 (Springer Berlin Heidelberg, Berlin, Heidelberg, 2001). Crossref, Google Scholar
30. G. L. Foresti and F. A. Pellegrino, Automatic visual recognition of deformable objects for grasping and manipulation, IEEE Trans. Syst. Man Cybern. C, Appl. Rev. 34(3) (2004) 325–333. Crossref, Web of Science, Google Scholar
31. A. Ortiz, J. Górriz, J. Ramírez and D. Salas-Gonzalez, MR brain image segmentation by growing hierarchical SOM and probability clustering, Electron. Lett. 47(10) (2011) 585. Crossref, Web of Science, Google Scholar
32. Z. Zhao, X. Zhang and Y. Fang, Stacked multilayer self-organizing map for background modeling, IEEE Trans. Image Proc. 24 (2015) 2841–2850. Crossref, Medline, Web of Science, Google Scholar
33. A. Rauber, D. Merkl and M. Dittenbach, The growing hierarchical self-organizing map: exploratory analysis of high-dimensional data, IEEE Trans. Neural Netw. 13 (2002) 1331–1341. Crossref, Medline, Web of Science, Google Scholar
34. C. Salge and D. Polani, Information-driven organization of visual receptive fields, Adv. Complex Syst. 12(3) (2009) 311–326. Link, Web of Science, Google Scholar
35. T. Najjar and O. Hasegawa, Hebbian network of selforganizing receptive field neurons as associative incremental learner, Int. J. Comput. Intell. Appl. 14(4) (2015) 1550023. Link, Web of Science, Google Scholar
36. H. Inoue, Self-organizing neural grove: Efficient multiple classifier system with pruned self-generating neural trees, in Constructive Neural Networks (Springer Berlin Heidelberg, 2009), pp. 281–291. Crossref, Google Scholar
37. O. T. Yildiz and E. Alpaydin, Omnivariate decision trees, IEEE Trans. Neural Netw. 12(6) (2001) 1539–1546. Crossref, Medline, Web of Science, Google Scholar
38. H. Bischof, Pyramidal Neural Networks (Psychology Press, 1995). Google Scholar
39. V. Cantoni and A. Petrosino, Neural recognition in a pyramidal structure, IEEE Trans. Neural Netw. 13(2) (2002) 472–480. Crossref, Medline, Web of Science, Google Scholar
40. Z.-H. Mao and S. G. Massaquoi, Dynamics of winner-take-all competition in recurrent neural networks with lateral inhibition, IEEE Trans. Neural Netw. 18 (2007) 55–69. Crossref, Medline, Web of Science, Google Scholar
41. A. M. Soares, B. J. T. Fernandes and C. J. A. Bastos-Filho, Lateral inhibition pyramidal neural networks designed by particle swarm optimization, in Artificial Neural Networks and Machine Learning – ICANN 2014. 24th International Conference on Artificial Neural Networks, Hamburg, Germany, September 15-19, 2014. Proceedings, eds. S. Wermter, C. Weber, W. Duch, T. Honkela, P. Koprinkova-Hristova, S. Magg, G. Palm and A. E. P. Villa, Lecture Notes in Computer Science, Vol. 8681 (Springer International Publishing, 2014), pp. 667–674. Google Scholar
42. H. Adeli and S. L. Hung, A concurrent adaptive conjugate gradient learning algorithm on mimd machines, J. Supercomput. Appl. 7(2) (1993) 155–166. Crossref, Web of Science, Google Scholar
43. H. Adeli and S. L. Hung, An adaptive conjugate gradient learning algorithm for effective training of multilayer neural networks, Appl. Math. Comput. 62(1) (1994) 81–102. Crossref, Web of Science, Google Scholar
44. S. L. Hung and H. Adeli, A parallel genetic/neural network learning algorithm for MIMD shared memory machines, IEEE Trans. Neural Netw. 5(6) (1994) 900–909. Crossref, Medline, Web of Science, Google Scholar
45. J. A. Hartigan, Clustering Algorithms, 99th edn. (John Wiley & Sons Inc., New York, NY, USA, 1975). Google Scholar
46. S. Hochreiter, Untersuchungen zu dynamischen neuronalen Netzen, Diploma thesis, Institut für Informatik, Lehrstuhl Prof. Brauer, Technische Universität München (1991) (in German). Google Scholar
47. C. M. Bishop, Neural Networks for Pattern Recognition (Clarendon, 2007). Google Scholar
48. M. Riedmiller and H. Braun, A direct adaptive method for faster backpropagation learning: The rprop algorithm, in Proc. IEEE Int. Conf. Neural Networks (1993) pp. 586–591. Google Scholar
49. B. J. T. Fernandes, Redes Neurais com Extracão Implícita de Características para Reconhecimento de Padrões Visuais, Ph.D. thesis, Universidade Federal de Pernambuco (2013) (in Portuguese). Google Scholar
50. A. P. Bradley, The use of the area under the ROC curve in the evaluation of machine learning algorithms, Pattern Recogn. 30 (1997) 1145–1159. Crossref, Web of Science, Google Scholar
51. J. A. Hanley and B. J. McNeil, The meaning and use of the area under a receiver operating characteristic (ROC) curve, Radiology 143(1) (1982) 29–36. Crossref, Medline, Web of Science, Google Scholar
52. X. H. Zhou, N. A. Obuchowski and D. K. McClish, Statistical Methods in Diagnostic Medicine, Vol. 712 (Wiley, 2011). Crossref, Google Scholar
53. F. Wilcoxon, Individual comparisons by ranking methods, Biometr. Bull. 1 (1945) 80–83. Crossref, Web of Science, Google Scholar
54. J. Suykens, T. V. Gestel, J. D. Brabanter, B. D. Moor and J. Vandewalle, Least Squares Support Vector Machines (World Scientific, Singapore). Google Scholar
55. F. Chollet, Keras (2015), Available at: https://github.com/fchollet/keras. Google Scholar
56. N. Otsu, A threshold selection method from gray-level histograms, IEEE Trans. Syst. Man Cybern. 9(1) (1979) 62–66. Crossref, Web of Science, Google Scholar
57. B. Heisele, T. Poggio and M. Pontil, Face detection in still gray images, CBCL Face Database ∖#1, Massachusetts Institute of Technology, Cambridge, MA (2000). Google Scholar
58. K. Ricanek Jr. and T. Tesafaye, Morph: A longitudinal image database of normal adult age-progression, IEEE 7th Int. Conf. Automatic Face and Gesture Recognition, Southampton, UK (2006), pp. 341–345. Google Scholar
59. P. Viola and M. J. Jones, Rapid object detection using a boosted cascade of simple features, Conf. Computer Vision and Pattern Recognition (2001), pp. 1-511–1-518. Google Scholar
60. G. Bradski (2000) The OpenCV Library, Dr. Dobb’s Journal of Software Tools. Google Scholar
61. T. Gehrig, Folds for controlled condition of gender classification (MORPH distribution) (2011), Available at http://_pa.cs.kit.edu/431.php, [Accessed on 23 May 2016]. Google Scholar
62. A. Ortiz, J. Munilla, J. M. Górriz and J. Ramírez, Ensembles of deep learning architectures for the early diagnosis of the alzheimers disease, Int. J. Neural Syst. 26(7) (2016) 1650025, 27478060. Link, Web of Science, Google Scholar
63. W. d. A. S. e. Silva, Deteccão de Elementos Figurados do Sangue em Imagens Utilizando Comitês de Classificadores Tipo Adaboost, Master’s thesis, Universidade de Pernambuco (2015) (in Portuguese). Google Scholar
64. American Society of Hematology (2012), Available at http://www.hematology.org/[accessed on 3 September 2016]. Google Scholar
65. R. Labati, V. Piuri and F. Scotti, All-idb: The acute lymphoblastic leukemia image database for image processing, IEEE 18th Int. Conf. Image Processing (2011), pp. 2045–2048. Google Scholar
66. M. Ahmadlou and H. Adeli, Enhanced probabilistic neural network with local decision circles: A robust classifier, Integr. Comput.-Aided Eng. 17(3) (2010) 197–210. Crossref, Web of Science, Google Scholar