World Scientific
  • Search
    Quick Search in Journals
    Access type:
    Quick Search anywhere
    Access type:
    Advanced Search
  • My Cart
  •   
Skip main navigation

Cookies Notification

We use cookies on this site to enhance your user experience. By continuing to browse the site, you consent to the use of our cookies. Learn More
×

System Upgrade on Tue, May 28th, 2024 at 2am (EDT)

Existing users will be able to log into the site and access content. However, E-commerce and registration of new users may not be available for up to 12 hours.
For online purchase, please visit us again. Contact us at [email protected] for any enquiries.
Research ArticleOpen Access

A Cerebellum-Inspired Learning Approach for Adaptive and Anticipatory Control

    https://doi.org/10.1142/S012906571950028XCited by:15 (Source: Crossref)
    Next

    Abstract

    The cerebellum, which is responsible for motor control and learning, has been suggested to act as a Smith predictor for compensation of time-delays by means of internal forward models. However, insights about how forward model predictions are integrated in the Smith predictor have not yet been unveiled. To fill this gap, a novel bio-inspired modular control architecture that merges a recurrent cerebellar-like loop for adaptive control and a Smith predictor controller is proposed. The goal is to provide accurate anticipatory corrections to the generation of the motor commands in spite of sensory delays and to validate the robustness of the proposed control method to input and physical dynamic changes. The outcome of the proposed architecture with other two control schemes that do not include the Smith control strategy or the cerebellar-like corrections are compared. The results obtained on four sets of experiments confirm that the cerebellum-like circuit provides more effective corrections when only the Smith strategy is adopted and that minor tuning in the parameters, fast adaptation and reproducible configuration are enabled.

    References

    • 1. A. J. Bastian, Learning to predict the future: The cerebellum adapts feedforward movement control, Curr. Opin. Neurobiol. 16(6) (2006) 645–649, Motor systems/Neurobiology of behavior. Medline, Web of Science, Google Scholar
    • 2. E. D’Angelo, A. Antonietti, S. Casali, C. Casellato, J. A. Garrido, N. R. Luque, L. Mapelli, S. Masoli, A. Pedrocchi, F. Prestori, M. F. Rizza and E. Ros, Modeling the cerebellar microcircuit: New strategies for a long-standing issue, Front. Cell. Neurosci. 10 (2016) 176. Medline, Web of Science, Google Scholar
    • 3. M. Manto, J. Bower, A. Conforto, J. Delgado-García, S. Da Guarda, M. Gerwig, C. Habas, N. Hagura, R. Ivry, P. Marien, M. Molinari, E. Naito, D. Nowak, N. Ben Taib, D. Pelisson, C. Tesche, C. Tilikete and D. Timmann, Consensus paper: Roles of the cerebellum in motor control-the diversity of ideas on cerebellar involvement in movement, Cerebellum 11(6) (2012) 457–487. Medline, Web of Science, Google Scholar
    • 4. H. Lalazar and E. Vaadia, Neural basis of sensorimotor learning: Modifying internal models, Current Opinion Neurobiol. 18(6) (2008) 573–581. Medline, Web of Science, Google Scholar
    • 5. T. J. Ebner, Cerebellum and Internal Models, Handbook of the Cerebellum and Cerebellar Disorders, eds. M. Manto, J. D. Schmahmann, F. Rossi, D. L. Gruol and N. Koibuchi (Springer Netherlands, Dordrecht, 2013), pp. 1279–1295. Google Scholar
    • 6. T. Ishikawa, S. Tomatsu, J. Izawa and S. Kakei, The cerebro-cerebellum: Could it be loci of forward models?, Neurosci. Res. 104 (2016) 72–79. Medline, Web of Science, Google Scholar
    • 7. D. M. Wolpert and Z. Ghahramani, Computational principles of movement neuroscience, Nat Neurosci 3 (2000) 1212–1217. Medline, Web of Science, Google Scholar
    • 8. A. Giovannucci, A. Badura, B. Deverett, F. Najafi, T. Pereira, Z. Gao, I. Ozden, A. Kloth, E. Pnevmatikakis, P. L, C. De Zeeuw, J. Medina and S. Wang, Cerebellar granule cells acquire a widespread predictive feedback signal during motor learning, Nat. Neurosci. 20(5) (2017) 727–734. Medline, Web of Science, Google Scholar
    • 9. M. Ito, The Cerebellum: Brain for an Implicit Self (FT Press, Upper Saddle River, N.J., 2012). Google Scholar
    • 10. R. C. Miall, D. J. Weir, D. Wolpert and J. F. Stein, Is the cerebellum a smith predictor?, J. Motor behav. 25(3) (1993) 203–216. Medline, Web of Science, Google Scholar
    • 11. O. Smith, Feedback Control Systems McGraw-Hill series in Control Systems Engineering (McGraw-Hill, 1958). Google Scholar
    • 12. J. Porrill, P. Dean and S. R. Anderson, Adaptive filters and internal models: Multilevel description of cerebellar function, Neural Netw. 47 (2013) 134–149. Medline, Web of Science, Google Scholar
    • 13. S. Tolu, M. Vanegas, J. A. Garrido, N. R. Luque and E. Ros, Adaptive and predictive control of a simulated robot arm, Int. J. Neural Syst. 23(3) (2013) 1350010. Link, Web of Science, Google Scholar
    • 14. M. Pacheco, R. Fogh, H. Lund and D. Christensen, Fable II: Design of a Modular Robot for Creative Learning, Proc. 2015 IEEE Int. Conf. Robotics and Automation. (IEEE, 2015), pp. 6134–6139. Google Scholar
    • 15. D. Marr, A theory of cerebellar cortex, J. Physiol. 202(2) (1969) 437–470. Medline, Web of Science, Google Scholar
    • 16. J. S. Albus, A theory of cerebellar function, Math. Biosci. 10(1) (1971) 25–61. Google Scholar
    • 17. M. Fujita, Adaptive filter model of the cerebellum, Biol. Cybernet. 45 (1982) 195–206. Medline, Web of Science, Google Scholar
    • 18. B. Siciliano and O. Khatib, Springer Handbook of Robotics, 2nd edn. (Springer Publishing Company, Incorporated, 2016). Google Scholar
    • 19. J. A. Garrido Alcazar, N. R. Luque, E. D’Angelo and E. Ros, Distributed cerebellar plasticity implements adaptable gain control in a manipulation task: A closed-loop robotic simulation, Front. Neural Circuits 7 (2013) 159. Medline, Web of Science, Google Scholar
    • 20. N. R. Luque, J. A. Garrido, R. R. Carrillo, S. Tolu and E. Ros, Adaptive cerebellar spiking model embedded in the control loop: Context switching and robustness against noise, Int. J. Neural Syst. 21(5) (2011) 385–401. Link, Web of Science, Google Scholar
    • 21. S. Tolu, M. Vanegas, N. R. Luque, J. A. Garrido and E. Ros, Bio-inspired adaptive feedback error learning architecture for motor control, Biolog. Cybern. 106(8–9) (2012) 507–522. Medline, Web of Science, Google Scholar
    • 22. D. Wolpert, R. Miall and M. Kawato, Internal models in the cerebellum, Trends Cogn. Sci. 2(9) (1998) 338–347. Medline, Web of Science, Google Scholar
    • 23. H. Imamizu, T. Kuroda, T. Yoshioka and M. Kawato, Functional magnetic resonance imaging examination of two modular architectures for switching multiple internal models, J. Neurosci. 24 (2004) 1173–1181. Medline, Web of Science, Google Scholar
    • 24. L. Lonini, L. Dipietro, L. Zollo, E. Guglielmelli and H. I. Krebs, An internal model for acquisition and retention of motor learning during arm reaching, Neural Comput. 21(7) (2009) 2009–2027. Medline, Web of Science, Google Scholar
    • 25. H. Imamizu and M. Kawato, Cerebellar internal models: Implications for the dexterous use of tools, Cerebellum 11(2) (2012) 325–335. Medline, Web of Science, Google Scholar
    • 26. M. Antonelli, A. J. Duran, E. Chinellato and A. P. del Pobil, Learning the visualoculomotor transformation: Effects on saccade control and space representation, Robot. Auton. Syst. 71 (2015) 13–22. Web of Science, Google Scholar
    • 27. T. Shibata and S. Schaal, Biomimetic gaze stabilization based on feedback-error-learning with nonparametric regression networks, Neural Netw. 14(2) (2001) 201–216. Medline, Web of Science, Google Scholar
    • 28. L. Vannucci, E. Falotico, S. Tolu, P. Dario, H. H. Lund and C. Laschi, Eye-head stabilization mechanism for a humanoid robot tested on human inertial data, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 9793 (2016) 341–352. Google Scholar
    • 29. L. Vannucci, E. Falotico, S. Tolu, V. Cacucciolo, P. Dario, H. Hautop Lund and C. Laschi, A comprehensive gaze stabilization controller based on cerebellar internal models, Bioinspiration Biomimetics 12(6) (2017) 1–15. Web of Science, Google Scholar
    • 30. I. B. Ojeda, S. Tolu and H. H. Lund, A scalable neuro-inspired robot controller integrating a machine learning algorithm and a spiking cerebellar-like network, Biomimetic and Biohybrid Systems - 6th Int. Conf., Living Machines 2017, Proc. Stanford, CA, USA, July 26-28, 2017 (2017), pp. 375–386. Google Scholar
    • 31. P. J. Porrill and Dean, Recurrent cerebellar loops simplify adaptive control of redundant and nonlinear motor systems, Neural Comput. 19(1) (2007) 170–193. Medline, Web of Science, Google Scholar
    • 32. A. Lenz, T. Balakrishnan, A. G. Pipe and C. Melhuish, An adaptive gaze stabilization controller inspired by the vestibulo-ocular reflex, Bioinspiration biomimetics 3(3) (2008) 035001. Medline, Web of Science, Google Scholar
    • 33. E. Franchi, E. Falotico, D. Zambrano, G. Muscolo, L. Marazzato, P. Dario and C. Laschi, A comparison between two bio-inspired adaptive models of vestibulo-ocular reflex (VOR) implemented on the iCub robot, 2010 10th IEEE-RAS Int. Conf. Humanoid Robots, Humanoids 2010 (Nashville, TN, USA, 2010), pp. 251–256. Google Scholar
    • 34. N. R. Luque, R. R. Carrillo, F. Naveros, J. A. Garrido and M. Sez-Lara, Integrated neural and robotic simulations. simulation of cerebellar neurobiological substrate for an object-oriented dynamic model abstraction process, Robot. Auton. Syst. 62(12) (2014) 1702–1716. Web of Science, Google Scholar
    • 35. M. C. Capolei, E. Angelidis, E. Falotico, H. Hautop Lund and S. Tolu, A biomimetic control method increases the adaptability of a humanoid robot acting in a dynamic environment, Front. Neurorobotics. 13(70) (2019) 1–15. Medline, Web of Science, Google Scholar
    • 36. A. Antonietti, C. Casellato, E. D’Angelo and A. Pedrocchi, Model-driven analysis of eyeblink classical conditioning reveals the underlying structure of cerebellar plasticity and neuronal activity, IEEE Trans. Neural Netw. Learn. Syst. 28 (2017) 2748–2762. Medline, Web of Science, Google Scholar
    • 37. A. Geminiani, C. Casellato, A. Antonietti, E. D’Angelo and A. Pedrocchi, A multiple-plasticity spiking neural network embedded in a closed-loop control system to model cerebellar pathologies, Int. J. Neural Syst. 28(05) (2018) 1750017. Link, Web of Science, Google Scholar
    • 38. L. Vannucci, S. Tolu, E. Falotico, P. Dario, H. Hautop Lund and C. Laschi, Adaptive gaze stabilization through cerebellar internal models in a humanoid robot, Proc. IEEE RAS and EMBS Int. Conf. Biomedical Robotics and Biomechatronics (Singapore, 2016), pp. 25–30. Google Scholar
    • 39. R. R. Carrillo, E. Ros, C. Boucheny and O. J.-M. Coenen, A real-time spiking cerebellum model for learning robot control, Biosystems 94(1) (2008) 18–27. Medline, Web of Science, Google Scholar
    • 40. C. Casellato, A. Antonietti, J. Garrido, R. R Carrillo, N. R. Luque, E. Ros, A. Pedrocchi and E. D’Angelo, Adaptive robotic control driven by a versatile spiking cerebellar network, PloS One 9(11) (2014) e112265. Medline, Web of Science, Google Scholar
    • 41. M. Kawato, Cerebellum: Models, Encyclopedia of Neuroscience, ed. in LR Squire, (2) (Elsevier Ltd., Oxford: Academic Press, 2009), pp. 757–767. Google Scholar
    • 42. M. Ito, Control of mental activities by internal models in the cerebellum, Nat. Rev. Neurosci. 9(4) (2008) 304–313. Medline, Web of Science, Google Scholar
    • 43. R. Miall and D. Wolpert, Forward models for physiological motor control, Neural Netw. 9(8) (1996) 1265–1279. Medline, Web of Science, Google Scholar
    • 44. P. Dean, J. Porrill, C.-F. Ekerot and H. Jorntell, The cerebellar microcircuit as an adaptive filter: Experimental and computational evidence, Nat. Rev. Neurosci. 11 (2010) 30–43. Medline, Web of Science, Google Scholar
    • 45. S. Schaal and N. Schweighofer, Computational motor control in humans and robots, Curr. Opinion Neurobiol. 15(6) (2005) 675–682. Medline, Web of Science, Google Scholar
    • 46. H. Imamizu, T. Kuroda, S. Miyauchi, T. Yoshioka and M. Kawato, Modular organization of internal models of tools in the human cerebellum, Proc. Nat. Acad. Sci. 100(9) (2003) 5461–5466. Medline, Web of Science, Google Scholar
    • 47. S. Vijayakumar and S. Schaal, Locally weighted projection regression: An o(n) algorithm for incremental real time learning in high dimensional space, in Proc. Seventeenth Int. Conf. Machine Learning (ICML 2000) (San Francisco, USA, 2000), pp. 1079–1086. Google Scholar
    • 48. N. R. Luque, J. A. Garrido, R. R. Carrillo, E. D’Angelo and E. Ros, Fast convergence of learning requires plasticity between inferior olive and deep cerebellar nuclei in a manipulation task: A closed-loop robotic simulation, Front. Comput. Neurosci. 8(97) (2014) 1–16. Medline, Web of Science, Google Scholar
    • 49. R. Apps and M. Garwicz, Anatomical and physiological foundations of cerebellar information processing, Nat. Rev. Neurosci. 6(4) (2005) 297–311. Medline, Web of Science, Google Scholar
    • 50. M. Ito, Cerebellar circuitry as a neuronal machine, Progress in Neurobiology 78(35) (2006) 272–303. Medline, Web of Science, Google Scholar
    • 51. H. Gomi and M. Kawato, Adaptive feedback control models of the vestibulocerebellum and spinocerebellum, Biol. Cybernet. 68(2) (1992) 105–114. Medline, Web of Science, Google Scholar
    • 52. I. Herreros, X. D. Arsiwalla and P. Verschure, A forward model at purkinje cell synapses facilitates cerebellar anticipatory motor control, Conf. Neutral Information Processing System 2016 (Barcelona, Spain, 2016). Google Scholar
    • 53. D. Nguyen-Tuong, M. Seeger and J. Peters, Model learning with local gaussian process regression, Adv. Robot. 23(15) (2009) 2015–2034. Web of Science, Google Scholar
    • 54. D. M. Wolpert and M. Kawato, Multiple paired forward and inverse models for motor control, Neural Netw. 11(7–8) (1998) 1317–1329. Medline, Web of Science, Google Scholar
    • 55. X. Wang, G. Zhang, F. Neri, T. Jiang, J. Zhao, M. Gheorghe, F. Ipate and R. Lefticaru, Design and implementation of membrane controllers for trajectory tracking of nonholonomic wheeled mobile robots, Integrated Comput. Aided Eng. 23(1) (2015) 15–30. Web of Science, Google Scholar
    • 56. L. Pan, G. Pun, G. Zhang and F. Neri, Spiking neural p systems with communication on request, Int. J. Neural Syst. 27(8) (2017) 1750042. Link, Web of Science, Google Scholar
    • 57. A. Antonietti, D. Martina, C. Casellato, E. DAngelo and A. Pedrocchi, Control of a humanoid nao robot by an adaptive bioinspired cerebellar module in 3d motion tasks, Comput. Int. neurosci. 2019 (2019) 1–15. Web of Science, Google Scholar
    • 58. E. Massi, L. Vannucci, U. Albanese, M. C. Capolei, A. Vandesompele, G. Urbain, A. M. Sabatini, J. Dambre, C. Laschi, S. Tolu and E. Falotico, Combining evolutionary and adaptive control strategies for quadruped robotic locomotion, Front. Neurorobotics. 13(71) (2019) 1–19. Medline, Web of Science, Google Scholar
    Remember to check out the Most Cited Articles!

    Check out our titles in neural networks today!