World Scientific
  • Search
    Quick Search in Journals
    Access type:
    Quick Search anywhere
    Access type:
    Advanced Search
  • My Cart
  •   
Skip main navigation

Cookies Notification

We use cookies on this site to enhance your user experience. By continuing to browse the site, you consent to the use of our cookies. Learn More
×
No Access

Decision making under uncertainty in a spiking neural network model of the basal ganglia

    https://doi.org/10.1142/S021963521650028XCited by:13 (Source: Crossref)
    Previous
    Next

    Abstract

    The mechanisms of decision-making and action selection are generally thought to be under the control of parallel cortico-subcortical loops connecting back to distinct areas of cortex through the basal ganglia and processing motor, cognitive and limbic modalities of decision-making. We have used these properties to develop and extend a connectionist model at a spiking neuron level based on a previous rate model approach. This model is demonstrated on decision-making tasks that have been studied in primates and the electrophysiology interpreted to show that the decision is made in two steps. To model this, we have used two parallel loops, each of which performs decision-making based on interactions between positive and negative feedback pathways. This model is able to perform two-level decision-making as in primates. We show here that, before learning, synaptic noise is sufficient to drive the decision-making process and that, after learning, the decision is based on the choice that has proven most likely to be rewarded. The model is then submitted to lesion tests, reversal learning and extinction protocols. We show that, under these conditions, it behaves in a consistent manner and provides predictions in accordance with observed experimental data.

    References

    • Albin, R.L., Penney, J.B. & Young, A.B. [1989] The functional anatomy of basal ganglia disorders. Trends Neurosci., 12, 366–375. Crossref, Medline, Web of Science, Google Scholar
    • Alexander, G.E., Long, M.R.D. & Strick, P.L. [1986] Parallel organization of functionally segregated circuits linking basal ganglia and cortex. Ann. Rev. Neurosci., 9, 357–381. Crossref, Medline, Web of Science, Google Scholar
    • Alexander, M.E. & Wickens, J.R. [1993] Analysis of striatal dynamics: The existence of two modes of behaviour. J. Theor. Biol., 163, 413–438. Crossref, Medline, Web of Science, Google Scholar
    • Bar-Gad, I. & Bergman, H. [2001] Stepping out of the box: Information processing in the neural networks of the basal ganglia. Curr. Opin. Neurobiol., 11, 689–695. Crossref, Medline, Web of Science, Google Scholar
    • Bar-Gad, I., Morris, G. & Bergman, H. [2003] Information processing, dimensionality reduction and reinforcement learning in the basal ganglia. Prog. Neurobiol., 71, 439–473. Crossref, Medline, Web of Science, Google Scholar
    • Barker, J.M., Taylor, J.R. & Chandler, L.J. [2014] A unifying model of the role of the infralimbic cortex in extinction and habits. Learn. Mem., 21, 441–448. Crossref, Medline, Web of Science, Google Scholar
    • Berthet, P., Hellgren-Kotaleski, J. & Lansner, A. [2012] Action selection performance of a reconfigurable basal ganglia inspired model with hebbian–bayesian go-nogo connectivity. Front. Behav. Neurosci., 6, 10–3389. Crossref, Medline, Web of Science, Google Scholar
    • Berthet, P., Lindahl, M., Tully, P.J., Hellgren-Kotaleski, J. & Lansner, A. [2016] Functional relevance of different basal ganglia pathways investigated in a spiking model with reward dependent plasticity. Front. Neural Circ., 10, 53. Medline, Web of Science, Google Scholar
    • Bevan, M.D., Eaton, S.A. & Bolam, J.P. [1997] Synaptic targets of physiologically, neurochemically and morphologically characterized neurons of the rate globus pallidus. in Society of Neuroscience Abstract, p. 196. Google Scholar
    • Bouton, M.E. [2004] Context and behavioral processes in extinction. Learn. Mem., 11, 485–494. Crossref, Medline, Web of Science, Google Scholar
    • Brette, R. [2015] Philosophy of the spike: Rate-based vs. spike-based theories of the brain. Front. Syst. Neurosci., 9, 151. Crossref, Medline, Web of Science, Google Scholar
    • Brown, L.L., Smith, D.M. & Goldbloom, L.M. [1998] Organizing principles of cortical integration in the rat neostriatum: corticostriate map of the body surface is an ordered lattice of curved laminae and radial points. J. Comput. Neurol., 392, 468–488. Crossref, Medline, Web of Science, Google Scholar
    • Chang, H.T., Wilson, C.J. & Kitai, S.T. [1981] Single neostriatal efferent axons in the globus pallidus: A light and electron microscopic study. Science, 313, 915–918. Crossref, Web of Science, Google Scholar
    • Chen, H., Zhuang, P., Miao, S.H., Yuan, G., Zhang, Y.Q. & Li, J.Y. [2010] Neuronal firing in the ventrolateral thalamus of patients with parkinson’s disease differs from that with essential tremor. Chin. Med. J., 123, 695–701. Crossref, Medline, Web of Science, Google Scholar
    • Chersi, F., Mirolli, M., Pezzulo, G. & Baldassarre, G. [2013] A spiking neuron model of the cortico-basal ganglia circuits for goal-directed and habitual action learning. Neural Netw., 41, 212–224. Crossref, Medline, Web of Science, Google Scholar
    • Cools, R., Clark, L., Owen, A.M. & Robbins, T.W. [2002] Defining the neural mechanisms of probabilistic reversal learning using event-related functional magnetic resonance imaging. J. Neurosci., 22, 4563–4567. Crossref, Medline, Web of Science, Google Scholar
    • Cools, R., Frank, M.J., Gibbs, S.E., Miyakawa, A., Jagust, W. & D’Esposito, M. [2009] Striatal dopamine predicts outcome-specific reversal learning and its sensitivity to dopaminergic drug administration. J. Neurosci., 29, 1538–1543. Crossref, Medline, Web of Science, Google Scholar
    • Costa, V.D., Tran, V.L., Turchi, J. & Averbeck, B.B. [2015] Reversal learning and dopamine: a bayesian perspective. J. Neurosci., 35, 2407–2416. Crossref, Medline, Web of Science, Google Scholar
    • Dombrovski, A.Y., Szanto, K., Clark, L., Aizenstein, H.J., Chase, H.W., Reynolds, C.F. & Siegle, G.J. [2015] Corticostriatothalamic reward prediction error signals and executive control in late-life depression. Psychol. Med., 45, 1413–1424. Crossref, Medline, Web of Science, Google Scholar
    • Frank, M.J., Loughry, B. & O’Reilly, R.C. [2001] Interactions between frontal cortex and basal ganglia in working memory: A computational model. Cognitive Affect. Behav. Neurosci., 1, 137–160. Crossref, Medline, Web of Science, Google Scholar
    • Gerfen, C.R. [1985] Compartmental organization of projections from the striatum to the substantia nigra in the rat. J. Comput. Neurol., 236, 454–476. Crossref, Medline, Web of Science, Google Scholar
    • Ghodratitoostani, I., Zana, Y., Delbem, A.C., Sani, S.S., Ekhtiari, H. & Sanchez, T.G. [2016] Theoretical tinnitus framework: A neurofunctional model. Front. Neurosci., 10, 370. Crossref, Medline, Web of Science, Google Scholar
    • Goodman, J. & Packard, M.G. [2015] The memory system engaged during acquisition determines the effectiveness of different extinction protocols. Front. Behav. Neurosci., 9, 314. Crossref, Medline, Web of Science, Google Scholar
    • Groves, P.M. [1983] A theory of the functional organization of the neostriatum and the neostriatal control of voluntary movement. Brain Res. Rev., 286, 109–132. Crossref, Google Scholar
    • Gurney, K., Prescott, T.J. & Redgrave, P. [2001] A computational model of action selection in the basal ganglia. i. a new functional anatomy. Biol. Cybern., 84, 401–410. Crossref, Medline, Web of Science, Google Scholar
    • Guthrie, M., Leblois, A., Garenne, A. & Boraud, T. [2013] Interaction between cognitive and motor cortico-basal ganglia loops during decision making: A computational study. J. Neurophysiol., 109 (12), 3025–3040. Crossref, Medline, Web of Science, Google Scholar
    • Hines, M.L., Markram, H. & Schurmann, F. [2008] Fully implicit parallel simulation of single neurons. J. Comput. Neurosci., 25, 439–448. Crossref, Medline, Web of Science, Google Scholar
    • Humphries, M.D., Stewart, R.D. & Gurney, K.N. [2006] A physiologically plausible model of action selection and oscillatory activity in the basal ganglia. J. Neurosci., 26, 12921–12942. Crossref, Medline, Web of Science, Google Scholar
    • Hunter, J.D. [2007] Matplotlib: A 2d graphics environment. Comput. Sci. Eng., 9, 90–95. Crossref, Web of Science, Google Scholar
    • Jacobs, A.L., Fridman, G., Douglas, R.M., Alam, N.M., Latham, P.E., Prusky, G.T. & Nirenberg, S. [2009] Ruling out and ruling in neural codes. Proc. Natl. Acad. Sci. U.S.A., 106, 5936–5941. Crossref, Medline, Web of Science, Google Scholar
    • Jaeger, D., Kita, H. & Wilson, C.J. [1994] Surround inhibition among projection neurons is weak or nonexistent in the rat neostriatum. J. Neurophysiol., 72, 2555–2558. Crossref, Medline, Web of Science, Google Scholar
    • Kerr, J.N.D. & Wickens, J.R. [2001] Dopamine d-1/d-5 receptor activation is required for long-term potentiation in the rat neostriatum in vitro. J. Neurophysiol., 85, 117–124. Crossref, Medline, Web of Science, Google Scholar
    • Kita, H. & Kita, S.T. [1994] The morphology of globus pallidus projection neurons in the rat: An intracellular staining study. Brain Res., 636, 308–319. Crossref, Medline, Web of Science, Google Scholar
    • Klopf, H. (ed.) [1988] Neural Models of Plasticity (John H. Byrne et William O. Berry). Google Scholar
    • Koos, T., Tepper, J. & Wilson, C.J. [2004] Comaparison of ipscs evoked by spiny and fast-spiking neurons in the neostriatum. J. Neurosci., 24, 7916–7922. Crossref, Medline, Web of Science, Google Scholar
    • Kotter, R. & Wickens, J.R. [1998] Striatal mechanisms in parkinson’s disease: New insights from computer modeling. Artifi. Intell. Med., 13, 37–55. Crossref, Medline, Web of Science, Google Scholar
    • Kropotov, J.D. & Etlinger, S.C. [1999] Selection of actions in the basal ganglia-thalamocortical circuits: Review and model. Int. J. Psychophysiol., 31, 197–217. Crossref, Medline, Web of Science, Google Scholar
    • Leblois, A., Boraud, T., Meissner, W., Bergman, H. & Hansel, D. [2006a] Competition between feedback loops underlies normal and pathological dynamics in the basal ganglia. J. Neurosci., 26, 3567–3583. Crossref, Medline, Web of Science, Google Scholar
    • Leblois, A., Meissner, W., Bezard, E., Bioulac, B., Gross, C.E. & Boraud, T. [2006b] Temporal and spatial alterations in gpi neuronal encoding might contribute to slow down movement in parkinsonian monkeys. Eur. J. Neurosci., 24, 1201–1208. Crossref, Medline, Web of Science, Google Scholar
    • Lee, D., Seo, H. & Jung, M.W. [2012] Neural basis of reinforcement learning and decision making. Ann. Rev. Neurosci., 35, 287–308. Crossref, Medline, Web of Science, Google Scholar
    • Levy, R. [1997] Re-evaluation of the functional anatomy of the basal ganglia in normal and parkinsonian states. Neuroscience, 76, 335–343. Crossref, Medline, Web of Science, Google Scholar
    • Lingawi, N.W. & Balleine, B.W. [2012] Amygdala central nucleus interacts with dorsolateral striatum to regulate the acquisition of habits. J. Neurosci., 32, 1073–1081. Crossref, Medline, Web of Science, Google Scholar
    • Mandali, A., Rengaswamy, M., Chakravarthy, V.S. & Moustafa, A.A. [2015] A spiking Basal Ganglia model of synchrony, exploration and decision making. Front. Neurosci., 9, 191. Crossref, Medline, Web of Science, Google Scholar
    • Medina, J.F., Garcia, K.S. & Mauk, M.D. [2001] A mechanism for savings in the cerebellum. J. Neurosci., 21, 4081–4089. Crossref, Medline, Web of Science, Google Scholar
    • Migliore, M., Cannia, C., Lytton, W.W., Markram, H. & Hines, M.L. [2006] Parallel networks simulations with neuron. J. Comput. Neurosci., 21, 119–129. Crossref, Medline, Web of Science, Google Scholar
    • Mink, J.W. [1996] The basal ganglia: Focused selection and inhibition of competing motor programs. Prog. Neurobiol., 50, 381–425. Crossref, Medline, Web of Science, Google Scholar
    • Morita, K. & Kawaguchi, Y. [2015] Computing reward-prediction error: An integrated account of cortical timing and basal-ganglia pathways for appetitive and aversive learning. Eur. J. Neurosci., 42, 2003–2021. Crossref, Medline, Web of Science, Google Scholar
    • Moyer, J.T., Halterman, B.L., Finkel, L.H. & Wolf, J.A. [2014] Lateral and feedforward inhibition suppress asynchronous activity in a large, biophysically-detailed computational model of the striatal network. Front. Comput. Neurosci., 8, 152. Crossref, Medline, Web of Science, Google Scholar
    • Nambu, A. [2011] Somatotopic organization of the primate basal ganglia. Front. Neuroanatomy, 5, 26. Crossref, Medline, Web of Science, Google Scholar
    • Neymotin, S.A., Lee, H., Park, E., Fenton, A.A. & Lytton, W.W. [2011] Emergence of physiological oscillation frequencies in a computer model of neocortex. Front. Syst. Neurosci., 5, 19. Medline, Google Scholar
    • Oorschot, D.E. [1996] Total number of neurons in the neostriatal, pallidal, subthalamic, and substantia nigral nuclei of the rat basal ganglia: A stereological study using the cavalieri and optical disector methods. J. Compar. Neurol., 366, 580–599. Crossref, Medline, Web of Science, Google Scholar
    • Papachristou, H., Nederkoorn, C., Beunen, S. & Jansen, A. [2013] Dissection of appetitive conditioning. does impulsivity play a role?, Appetite, 69, 46–53. Crossref, Medline, Web of Science, Google Scholar
    • Pasquereau, B., Nadjar, A., Arkadir, D., Bezard, E., Goillandeau, M., Bioulac, B., Gross, C.E. & Boraud, T. [2007] Shaping of motor responses by incentive values through the basal ganglia. J. Neurosci., 27, 1176–1183. Crossref, Medline, Web of Science, Google Scholar
    • Pawlak, V. & Kerr, J.N.D. [2008] Dopamine receptor activation is required for corticostriatal spike-timing dependent plasticity. J. Neurosci., 28, 2435–2446. Crossref, Medline, Web of Science, Google Scholar
    • Percheron, G., Yelnik, J. & Franois, C.A. [1984] A golgi analysis of the primate globus pallidus. iii. spatial organization of the striato-pallidal complex. J. Compar. Neurol., 227, 214–227. Crossref, Medline, Web of Science, Google Scholar
    • Piron, C., Kase, D., Topalidou, M., Goillandeau, M., Orignac, H., N’Guyen, T.H., Rougier, N. & Boraud, T. [2016] The globus pallidus pars interna in goal-oriented and routine behaviors: Resolving a long-standing paradox. Move. Disord., 31 (8), 1146–1154. Crossref, Medline, Web of Science, Google Scholar
    • Portelli, G., Barrett, J.M., Hilgen, G., Masquelier, T., Maccione, A., Di Marco, S., Berdondini, L., Kornprobst, P. & Sernagor, E. [2016] Rank order coding: a retinal information decoding strategy revealed by large-scale multielectrode array retinal recordings. eNeuro, 3 (3), 18. Crossref, Web of Science, Google Scholar
    • R Core Team [2015] R: A Language and Environment for Statistical Computing, R Foundation for Statistical Computing, Vienna, Austria, URL https://www.R-project.org/. Google Scholar
    • Redgrave, P., Prescott, T.J. & Gurney, K. [1999] The basal ganglia: a vertebrate solution to the selection problem?, Neuroscience, 89, 1009–1023. Crossref, Medline, Web of Science, Google Scholar
    • Reynolds, J., Hyland, B. & Wickens, J.R. [2001] A cellular mechanism of reward-related learning. Nature, 413, 67–70. Crossref, Medline, Web of Science, Google Scholar
    • Saal, H.P., Harvey, M.A. & Bensmaia, S.J. [2015] Rate and timing of cortical responses driven by separate sensory channels. Elife, 4, e10450. Crossref, Medline, Web of Science, Google Scholar
    • Sandoz, J.C. & Pham-Delegue, M.H. [2004] Spontaneous recovery after extinction of the conditioned proboscis extension response in the honeybee. Learn. Mem., 11, 586–597. Crossref, Medline, Web of Science, Google Scholar
    • Sarvestani, I.K., Kozlov, A., Harischandra, N., Grillner, S. & rjan Ekeberg [2013] A computational model of visually guided locomotion in lamprey. Biol. Cybern., 107, 497–512. Crossref, Medline, Web of Science, Google Scholar
    • Schaette, R. & Kempter, R. [2012] Computational models of neurophysiological correlates of tinnitus. Front. Syst. Neurosci., 6, 34. Crossref, Medline, Google Scholar
    • Schroll, H., Vitay, J. & Hamker, F.H. [2014] Dysfunctional and compensatory synaptic plasticity in parkinson’s disease. Eur. J. Neurosci., 39, 688–702. Crossref, Medline, Web of Science, Google Scholar
    • Shohamy, D., Myers, C.E., Hopkins, R.O., Sage, J. & Gluck, M.A. [2009] Distinct hippocampal and basal ganglia contributions to probabilistic learning and reversal. J. Cogn. Neurosci., 21, 1821–1833. Crossref, Medline, Web of Science, Google Scholar
    • Smith, Y. & Bolam, J.P. [1989] Neurons of the substantia nigra reticulata receive a dense gaba-containing input from the globus pallidus in the rat. Brain Res., 493, 160–167. Crossref, Medline, Web of Science, Google Scholar
    • Sukumar, D., Rengaswamy, M. & Chakravarthy, S.V. [2012] Modeling the contributions of basal ganglia and hippocampus to spatial navigation using reinforcement learning. PLoS One, 7, e47467. Crossref, Medline, Web of Science, Google Scholar
    • Suri, R.E. & Schultz, W. [1999] A neural network model with dopamine-like reinforcement signal that learns a spatial delayed response task. Neuroscience, 91, 871–890. Crossref, Medline, Web of Science, Google Scholar
    • Tunstall, M., Oorschot, D.E., Kean, A. & Wickens, J.R. [2002] Inhibitory interactions between spiny projection neurons in the rat striatum. J. Neurophysiol., 88, 1263–1269. Crossref, Medline, Web of Science, Google Scholar
    • Wickens, J.R. & Arbuthnott, G.W. [1993] The corticostriatal system on computer simulation: an intermediate mechanism for sequencing of actions. Prog. Brain Res., 99, 325–339. Crossref, Medline, Web of Science, Google Scholar
    • Woodward, D., Kirillov, A., Myre, C. & Sawyer, S. [1995] Neostriatal circuitry as a scalar memory: Modeling and ensemble neuron recording. In: J.C. Houk, J.L. Davis & D.G. Beiser eds. Models of Information Processing in the Basal Ganglia. Cambridge, MA: The MIT Press, pp. 315–336. Google Scholar
    • Xue, G., Xue, F., Droutman, V., Lu, Z.L., Bechara, A. & Read, S. [2013] Common neural mechanisms underlying reversal learning by reward and punishment. PLoS ONE, 8, e82169. Crossref, Medline, Web of Science, Google Scholar
    • Yang, S., Wang, J., Li, S., Deng, B., Wei, X., Yu, H. & Li, H. [2015] Cost-efficient FPGA implementation of basal ganglia and their Parkinsonian analysis. Neural Netw., 71, 62–75. Crossref, Medline, Web of Science, Google Scholar