World Scientific
  • Search
  •   
Skip main navigation

Cookies Notification

We use cookies on this site to enhance your user experience. By continuing to browse the site, you consent to the use of our cookies. Learn More
×
Our website is made possible by displaying certain online content using javascript.
In order to view the full content, please disable your ad blocker or whitelist our website www.worldscientific.com.

System Upgrade on Tue, Oct 25th, 2022 at 2am (EDT)

Existing users will be able to log into the site and access content. However, E-commerce and registration of new users may not be available for up to 12 hours.
For online purchase, please visit us again. Contact us at [email protected] for any enquiries.

Decision making under uncertainty in a spiking neural network model of the basal ganglia

    https://doi.org/10.1142/S021963521650028XCited by:12 (Source: Crossref)

    The mechanisms of decision-making and action selection are generally thought to be under the control of parallel cortico-subcortical loops connecting back to distinct areas of cortex through the basal ganglia and processing motor, cognitive and limbic modalities of decision-making. We have used these properties to develop and extend a connectionist model at a spiking neuron level based on a previous rate model approach. This model is demonstrated on decision-making tasks that have been studied in primates and the electrophysiology interpreted to show that the decision is made in two steps. To model this, we have used two parallel loops, each of which performs decision-making based on interactions between positive and negative feedback pathways. This model is able to perform two-level decision-making as in primates. We show here that, before learning, synaptic noise is sufficient to drive the decision-making process and that, after learning, the decision is based on the choice that has proven most likely to be rewarded. The model is then submitted to lesion tests, reversal learning and extinction protocols. We show that, under these conditions, it behaves in a consistent manner and provides predictions in accordance with observed experimental data.

    References

    • Albin, R.L., Penney, J.B. & Young, A.B. [1989] The functional anatomy of basal ganglia disorders. Trends Neurosci., 12, 366–375. Crossref, Medline, ISIGoogle Scholar
    • Alexander, G.E., Long, M.R.D. & Strick, P.L. [1986] Parallel organization of functionally segregated circuits linking basal ganglia and cortex. Ann. Rev. Neurosci., 9, 357–381. Crossref, Medline, ISIGoogle Scholar
    • Alexander, M.E. & Wickens, J.R. [1993] Analysis of striatal dynamics: The existence of two modes of behaviour. J. Theor. Biol., 163, 413–438. Crossref, Medline, ISIGoogle Scholar
    • Bar-Gad, I. & Bergman, H. [2001] Stepping out of the box: Information processing in the neural networks of the basal ganglia. Curr. Opin. Neurobiol., 11, 689–695. Crossref, Medline, ISIGoogle Scholar
    • Bar-Gad, I., Morris, G. & Bergman, H. [2003] Information processing, dimensionality reduction and reinforcement learning in the basal ganglia. Prog. Neurobiol., 71, 439–473. Crossref, Medline, ISIGoogle Scholar
    • Barker, J.M., Taylor, J.R. & Chandler, L.J. [2014] A unifying model of the role of the infralimbic cortex in extinction and habits. Learn. Mem., 21, 441–448. Crossref, Medline, ISIGoogle Scholar
    • Berthet, P., Hellgren-Kotaleski, J. & Lansner, A. [2012] Action selection performance of a reconfigurable basal ganglia inspired model with hebbian–bayesian go-nogo connectivity. Front. Behav. Neurosci., 6, 10–3389. Crossref, Medline, ISIGoogle Scholar
    • Berthet, P., Lindahl, M., Tully, P.J., Hellgren-Kotaleski, J. & Lansner, A. [2016] Functional relevance of different basal ganglia pathways investigated in a spiking model with reward dependent plasticity. Front. Neural Circ., 10, 53. Medline, ISIGoogle Scholar
    • Bevan, M.D., Eaton, S.A. & Bolam, J.P. [1997] Synaptic targets of physiologically, neurochemically and morphologically characterized neurons of the rate globus pallidus. in Society of Neuroscience Abstract, p. 196. Google Scholar
    • Bouton, M.E. [2004] Context and behavioral processes in extinction. Learn. Mem., 11, 485–494. Crossref, Medline, ISIGoogle Scholar
    • Brette, R. [2015] Philosophy of the spike: Rate-based vs. spike-based theories of the brain. Front. Syst. Neurosci., 9, 151. Crossref, Medline, ISIGoogle Scholar
    • Brown, L.L., Smith, D.M. & Goldbloom, L.M. [1998] Organizing principles of cortical integration in the rat neostriatum: corticostriate map of the body surface is an ordered lattice of curved laminae and radial points. J. Comput. Neurol., 392, 468–488. Crossref, Medline, ISIGoogle Scholar
    • Chang, H.T., Wilson, C.J. & Kitai, S.T. [1981] Single neostriatal efferent axons in the globus pallidus: A light and electron microscopic study. Science, 313, 915–918. Crossref, ISIGoogle Scholar
    • Chen, H., Zhuang, P., Miao, S.H., Yuan, G., Zhang, Y.Q. & Li, J.Y. [2010] Neuronal firing in the ventrolateral thalamus of patients with parkinson’s disease differs from that with essential tremor. Chin. Med. J., 123, 695–701. Crossref, Medline, ISIGoogle Scholar
    • Chersi, F., Mirolli, M., Pezzulo, G. & Baldassarre, G. [2013] A spiking neuron model of the cortico-basal ganglia circuits for goal-directed and habitual action learning. Neural Netw., 41, 212–224. Crossref, Medline, ISIGoogle Scholar
    • Cools, R., Clark, L., Owen, A.M. & Robbins, T.W. [2002] Defining the neural mechanisms of probabilistic reversal learning using event-related functional magnetic resonance imaging. J. Neurosci., 22, 4563–4567. Crossref, Medline, ISIGoogle Scholar
    • Cools, R., Frank, M.J., Gibbs, S.E., Miyakawa, A., Jagust, W. & D’Esposito, M. [2009] Striatal dopamine predicts outcome-specific reversal learning and its sensitivity to dopaminergic drug administration. J. Neurosci., 29, 1538–1543. Crossref, Medline, ISIGoogle Scholar
    • Costa, V.D., Tran, V.L., Turchi, J. & Averbeck, B.B. [2015] Reversal learning and dopamine: a bayesian perspective. J. Neurosci., 35, 2407–2416. Crossref, Medline, ISIGoogle Scholar
    • Dombrovski, A.Y., Szanto, K., Clark, L., Aizenstein, H.J., Chase, H.W., Reynolds, C.F. & Siegle, G.J. [2015] Corticostriatothalamic reward prediction error signals and executive control in late-life depression. Psychol. Med., 45, 1413–1424. Crossref, Medline, ISIGoogle Scholar
    • Frank, M.J., Loughry, B. & O’Reilly, R.C. [2001] Interactions between frontal cortex and basal ganglia in working memory: A computational model. Cognitive Affect. Behav. Neurosci., 1, 137–160. Crossref, Medline, ISIGoogle Scholar
    • Gerfen, C.R. [1985] Compartmental organization of projections from the striatum to the substantia nigra in the rat. J. Comput. Neurol., 236, 454–476. Crossref, Medline, ISIGoogle Scholar
    • Ghodratitoostani, I., Zana, Y., Delbem, A.C., Sani, S.S., Ekhtiari, H. & Sanchez, T.G. [2016] Theoretical tinnitus framework: A neurofunctional model. Front. Neurosci., 10, 370. Crossref, Medline, ISIGoogle Scholar
    • Goodman, J. & Packard, M.G. [2015] The memory system engaged during acquisition determines the effectiveness of different extinction protocols. Front. Behav. Neurosci., 9, 314. Crossref, Medline, ISIGoogle Scholar
    • Groves, P.M. [1983] A theory of the functional organization of the neostriatum and the neostriatal control of voluntary movement. Brain Res. Rev., 286, 109–132. CrossrefGoogle Scholar
    • Gurney, K., Prescott, T.J. & Redgrave, P. [2001] A computational model of action selection in the basal ganglia. i. a new functional anatomy. Biol. Cybern., 84, 401–410. Crossref, Medline, ISIGoogle Scholar
    • Guthrie, M., Leblois, A., Garenne, A. & Boraud, T. [2013] Interaction between cognitive and motor cortico-basal ganglia loops during decision making: A computational study. J. Neurophysiol., 109 (12), 3025–3040. Crossref, Medline, ISIGoogle Scholar
    • Hines, M.L., Markram, H. & Schurmann, F. [2008] Fully implicit parallel simulation of single neurons. J. Comput. Neurosci., 25, 439–448. Crossref, Medline, ISIGoogle Scholar
    • Humphries, M.D., Stewart, R.D. & Gurney, K.N. [2006] A physiologically plausible model of action selection and oscillatory activity in the basal ganglia. J. Neurosci., 26, 12921–12942. Crossref, Medline, ISIGoogle Scholar
    • Hunter, J.D. [2007] Matplotlib: A 2d graphics environment. Comput. Sci. Eng., 9, 90–95. Crossref, ISIGoogle Scholar
    • Jacobs, A.L., Fridman, G., Douglas, R.M., Alam, N.M., Latham, P.E., Prusky, G.T. & Nirenberg, S. [2009] Ruling out and ruling in neural codes. Proc. Natl. Acad. Sci. U.S.A., 106, 5936–5941. Crossref, Medline, ISIGoogle Scholar
    • Jaeger, D., Kita, H. & Wilson, C.J. [1994] Surround inhibition among projection neurons is weak or nonexistent in the rat neostriatum. J. Neurophysiol., 72, 2555–2558. Crossref, Medline, ISIGoogle Scholar
    • Kerr, J.N.D. & Wickens, J.R. [2001] Dopamine d-1/d-5 receptor activation is required for long-term potentiation in the rat neostriatum in vitro. J. Neurophysiol., 85, 117–124. Crossref, Medline, ISIGoogle Scholar
    • Kita, H. & Kita, S.T. [1994] The morphology of globus pallidus projection neurons in the rat: An intracellular staining study. Brain Res., 636, 308–319. Crossref, Medline, ISIGoogle Scholar
    • Klopf, H. (ed.) [1988] Neural Models of Plasticity (John H. Byrne et William O. Berry). Google Scholar
    • Koos, T., Tepper, J. & Wilson, C.J. [2004] Comaparison of ipscs evoked by spiny and fast-spiking neurons in the neostriatum. J. Neurosci., 24, 7916–7922. Crossref, Medline, ISIGoogle Scholar
    • Kotter, R. & Wickens, J.R. [1998] Striatal mechanisms in parkinson’s disease: New insights from computer modeling. Artifi. Intell. Med., 13, 37–55. Crossref, Medline, ISIGoogle Scholar
    • Kropotov, J.D. & Etlinger, S.C. [1999] Selection of actions in the basal ganglia-thalamocortical circuits: Review and model. Int. J. Psychophysiol., 31, 197–217. Crossref, Medline, ISIGoogle Scholar
    • Leblois, A., Boraud, T., Meissner, W., Bergman, H. & Hansel, D. [2006a] Competition between feedback loops underlies normal and pathological dynamics in the basal ganglia. J. Neurosci., 26, 3567–3583. Crossref, Medline, ISIGoogle Scholar
    • Leblois, A., Meissner, W., Bezard, E., Bioulac, B., Gross, C.E. & Boraud, T. [2006b] Temporal and spatial alterations in gpi neuronal encoding might contribute to slow down movement in parkinsonian monkeys. Eur. J. Neurosci., 24, 1201–1208. Crossref, Medline, ISIGoogle Scholar
    • Lee, D., Seo, H. & Jung, M.W. [2012] Neural basis of reinforcement learning and decision making. Ann. Rev. Neurosci., 35, 287–308. Crossref, Medline, ISIGoogle Scholar
    • Levy, R. [1997] Re-evaluation of the functional anatomy of the basal ganglia in normal and parkinsonian states. Neuroscience, 76, 335–343. Crossref, Medline, ISIGoogle Scholar
    • Lingawi, N.W. & Balleine, B.W. [2012] Amygdala central nucleus interacts with dorsolateral striatum to regulate the acquisition of habits. J. Neurosci., 32, 1073–1081. Crossref, Medline, ISIGoogle Scholar
    • Mandali, A., Rengaswamy, M., Chakravarthy, V.S. & Moustafa, A.A. [2015] A spiking Basal Ganglia model of synchrony, exploration and decision making. Front. Neurosci., 9, 191. Crossref, Medline, ISIGoogle Scholar
    • Medina, J.F., Garcia, K.S. & Mauk, M.D. [2001] A mechanism for savings in the cerebellum. J. Neurosci., 21, 4081–4089. Crossref, Medline, ISIGoogle Scholar
    • Migliore, M., Cannia, C., Lytton, W.W., Markram, H. & Hines, M.L. [2006] Parallel networks simulations with neuron. J. Comput. Neurosci., 21, 119–129. Crossref, Medline, ISIGoogle Scholar
    • Mink, J.W. [1996] The basal ganglia: Focused selection and inhibition of competing motor programs. Prog. Neurobiol., 50, 381–425. Crossref, Medline, ISIGoogle Scholar
    • Morita, K. & Kawaguchi, Y. [2015] Computing reward-prediction error: An integrated account of cortical timing and basal-ganglia pathways for appetitive and aversive learning. Eur. J. Neurosci., 42, 2003–2021. Crossref, Medline, ISIGoogle Scholar
    • Moyer, J.T., Halterman, B.L., Finkel, L.H. & Wolf, J.A. [2014] Lateral and feedforward inhibition suppress asynchronous activity in a large, biophysically-detailed computational model of the striatal network. Front. Comput. Neurosci., 8, 152. Crossref, Medline, ISIGoogle Scholar
    • Nambu, A. [2011] Somatotopic organization of the primate basal ganglia. Front. Neuroanatomy, 5, 26. Crossref, Medline, ISIGoogle Scholar
    • Neymotin, S.A., Lee, H., Park, E., Fenton, A.A. & Lytton, W.W. [2011] Emergence of physiological oscillation frequencies in a computer model of neocortex. Front. Syst. Neurosci., 5, 19. MedlineGoogle Scholar
    • Oorschot, D.E. [1996] Total number of neurons in the neostriatal, pallidal, subthalamic, and substantia nigral nuclei of the rat basal ganglia: A stereological study using the cavalieri and optical disector methods. J. Compar. Neurol., 366, 580–599. Crossref, Medline, ISIGoogle Scholar
    • Papachristou, H., Nederkoorn, C., Beunen, S. & Jansen, A. [2013] Dissection of appetitive conditioning. does impulsivity play a role?, Appetite, 69, 46–53. Crossref, Medline, ISIGoogle Scholar
    • Pasquereau, B., Nadjar, A., Arkadir, D., Bezard, E., Goillandeau, M., Bioulac, B., Gross, C.E. & Boraud, T. [2007] Shaping of motor responses by incentive values through the basal ganglia. J. Neurosci., 27, 1176–1183. Crossref, Medline, ISIGoogle Scholar
    • Pawlak, V. & Kerr, J.N.D. [2008] Dopamine receptor activation is required for corticostriatal spike-timing dependent plasticity. J. Neurosci., 28, 2435–2446. Crossref, Medline, ISIGoogle Scholar
    • Percheron, G., Yelnik, J. & Franois, C.A. [1984] A golgi analysis of the primate globus pallidus. iii. spatial organization of the striato-pallidal complex. J. Compar. Neurol., 227, 214–227. Crossref, Medline, ISIGoogle Scholar
    • Piron, C., Kase, D., Topalidou, M., Goillandeau, M., Orignac, H., N’Guyen, T.H., Rougier, N. & Boraud, T. [2016] The globus pallidus pars interna in goal-oriented and routine behaviors: Resolving a long-standing paradox. Move. Disord., 31 (8), 1146–1154. Crossref, Medline, ISIGoogle Scholar
    • Portelli, G., Barrett, J.M., Hilgen, G., Masquelier, T., Maccione, A., Di Marco, S., Berdondini, L., Kornprobst, P. & Sernagor, E. [2016] Rank order coding: a retinal information decoding strategy revealed by large-scale multielectrode array retinal recordings. eNeuro, 3 (3), 18. Crossref, ISIGoogle Scholar
    • R Core Team [2015] R: A Language and Environment for Statistical Computing, R Foundation for Statistical Computing, Vienna, Austria, URL https://www.R-project.org/. Google Scholar
    • Redgrave, P., Prescott, T.J. & Gurney, K. [1999] The basal ganglia: a vertebrate solution to the selection problem?, Neuroscience, 89, 1009–1023. Crossref, Medline, ISIGoogle Scholar
    • Reynolds, J., Hyland, B. & Wickens, J.R. [2001] A cellular mechanism of reward-related learning. Nature, 413, 67–70. Crossref, Medline, ISIGoogle Scholar
    • Saal, H.P., Harvey, M.A. & Bensmaia, S.J. [2015] Rate and timing of cortical responses driven by separate sensory channels. Elife, 4, e10450. Crossref, Medline, ISIGoogle Scholar
    • Sandoz, J.C. & Pham-Delegue, M.H. [2004] Spontaneous recovery after extinction of the conditioned proboscis extension response in the honeybee. Learn. Mem., 11, 586–597. Crossref, Medline, ISIGoogle Scholar
    • Sarvestani, I.K., Kozlov, A., Harischandra, N., Grillner, S. & rjan Ekeberg [2013] A computational model of visually guided locomotion in lamprey. Biol. Cybern., 107, 497–512. Crossref, Medline, ISIGoogle Scholar
    • Schaette, R. & Kempter, R. [2012] Computational models of neurophysiological correlates of tinnitus. Front. Syst. Neurosci., 6, 34. Crossref, MedlineGoogle Scholar
    • Schroll, H., Vitay, J. & Hamker, F.H. [2014] Dysfunctional and compensatory synaptic plasticity in parkinson’s disease. Eur. J. Neurosci., 39, 688–702. Crossref, Medline, ISIGoogle Scholar
    • Shohamy, D., Myers, C.E., Hopkins, R.O., Sage, J. & Gluck, M.A. [2009] Distinct hippocampal and basal ganglia contributions to probabilistic learning and reversal. J. Cogn. Neurosci., 21, 1821–1833. Crossref, Medline, ISIGoogle Scholar
    • Smith, Y. & Bolam, J.P. [1989] Neurons of the substantia nigra reticulata receive a dense gaba-containing input from the globus pallidus in the rat. Brain Res., 493, 160–167. Crossref, Medline, ISIGoogle Scholar
    • Sukumar, D., Rengaswamy, M. & Chakravarthy, S.V. [2012] Modeling the contributions of basal ganglia and hippocampus to spatial navigation using reinforcement learning. PLoS One, 7, e47467. Crossref, Medline, ISIGoogle Scholar
    • Suri, R.E. & Schultz, W. [1999] A neural network model with dopamine-like reinforcement signal that learns a spatial delayed response task. Neuroscience, 91, 871–890. Crossref, Medline, ISIGoogle Scholar
    • Tunstall, M., Oorschot, D.E., Kean, A. & Wickens, J.R. [2002] Inhibitory interactions between spiny projection neurons in the rat striatum. J. Neurophysiol., 88, 1263–1269. Crossref, Medline, ISIGoogle Scholar
    • Wickens, J.R. & Arbuthnott, G.W. [1993] The corticostriatal system on computer simulation: an intermediate mechanism for sequencing of actions. Prog. Brain Res., 99, 325–339. Crossref, Medline, ISIGoogle Scholar
    • Woodward, D., Kirillov, A., Myre, C. & Sawyer, S. [1995] Neostriatal circuitry as a scalar memory: Modeling and ensemble neuron recording. In: J.C. HoukJ.L. DavisD.G. Beiser eds. Models of Information Processing in the Basal Ganglia. Cambridge, MA: The MIT Press, pp. 315–336. Google Scholar
    • Xue, G., Xue, F., Droutman, V., Lu, Z.L., Bechara, A. & Read, S. [2013] Common neural mechanisms underlying reversal learning by reward and punishment. PLoS ONE, 8, e82169. Crossref, Medline, ISIGoogle Scholar
    • Yang, S., Wang, J., Li, S., Deng, B., Wei, X., Yu, H. & Li, H. [2015] Cost-efficient FPGA implementation of basal ganglia and their Parkinsonian analysis. Neural Netw., 71, 62–75. Crossref, Medline, ISIGoogle Scholar