World Scientific
Skip main navigation

Cookies Notification

We use cookies on this site to enhance your user experience. By continuing to browse the site, you consent to the use of our cookies. Learn More
×
Our website is made possible by displaying certain online content using javascript.
In order to view the full content, please disable your ad blocker or whitelist our website www.worldscientific.com.

System Upgrade on Mon, Jun 21st, 2021 at 1am (EDT)

During this period, the E-commerce and registration of new users may not be available for up to 6 hours.
For online purchase, please visit us again. Contact us at [email protected] for any enquiries.
Neural and Information DynamicsNo Access

METABOLIC COST AS AN ORGANIZING PRINCIPLE FOR COOPERATIVE LEARNING

    This article investigates how neurons can use metabolic cost to facilitate learning at a population level. Although decision-making by individual neurons has been extensively studied, questions regarding how neurons should behave to cooperate effectively remain largely unaddressed. Under assumptions that capture a few basic features of cortical neurons, we show that constraining reward maximization by metabolic cost aligns the information content of actions with their expected reward. Thus, metabolic cost provides a mechanism whereby neurons encode expected reward into their outputs. Further, aside from reducing energy expenditures, imposing a tight metabolic constraint also increases the accuracy of empirical estimates of rewards, increasing the robustness of distributed learning. Finally, we present two implementations of metabolically constrained learning that confirm our theoretical finding. These results suggest that metabolic cost may be an organizing principle underlying the neural code, and may also provide a useful guide to the design and analysis of other cooperating populations.

    References

    • D.   Attwell and S. B.   Laughlin , J. Cereb. Blood Flow Metab.   21 , 1133 ( 2001 ) . Crossref, ISIGoogle Scholar
    • D.   Balduzzi and M.   Besserve , Adv. Neural Inform. Process. Syst. (NIPS)   25 , 2465 ( 2012 ) . Google Scholar
    • D.   Balduzzi and G.   Tononi , PLoS Comput. Biol.   4 , e1000091 ( 2008 ) . Crossref, ISIGoogle Scholar
    • D.   Balduzzi and G.   Tononi , PLoS Comput. Biol.   5 , e1000462 ( 2009 ) . Crossref, ISIGoogle Scholar
    • D.   Balduzzi and G.   Tononi , Theory Biosci.   132 , 27 ( 2012 ) . Crossref, ISIGoogle Scholar
    • R.   Bellman , Dynamic Programming ( Princeton University Press , 1957 ) . Google Scholar
    • L.   Buesing and W.   Maass , Adv. Neural Inform. Process. Syst. (NIPS)   20 , 193 ( 2007 ) . Google Scholar
    • Y.   Dan and M.-M.   Poo , Neuron   44 , 23 ( 2004 ) . Crossref, ISIGoogle Scholar
    • Y.   Dan and M.-M.   Poo , Physiol. Rev.   86 , 1033 ( 2006 ) . Crossref, ISIGoogle Scholar
    • S.   Geman , E.   Bienenstock and R.   Doursat , Neural Comput.   4 , 1 ( 1992 ) . Crossref, ISIGoogle Scholar
    • G. F.   Gilestro , G.   Tononi and C.   Cirelli , Science   324 , 109 ( 2009 ) . Crossref, ISIGoogle Scholar
    • J. A.   Gottfried , J.   O'Doherty and R. J.   Dolan , Science   301 , 1104 ( 2003 ) . Crossref, ISIGoogle Scholar
    • A.   Hasenstaub et al. , Proc. Natl. Acad. Sci. USA   107 , 12329 ( 2010 ) . Crossref, ISIGoogle Scholar
    • S.   Maret et al. , Nat. Neurosci.   14 , 1418 ( 2011 ) . Crossref, ISIGoogle Scholar
    • H.   Markram et al. , Science   275 , 213 ( 1997 ) . Crossref, ISIGoogle Scholar
    • A.   Nere et al. , PLoS One   7 , e36958 ( 2012 ) . Crossref, ISIGoogle Scholar
    • B. A.   Olshausen and D. J.   Field , Vision Res.   37 , 3311 ( 1997 ) . Crossref, ISIGoogle Scholar
    • B. A.   Olshausen and D. J.   Field , Curr. Opin. Neurobiol.   14 , 481 ( 2004 ) . Crossref, ISIGoogle Scholar
    • P. A. Ortega and D. A. Braun, Information, utility and bounded rationality, The Fourth Conf. Artificial General Intelligence (2011) pp. 269–274. Google Scholar
    • V.   Pawlak et al. , Front. Syn. Neurosci.   2 , ( 2010 ) . CrossrefGoogle Scholar
    • J.   Rubin , O.   Shamir and N.   Tishby , Decision Making with Imperfect Decision Makers , eds. T. V.   Guy , M.   Kárný and D.   Wolpert ( Springer , 2011 ) . Google Scholar
    • S.   Russell and P.   Norvig , Artificial Intelligence: A Modern Approach , 3rd edn. ( Prentice Hall , 2009 ) . Google Scholar
    • Y.   Seldin and N.   Tishby , Multi-classification by categorical features via clustering , Proc. 25th Int. Conf. Machine Learning ( 2008 ) . Google Scholar
    • H. S.   Seung , Neuron   40 , 1063 ( 2003 ) . Crossref, ISIGoogle Scholar
    • S.   Song , K. D.   Miller and L. F.   Abbott , Nat. Neurosci.   3 , 919 ( 2000 ) . Crossref, ISIGoogle Scholar
    • R. S.   Sutton and A. G.   Barto , Reinforcement Learning: An Introduction ( MIT Press , 1998 ) . Google Scholar
    • N. Tishby, F. Pereira and W. Bialek, The information bottleneck method, Proc. 37th Annual Allerton Conf. Communication, Control and Computing, eds. B. Hajek and R. Sreenivas (1999) pp. 368–377. Google Scholar
    • G.   Tononi and C.   Cirelli , Brain Res. Bull.   62 , 143 ( 2003 ) . Crossref, ISIGoogle Scholar
    • V.   Vapnik , Estimation of Dependencies Based on Empirical Data ( Springer , 1982 ) . Google Scholar
    • V. V.   Vyazovskiy et al. , Neuron   63 , 865 ( 2009 ) . Crossref, ISIGoogle Scholar
    • C.   Watkins and P.   Dayan , Mach. Learn.   8 , 279 ( 1992 ) . Crossref, ISIGoogle Scholar
    Published: 9 April 2013
    Remember to check out the Most Cited Articles!

    Check out our titles in Complex Systems today!