The application of a neuroeconomic approach to the study of reward-related processes
has provided significant insights in our understanding of human learning and decision
making. Much of this research has focused primarily on the contributions of the corticostriatal
circuitry, involved in trial-and-error reward learning. As a result, less consideration
has been allotted to the potential influence of different neural mechanisms such as
the hippocampus or to more common ways in human society in which information is acquired
and utilized to reach a decision, such as through explicit instruction rather than
trial-and-error learning. This review examines the individual contributions of multiple
learning and memory neural systems and their interactions during human decision making
in both normal and neuropsychiatric populations. Specifically, the anatomical and
functional connectivity across multiple memory systems are highlighted to suggest
that probing the role of the hippocampus and its interactions with the corticostriatal
circuitry via the application of model-based neuroeconomic approaches may provide
novel insights into neuropsychiatric populations that suffer from damage to one of
these structures and as a consequence have deficits in learning, memory, or decision
making.
Key Words
To read this article in full you will need to make a payment
Purchase one-time access:
Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online accessOne-time access price info
- For academic or personal research use, select 'Academic and Personal'
- For corporate R&D use, select 'Corporate R&D Professionals'
Subscribe:
Subscribe to Biological PsychiatryAlready a print subscriber? Claim online access
Already an online subscriber? Sign in
Register: Create an account
Institutional Access: Sign in to ScienceDirect
References
- Neuroeconomics Decision Making and the Brain.Academic Press, London2009
- Reinforcement learning: The good, the bad and the ugly.Curr Opin Neurobiol. 2008; 18: 185-196
- Multiple systems in decision making: A neurocomputational perspective.Curr Direct Psychol Sci. 2009; 18: 73-77
- Neuroeconomics: Cross-currents in research on decision-making.Trends Cogn Sci. 2006; 10: 108-116
- The evolution of multiple memory systems.Psychol Rev. 1987; 94: 439-454
- Structure and function of declarative and nondeclarative memory systems.Proc Natl Acad Sci U S A. 1996; 93: 13515-13522
- The role of the dorsal striatum in reward and decision-making.J Neurosci. 2007; 27: 8161-8165
- Basal ganglia output and cognition: Evidence from anatomical, behavioral, and clinical studies.Brain Cogn. 2000; 42: 183-200
- A framework for studying the neurobiology of value-based decision making.Nat Rev Neurosci. 2008; 9: 545-556
- Mechanisms of cognitive set flexibility in Parkinson's disease.Brain. 2001; 124: 2503-2512
- By carrot or by stick: Cognitive reinforcement learning in parkinsonism.Science. 2004; 306: 1940-1943
- Probabilistic reversal learning is impaired in Parkinson's disease.Neuroscience. 2009; 163: 1092-1101
- Selective impairment of prediction error signaling in human dorsolateral but not ventral striatum in Parkinson's disease patients: Evidence from a model-based fMRI study.Neuroimage. 2010; 49: 772-781
- Distinct hippocampal and basal ganglia contributions to probabilistic learning and reversal.J Cogn Neurosci. 2009; 21: 1821-1833
- Role of the basal ganglia in category learning: How do patients with Parkinson's disease learn?.Behav Neurosci. 2004; 118: 676-686
- Dopaminergic drugs modulate learning rates and perseveration in Parkinson's patients in a dynamic foraging task.J Neurosci. 2009; 29: 15104-15114
- Parkinson's disease.N Engl J Med. 1998; 339: 1044-1053
- Parkinson's disease.N Engl J Med. 1998; 339: 1130-1143
- Learning and memory functions of the basal ganglia.Annu Rev Neurosci. 2002; 25: 563-593
- Declarative and nondeclarative memory: Multiple brain systems supporting learning and memory.J Cogn Neurosci. 1992; 4: 232-243
- Memory systems of the brain: A brief history and current perspective.Neurobiol Learn Mem. 2004; 82: 171-177
- Dissociable properties of memory systems: differences in the flexibility of declarative and nondeclarative knowledge.Behav Neurosci. 1996; 110: 861-871
- Differential effects of fornix and caudate nucleus lesions on two radial maze tasks: Evidence for multiple memory systems.J Neurosci. 1989; 9: 1465-1472
- Double dissociation of fornix and caudate nucleus lesions on acquisition of two water maze tasks: Further evidence for multiple memory systems.Behav Neurosci. 1992; 106: 439-446
- Multiple parallel memory systems in the brain of the rat.Neurobiol Learn Mem. 2002; 77: 125-184
- Damage limited to the hippocampal region produces long-lasting memory impairment in monkeys.J Neurosci. 1995; 15: 3796-3807
- Medial temporal lesions in monkeys impair memory on a variety of tasks sensitive to human amnesia.Behav Neurosci. 1985; 99: 22-34
- Preserved learning in monkeys with medial temporal lesions: Sparing of motor and cognitive skills.J Neurosci. 1984; 4: 1072-1085
- Lasting consequences of bilateral medial temporal lobectomy—clincal course and experimental findings in HM.Semin Neurol. 1984; 4: 249-259
- A neostriatal habit learning system in humans.Science. 1996; 273: 1399-1402
- Dissociating hippocampal versus basal ganglia contributions to learning and transfer.J Cogn Neurosci. 2003; 15: 185-193
- Cortico-striatal contributions to feedback-based learning: converging data from neuroimaging and neuropsychology.Brain. 2004; 127: 851-859
- A model for memory systems based on processing modes rather than consciousness.Nat Rev Neurosci. 2010; 11: 523-532
- Implicit and explicit memory in Alzheimer's disease and Parkinson's disease.J Clin Exp Neuropsychol. 1991; 13: 339-358
- Neuropsychological evidence for multiple implicit memory systems: A comparison of Alzheimer's, Huntington's, and Parkinson's disease patients.J Neurosci. 1989; 9: 582-587
- Probabilistic categorization: How do normal participants and amnesic patients do it?.Neurosci Biobehav Rev. 2008; 32: 237-248
- Visuospatial memory deficits at different stages of Parkinson's disease.Neuropsychologia. 1993; 31: 627-644
- Relaxing decision criteria does not improve recognition memory in amnesic patients.Mem Cogn. 1999; 27: 501-511
- The effect of feedback on non-motor probabilistic classification learning in Parkinson's disease.Neuropsychologia. 2008; 46: 2683-2695
- Model-based fMRI and its application to reward learning and decision making.Ann N Y Acad Sci. 2007; 1104: 35-53
- Reinforcement Learning.MIT Press, Cambridge, MA1998
- A theory of Pavlovian conditioning: Variations in the effectiveness of reinforcement and nonreinforcement.in: Black A.H. Prokasy W.F. Classical Conditioning II: Current Research and Theory. Appleton Century Crofts, New York1972: 64-99
- Reward-related responses in the human striatum.Ann N Y Acad Sci. 2007; 1104: 70-88
- The reward circuit: Linking primate anatomy and human imaging.Neuropsychopharmacology. 2010; 35: 4-26
- Neural economics and the biological substrates of valuation.Neuron. 2002; 36: 265-284
- Dissociable roles of ventral and dorsal striatum in instrumental conditioning.Science. 2004; 304: 452-454
- Striatal activation during acquisition of a cognitive skill.Neuropsychology. 1999; 13: 564-574
- A specific role for posterior dorsolateral striatum in human habit learning.Eur J Neurosci. 2009; 29: 2225-2232
- The role of the basal ganglia in habit formation.Nat Rev Neurosci. 2006; 7: 464-476
- Reward representations and reward-related learning in the human brain: insights from neuroimaging.Curr Opin Neurobiol. 2004; 14: 769-776
- Modulation of caudate activity by action contingency.Neuron. 2004; 41: 281-292
- Putting a spin on the dorsal-ventral divide of the striatum.Trends Neurosci. 2004; 27: 468-474
- Distributed neural representation of expected value.J Neurosci. 2005; 25: 4806-4812
- A neural substrate of prediction and reward.Science. 1997; 275: 1593-1599
- BOLD responses reflecting dopaminergic signals in the human ventral tegmental area.Science. 2008; 319: 1264-1267
- Temporal prediction errors in a passive learning task activate human striatum.Neuron. 2003; 38: 339-346
- Temporal difference models and reward-related learning in the human brain.Neuron. 2003; 38: 329-337
- Anticipation of increasing monetary reward selectively recruits nucleus accumbens.J Neurosci. 2001; 21 (RC159)
- Tracking the hemodynamic responses to reward and punishment in the striatum.J Neurophysiol. 2000; 84: 3072-3077
- Different neural correlates of reward expectation and reward expectation error in the putamen and caudate nucleus during stimulus-action-reward association learning.J Neurophysiol. 2006; 95: 948-959
- Reinforcement learning signals in the human striatum distinguish learners from nonlearners during reward-based decision making.J Neurosci. 2007; 27: 12860-12867
- Cortical substrates for exploratory decisions in humans.Nature. 2006; 441: 876-879
- The role of the basal ganglia in learning and memory: Insight from Parkinson's disease.Neurobiol Learn Mem. 2011; 96: 624-636
- Dopamine-dependent prediction errors underpin reward-seeking behaviour in humans.Nature. 2006; 442: 1042-1045
- Pathological gambling caused by drugs used to treat Parkinson disease.Arch Neurol. 2005; 62: 1377-1381
- Pathologic gambling in Parkinson's disease: A behavioral manifestation of pharmacologic treatment?.Mov Disord. 2000; 15: 869-872
- Medication-related impulse control and repetitive behaviors in Parkinson disease.Arch Neurol. 2007; 64: 1089-1096
- Enhanced or impaired cognitive function in Parkinson's disease as a function of dopaminergic medication and task demands.Cereb Cortex. 2001; 11: 1136-1143
- “Frontal” cognitive function in patients with Parkinson's disease “on” and “off” levodopa.Brain. 1988; 111: 299-321
- Probabilistic learning and reversal deficits in patients with Parkinson's disease or frontal or temporal lobe lesions: Possible adverse effects of dopaminergic medication.Neuropsychologia. 2000; 38: 596-612
- The medial temporal lobe.Annu Rev Neurosci. 2004; 27: 279-306
- Contribution of human hippocampal region to novelty detection.Nature. 1996; 383: 256-259
- The hippocampal-VTA loop: Controlling the entry of information into long-term memory.Neuron. 2005; 46: 703-713
- Integrating memories in the human brain: Hippocampal-midbrain encoding of overlapping events.Neuron. 2008; 60: 378-389
- The striatum and beyond: Hippocampal contributions to decision making.in: Delgado M.R. Phelps E.A. Robbins T.W. Attention and Performance XXII. Oxford University Press, Oxford2011
- The human hippocampus and spatial and episodic memory.Neuron. 2002; 35: 625-641
- Triple dissociation of information processing in dorsal striatum, ventral striatum, and hippocampus on a learned spatial decision task.Neuron. 2010; 67: 25-32
- What's new with the amnesic patient H.M.?.Nat Rev Neurosci. 2002; 3: 153-160
- Reward-motivated learning: Mesolimbic activation precedes memory formation.Neuron. 2006; 50: 507-517
- Hippocampal replay contributes to within session learning in a temporal difference reinforcement learning model.Neural Netw. 2005; 18: 1163-1171
- Integrating hippocampus and striatum in decision-making.Curr Opin Neurobiol. 2007; 17: 692-697
- Correlates of reward-predictive value in learning-related hippocampal neural activity.Hippocampus. 2009; 19: 487-506
- Dopamine and adaptive memory.Trends Cogn Sci. 2010; 14: 464-472
- Trial outcome and associative learning signals in the monkey hippocampus.Neuron. 2009; 61: 930-940
- Anticipation of novelty recruits reward system and hippocampus while promoting recollection.Neuroimage. 2007; 38: 194-202
- Reward-related FMRI activation of dopaminergic midbrain is associated with enhanced hippocampus-dependent long-term memory formation.Neuron. 2005; 45: 459-467
- The patterns of afferent innervation of the core and shell in the “accumbens” part of the rat ventral striatum: Immunohistochemical detection of retrogradely transported fluoro-gold.J Comp Neurol. 1993; 338: 255-278
- Organization of the projections from the subiculum to the ventral striatum in the rat.Neuroscience. 1987; 23: 103-120
- How do memory systems interact?.Neurobiol Learn Mem. 2004; 82: 324-332
- Catecholaminergic innervation of the hippocampus in the cynomolgus monkey.J Comp Neurol. 1990; 298: 250-263
- The projections of the ventral tegmental area and adjacent regions: A combined fluorescent retrograde tracer and immunofluorescence study in the rat.Brain Res Bull. 1982; 9: 321-353
- Linking context with reward: A functional circuit from hippocampal CA3 to ventral tegmental area.Science. 2011; 333: 353-357
- Quantitative modeling of category learning in amnesic patients.J Int Neuropsychol Soc. 2001; 7: 1-19
- Hippocampal lesions facilitate instrumental learning with delayed reinforcement but induce impulsive choice in rats.BMC Neurosci. 2005; 6: 36
- How instructed knowledge modulates the neural systems of reward learning.Proc Natl Acad Sci U S A. 2011; 108: 55-60
- Parallel contributions of distinct human memory systems during probabilistic learning.Neuroimage. 2011; 55: 266-276
- The hippocampus codes the uncertainty of cue-outcome associations: An intracranial electrophysiological study in humans.J Neurosci. 2009; 29: 5287-5294
- Hippocampal neurons responding to first-time dislocation of a target object.Neuron. 2002; 35: 555-566
- Novelty signals: A window into hippocampal information processing.Trends Cogn Sci. 2009; 13: 47-54
- Associative retrieval processes in the human medial temporal lobe: Hippocampal retrieval success and CA1 mismatch detection.Learn Mem. 2011; 18: 523-528
- An unexpected sequence of events: Mismatch detection in the human hippocampus.PLoS Biol. 2006; 4: e424
- Feedback timing modulates brain systems for learning in humans.J Neurosci. 2011; 31: 13157-13167
- Trust, reciprocity, and social history.Games Econ Behav. 1995; 10: 122-142
- Oxytocin shapes the neural circuitry of trust and trust adaptation in humans.Neuron. 2008; 58: 639-650
- Perceptions of moral character modulate the neural systems of reward during the trust game.Nat Neurosci. 2005; 8: 1611-1618
- Getting to know you: Reputation and trust in a two-person economic exchange.Science. 2005; 308: 78-83
- Seeing is believing: Trustworthiness as a dynamic belief.Cogn Psychol. 2010; 61: 87-105
- A double dissociation revealing bidirectional competition between striatum and hippocampus during learning.Proc Natl Acad Sci U S A. 2008; 105: 17163-17168
- Interactive memory systems in the human brain.Nature. 2001; 414: 546-550
- The role of the striatum and hippocampus in planning: A PET activation study in Parkinson's disease.Brain. 2001; 124: 1020-1032
- Interaction between the human hippocampus and the caudate nucleus during route recognition.Neuron. 2004; 43: 427-435
- Cooperation between the hippocampus and the striatum during episodic encoding.J Cogn Neurosci. 2011; 23: 1597-1608
- Parallel striatal and hippocampal systems for landmarks and boundaries in spatial memory.Proc Natl Acad Sci U S A. 2008; 105: 5915-5920
- Feedback signals in the caudate reflect goal achievement on a declarative memory task.Neuroimage. 2008; 41: 1154-1167
- Human and rodent homologies in action control: Corticostriatal determinants of goal-directed and habitual action.Neuropsychopharmacology. 2010; 35: 48-69
- The neural correlates of subjective value during intertemporal choice.Nat Neurosci. 2007; 10: 1625-1633
- Prospect theory: An analysis of decision under risk.Econometrica. 1979; 47: 263-292
- Time discounting for primary rewards.J Neurosci. 2007; 27: 5796-5804
- Separate neural systems value immediate and delayed monetary rewards.Science. 2004; 306: 503-507
- The effect of lesions of the insular cortex on instrumental conditioning: evidence for a role in incentive memory.J Neurosci. 2000; 20: 8954-8964
- Model-based influences on humans' choices and striatal prediction errors.Neuron. 2011; 69: 1204-1215
- States versus rewards: dissociable neural prediction error signals underlying model-based and model-free reinforcement learning.Neuron. 2010; 66: 585-595
- The role of value systems in decision making.in: Engel C. Singer W. Better than Conscious? Decision Making, the Human Mind, and Implications for Institutions MIT Press, Cambridge, MA2008: 51-70
- The human orbitofrontal cortex: Linking reward to hedonic experience.Nat Rev Neurosci. 2005; 6: 691-702
- Emotion, decision making, and the amygdala.Neuron. 2008; 58: 662-671
- Contributions of the amygdala to emotion processing: From animal models to human behavior.Neuron. 2005; 48: 175-187
- The amygdala modulates the consolidation of memories of emotionally arousing experiences.Annu Rev Neurosci. 2004; 27: 1-28
- Neural systems responding to degrees of uncertainty in human decision-making.Science. 2005; 310: 1680-1683
- Frames, biases, and rational decision-making in the human brain.Science. 2006; 313: 684-687
- Amygdala modulation of multiple memory systems: Hippocampus and caudate-putamen.Neurobiol Learn Mem. 1998; 69: 163-203
- Dorsolateral prefrontal cortex drives mesolimbic dopaminergic regions to initiate motivated behavior.J Neurosci. 2011; 31: 10340-10346
- Instructional control of reinforcement learning: A behavioral and neurocomputational investigation.Brain Res. 2009; 1299: 74-94
- Neural mechanisms of observational learning.Proc Natl Acad Sci U S A. 2010; 107: 14431-14436
Article info
Publication history
Published online: February 27, 2012
Accepted:
January 19,
2012
Received in revised form:
December 24,
2011
Received:
September 8,
2011
Identification
Copyright
© 2012 Society of Biological Psychiatry. Published by Elsevier Inc. All rights reserved.
