Background
Sequential reward-seeking actions are readily learned despite the temporal gap between
the earliest (distal) action in the sequence and the reward delivery. Fast dopamine
signaling is hypothesized to mediate this form of learning by reporting errors in
reward prediction. However, such a role for dopamine release in voluntarily initiated
action sequences remains to be demonstrated.
Methods
Using fast-scan cyclic voltammetry, we monitored phasic mesolimbic dopamine release,
in real time, as rats performed a self-initiated sequence of lever presses to earn
sucrose rewards. Before testing, rats received either 0 (n = 11), 5 (n = 11), or 10 (n = 8) days of action sequence training.
Results
For rats acquiring the action sequence task at test, dopamine release was strongly
elicited by response-contingent (but unexpected) rewards. With learning, a significant
elevation in dopamine release preceded performance of the proximal action and subsequently
came to precede the distal action. This predistal dopamine release response was also
observed in rats previously trained on the action sequence task, and the amplitude
of this signal predicted the latency with which rats completed the action sequence.
Importantly, the dopamine response to contingent reward delivery was not observed
in rats given extensive pretraining. Pharmacological analysis confirmed that task
performance was dopamine-dependent.
Conclusions
These data suggest that phasic mesolimbic dopamine release mediates the influence
that rewards exert over the performance of self-paced, sequentially-organized behavior
and sheds light on how dopamine signaling abnormalities may contribute to disorders
of behavioral control.
Key Words
To read this article in full you will need to make a payment
Purchase one-time access:
Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online accessOne-time access price info
- For academic or personal research use, select 'Academic and Personal'
- For corporate R&D use, select 'Corporate R&D Professionals'
Subscribe:
Subscribe to Biological PsychiatryAlready a print subscriber? Claim online access
Already an online subscriber? Sign in
Register: Create an account
Institutional Access: Sign in to ScienceDirect
References
- Principles of Behavior.Appleton, New York1943
- Steps toward artificial intelligence.Proceedings of the IRE. 1961; 49: 8-30
- Parkinson's disease: Clinical features and diagnosis.J Neurol Neurosurg Psychiatry. 2008; 79: 368-376
- The neurobiology of addiction: Implications for voluntary control of behavior.Am J Bioeth. 2007; 7: 8-11
- Imaging dopamine's role in drug abuse and addiction.Neuropharmacology. 2009; 56: 3-8
- Neurocircuitry of addiction.Neuropsychopharmacology. 2010; 35: 217-238
- A neural substrate of prediction and reward.Science. 1997; 275: 1593-1599
- The reward signal of midbrain dopamine neurons.News Physiol Sci. 1999; 14: 249-255
- Getting formal with dopamine and reward.Neuron. 2002; 36: 241-263
- Responses of monkey dopamine neurons during learning of behavioral reactions.J Neurophysiol. 1992; 67: 145-163
- Reward-related activity in the monkey striatum and substantia nigra.Prog Brain Res. 1993; 99: 227-235
- Dopamine in motivational control: Rewarding, aversive, and alerting.Neuron. 2010; 68: 815-834
- A selective role for dopamine in stimulus-reward learning.Nature. 2011; 469: 53-57
- Associative learning mediates dynamic shifts in dopamine signaling in the nucleus accumbens.Nat Neurosci. 2007; 10: 1020-1028
- Dopamine operates as a subsecond modulator of food seeking.J Neurosci. 2004; 24: 1265-1271
- Basolateral amygdala modulates terminal dopamine release in the nucleus accumbens and conditioned responding.Biol Psychiatry. 2010; 67: 737-744
- A possible role of midbrain dopamine neurons in short- and long-term adaptation of saccades to position-reward mapping.J Neurophysiol. 2004; 92: 2520-2529
- Responses of monkey dopamine neurons to reward and conditioned stimuli during successive steps of learning a delayed response task.J Neurosci. 1993; 13: 900-913
- Dopamine responses comply with basic assumptions of formal learning theory.Nature. 2001; 412: 43-48
- A framework for mesencephalic dopamine systems based on predictive Hebbian learning.J Neurosci. 1996; 16: 1936-1947
- Learning of sequential movements by neural network model with dopamine-like reinforcement signal.Exp Brain Res. 1998; 121: 350-354
- Actor-critic models of the basal ganglia: New anatomical and computational perspectives.Neural Netw. 2002; 15: 535-547
- Seeking-taking chain schedules of cocaine and sucrose self-administration: Effects of reward size, reward omission, and α-flupenthixol.([published online ahead of print October 12]) Psychopharmacology (Berl), 2011
- Nucleus accumbens dopamine depletions make animals highly sensitive to high fixed ratio requirements but do not impair primary food reinforcement.Neuroscience. 2001; 105: 863-870
- What is the role of dopamine in reward: Hedonic impact, reward learning, or incentive salience?.Brain Res Brain Res Rev. 1998; 28: 309-369
- Dissociation of Pavlovian and instrumental incentive learning under dopamine antagonists.Behav Neurosci. 2000; 114: 468-483
- Differential dependence of Pavlovian incentive motivation and instrumental incentive learning processes on dopamine signaling.Learn Mem. 2011; 18: 475-483
- Dopamine receptor blockade attenuates the general incentive motivational effects of noncontingently delivered rewards and reward-paired cues without affecting their ability to bias action selection.Neuropsychopharmacology. 2012; 37: 408-519
- Dopamine neurons of the monkey midbrain: Contingencies of responses to active touch during self-initiated arm movements.J Neurophysiol. 1990; 63: 592-606
- Start/stop signals emerge in nigrostriatal circuits during sequence learning.Nature. 2010; 466: 457-462
- Rapid dopamine signaling in the nucleus accumbens during contingent and noncontingent cocaine administration.Neuropsychopharmacology. 2005; 30: 853-863
- Subsecond dopamine release promotes cocaine seeking.Nature. 2003; 422: 614-618
- Addiction as a computational process gone awry.Science. 2004; 306: 1944-1947
- Cocaine-induced alterations in dopamine receptor signaling: Implications for reinforcement and reinstatement.Pharmacol Ther. 2005; 106: 389-403
- Somatosensory input to dopamine neurones of the monkey midbrain: Responses to pain pinch under anaesthesia and to active touch in behavioural context.Prog Brain Res. 1989; 80 (discussion 465–466): 473-478
- A role for mesencephalic dopamine in activation: Commentary on Berridge (2006).Psychopharmacology (Berl). 2007; 191: 433-437
- Effort-related functions of nucleus accumbens dopamine and associated forebrain circuits.Psychopharmacology (Berl). 2007; 191: 461-482
- Extracellular dopamine levels in striatal subregions track shifts in motivation and response cost during instrumental conditioning.J Neurosci. 2011; 31: 200-207
- The organization of midbrain projections to the striatum in the primate: Sensorimotor-related striatum versus ventral striatum.Neuroscience. 1994; 59: 625-640
- The organization of midbrain projections to the ventral striatum in the primate.Neuroscience. 1994; 59: 609-623
- Dopamine cells respond to predicted events during classical conditioning: Evidence for eligibility traces in the reward-learning network.J Neurosci. 2005; 25: 6235-6242
- Neuronal activity in the ventral tegmental area (VTA) during motivated bar press feeding in the monkey.Brain Res. 1987; 413: 302-313
- Neural encoding of cocaine-seeking behavior is coincident with phasic dopamine release in the accumbens core and shell.Eur J Neurosci. 2009; 30: 1117-1127
- Sensitization of rapid dopamine signaling in the nucleus accumbens core and shell after repeated cocaine in rats.J Neurophysiol. 2010; 104: 922-931
- Effect of cocaine on the disposition of noradrenaline labelled with tritium.Nature. 1960; 187: 604-605
- Learning to predict by methods of temporal differences.Machine Learning. 1988; 3: 9-44
- Sequential decision problems and neural networks.in: Touretzky D.S. Advances in Neural Information Processing Systems. MIT Press, Cambridge, MA1989: 686-693
- A model of how the basal ganglia generate and use neural signals that predict reinforcement.in: Houk J.C. Davis J.L. Beiser D.G. Models of Information Processing in the Basal Ganglia. MIT Press, Cambridge, MA1995: 249-270
- The debate over dopamine's role in reward: The case for incentive salience.Psychopharmacology (Berl). 2007; 191: 391-431
- Dissecting components of reward: 'Liking', 'wanting', and learning.Curr Opin Pharmacol. 2009; 9: 65-73
- The role of nucleus accumbens dopamine in motivated behavior: A unifying interpretation with special reference to reward-seeking.Brain Res Brain Res Rev. 1999; 31: 6-41
- A computational substrate for incentive salience.Trends Neurosci. 2003; 26: 423-428
- A neural computational model of incentive salience.PLoS Comput Biol. 2009; 5: e1000437
- Tonic dopamine: Opportunity costs and the control of response vigor.Psychopharmacology (Berl). 2007; 191: 507-520
- Motor sequence learning in primate: Role of the D2 receptor in movement chunking during consolidation.Behav Brain Res. 2009; 198: 231-239
- Raclopride-induced motor consolidation impairment in primates: Role of the dopamine type-2 receptor in movement chunking into integrated sequences.Exp Brain Res. 2007; 182: 499-508
- Evidence of action sequence chunking in goal-directed instrumental conditioning and its dependence on the dorsomedial prefrontal cortex.J Neurosci. 2009; 29: 8280-8287
- The basal ganglia and chunking of action repertoires.Neurobiol Learn Mem. 1998; 70: 119-136
- The Rat Brain in Stereotaxic Coordinates. 4th ed. Academic Press, San Diego1998
Article info
Publication history
Published online: February 06, 2012
Accepted:
December 20,
2011
Received in revised form:
November 19,
2011
Received:
October 18,
2011
Footnotes
Authors KMW and SBO contributed equally to this work.
Identification
Copyright
© 2012 Society of Biological Psychiatry. Published by Elsevier Inc. All rights reserved.
ScienceDirect
Access this article on ScienceDirectLinked Article
- A Step-by-Step Guide to DopamineBiological PsychiatryVol. 71Issue 10
- PreviewThere is an odd irony associated with the by-now almost ineluctable tie between reinforcement learning's temporal difference learning rule (1) and the phasic activity of dopamine neurons (2). Although temporal difference learning was designed to enable the acquisition of whole sequences of actions and predictions, a task that its ancestors (3) would flub, there are very few direct tests of this characteristic. In a penetrating new study, Wassum and colleagues (4) measured and manipulated dopamine in a sequence learning task for sucrose reward, revealing four correlates of the neuromodulator: its fine- and gross-scale dynamics during learning, its involvement in two aspects of energizing behavior, and its necessity for learning.
- Full-Text
- Preview
