Abstract
A central goal of computational psychiatry is to identify systematic relationships
between transdiagnostic dimensions of psychiatric symptomatology and the latent learning
and decision-making computations that inform individuals’ thoughts, feelings, and
choices. Most psychiatric disorders emerge prior to adulthood, yet little work has
extended these computational approaches to study the development of psychopathology.
Here, we lay out a roadmap for future studies implementing this approach by developing
empirically and theoretically informed hypotheses about how developmental changes
in model-based control of action and Pavlovian learning processes may modulate vulnerability
to anxiety and addiction. We highlight how insights from studies leveraging computational
approaches to characterize the normative developmental trajectories of clinically
relevant learning and decision-making processes may suggest promising avenues for
future developmental computational psychiatry research.
To read this article in full you will need to make a payment
Purchase one-time access:
Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online accessOne-time access price info
- For academic or personal research use, select 'Academic and Personal'
- For corporate R&D use, select 'Corporate R&D Professionals'
Subscribe:
Subscribe to Biological PsychiatryAlready a print subscriber? Claim online access
Already an online subscriber? Sign in
Register: Create an account
Institutional Access: Sign in to ScienceDirect
References
- Our future: A Lancet commission on adolescent health and wellbeing.Lancet. 2016; 387: 2423-2478
- Why do many psychiatric disorders emerge during adolescence?.Nat Rev Neurosci. 2008; 9: 947-957
- Lifetime prevalence and age-of-onset distributions of DSM-IV disorders in the National Comorbidity Survey Replication.Arch Gen Psychiatry. 2005; 62: 593-602
- Age of onset, clinical characteristics, and 15-year course of anxiety disorders in a prospective, longitudinal, observational study.J Affect Disord. 2011; 132: 260-264
- Age at drinking onset and alcohol dependence: Age at onset, duration, and severity.Arch Pediatr Adolesc Med. 2006; 160: 739-746
- Annual Research Review: Developmental computational psychiatry.J Child Psychol Psychiatry. 2019; 60: 412-426
- Computational psychiatry needs time and context.Annu Rev Psychol. 2022; 73: 243-270
- Computational psychiatry as a bridge from neuroscience to clinical applications.Nat Neurosci. 2016; 19: 404-413
- Computational phenotyping: Using models to understand individual differences in personality, development, and mental illness.Personal Neurosci. 2018; 1: e18
- The RDoC framework: Facilitating transition from ICD/DSM to dimensional approaches that integrate neuroscience and psychopathology.World Psychiatry. 2014; 13: 28-35
- Building a new field of computational psychiatry.Biol Psychiatry. 2017; 82: 388-390
- Neurodevelopmental heterogeneity and computational approaches for understanding autism.Transl Psychiatry. 2019; 9: 63
- Towards a computational psychiatry of juvenile obsessive–compulsive disorder.Neurosci Biobehav Rev. 2020; 118: 631-642
- Decision-making ability, psychopathology, and brain connectivity.Neuron. 2021; 109: 2025-2040.e7
- Compulsivity is linked to reduced adolescent development of goal-directed control and frontostriatal functional connectivity.Proc Natl Acad Sci U S A. 2020; 117: 25911-25922
- Reinforcement learning across development: What insights can we draw from a decade of research?.Dev Cogn Neurosci. 2019; 40100733
- Developmental epidemiology of anxiety disorders.Child Adolesc Psychiatr Clin N Am. 2012; 21: 457-478
- Prevalence of anxiety disorders and their comorbidity with mood and addictive disorders.Br J Psychiatry. 1998; 34: 24-28
- Anxiety, depression, and decision making: A computational perspective.Annu Rev Neurosci. 2018; 41: 371-388
- Computational theory-driven studies of reinforcement learning and decision-making in addiction: What have we learned?.Curr Opin Behav Sci. 2021; 38: 40-48
- Neurocircuitry of addiction.Neuropsychopharmacology. 2010; 35: 217-238
- Use and abuse of alcohol and illicit drugs in US adolescents: Results of the National Comorbidity Survey-Adolescent Supplement.Arch Gen Psychiatry. 2012; 69: 390-398
- From first drug use to drug dependence; developmental periods of risk for dependence upon marijuana, cocaine, and alcohol.Neuropsychopharmacology. 2002; 26: 479-488
- What is an anxiety disorder?.Focus. 2011; 9: 369-388
- Twelve-month and lifetime prevalence and lifetime morbid risk of anxiety and mood disorders in the United States.Int J Methods Psychiatr Res. 2012; 21: 169-184
- Reinforcement learning: An introduction.IEEE Trans Neural Netw. 1998; 9 (1054–1054)
- Actions and habits: The development of behavioural autonomy.Phil Trans R Soc Lond B. 1985; 308: 67-78
Raab HA, Hartley CA (2018): The development of goal-directed decision-making. In: Morris R, Bornstein A, Shenhav A, editors. Goal-Directed Decision Making. New York: Academic Press, 279–308. Available at: https://www.sciencedirect.com/science/article/pii/B9780128120989000139.
- The control of instrumental action following outcome devaluation in young children aged between 1 and 4 years.J Exp Psychol Gen. 2008; 137: 39-51
- Parallel maturation of goal-directed behavior and dopaminergic systems during adolescence.J Neurosci. 2012; 32: 16223-16232
- Model-based influences on humans’ choices and striatal prediction errors.Neuron. 2011; 69: 1204-1215
- Computational and behavioral markers of model-based decision making in childhood.Dev Sci. 2023; 26: e13295
- Valence bias in metacontrol of decision making in adolescents and young adults.Child Dev. 2022; 93: e103-e116
- From creatures of habit to goal-directed learners: Tracking the developmental emergence of model-based reinforcement learning.Psychol Sci. 2016; 27: 848-858
- Cognitive components underpinning the development of model-based learning.Dev Cogn Neurosci. 2017; 25: 272-280
- Moving developmental research online: Comparing in-lab and web-based studies of model-based reinforcement learning.Collabra Psychol. 2020; 6: 17213
- Cost–benefit arbitration between multiple reinforcement-learning systems.Psychol Sci. 2017; 28: 1321-1333
- Model-based control in dimensional psychiatry.Biol Psychiatry. 2017; 82: 391-400
- Addiction is driven by excessive goal-directed drug choice under negative affect: Translational critique of habit and compulsion theory.Neuropsychopharmacology. 2020; 45: 720-735
- “Chasing the first high”: Memory sampling in drug choice.Neuropsychopharmacology. 2020; 45: 907-915
- Habitual alcohol seeking: Time course and the contribution of subregions of the dorsal striatum.Biol Psychiatry. 2012; 72: 389-395
- Shift from goal-directed to habitual cocaine seeking after prolonged experience in rats.J Neurosci. 2010; 30: 15457-15463
- Post-training cocaine administration facilitates habit learning and requires the infralimbic cortex and dorsolateral striatum.Neurobiol Learn Mem. 2015; 118: 105-112
- Amphetamine exposure enhances habit formation.J Neurosci. 2006; 26: 3805-3812
- Individual differences in the engagement of habitual control over alcohol seeking predict the development of compulsive alcohol seeking and drinking.Addict Biol. 2021; 26e13041
- Behavioral and neuroimaging evidence for overreliance on habit learning in alcohol-dependent patients.Transl Psychiatry. 2013; 3: e337
- Carrots and sticks fail to change behavior in cocaine addiction.Science. 2016; 352: 1468-1471
- Model-based and model-free decisions in alcohol dependence.Neuropsychobiology. 2014; 70: 122-131
- The involuntary nature of binge drinking: Goal directedness and awareness of intention.Addict Biol. 2018; 23: 515-526
- Disorders of compulsivity: A common bias towards learning habits.Mol Psychiatry. 2015; 20: 345-352
- Reduced model-based decision-making in gambling disorder.Sci Rep. 2019; 919625
- Intact corticostriatal control of goal-directed action in alcohol use disorder: A Pavlovian-to-instrumental transfer and outcome-devaluation study.Sci Rep. 2020; 10: 4949
- No association of goal-directed and habitual control with alcohol consumption in young adults.Addict Biol. 2018; 23: 379-393
- Characterizing a psychiatric symptom dimension related to deficits in goal-directed control.eLife. 2016; 5: e11305
- Comparison of the association between goal-directed planning and self-reported compulsivity vs obsessive–compulsive disorder diagnosis.JAMA Psychiatry. 2020; 77: 77-85
- Transitions from first substance use to substance use disorders in adolescence: Is early onset associated with a rapid escalation?.Drug Alcohol Depend. 2009; 99: 68-78
- Early-onset drug use and risk for drug dependence problems.Addict Behav. 2009; 34: 319-322
- Model-based and model-free control predicts alcohol consumption developmental trajectory in young adults: A 3-year prospective study.Biol Psychiatry. 2021; 89: 980-989
- Online gaming addiction in children and adolescents: A review of empirical research.J Behav Addict. 2012; 1: 3-22
- A systematic review of the impact of COVID-19 on the game addiction of children and adolescents.Front Psychiatry. 2022; 13976601
- Relationship between problematic gaming and age at the onset of habitual gaming.Pediatr Int. 2020; 62: 1275-1281
- Applying a behavioral economic framework to understanding adolescent smoking.Psychol Addict Behav. 2004; 18: 64-73
- A review of psychosocial factors linked to adolescent substance use.Pharmacol Biochem Behav. 2020; 196172969
- When planning to survive goes wrong: Predicting the future and replaying the past in anxiety and PTSD.Curr Opin Behav Sci. 2018; 24: 89-95
- Goals and habits in the brain.Neuron. 2013; 80: 312-325
- Anxiety, depression, and judgments about the probability of future negative and positive events in children.J Anxiety Disord. 2006; 20: 252-261
- Increased and biased deliberation in social anxiety.Nat Hum Behav. 2022; 6: 146-154
- When does the gender difference in rumination begin? Gender and age differences in the use of rumination by adolescents.J Youth Adolescence. 2008; 37: 180-192
- Interactive development of adaptive learning and memory.Annu Rev Dev Psychol. 2021; 3: 59-85
- The cognitive content of thought-listed worry episodes in clinic-referred anxious and nonreferred children.J Clin Child Adolesc Psychol. 2004; 33: 613-622
- Rumination as a mechanism linking stressful life events to symptoms of depression and anxiety: Longitudinal evidence in early adolescents and adults.J Abnorm Psychol. 2013; 122: 339-352
- Rumination as a transdiagnostic factor in depression and anxiety.Behav Res Ther. 2011; 49: 186-193
- Behavioral studies of Pavlovian conditioning.Annu Rev Neurosci. 1988; 11: 329-352
- Drug addiction: Updating actions to habits to compulsions ten years on.Annu Rev Psychol. 2016; 67: 23-50
- State-of-the-art and future directions for extinction as a translational model for fear and anxiety.Philos Trans R Soc Lond B Biol Sci. 2018; 373
- The neural basis of drug craving: An incentive-sensitization theory of addiction.Brain Res Brain Res Rev. 1993; 18: 247-291
- Facilitation of instrumental behavior by a Pavlovian appetitive conditioned stimulus.J Exp Psychol Anim Behav Process. 1983; 9: 225-247
- Ethanol-associated cues produce general Pavlovian-instrumental transfer.Alcohol Clin Exp Res. 2007; 31: 766-774
- Changes in the influence of alcohol-paired stimuli on alcohol seeking across extended training.Front Psychiatry. 2016; 7: 169
- Nucleus accumbens core and shell are differentially involved in general and outcome-specific forms of Pavlovian-instrumental transfer with alcohol and sucrose rewards.Eur J Neurosci. 2016; 43: 1229-1236
- The role of Pavlovian cues in alcohol seeking in dependent and nondependent rats.J Stud Alcohol. 2005; 66: 53-61
- The effects of amphetamine exposure on outcome-selective Pavlovian-instrumental transfer in rats.Psychopharmacology. 2012; 223: 361-370
- Neural correlates of Pavlovian-to-instrumental transfer in the nucleus accumbens shell are selectively potentiated following cocaine self-administration.Eur J Neurosci. 2011; 33: 2274-2287
- Impact of repeated intravenous cocaine administration on incentive motivation depends on mode of drug delivery.Addict Biol. 2014; 19: 965-971
- Repeated cocaine exposure facilitates the expression of incentive motivation and induces habitual control in rats.PLoS One. 2013; 8e61355
- Phasic mesolimbic dopamine signaling encodes the facilitation of incentive motivation produced by repeated cocaine exposure.Neuropsychopharmacology. 2014; 39: 2441-2449
- Incentive sensitization by previous amphetamine exposure: Increased cue-triggered “wanting” for sucrose reward.J Neurosci. 2001; 21: 7831-7840
- Pavlovian-to-instrumental transfer in alcohol dependence: A pilot study.Neuropsychobiology. 2014; 70: 111-121
- Pavlovian-to-instrumental transfer effects in the nucleus accumbens relate to relapse in alcohol dependence.Addict Biol. 2016; 21: 719-731
- Strong seduction: Impulsivity and the impact of contextual cues on instrumental behavior in alcohol dependence.Transl Psychiatry. 2017; 7: e1183
- Pavlovian-to-instrumental transfer and alcohol consumption in young male social drinkers: Behavioral, neural and polygenic correlates.J Clin Med. 2019; 8: 1188
- Susceptibility to interference between Pavlovian and instrumental control is associated with early hazardous alcohol use.Addict Biol. 2021; 26e12983
- Dysfunctional approach behavior triggered by alcohol-unrelated Pavlovian cues predicts long-term relapse in alcohol dependence.Addict Biol. 2020; 25e12703
- When habits are dangerous: Alcohol expectancies and habitual decision making predict relapse in alcohol dependence.Biol Psychiatry. 2017; 82: 847-856
- The association of non-drug-related Pavlovian-to-instrumental transfer effect in nucleus accumbens with relapse in alcohol dependence: A replication.Biol Psychiatry. 2023; 93: 558-565
- Alcohol approach bias is associated with both behavioral and neural Pavlovian-to-instrumental transfer effects in alcohol-dependent patients [published online Apr 14].Biol Psychiatry Glob Open Sci. 2022;
- Modeling avoidance in mood and anxiety disorders using reinforcement learning.Biol Psychiatry. 2017; 82: 532-539
- The impact of traumatic stress on Pavlovian biases.Psychol Med. 2018; 48: 327-336
- Adolescents exhibit reduced Pavlovian biases on instrumental learning.Sci Rep. 2020; 1015770
- Change, stability, and instability in the Pavlovian guidance of behaviour from adolescence to young adulthood.PLoS Comput Biol. 2018; 14e1006679
- Bonsai trees in your head: How the Pavlovian system sculpts goal-directed choices by pruning decision trees.PLoS Comput Biol. 2012; 8e1002410
- Cognitive prostheses for goal achievement.Nat Hum Behav. 2019; 3: 1096-1106
- The role of attentional bias in the effect of food advertising on actual food intake among children.Appetite. 2015; 84: 251-258
- Overweight children overeat after exposure to food cues.Eat Behav. 2003; 4: 197-209
- Alcohol craving in adolescents: Bridging the laboratory and natural environment.Psychopharmacology. 2014; 231: 1841-1851
- Model-based and model-free Pavlovian reward learning: Revaluation, revision, and revelation.Cogn Affect Behav Neurosci. 2014; 14: 473-492
- Individual differences in the attribution of incentive salience to reward-related cues: Implications for addiction.Neuropharmacology. 2009; 56: 139-148
- The role of associative fear and avoidance learning in anxiety disorders: Gaps and directions for future research.Neurosci Biobehav Rev. 2018; 88: 117-140
- Context and behavioral processes in extinction.Learn Mem. 2004; 11: 485-494
- Toward a universal law of generalization for psychological science.Science. 1987; 237: 1317-1323
- Context, learning, and extinction.Psychol Rev. 2010; 117: 197-209
- Individual differences in learning predict the return of fear.Learn Behav. 2015; 43: 243-250
- A meta-analysis of conditioned fear generalization in anxiety-related disorders.Neuropsychopharmacology. 2022; 47: 1652-1661
- Updated meta-analysis of classical fear conditioning in the anxiety disorders.Depress Anxiety. 2015; 32: 239-253
- Trait anxiety is associated with hidden state inference during aversive reversal learning.bioRxiv. 2022; https://doi.org/10.1101/2022.04.01.483303
- Latent cause inference during extinction learning in trauma-exposed individuals with and without PTSD [published online Mar 8].Psychol Med. 2021;
- Sensitive periods in affective development: Nonlinear maturation of fear learning.Neuropsychopharmacology. 2015; 40: 50-60
- A developmental dissociation in reinstatement of an extinguished fear response in rats.Neurobiol Learn Mem. 2007; 88: 48-57
- Extinction in the developing rat: An examination of renewal effects.Dev Psychobiol. 2007; 49: 565-575
- Perineuronal nets protect fear memories from erasure.Science. 2009; 325: 1258-1261
- Fear extinction across development: The involvement of the medial prefrontal cortex as assessed by temporary inactivation and immunohistochemistry.J Neurosci. 2009; 29: 10802-10808
- The effect of temporary amygdala inactivation on extinction and reextinction of fear in the developing rat: Unlearning as a potential mechanism for extinction early in development.J Neurosci. 2008; 28: 1282-1290
- Altered fear learning across development in both mouse and human.Proc Natl Acad Sci U S A. 2012; 109: 16318-16323
- A window of vulnerability: Impaired fear extinction in adolescence.Neurobiol Learn Mem. 2014; 113: 90-100
- Distinct neural signatures of threat learning in adolescents and adults.Proc Natl Acad Sci U S A. 2011; 108: 4500-4505
- A developmental analysis of threat/safety learning and extinction recall during middle childhood.J Exp Child Psychol. 2016; 146: 95-105
- The development of fear learning and generalization in 8-13 year-olds.Dev Psychobiol. 2012; 54: 675-684
- Developmental aspects of fear: Comparing the acquisition and generalization of conditioned fear in children and adults.Dev Psychobiol. 2016; 58: 471-481
- Fear conditioning and stimulus generalization in association with age in children and adolescents.Eur Child Adolesc Psychiatry. 2022; 31: 1581-1590
- Hippocampal maturation drives memory from generalization to specificity.Trends Cogn Sci. 2018; 22: 676-686
- The computational nature of memory modification.eLife. 2017; 6: e23763
- The spread of fear: Symbolic generalization mediates graded threat-avoidance in specific phobia.Q J Exp Psychol (Hove). 2014; 67: 247-259
- Childhood generalized specific phobia as an early marker of internalizing psychopathology across the lifespan: Results from the World Mental Health Surveys.BMC Med. 2019; 17: 101
- Explaining the effectiveness of fear extinction through latent-cause inference.arXiv. 2022;
- Gradual extinction prevents the return of fear: Implications for the discovery of state.Front Behav Neurosci. 2013; 7: 164
- Anxious individuals have difficulty learning the causal statistics of aversive environments.Nat Neurosci. 2015; 18: 590-596
- Impaired adaptation of learning to contingency volatility in internalizing psychopathology.eLife. 2020; 9: e61387
- Mood as representation of momentum.Trends Cogn Sci. 2016; 20: 15-24
- Oxytocin-dependent reopening of a social reward learning critical period with MDMA.Nature. 2019; 569: 116-120
- MDMA-assisted therapy for severe PTSD: A randomized, double-blind, placebo-controlled phase 3 study.Nat Med. 2021; 27: 1025-1033
- Forming beliefs: Why valence matters.Trends Cogn Sci. 2016; 20: 25-33
- The computational roots of positivity and confirmation biases in reinforcement learning.Trends Cogn Sci. 2022; 26: 607-621
- Reinforcement learning in patients with mood and anxiety disorders vs control individuals: A systematic review and meta-analysis.JAMA Psychiatry. 2022; 79: 313-322
- The interpretation of computational model parameters depends on the context.Elife. 2022; 11e75474
- Children are full of optimism, but those rose-tinted glasses are fading-Reduced learning from negative outcomes drives hyperoptimism in children.J Exp Psychol Gen. 2022; 151: 1843-1853
- Valence biases in reinforcement learning shift across adolescence and modulate subsequent memory.eLife. 2022; 11: e64620
- Human development of the ability to learn from bad news.Proc Natl Acad Sci U S A. 2013; 110: 16396-16401
- Optimistic update bias increases in older age.Psychol Med. 2014; 44: 2003-2012
- Depressive symptoms bias the prediction-error enhancement of memory towards negative events in reinforcement learning.Psychopharmacology. 2019; 236: 2425-2435
- Learning the value of information in an uncertain world.Nat Neurosci. 2007; 10: 1214-1221
- Computational dysfunctions in anxiety: Failure to differentiate signal from noise.Biol Psychiatry. 2017; 82: 440-446
- Anxiety impedes adaptive social learning under uncertainty.Psychol Sci. 2020; 31: 592-603
- Flexibility in valenced reinforcement learning computations across development.Child Dev. 2022; 93: 1601-1615
- Metacognition and cognitive monitoring: A new area of cognitive–developmental inquiry.Am Psychol. 1979; 34: 906-911
- Abnormalities of confidence in psychiatry: An overview and future perspectives.Transl Psychiatry. 2019; 9: 268
- How local and global metacognition shape mental health.Biol Psychiatry. 2021; 90: 436-446
- The development of metacognitive ability in adolescence.Conscious Cogn. 2013; 22: 264-271
- I know better! Emerging metacognition allows adolescents to ignore false advice.Dev Sci. 2021; 24e13101
- The metacognitive model of generalized anxiety disorder in children and adolescents.Clin Child Fam Psychol Rev. 2010; 13: 151-163
- Hunting for what works: Adolescents in addiction treatment.Alcohol Clin Exp Res. 2019; 43: 578-592
- Influences of early-life stress on frontolimbic circuitry: Harnessing a dimensional approach to elucidate the effects of heterogeneity in stress exposure.Dev Psychobiol. 2021; 63: 153-172
- Biological embedding of childhood adversity: From physiological mechanisms to clinical implications.BMC Med. 2017; 15: 135
- Effects of stress throughout the lifespan on the brain, behaviour and cognition.Nat Rev Neurosci. 2009; 10: 434-445
- Stress and the developing adolescent brain.Neuroscience. 2012; 249: 162-171
- The role of early life stress in HPA axis and anxiety.in: Kim Y.-K. Anxiety Disorders: Rethinking and Understanding Recent Discoveries. Springer Singapore, Singapore2020: 141-153
- Stress as a common risk factor for obesity and addiction.Biol Psychiatry. 2013; 73: 827-835
- Childhood adversities and first onset of psychiatric disorders in a national sample of US adolescents.Arch Gen Psychiatry. 2012; 69: 1151-1160
- Chronic stress causes frontostriatal reorganization and affects decision-making.Science. 2009; 325: 621-625
- Simultaneous glucocorticoid and noradrenergic activity disrupts the neural basis of goal-directed action in the human brain.J Neurosci. 2012; 32: 10146-10155
- Socially evaluated cold pressor stress after instrumental learning favors habits over goal-directed action.Psychoneuroendocrinology. 2010; 35: 977-986
- Stress prompts habit behavior in humans.J Neurosci. 2009; 29: 7191-7198
- Stress reduces both model-based and model-free neural computations during flexible learning.Neuroimage. 2021; 229117747
- Working-memory capacity protects model-based learning from stress.Proc Natl Acad Sci U S A. 2013; 110: 20941-20946
- Stress transiently affects Pavlovian-to-instrumental transfer.Front Neurosci. 2012; 6: 93
- The effects of acute stress on Pavlovian-instrumental transfer in rats.Cogn Affect Behav Neurosci. 2013; 13: 174-185
- Does acute stress influence the Pavlovian-to-instrumental transfer effect? Implications for substance use disorders.Psychopharmacology. 2020; 237: 2305-2316
- Forced swim stress but not exogenous corticosterone could induce the reinstatement of extinguished morphine conditioned place preference in rats: Involvement of glucocorticoid receptors in the basolateral amygdala.Behav Brain Res. 2014; 264: 43-50
- Reinstatement of Pavlovian responses to alcohol cues by stress.Psychopharmacology (Berl). 2023; 240: 531-545
- Stress-induced reinstatement of drug seeking: 20 years of progress.Neuropsychopharmacology. 2016; 41: 335-356
- The influence of acute stress on the regulation of conditioned fear.Neurobiol Stress. 2015; 1: 134-146
- Threat of shock and aversive inhibition: Induced anxiety modulates Pavlovian-instrumental interactions.J Exp Psychol Gen. 2017; 146: 1694-1704
Article info
Publication history
Published online: February 10, 2023
Accepted:
February 7,
2023
Received in revised form:
February 5,
2023
Received:
October 20,
2022
Identification
Copyright
© 2023 Society of Biological Psychiatry.
