Advertisement

Spatiotemporal precision of neuroimaging in psychiatry

  • Jessica McFadyen
    Correspondence
    corresponding author:
    Affiliations
    The UCL Max Planck Centre for Computational Psychiatry and Ageing Research, University College London, London, UK

    Wellcome Centre for Human Neuroimaging, University College London, London, UK
    Search for articles by this author
  • Raymond J. Dolan
    Affiliations
    The UCL Max Planck Centre for Computational Psychiatry and Ageing Research, University College London, London, UK

    Wellcome Centre for Human Neuroimaging, University College London, London, UK

    State Key Laboratory of Cognitive Neuroscience and Learning, IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing, 100875, China
    Search for articles by this author
Open AccessPublished:August 22, 2022DOI:https://doi.org/10.1016/j.biopsych.2022.08.016

      Abstract

      Aberrant patterns of cognition, perception, and behaviour seen in psychiatric disorders are thought to be driven by a complex interplay of neural processes that evolve at a rapid temporal scale. Understanding these dynamic processes in vivo in humans has been hampered by a trade-off between the spatial and temporal resolution inherent to current neuroimaging technology. A recent trend in psychiatric research has been the use of high temporal resolution imaging, particularly magnetoencephalography (MEG), often in conjunction with sophisticated machine learning decoding techniques. Developments here promise novel insights into the spatiotemporal dynamics of cognitive phenomena, including domains relevant to psychiatric illness such as reward and avoidance learning, memory, and planning. This review considers recent advances afforded by exploiting this increased spatiotemporal precision, with specific reference to applications the seek to drive a mechanistic understanding of psychopathology and the realisation of preclinical translation.

      Keywords

      An important goal within cognitive neuroscience is to determine the precise neurophysiological features that contribute to the expression of psychiatric phenomena, with an ultimate goal to inform psychiatric diagnosis and treatment. Given the multitude of neuroimaging tools accessible to researchers today, it may seem surprising that neuroimaging research has had scant impact on clinical psychiatry (
      • Kapur S.
      • Phillips A.G.
      • Insel T.R.
      Why has it taken so long for biological psychiatry to develop clinical tests and what to do about it?.
      ,
      • Aryutova K.
      • Paunova R.
      • Kandilarova S.
      • Todeva-Radneva A.
      • Stoyanov D.
      Implications from translational cross-validation of clinical assessment tools for diagnosis and treatment in psychiatry.
      ). Several non-competing explanations have been put forward (

      Abramovitch, Schweiger (n.d.): Misuse of cognitive neuropsychology in psychiatry research: the intoxicating appeal of neo-reductionism. Behav Ther . Retrieved from http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.710.7869&rep=rep1&type=pdf

      ), pointing to either the historical limitations of neuroimaging analyses and their interpretation (
      • Specht K.
      Current Challenges in Translational and Clinical fMRI and Future Directions.
      ,
      • First M.B.
      • Drevets W.C.
      • Carter C.
      • Dickstein D.P.
      • Kasoff L.
      • Kim K.L.
      • et al.
      Clinical Applications of Neuroimaging in Psychiatric Disorders.
      ,

      Fried EI (2021, August): Studying mental health problems as systems, not syndromes. https://doi.org/10.31234/osf.io/k4mhv

      ,
      • Neuner I.
      • Veselinović T.
      • Ramkiran S.
      • Rajkumar R.
      • Schnellbaecher G.J.
      • Shah N.J.
      7T ultra-high-field neuroimaging for mental health: an emerging tool for precision psychiatry?.
      ,
      • Etkin A.
      A Reckoning and Research Agenda for Neuroimaging in Psychiatry.
      ,
      • Saggar M.
      • Uddin L.Q.
      Pushing the Boundaries of Psychiatric Neuroimaging to Ground Diagnosis in Biology.
      ), or to the restrictive, subjective, and arbitrary nature of clinical diagnosis (

      Fried EI (2021, August): Studying mental health problems as systems, not syndromes. https://doi.org/10.31234/osf.io/k4mhv

      ,
      • Etkin A.
      A Reckoning and Research Agenda for Neuroimaging in Psychiatry.
      ,
      • Henderson T.A.
      • van Lierop M.J.
      • McLean M.
      • Uszler J.M.
      • Thornton J.F.
      • Siow Y.-H.
      • et al.
      Functional Neuroimaging in Psychiatry—Aiding in Diagnosis and Guiding Treatment. What the American Psychiatric Association Does Not Know.
      ). Here, we focus on the former. We argue that the utility of neuroimaging in psychiatry has reached an inflection point upon which recent methodological advancements can now dramatically improve the spatiotemporal precision of functional brain mapping, opening new approaches to elucidating the neurocognitive dynamics underlying complex human behaviour and psychopathology.
      Our ability to precisely capture spatiotemporal patterns of neural activity has, until recently, been limited by two primary obstacles. One relates to a trade-off between spatial and temporal resolution that is inherent to a reliance on non-invasive neuroimaging approaches. This limits the ability of any single methodology to provide a complete picture of both the “where” and “when” of the neural processes that underlie complex human cognition and behaviour, potentially obscuring core aspects of neural dynamics that play causal roles in the genesis of psychiatric illnesses.
      A second obstacle is the extent to which it is possible to ascribe precise mechanistic significance to in vivo recorded brain activity; in other words, the “what” and “how” of a neural process. For example, increased blood-oxygen-level dependent (BOLD) signal in the striatum after receipt of a reward is interpreted as indicating a functional role for this structure in reward processing, but this observation lacks specificity as to what that functional role actually is (
      • Adams R.A.
      • Huys Q.J.M.
      • Roiser J.P.
      Computational Psychiatry: towards a mathematically informed understanding of mental illness.
      ). Mechanistic specificity can be gained from designing highly controlled experiments that attempt to isolate a precise cognitive function, usually informed by a computational model, though this often entails reduced ecological validity and generalisability (
      • Mobbs D.
      • Wise T.
      • Suthana N.
      • Guzmán N.
      • Kriegeskorte N.
      • Leibo J.Z.
      Promises and challenges of human computational ethology.
      ,
      • Zaki J.
      • Ochsner K.
      The need for a cognitive neuroscience of naturalistic social cognition.
      ).
      The dynamic nature and real-world relevance of features that characterise psychiatric disorders mean that both spatiotemporal and functional precision are crucial to improving our understanding and, ultimately, guiding development of targeted treatments (
      • Hitchcock P.F.
      • Fried E.I.
      • Frank M.J.
      Computational Psychiatry Needs Time and Context.
      ). In this review, we outline current trends in human neuroimaging that advance a quest for increased spatiotemporal precision. First, we provide an overview of the current spatiotemporal resolution achievable in neuroimaging. Second, we illustrate how to enhance spatiotemporal precision by extracting meaningful state representations from neuroimaging data, as well as how to track the dynamic reinstatement of these processes in the brain, taking recent breakthroughs in the detection of hippocampal replay using magnetoencephalography (MEG) as a case example. Finally, we explore how uncovering the spatiotemporal dynamics of mechanistically-relevant neural activity can be combined with generative modelling of pathological behaviour and cognition, with specific relevance to the burgeoning field of computational psychiatry (
      • Huys Q.J.M.
      • Maia T.V.
      • Paulus M.P.
      Computational Psychiatry: From Mechanistic Insights to the Development of New Treatments.
      ).

      Spatiotemporal precision of neuroimaging

      Non-invasive neuroimaging methods range from modern ultra-high-field MRI that delivers a spatial resolution as fine as 0.5 millimetres (
      • De Martino F.
      • Yacoub E.
      • Kemper V.
      • Moerel M.
      • Uludağ K.
      • De Weerd P.
      • et al.
      The impact of ultra-high field MRI on cognitive and computational neuroimaging.
      ), to older technologies such as electroencephalography (EEG) and MEG that provide measurements of mass neural activity at a millisecond resolution (
      • Cohen D.
      Magnetoencephalography: evidence of magnetic fields produced by alpha-rhythm currents.
      ,
      • Pravdich-Neminsky W.
      Ein Versuch der Registrierung der Elektrischen Gehirnerscheinungen.
      ). Each of these modalities have strengths and weaknesses with regards to spatial and temporal resolution, in addition to factors such as tolerance in freedom of movement (
      • Boto E.
      • Holmes N.
      • Leggett J.
      • Roberts G.
      • Shah V.
      • Meyer S.S.
      • Brookes M.J.
      Moving brain imaging towards real-world applications using a wearable MEG system.
      ) and the precise physiological processes used to index neural activity.
      In psychiatry, it can be conjectured that processes underlying psychopathology encompass rapidly-evolving and spatially-specific neural dynamics. For example, disordered belief formation in schizophrenia has been ascribed to aberrant activity in prefrontal cortex and hippocampus related to reduced synaptic gain, causing an imprecise coding of prior beliefs which, in turn, influences neural responses to surprising stimuli as early as 50 ms post-stimulus onset (
      • Adams R.A.
      • Huys Q.J.M.
      • Roiser J.P.
      Computational Psychiatry: towards a mathematically informed understanding of mental illness.
      ). Similarly, depression has been thought of as a “disconnection” syndrome, where connectivity between anatomically-discrete brain regions is reduced (
      • Liao Y.
      • Huang X.
      • Wu Q.
      • Yang C.
      • Kuang W.
      • Du M.
      • et al.
      Is depression a disconnection syndrome? Meta-analysis of diffusion tensor imaging studies in patients with MDD.
      ,
      • Kaiser R.H.
      • Andrews-Hanna J.R.
      • Wager T.D.
      • Pizzagalli D.A.
      Large-Scale Network Dysfunction in Major Depressive Disorder: A Meta-analysis of Resting-State Functional Connectivity.
      ) but where the rapid, dynamic evolution of this connectivity (i.e., sub-second transient changes in distinct spatial neuronal populations) differ between clinical subtypes (
      • Li J.
      • Li N.
      • Shao X.
      • Chen J.
      • Hao Y.
      • Li X.
      • Hu B.
      Altered Brain Dynamics and Their Ability for Major Depression Detection using EEG Microstates Analysis.
      ,
      • Murphy M.
      • Whitton A.E.
      • Deccy S.
      • Ironside M.L.
      • Rutherford A.
      • Beltzer M.
      • et al.
      Abnormalities in electroencephalographic microstates are state and trait markers of major depressive disorder.
      ), providing a potential biomarker for the efficacy of electroconvulsive therapy (
      • Xin Y.
      • Bai T.
      • Zhang T.
      • Chen Y.
      • Wang K.
      • Yu S.
      • et al.
      Electroconvulsive therapy modulates critical brain dynamics in major depressive disorder patients.
      ). Thus, despite apparent progress using conventional approaches it is nevertheless the case that fundamental research questions related to neural dynamics likely require a level of spatiotemporal precision that has historically been extremely difficult to realise (
      • McFadyen J.
      • Dolan R.J.
      • Garrido M.I.
      The influence of subcortical shortcuts on disordered sensory and cognitive processing.
      ).

      Multimodal imaging

      Considerable effort has been invested in attaining higher spatiotemporal precision by deriving converging results from separate neuroimaging methodologies with complementary spatial and temporal resolutions, either recorded simultaneously (e.g., simultaneous EEG-fMRI) or in separate sessions (e.g., MEG, followed by fMRI) (

      Uludağ K, Roebroeck A (2014): General overview on the merits of multimodal neuroimaging data fusion. Neuroimage 102 Pt 1: 3–10.

      ). In many cases, this multimodal approach to neuroimaging has been informative about brain dynamics underlying psychopathology (
      • Calhoun V.D.
      • Sui J.
      Multimodal Fusion of Brain Imaging Data: A Key to Finding the Missing Link(s) in Complex Mental Illness.
      ). For instance, the amplitude of a fast-latency signature of reward processing detected with EEG correlates with BOLD signal in striatum and, together, this fast striatal reward responsivity is reported as blunted in a subtype of depression characterised by impaired mood reactivity (
      • Foti D.
      • Carlson J.M.
      • Sauder C.L.
      • Proudfit G.H.
      Reward dysfunction in major depression: multimodal neuroimaging evidence for refining the melancholic phenotype.
      ). Thus, multimodal imaging has the potential to enhance detectability of subtle, neurobiological effects that would otherwise be difficult to detect through reliance on a single modality (

      Uludağ K, Roebroeck A (2014): General overview on the merits of multimodal neuroimaging data fusion. Neuroimage 102 Pt 1: 3–10.

      ,
      • Keren H.
      • O’Callaghan G.
      • Vidal-Ribas P.
      • Buzzell G.A.
      • Brotman M.A.
      • Leibenluft E.
      • et al.
      Reward Processing in Depression: A Conceptual and Meta-Analytic Review Across fMRI and EEG Studies.
      ). Multimodal imaging studies, however, impose a significantly higher demand on resources, and a lack of a unifying model can lead to difficulties with interpreting convergent or discrepant multimodal findings (
      • Calhoun V.D.
      • Sui J.
      Multimodal Fusion of Brain Imaging Data: A Key to Finding the Missing Link(s) in Complex Mental Illness.
      ,
      • Zhang Y.-D.
      • Dong Z.
      • Wang S.-H.
      • Yu X.
      • Yao X.
      • Zhou Q.
      • et al.
      Advances in multimodal data fusion in neuroimaging: Overview, challenges, and novel orientation.
      ).

      Increasing granularity using statistical learning

      A recently developed approach to enhancing spatiotemporal precision of a single neuroimaging modality involves the exploitation of machine (or “statistical”) learning, which harnesses a range of statistical techniques to distinguish between neural or behavioural states. This approach has demonstrated that even the most nuanced fluctuations in spatiotemporal neural data may contain relevant information (
      • Stokes M.G.
      • Wolff M.J.
      • Spaak E.
      Decoding Rich Spatial Information with High Temporal Resolution.
      ). These nuances, such as small differences in the angle of neighbouring dipoles in MEG data, create statistically-separable patterns that are identifiable using multivariate pattern classification algorithms.
      An early example of a machine learning approach to neuroimaging data involved decoding visual orientation from human visual cortex using multi-voxel pattern analysis (MVPA) of functional MRI data (
      • Kamitani Y.
      • Tong F.
      Decoding the visual and subjective contents of the human brain.
      ). Although orientation-selective cortical columns are much smaller than the spatial resolution of functional MRI (3 mm3), orientation selectivity can be reliably estimated from signals generated by entire ensembles of voxels. Remarkably, orientation selectivity (
      • Cichy R.M.
      • Ramirez F.M.
      • Pantazis D.
      Can visual information encoded in cortical columns be decoded from magnetoencephalography data in humans?.
      ) and retinotopic maps in primary visual cortex (
      • Nasiotis K.
      • Clavagnier S.
      • Baillet S.
      • Pack C.C.
      High-resolution retinotopic maps estimated with magnetoencephalography.
      ) have now been reliably estimated from MEG data using support vector machine (SVM) classifiers, despite source-reconstructed MEG having a resolution in the order of several millimetres at the cortical surface. This example demonstrates that modern analytic approaches can exploit subtle variation in coarse spatial or temporal information to detect, and classify, neural processes that unfold at a finer scale than the resolution of the imaging modality itself. Such a feat can be achieved by biology-agnostic machine learning methods that distil spatiotemporal information from rich sources of neuroimaging data (as just described), and also by biophysically-realistic models that utilise prior knowledge of neurophysiological activity (provided by other modalities; e.g., invasive electrophysiological recordings in animals), to capture traces of such processes present in non-invasive human neuroimaging data (e.g., dynamic causal modelling of fMRI and M/EEG; (
      • Friston K.J.
      • Harrison L.
      • Penny W.
      Dynamic causal modelling.
      )). Thus, both biologically-informed models and biology-agnostic machine learning methods can be used to offset spatiotemporal constraints of current neuroimaging methodologies.

      Hippocampal replay as a case example

      A striking example of the use of statistical learning to extract precise spatiotemporal information from MEG data comes from pioneering studies demonstrating hippocampal replay in humans (
      • Liu Y.
      • Nour M.M.
      • Schuck N.W.
      • Behrens T.E.J.
      • Dolan R.J.
      Decoding cognition from spontaneous neural activity.
      ). A central tenet of this review is that non-invasive measurement of hippocampal replay in humans is likely to represent a major advance not only for cognitive neuroscience but also biological psychiatry. The approach indicates that neuroimaging data can provide a sufficiently rich source of spatiotemporal information to signal rapid, dynamic, shifts in mental states, thereby allowing for a more precise estimate of when and where cognitive processes unfold in the brain. Below, we detail this approach and discuss how it has been, and can be, exploited to further the field of biological psychiatry.

      The methodological challenge of replay

      Replay was first observed in rodents in the 1990s where, during post-task rest, hippocampal place cells indexing the trajectory of an animal through an environment rapidly reactivated in the same order in which these locations were experienced, albeit with a pronounced temporal compression (
      • Skaggs W.E.
      • McNaughton B.L.
      Replay of neuronal firing sequences in rat hippocampus during sleep following spatial experience.
      ,
      • Nádasdy Z.
      • Hirase H.
      • Czurkó A.
      • Csicsvari J.
      • Buzsáki G.
      Replay and time compression of recurring spike sequences in the hippocampus.
      ,
      • Wilson M.A.
      • McNaughton B.L.
      Reactivation of hippocampal ensemble memories during sleep.
      ). This spontaneous and rapid unfolding activity pattern was subsequently shown to play a causal role in memory consolidation (
      • Jadhav S.P.
      • Kemere C.
      • German P.W.
      • Frank L.M.
      Awake hippocampal sharp-wave ripples support spatial memory.
      ,
      • Dupret D.
      • O’Neill J.
      • Pleydell-Bouverie B.
      • Csicsvari J.
      The reorganization and reactivation of hippocampal maps predict spatial memory performance.
      ,
      • Ego-Stengel V.
      • Wilson M.A.
      Disruption of ripple-associated hippocampal activity during rest impairs spatial learning in the rat.
      ,
      • Girardeau G.
      • Benchenane K.
      • Wiener S.I.
      • Buzsáki G.
      • Zugaro M.B.
      Selective suppression of hippocampal ripples impairs spatial memory.
      ), and has been linked to higher-order cognitive functions such reward learning (
      • Diba K.
      • Buzsáki G.
      Forward and reverse hippocampal place-cell sequences during ripples.
      ,
      • Foster D.J.
      • Wilson M.A.
      Reverse replay of behavioural sequences in hippocampal place cells during the awake state.
      ,
      • Singer A.C.
      • Frank L.M.
      Rewarded outcomes enhance reactivation of experience in the hippocampus.
      ,
      • Ambrose R.E.
      • Pfeiffer B.E.
      • Foster D.J.
      Reverse Replay of Hippocampal Place Cells Is Uniquely Modulated by Changing Reward.
      ,
      • Carey A.A.
      • Tanaka Y.
      • van der Meer M.A.A.
      Reward revaluation biases hippocampal replay content away from the preferred outcome.
      ,
      • Michon F.
      • Sun J.-J.
      • Kim C.Y.
      • Ciliberti D.
      • Kloosterman F.
      Post-learning Hippocampal Replay Selectively Reinforces Spatial Memory for Highly Rewarded Locations.
      ,
      • Mou X.
      • Pokhrel A.
      • Suresh P.
      • Ji D.
      Observational learning promotes hippocampal remote awake replay toward future reward locations.
      ) and planning (
      • Gupta A.S.
      • van der Meer M.A.A.
      • Touretzky D.S.
      • Redish A.D.
      Hippocampal replay is not a simple function of experience.
      ,
      • Mattar M.G.
      • Daw N.D.
      Prioritized memory access explains planning and hippocampal replay.
      ,
      • Ólafsdóttir H.F.
      • Bush D.
      • Barry C.
      The Role of Hippocampal Replay in Memory and Planning.
      ,
      • Ólafsdóttir H.F.
      • Carpenter F.
      • Barry C.
      Task Demands Predict a Dynamic Switch in the Content of Awake Hippocampal Replay.
      ,
      • Shin J.D.
      • Tang W.
      • Jadhav S.P.
      Dynamics of Awake Hippocampal-Prefrontal Replay for Spatial Learning and Memory-Guided Decision Making.
      ).
      In humans, measuring hippocampal replay non-invasively presents a considerable methodological challenge, as one of its putative source (the hippocampus) is located deep within the brain, and the speed with which replay events unfold is extremely fast (in animals, the sequential reactivation of place cells indexing discrete locations is typically separated by tens of milliseconds). This challenge is shared by neuroimaging research in psychiatry, where there is often a need for both spatial and temporal precision. For example, in mood disorders, fast latency activity in deep brain structures, such as the amygdala, allegedly play a pivotal role in the genesis and maintenance of symptoms but is notoriously difficult to measure in vivo (
      • McFadyen J.
      • Dolan R.J.
      • Garrido M.I.
      The influence of subcortical shortcuts on disordered sensory and cognitive processing.
      ). Moreover, replay by its very nature involves reactivation of anatomically-specific neural populations (e.g., specific place cells) that represent specific mental states (e.g., different locations in space). Thus, measuring replay in humans from non-invasive neuroimaging data necessitates innovative approaches, such as the exploitation of statistical learning to extract fast sequential reactivation of state representations (
      • Liu Y.
      • Dolan R.J.
      • Higgins C.
      • Penagos H.
      • Woolrich M.W.
      • Freyja Ólafsdóttir H.
      • et al.
      Temporally delayed linear modelling (TDLM) measures replay in both animals and humans.
      ,
      • Kurth-Nelson Z.
      • Economides M.
      • Dolan R.J.
      • Dayan P.
      Fast Sequences of Non-spatial State Representations in Humans.
      ).

      Measuring hippocampal replay

      An approach to quantifying replay from non-invasive neuroimaging data is Temporally Delayed Linear Modelling (TDLM) (
      • Liu Y.
      • Dolan R.J.
      • Higgins C.
      • Penagos H.
      • Woolrich M.W.
      • Freyja Ólafsdóttir H.
      • et al.
      Temporally delayed linear modelling (TDLM) measures replay in both animals and humans.
      ), which estimates evidence for sequential state reactivation. TDLM capitalises on the fact that reactivation of a particular state within the hippocampus causes a cascade of related activity across a distributed network that includes the entorhinal cortex (
      • Chenani A.
      • Sabariego M.
      • Schlesiger M.I.
      • Leutgeb J.K.
      • Leutgeb S.
      • Leibold C.
      Hippocampal CA1 replay becomes less prominent but more rigid without inputs from medial entorhinal cortex.
      ), medial temporal cortex (
      • Vaz A.P.
      • Wittig Jr., J.H.
      • Inati S.K.
      • Zaghloul K.A.
      Replay of cortical spiking sequences during human memory retrieval.
      ), visual cortex (
      • Ji D.
      • Wilson M.A.
      Coordinated memory replay in the visual cortex and hippocampus during sleep.
      ), and prefrontal cortex (
      • Peyrache A.
      • Khamassi M.
      • Benchenane K.
      • Wiener S.I.
      • Battaglia F.P.
      Replay of rule-learning related neural patterns in the prefrontal cortex during sleep.
      ,
      • Kaefer K.
      • Nardin M.
      • Blahna K.
      • Csicsvari J.
      Replay of Behavioral Sequences in the Medial Prefrontal Cortex during Rule Switching.
      ,
      • Yu J.Y.
      • Liu D.F.
      • Loback A.
      • Grossrubatscher I.
      • Frank L.M.
      Specific hippocampal representations are linked to generalized cortical representations in memory.
      ,
      • Berners-Lee A.
      • Wu X.
      • Foster D.J.
      Prefrontal cortical neurons are selective for non-local hippocampal representations during replay and behavior.
      ). Thus, while hippocampal activity can be challenging to identify from MEG recordings (but far from impossible: see 71,72), information related to a specific memory or state can be decoded from unique spatial patterns of neural activity to uncover rapid, sequential reactivation of prior experiences (
      • Kurth-Nelson Z.
      • Economides M.
      • Dolan R.J.
      • Dayan P.
      Fast Sequences of Non-spatial State Representations in Humans.
      ,
      • Jafarpour A.
      • Fuentemilla L.
      • Horner A.J.
      • Penny W.
      • Duzel E.
      Replay of very early encoding representations during recollection.
      ,
      • Liu Y.
      • Mattar M.G.
      • Behrens T.E.J.
      • Daw N.D.
      • Dolan R.J.
      Experience replay is associated with efficient nonlocal learning.
      ,
      • Wimmer G.E.
      • Liu Y.
      • Vehar N.
      • Behrens T.E.J.
      • Dolan R.J.
      Episodic memory retrieval success is associated with rapid replay of episode content.
      ,
      • Liu Y.
      • Dolan R.J.
      • Kurth-Nelson Z.
      • Behrens T.E.J.
      Human Replay Spontaneously Reorganizes Experience.
      ,
      • Michelmann S.
      • Staresina B.P.
      • Bowman H.
      • Hanslmayr S.
      Speed of time-compressed forward replay flexibly changes in human episodic memory.
      ,
      • Nour M.M.
      • Liu Y.
      • Arumuham A.
      • Kurth-Nelson Z.
      • Dolan R.J.
      Impaired neural replay of inferred relationships in schizophrenia.
      ,
      • Eldar E.
      • Lièvre G.
      • Dayan P.
      • Dolan R.J.
      The roles of online and offline replay in planning.
      ). This ability to detect subtle but relevant spatial information increases both temporal and representational precision (e.g., specific memories) even at relatively low spatial resolution. Importantly, in psychiatry research, representational precision might often be considered more valuable than spatial precision, such as when investigating whether a therapeutic intervention instantiates a change in cognitive processes.
      How can specific states be isolated and captured? Investigators commonly use visual stimuli presented in a particular order to represent distinct “states”. A key idea here is that the brain organises information — spatial or otherwise — into “cognitive maps” constructed from information like conceptual associations or temporal-order relationships (
      • Behrens T.E.J.
      • Muller T.H.
      • Whittington J.C.R.
      • Mark S.
      • Baram A.B.
      • Stachenfeld K.L.
      • Kurth-Nelson Z.
      What Is a Cognitive Map? Organizing Knowledge for Flexible Behavior.
      ). By using visually- and conceptually-unique images, machine learning algorithms can accurately and reliably classify spatial patterns of neural activity associated with viewing each image (Fig. 1A). The sheer size of the visual system in the human brain means that visual stimuli can be classified from distributed spatiotemporal activity generated primarily from occipital and temporal cortices, with classification accuracy typically in the range of 37% to 50%, which is 3 to 8 times higher than what would be expected by chance (
      • Liu Y.
      • Mattar M.G.
      • Behrens T.E.J.
      • Daw N.D.
      • Dolan R.J.
      Experience replay is associated with efficient nonlocal learning.
      ,
      • Liu Y.
      • Dolan R.J.
      • Kurth-Nelson Z.
      • Behrens T.E.J.
      Human Replay Spontaneously Reorganizes Experience.
      ,
      • Nour M.M.
      • Liu Y.
      • Arumuham A.
      • Kurth-Nelson Z.
      • Dolan R.J.
      Impaired neural replay of inferred relationships in schizophrenia.
      ,
      • McFadyen J.
      • Liu Y.
      • Dolan R.J.
      Differential replay for reward and punishment paths predicts approach and avoidance.
      ). Classifiers are typically trained on neural activity patterns recorded during an initial functional localiser, when participants view images before learning about task-related temporal-order relationships (
      • Liu Y.
      • Dolan R.J.
      • Higgins C.
      • Penagos H.
      • Woolrich M.W.
      • Freyja Ólafsdóttir H.
      • et al.
      Temporally delayed linear modelling (TDLM) measures replay in both animals and humans.
      ). Hence, this constitutes a “supervised” machine learning approach, where identity labels are known (e.g., whether participants were viewing image A or image B). The associated MEG sensor patterns then provide a reliable estimate of activity when these states are subsequently reactivated, for example during a cognitive task such as planning (online) or during a rest period (offline) (Fig. 2). Both hippocampus and medial temporal lobe, as well as visual cortex, have been identified as likely sources of such replay events in humans (
      • Liu Y.
      • Mattar M.G.
      • Behrens T.E.J.
      • Daw N.D.
      • Dolan R.J.
      Experience replay is associated with efficient nonlocal learning.
      ,
      • Wimmer G.E.
      • Liu Y.
      • Vehar N.
      • Behrens T.E.J.
      • Dolan R.J.
      Episodic memory retrieval success is associated with rapid replay of episode content.
      ,
      • Nour M.M.
      • Liu Y.
      • Arumuham A.
      • Kurth-Nelson Z.
      • Dolan R.J.
      Impaired neural replay of inferred relationships in schizophrenia.
      ).
      Figure thumbnail gr1
      Figure 1Capturing mental states using statistical learning. (A) Mental states, such as viewing an image, can be differentiated by the unique patterns of evoked spatiotemporal brain activity, captured with MEG. These spatiotemporal state classifiers can then be applied to MEG data during a task of interest (e.g., decision-making), revealing the time-course of state reactivation associated with specific aspects of cognition and behaviour. (B) Visual orientation can be classified from MEG and EEG sensor data due to unique configurations of angled dipoles along the cortical surface. Adapted from Stokes et al. (2015). (C) Different mental states may also evoke different neural network configurations, producing unique patterns of activity across MEG sensors.
      Overall, investigating replay in the human brain exemplifies how a rapidly-evolving neurophysiological signal can be detected and characterised at an extremely fine temporal resolution. More importantly, these studies provide a representational specificity (e.g., states in a cognitive map) that is not easily obtained using traditional neuroimaging analyses. This implies that a “representation-rich” characterisation of neuroimaging data can greatly enhance the granularity of observable neural dynamics (
      • Liu Y.
      • Nour M.M.
      • Schuck N.W.
      • Behrens T.E.J.
      • Dolan R.J.
      Decoding cognition from spontaneous neural activity.
      ), allowing exploration of more abstract neural processes underlying complex cognition.

      Mechanistic specificity

      Computational modelling of behaviour

      The ability to uncover hidden spatiotemporal dynamics of cognition from neuroimaging data has the potential to unlock crucial information about psychiatric disorders that might otherwise be undetectable from behaviour alone. As an example, consider the cognitive processes that contribute to planning. These include an ability to learn and retrieve a cognitive model of the environment that captures how states are connected, the consequences of taking different actions at different states, and the effective appraisal of prospective reward and loss (
      • Mattar M.G.
      • Lengyel M.
      Planning in the brain.
      ). Computations such as these evolve dynamically over time, where one type of processing (e.g., the accessibility of an aversive memory) may influence another (e.g., the perceived probability of being punished) (
      • Gagne C.
      • Dayan P.
      • Bishop S.J.
      When planning to survive goes wrong: predicting the future and replaying the past in anxiety and PTSD.
      ). These dynamics are pervasive in existing computational psychiatry models of behaviour, which reveal information about how specific cognitive mechanisms operate differently in psychiatric disorders (
      • Huys Q.J.M.
      • Browning M.
      • Paulus M.P.
      • Frank M.J.
      Advances in the computational understanding of mental illness.
      ).
      Spatiotemporally-precise neuroimaging can bestow cognitive models with biological plausibility, revealing how modelled dynamics of cognition (where cognition is either a construct, as in algorithmic models like reinforcement learning, or a biophysically-realistic process, as in synthetic models like attractor network models) are supported by the temporal profile of network activity (
      • Huys Q.J.M.
      • Maia T.V.
      • Frank M.J.
      Computational psychiatry as a bridge from neuroscience to clinical applications.
      ). Therefore, it seems reasonable to conjecture that clinical translation of computational psychiatry may be catalysed by approaches to neuroimaging analysis that enhance spatiotemporal precision by: a) validating the dynamics of theory-driven cognitive processes through convergent biological evidence, b) assigning a neurophysiological basis to modelled cognitive mechanisms, potentially revealing targets for treatment, and c) enhancing the informational content of models by revealing hidden states. Below, we describe recent studies that pair spatiotemporally-precise neuroimaging, such as sequential state reactivation during replay, with computational psychiatry models, with a particular focus on structural inference and reward learning.

      Inferring environment structure

      Decoded state representations shed light on how we learn, store, and retrieve structured representations of our environment. The spontaneous reactivation of sequences — both experienced and imagined — is implicated in constructing and utilising internal representations of the environment. For instance, an ordered reactivation of previously-experienced states during a post-task rest period has been shown to correspond not to an experienced structure, but instead to an inferred structure that participants abstracted based on a learned task rule (
      • Liu Y.
      • Dolan R.J.
      • Kurth-Nelson Z.
      • Behrens T.E.J.
      Human Replay Spontaneously Reorganizes Experience.
      ,
      • Nour M.M.
      • Liu Y.
      • Arumuham A.
      • Kurth-Nelson Z.
      • Dolan R.J.
      Impaired neural replay of inferred relationships in schizophrenia.
      ). This sensitivity of reactivated state representations to inferred structural features implies that MEG-decoded replay can provide a neurobiological signature of an ability to structurally reorganise our model of the world.
      A breakdown in structural inference has been conjectured to underlie psychiatric symptoms that indicate inflexible or repetitive thinking, including compulsive behaviour in obsessive-compulsive disorder, detrimental drug consumption in addiction disorders, and incoherent thought in schizophrenia (
      • Groman S.M.
      • Massi B.
      • Mathias S.R.
      • Lee D.
      • Taylor J.R.
      Model-Free and Model-Based Influences in Addiction-Related Behaviors.
      ,
      • Chen C.
      • Takahashi T.
      • Nakagawa S.
      • Inoue T.
      • Kusumi I.
      Reinforcement learning in depression: A review of computational research.
      ,
      • Daw N.
      Model-Based and Model-Free Learning in Anorexia Nervosa and Other Disorders.
      ,
      • Bishop S.J.
      • Gagne C.
      Anxiety, Depression, and Decision Making: A Computational Perspective.
      ,
      • Voon V.
      • Reiter A.
      • Sebold M.
      • Groman S.
      Model-Based Control in Dimensional Psychiatry.
      ). This accords with findings of relatively stronger evidence for model-free decision-making (i.e., habitual behaviour that disregards environment structure), compared to model-based control (i.e., deliberate behaviour that grants flexibility and accuracy at the cost of increased cognitive load) (
      • Voon V.
      • Reiter A.
      • Sebold M.
      • Groman S.
      Model-Based Control in Dimensional Psychiatry.
      ), in these clinical populations.
      In schizophrenia, we can ask whether a putative deficit in structural inference is reflected in spontaneous neural replay. After completing a task in which the temporal order of a stimulus sequence needs to be inferred, even though the “true” order is never experienced, patients with schizophrenia show weaker evidence for reorganisation of ordered state reactivation during rest compared with healthy controls, an effect that localises to hippocampus and corresponds with behaviour (
      • Nour M.M.
      • Liu Y.
      • Arumuham A.
      • Kurth-Nelson Z.
      • Dolan R.J.
      Impaired neural replay of inferred relationships in schizophrenia.
      ). This finding is consistent with a theory of reduced synaptic gain in schizophrenia, which is thought to significantly impact synaptic plasticity and attractor dynamics within hippocampus (
      • Musa A.
      • Khan S.
      • Mujahid M.
      • El-Gaby M.
      The shallow cognitive map hypothesis: A hippocampal framework for thought disorder in schizophrenia.
      ,
      • Nour M.M.
      • Dolan R.J.
      January 15): Synaptic Gain Abnormalities in Schizophrenia and the Potential Relevance for Cognition.
      ,
      • Adams R.A.
      • Pinotsis D.
      • Tsirlis K.
      • Unruh L.
      • Mahajan A.
      • Horas A.M.
      • et al.
      Computational Modeling of Electroencephalography and Functional Magnetic Resonance Imaging Paradigms Indicates a Consistent Loss of Pyramidal Cell Synaptic Gain in Schizophrenia.
      ). This points to a link between an observable cognitive process (impaired structural inference, possibly manifesting as incoherent thought) and a previously unobservable neurophysiological process (replay of an inferred cognitive map in hippocampus) that might guide prognosis, as well as pharmacological and therapeutic treatment (
      • Musa A.
      • Khan S.
      • Mujahid M.
      • El-Gaby M.
      The shallow cognitive map hypothesis: A hippocampal framework for thought disorder in schizophrenia.
      ).

      Making inferences under uncertainty

      A feature of several psychiatric disorders is an impaired ability to update beliefs about the structure of an environment when something changes unexpectedly. For instance, behavioural modelling of decision-making has shown that paranoia and delusions can be explained by having a general expectation that stimulus-outcome contingencies will change more frequently, resulting in poorer learning in volatile environments (
      • Reed E.J.
      • Uddenberg S.
      • Suthaharan P.
      • Mathys C.D.
      • Taylor J.R.
      • Groman S.M.
      • Corlett P.R.
      Paranoia as a deficit in non-social belief updating.
      ,
      • Suthaharan P.
      • Reed E.J.
      • Leptourgos P.
      • Kenney J.G.
      • Uddenberg S.
      • Mathys C.D.
      • et al.
      Paranoia and belief updating during the COVID-19 crisis.
      ,
      • Adams R.A.
      • Vincent P.
      • Benrimoh D.
      • Friston K.J.
      • Parr T.
      Everything is connected: Inference and attractors in delusions.
      ,
      • Sheffield J.M.
      • Suthaharan P.
      • Leptourgos P.
      • Corlett P.R.
      Belief Updating and Paranoia in Individuals with Schizophrenia.
      ,
      • Katthagen T.
      • Fromm S.
      • Wieland L.
      • Schlagenhauf F.
      Models of Dynamic Belief Updating in Psychosis-A Review Across Different Computational Approaches.
      ). This translates to an overweighting of unlikely explanations (i.e., paranoid delusions), the quality of which depends on a complex interplay of other parameters such as mood, prior habits, and whether beliefs pertain to social interaction (
      • Adams R.A.
      • Vincent P.
      • Benrimoh D.
      • Friston K.J.
      • Parr T.
      Everything is connected: Inference and attractors in delusions.
      ).
      Dysfunctional belief updating is a target of cognitive behavioural therapy (CBT), which reports success in correcting beliefs about risk and uncertainty in the context of obsessive-compulsive (OCD) disorder (
      • McKay D.
      • Sookman D.
      • Neziroglu F.
      • Wilhelm S.
      • Stein D.J.
      • Kyrios M.
      • et al.
      Efficacy of cognitive-behavioral therapy for obsessive–compulsive disorder.
      ), as well as in reducing negative beliefs in depression through “cognitive restructuring” methods (
      • Cuijpers P.
      • Berking M.
      • Andersson G.
      • Quigley L.
      • Kleiboer A.
      • Dobson K.S.
      A meta-analysis of cognitive-behavioural therapy for adult depression, alone and in comparison with other treatments.
      ). There are, however, instances where CBT inexplicably fails, such as with the long-term persistence of paranoid delusions (
      • Fried E.I.
      • Koenders M.A.
      • Blom J.D.
      Bleuler revisited: on persecutory delusions and their resistance to therapy.
      ) and with treatment resistance in specific subtypes of OCD (
      • Sookman D.
      • Steketee G.
      Directions in Specialized Cognitive Behavior Therapy for Resistant Obsessive-Compulsive Disorder: Theory and Practice of Two Approaches.
      ), even when administered alongside pharmacotherapy. The ability to derive a precise neural signature of how beliefs evolve over time, much in the same way that state representations are decoded to indicate neural replay (
      • Liu Y.
      • Dolan R.J.
      • Higgins C.
      • Penagos H.
      • Woolrich M.W.
      • Freyja Ólafsdóttir H.
      • et al.
      Temporally delayed linear modelling (TDLM) measures replay in both animals and humans.
      ), can in principle help reveal whether cognitive restructuring in CBT is having a significant impact on generative processes throughout the course of treatment, potentially serving also as a post-treatment predictor of relapse.
      Research on healthy participants has demonstrated that dynamic belief updating can indeed be detected via spatiotemporal decoding of MEG data. Weiss et al. (2021) investigated the computational and neural mechanisms underlying structural inference in uncertain environments with and without an ability to control how information was sampled (
      • Weiss A.
      • Chambon V.
      • Lee J.K.
      • Drugowitsch J.
      • Wyart V.
      Interacting with volatile environments stabilizes hidden-state inference and its brain signatures.
      ). They found that being able to choose which information to sample made environments appear more stable, echoing beliefs people with OCD hold about compulsive and repetitive behaviours (
      • Fradkin I.
      • Adams R.A.
      • Parr T.
      • Roiser J.P.
      • Huppert J.D.
      Searching for an anchor in an unpredictable world: A computational model of obsessive compulsive disorder.
      ). Moreover, MEG pattern classification revealed crucial temporal and spatial dynamics of how evidence was evaluated against current beliefs during information gathering. Specifically, activity in temporal and visual cortex encoded how consistent each piece of evidence was with current beliefs, revealing changes of mind that occurred throughout a trial prior to making a response. These changes of mind were delayed when participants had control over information sampling, consistent with participants reportedly viewing these environments as being more stable. This work elegantly demonstrates how neural pattern classification can reveal temporally-precise trajectories of beliefs with a neuroanatomical grounding, which could provide novel information about such cognitive processes in conditions such as OCD (
      • Weiss A.
      • Chambon V.
      • Lee J.K.
      • Drugowitsch J.
      • Wyart V.
      Interacting with volatile environments stabilizes hidden-state inference and its brain signatures.
      ,

      Rouault M, Weiss A, Lee JK, Drugowitsch J, Chambon V, Wyart V (2021): Controllability reveals defining features of information seeking. Retrieved from https://europepmc.org/article/ppr/ppr260674

      ).

      Tracking the dynamics of reward learning

      Disordered belief updating leads to dysfunctional decision-making, which is a cause of disruption to everyday life in people with certain psychiatric disorders (
      • Bishop S.J.
      • Gagne C.
      Anxiety, Depression, and Decision Making: A Computational Perspective.
      ). In mood disorders, a bias towards using negative information to update beliefs (which we can consider analogous to “learning”) (
      • Pike A.C.
      • Robinson O.J.
      Reinforcement Learning in Patients With Mood and Anxiety Disorders vs Control Individuals: A Systematic Review and Meta-analysis.
      ) can be computationally deduced (e.g., by reinforcement learning models) from patterns of dysfunctional decision-making, such as increased risk aversion in anxiety and reduced reward-seeking behaviour in depression (
      • Bishop S.J.
      • Gagne C.
      Anxiety, Depression, and Decision Making: A Computational Perspective.
      ). Neuroimaging can complement such computational models of decision-making in psychopathology by measuring a “reward prediction error” signal (i.e., the difference between the reward that was received and the reward that was expected), a key computational component in reinforcement learning and active inference models (
      • FitzGerald T.H.B.
      • Dolan R.J.
      • Friston K.
      Dopamine, reward learning, and active inference.
      ). Reward prediction error signals localise to specific neurochemical circuitry (e.g., dopaminergic pathways) and are observable in both M/EEG (

      Liuzzi L, Chang KK, Zheng C, Keren H, Saha D, Nielson DM, Stringaris A (2021): Magnetoencephalographic Correlates of Mood and Reward Dynamics in Human Adolescents. Cereb Cortex. https://doi.org/10.1093/cercor/bhab417

      ,
      • Sambrook T.D.
      • Hardwick B.
      • Wills A.J.
      • Goslin J.
      Model-free and model-based reward prediction errors in EEG.
      ) and fMRI (
      • Kahnt T.
      A decade of decoding reward-related fMRI signals and where we go from here.
      ).
      Reward prediction error signals, detected with fMRI, accurately predict response to CBT in depression, where an increased responsivity of amygdala and striatum to unexpected rewards has been interpreted as indicating a susceptibility to subsequent belief updating during cognitive restructuring during CBT (
      • Queirazza F.
      • Fouragnan E.
      • Steele J.D.
      • Cavanagh J.
      • Philiastides M.G.
      Neural correlates of weighted reward prediction error during reinforcement learning classify response to cognitive behavioral therapy in depression.
      ). In contrast, reward prediction errors derived from computational modelling of behaviour alone have not yet been shown to predict treatment response, highlighting the power of mechanism-focused neuroimaging analysis for detecting subtle but clinically relevant effects. Extending this, we might consider that belief updating occurs not only at outcome receipt (when reward prediction errors occur) but also in anticipation of an event (e.g., worrying about the future in anxiety) (
      • Gagne C.
      • Dayan P.
      • Bishop S.J.
      When planning to survive goes wrong: predicting the future and replaying the past in anxiety and PTSD.
      ) and when recollecting and re-interpreting past events (e.g., rumination in depression or “post-event processing” in social anxiety) (
      • Gagne C.
      • Dayan P.
      • Bishop S.J.
      When planning to survive goes wrong: predicting the future and replaying the past in anxiety and PTSD.
      ,
      • Hitchcock P.
      • Forman E.
      • Rothstein N.
      • Zhang F.
      • Kounios J.
      • Niv Y.
      • Sims C.
      Rumination Derails Reinforcement Learning With Possible Implications for Ineffective Behavior.
      ). Uncovering hidden temporal dynamics of belief updating could broaden our understanding of how events are evaluated and deliberated upon before and after decision-making, potentially enabling a closer mapping to specific symptoms such as rumination and worry.
      In animals, understanding the temporal dynamics of reward learning has benefited from machine learning. An elegant example is that of Rich and Wallis (2016), who used linear discriminant analysis (LDA) to capture patterns of neural firing in OFC corresponding to four potential choice options, each represented by unique images. While the animals deliberated on their choice, neural activity patterns in OFC alternated approximately every 230 ms between the chosen and unchosen option at each trial, with the chosen option becoming increasingly decodable across deliberation time. This also corresponded to fewer switches towards an unchosen option, as well as faster decision-making and less deliberation (
      • Rich E.L.
      • Wallis J.D.
      Decoding subjective decisions from orbitofrontal cortex.
      ). Building on this, recent studies have classified patterns of activity in OFC that represent not only the dynamics of outcome representations, but also features such as task structure (e.g., preconditioned associations between states, predictions of upcoming states) and the expected reward value of each state (
      • Zhou J.
      • Gardner M.P.H.
      • Stalnaker T.A.
      • Ramus S.J.
      • Wikenheiser A.M.
      • Niv Y.
      • Schoenbaum G.
      Rat Orbitofrontal Ensemble Activity Contains Multiplexed but Dissociable Representations of Value and Task Structure in an Odor Sequence Task.
      ,
      • Wang F.
      • Schoenbaum G.
      • Kahnt T.
      Interactions between human orbitofrontal cortex and hippocampus support model-based inference.
      ,
      • Elliott Wimmer G.
      • Büchel C.
      Learning of distant state predictions by the orbitofrontal cortex in humans.
      ).
      Tracking representations of reward over time provide added value to computational models of decision-making. For example, Eldar et al. (2018) investigated whether a person’s mood relates to differences in receptivity to reward, a process thought to play a significant role in the onset of depression and bipolar disorder (
      • Alloy L.B.
      • Olino T.
      • Freed R.D.
      • Nusslock R.
      Role of Reward Sensitivity and Processing in Major Depressive and Bipolar Spectrum Disorders.
      ,
      • Eldar E.
      • Niv Y.
      Interaction between emotional state and learning underlies mood instability.
      ,
      • Huys Q.J.
      • Pizzagalli D.A.
      • Bogdan R.
      • Dayan P.
      Mapping anhedonia onto reinforcement learning: a behavioural meta-analysis.
      ). Here, reinforcement learning models suggested two underlying mechanisms of reward learning: a fast learning process that rapidly forgot, and a slower learning process that persisted across multiple days. This model then formed the basis for a parameterised data set containing trial-by-trial estimates of the prediction errors produced by fast and slow learning rates and where a statistical learning analysis showed these two types of prediction errors were decodable from heart rate and EEG data (recorded from a single wearable electrode) collected over the course of the experiment. Crucially, these physiological representations of prediction error accurately predicted self-reported mood at short and long timescales, revealing a relationship not evident from behaviour alone (
      • Eldar E.
      • Roth C.
      • Dayan P.
      • Dolan R.J.
      Decodability of Reward Learning Signals Predicts Mood Fluctuations.
      ).
      An increasing number of studies now use decoded state representations to investigate how reward is algorithmically processed, with considerable potential for understanding mood disorders such as depression and anxiety (
      • Huys Q.J.M.
      • Daw N.D.
      • Dayan P.
      Depression: a decision-theoretic analysis.
      ). One formulation of value-guided decision-making is the “successor representation”(
      • Dayan P.
      Improving Generalization for Temporal Difference Learning: The Successor Representation.
      ), which describes how we build a predictive map of state values. Recent decoding of functional MRI data has shown that, during decision-making, the successor representation predicts which states are reactivated in the brain more accurately than other behavioural models (
      • Russek E.M.
      • Momennejad I.
      • Botvinick M.M.
      • Gershman S.J.
      • Daw N.D.
      Neural evidence for the successor representation in choice evaluation.
      ). In a similar vein, MEG investigations have shown that neural reactivation of outcomes during choice deliberation is modulated by both the subjective value and probability of an outcome (

      Russek E, Moran R, Liu Y, Dolan RJ, Huys QJM (2021, October 10): Selective outcome reinstatement during evaluation drives heuristics in risky choice. PsyArXiv. https://doi.org/10.31234/osf.io/kb6ew

      ), and predicts subsequent choice behaviour (
      • Castegnetti G.
      • Tzovara A.
      • Khemka S.
      • Melinščak F.
      • Barnes G.R.
      • Dolan R.J.
      • Bach D.R.
      Representation of probabilistic outcomes during risky decision-making.
      ).

      Conclusion

      We highlight a recent trend in the application of statistical learning to neuroimaging data, particularly MEG, where the goal has been to uncover rapid reactivation of state representations that might otherwise go undetected, either due to spatiotemporal limitations of neuroimaging modalities or the complexity of the evolving state representation. These decoded representations can serve as rich and dynamic support for, or latent variables within, computational models of complex cognitive processes, allowing investigation of a range of candidate processes that may go awry in psychiatric disorders. When combined with neurophysiological recordings, such as MEG, pattern classification provides a level of spatiotemporal precision that is virtually impossible to gain from behaviour-only models or from conventional neuroimaging analyses. In turn, combining neural decoding of states with computational models of behaviour or cognition provides a level of representational precision not easily attained using conventional neuroimaging analysis alone. Moreover, by classifying holistic mental states, researchers can access highly temporally-resolved signatures of disorder-related representations, opening new avenues for examining cognition and behaviour in ecological contexts that involve a high degree of representational complexity, including indexing the impact of treatments.
      Table 1, Table 2
      Table 1Key terms and definitions
      TermDefinition
      Machine learningA methodological approach in which an algorithm (e.g., a support vector machine) is iteratively improved to capture relationships between variables in a training data set (
      • Liu Y.
      • Nour M.M.
      • Schuck N.W.
      • Behrens T.E.J.
      • Dolan R.J.
      Decoding cognition from spontaneous neural activity.
      ). The optimised algorithm is then applied to a test data set to predict the same relationships. Machine learning may be supervised or unsupervised, and is generally model-agnostic.
      Statistical learningA branch of machine learning in which a suitable statistical model (e.g., logistic regression) is deliberately selected and fit to a training data set in order to infer relationships between variables, in accordance with the assumptions of the selected model (
      • Skaggs W.E.
      • McNaughton B.L.
      Replay of neuronal firing sequences in rat hippocampus during sleep following spatial experience.
      ). The optimised model may then be used to predict relationships in a test data set.
      Multi-voxel pattern analysis (MVPA)A supervised classification problem that captures the relationship between spatial patterns of BOLD signal across voxels and a particular experimental condition in a training data set (
      • Liu Y.
      • Nour M.M.
      • Schuck N.W.
      • Behrens T.E.J.
      • Dolan R.J.
      Decoding cognition from spontaneous neural activity.
      ). These spatial patterns can then be detected by applying classifiers to a test data set.
      Neural representationA spatiotemporal pattern of neural activity that is reliably evoked by a specific mental or physical state, indicating that the pattern “encodes” the state (
      • Nádasdy Z.
      • Hirase H.
      • Czurkó A.
      • Csicsvari J.
      • Buzsáki G.
      Replay and time compression of recurring spike sequences in the hippocampus.
      ).
      Cognitive mapA neural representation of how different states relate to each other (
      • Wilson M.A.
      • McNaughton B.L.
      Reactivation of hippocampal ensemble memories during sleep.
      ).
      Structural inferenceThe ability to infer how an environment is structured, given previous experience of state-to-state transitions, as well as any higher-order information (
      • Wilson M.A.
      • McNaughton B.L.
      Reactivation of hippocampal ensemble memories during sleep.
      ). In other words, the ability to construct, utilise, and update a cognitive map.
      ReplayA neurophysiological phenomenon whereby neural representations of states are reactivated in a specific order, indicating their relationships within a cognitive map (
      • Jadhav S.P.
      • Kemere C.
      • German P.W.
      • Frank L.M.
      Awake hippocampal sharp-wave ripples support spatial memory.
      ).
      Computational psychiatryA field of research in which generative mathematical models are constructed to explain the relationships between behaviour, cognition, environment, and underlying neurobiology of psychiatric disorders (
      • Cohen D.
      Magnetoencephalography: evidence of magnetic fields produced by alpha-rhythm currents.
      ).
      Reinforcement learningA computational model describing how decision-making is influenced by past experiences of reward (
      • Dupret D.
      • O’Neill J.
      • Pleydell-Bouverie B.
      • Csicsvari J.
      The reorganization and reactivation of hippocampal maps predict spatial memory performance.
      ).
      Cognitive behavioural therapy (CBT)A talking therapy that aims to reduce symptoms of mental disorders by challenging dysfunctional beliefs (cognition) and their associated behaviours (
      • Ego-Stengel V.
      • Wilson M.A.
      Disruption of ripple-associated hippocampal activity during rest impairs spatial learning in the rat.
      ).
      Table 2Outstanding questions in psychiatry that may be addressed by using increasing spatiotemporal resolution of neuroimaging data
      Research questionExisting dataPotential use cases
      What are the fine-grained neurobiological causes of psychiatric symptoms, and can knowledge of this assist with prognosis and/or treatment?Schizophrenia: Disorganised replay suggests a neurophysiological basis for impaired structural inference, implying abnormal NMDA receptor function in hippocampus (
      • Groman S.M.
      • Massi B.
      • Mathias S.R.
      • Lee D.
      • Taylor J.R.
      Model-Free and Model-Based Influences in Addiction-Related Behaviors.
      ,
      • McKay D.
      • Sookman D.
      • Neziroglu F.
      • Wilhelm S.
      • Stein D.J.
      • Kyrios M.
      • et al.
      Efficacy of cognitive-behavioral therapy for obsessive–compulsive disorder.
      ).

      Schizophrenia: Multimodal imaging shows a coupling of computationally-derived belief updates with BOLD signal in striatum that relates to dopamine receptor functionality measured with positron-emission tomography (PET) (132).

      Depression: Functional connectivity measured with fMRI in depression is markedly reduced at rest (
      • Calhoun V.D.
      • Sui J.
      Multimodal Fusion of Brain Imaging Data: A Key to Finding the Missing Link(s) in Complex Mental Illness.
      ,
      • Foti D.
      • Carlson J.M.
      • Sauder C.L.
      • Proudfit G.H.
      Reward dysfunction in major depression: multimodal neuroimaging evidence for refining the melancholic phenotype.
      ). Sub-second transient changes in microstates of functional connectivity detected with EEG is significantly different between clinical subtypes of depression (
      • Keren H.
      • O’Callaghan G.
      • Vidal-Ribas P.
      • Buzzell G.A.
      • Brotman M.A.
      • Leibenluft E.
      • et al.
      Reward Processing in Depression: A Conceptual and Meta-Analytic Review Across fMRI and EEG Studies.
      ,
      • Zhang Y.-D.
      • Dong Z.
      • Wang S.-H.
      • Yu X.
      • Yao X.
      • Zhou Q.
      • et al.
      Advances in multimodal data fusion in neuroimaging: Overview, challenges, and novel orientation.
      ).
      Schizophrenia: Replay of reorganised state sequences may be used as an indicator of the efficacy of dopaminergic antagonists on increasing synaptic gain in hippocampus, supporting structural inference capabilities.

      Depression: MEG may be used as a more spatially-precise measure of rapid changes in microstates of functional connectivity, a measure that could help to predict patient-specific efficacy of electroconvulsive therapy (
      • Stokes M.G.
      • Wolff M.J.
      • Spaak E.
      Decoding Rich Spatial Information with High Temporal Resolution.
      ).
      How can we better estimate the efficacy of CBT in restructuring dysfunctional beliefs?Depression: Reward prediction error signals related to learning in amygdala and striatum (measured with fMRI) predict response of depressed patients to CBT (
      • Eldar E.
      • Niv Y.
      Interaction between emotional state and learning underlies mood instability.
      ).

      General: The perceived congruence between current evidence and prior beliefs can be decoded from MEG activity and used to indicate the time course of belief updating and subsequent decision-making (
      • Kahnt T.
      A decade of decoding reward-related fMRI signals and where we go from here.
      ).
      Depression: By using decoding to track how rewarding outcomes are neurally represented during choice deliberation, we could assess the efficacy of CBT in increasing representation of reward in a manner that relates to improved choice behaviour.

      OCD: Neural signatures of belief updating could indicate how acting on an environment to sample information (as is the case in compulsive behaviour) influences beliefs about uncertain environments, and whether this is influenced by CBT (
      • Kahnt T.
      A decade of decoding reward-related fMRI signals and where we go from here.
      ).
      How do thought patterns (conscious or unconscious) differ between clinical subtypes, and can this guide personalised therapy?Anxiety: Replay supports flexible avoidance of potential threat by simulating inferred trajectories to threat (133).

      General: Replay reflects an ability to infer trajectories that lead to future reward in changing environments (
      • Mattar M.G.
      • Lengyel M.
      Planning in the brain.
      ).
      Anxiety: Patients with anxiety may differ in whether they anxiously anticipate the future or ruminate on the past, which could reflect different magnitudes of forwards replay of paths leading to threat versus backwards replay after outcome receipt. These signatures, if present, could therefore serve as biological markers of anxiety subtypes.

      Uncited reference

      • Haxby J.V.
      • Gobbini M.I.
      • Furey M.L.
      • Ishai A.
      • Schouten J.L.
      • Pietrini P.
      Distributed and overlapping representations of faces and objects in ventral temporal cortex.
      ,
      • deCharms R.C.
      • Zador A.
      Neural representation and the cortical code.
      ,
      • Momennejad I.
      Learning Structures: Predictive Representations, Replay, and Generalization.
      ,
      • Sutton R.S.
      • Barto A.G.
      • Others
      Reinforcement learning.
      ,
      • Nair A.
      • Rutledge R.B.
      • Mason L.
      Under the Hood: Using Computational Psychiatry to Make Psychological Therapies More Mechanism-Focused.
      ,
      • Tierney T.M.
      • Levy A.
      • Barry D.N.
      • Meyer S.S.
      • Shigihara Y.
      • Everatt M.
      • et al.
      Mouth magnetoencephalography: A unique perspective on the human hippocampus.
      ,
      • Ruzich E.
      • Crespo-García M.
      • Dalal S.S.
      • Schneiderman J.F.
      Characterizing hippocampal dynamics with MEG: A systematic review and evidence-based guidelines.
      ,
      • Nour M.M.
      • Dahoun T.
      • Schwartenbeck P.
      • Adams R.A.
      • FitzGerald T.H.B.
      • Coello C.
      • et al.
      Dopaminergic basis for signaling belief updates, but not surprise, and the link to paranoia.
      ,
      • Wise T.
      • Liu Y.
      • Chowdhury F.
      • Dolan R.J.
      Model-based aversive learning in humans is supported by preferential task state reactivation.
      .

      Acknowledgments

      This work was carried out within the framework of a Max Planck UCL Center for Computational Psychiatry and Ageing Research, supported by the Max Planck Society (MPS). RJD and JM are supported by a Wellcome Investigator Award, 098362/Z/12/Z. The Wellcome Centre for Human Neuroimaging (WCHN) is supported by core funding from the Wellcome Trust (203147/Z/16/Z).
      Disclosures
      The authors report no biomedical financial interests or potential conflicts of interest.

      References

        • Kapur S.
        • Phillips A.G.
        • Insel T.R.
        Why has it taken so long for biological psychiatry to develop clinical tests and what to do about it?.
        Mol Psychiatry. 2012; 17: 1174-1179
        • Aryutova K.
        • Paunova R.
        • Kandilarova S.
        • Todeva-Radneva A.
        • Stoyanov D.
        Implications from translational cross-validation of clinical assessment tools for diagnosis and treatment in psychiatry.
        World J Psychiatry. 2021; 11: 169-180
      1. Abramovitch, Schweiger (n.d.): Misuse of cognitive neuropsychology in psychiatry research: the intoxicating appeal of neo-reductionism. Behav Ther . Retrieved from http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.710.7869&rep=rep1&type=pdf

        • Specht K.
        Current Challenges in Translational and Clinical fMRI and Future Directions.
        Front Psychiatry. 2019; 10: 924
        • First M.B.
        • Drevets W.C.
        • Carter C.
        • Dickstein D.P.
        • Kasoff L.
        • Kim K.L.
        • et al.
        Clinical Applications of Neuroimaging in Psychiatric Disorders.
        Am J Psychiatry. 2018; 175: 915-916
      2. Fried EI (2021, August): Studying mental health problems as systems, not syndromes. https://doi.org/10.31234/osf.io/k4mhv

        • Neuner I.
        • Veselinović T.
        • Ramkiran S.
        • Rajkumar R.
        • Schnellbaecher G.J.
        • Shah N.J.
        7T ultra-high-field neuroimaging for mental health: an emerging tool for precision psychiatry?.
        Transl Psychiatry. 2022; 12: 36
        • Etkin A.
        A Reckoning and Research Agenda for Neuroimaging in Psychiatry.
        American Journal of Psychiatry. 2019; 176: 507-511
        • Saggar M.
        • Uddin L.Q.
        Pushing the Boundaries of Psychiatric Neuroimaging to Ground Diagnosis in Biology.
        eNeuro. 2019; 6https://doi.org/10.1523/ENEURO.0384-19.2019
        • Henderson T.A.
        • van Lierop M.J.
        • McLean M.
        • Uszler J.M.
        • Thornton J.F.
        • Siow Y.-H.
        • et al.
        Functional Neuroimaging in Psychiatry—Aiding in Diagnosis and Guiding Treatment. What the American Psychiatric Association Does Not Know.
        Front Psychiatry. 2020; 11https://doi.org/10.3389/fpsyt.2020.00276
        • Adams R.A.
        • Huys Q.J.M.
        • Roiser J.P.
        Computational Psychiatry: towards a mathematically informed understanding of mental illness.
        J Neurol Neurosurg Psychiatry. 2016; 87: 53-63
        • Mobbs D.
        • Wise T.
        • Suthana N.
        • Guzmán N.
        • Kriegeskorte N.
        • Leibo J.Z.
        Promises and challenges of human computational ethology.
        Neuron. 2021; 109: 2224-2238
        • Zaki J.
        • Ochsner K.
        The need for a cognitive neuroscience of naturalistic social cognition.
        Ann N Y Acad Sci. 2009; 1167: 16-30
        • Hitchcock P.F.
        • Fried E.I.
        • Frank M.J.
        Computational Psychiatry Needs Time and Context.
        Annu Rev Psychol. 2022; 73: 243-270
        • Huys Q.J.M.
        • Maia T.V.
        • Paulus M.P.
        Computational Psychiatry: From Mechanistic Insights to the Development of New Treatments.
        Biol Psychiatry Cogn Neurosci Neuroimaging. 2016; 1: 382-385
        • De Martino F.
        • Yacoub E.
        • Kemper V.
        • Moerel M.
        • Uludağ K.
        • De Weerd P.
        • et al.
        The impact of ultra-high field MRI on cognitive and computational neuroimaging.
        Neuroimage. 2018; 168: 366-382
        • Cohen D.
        Magnetoencephalography: evidence of magnetic fields produced by alpha-rhythm currents.
        Science. 1968; 161: 784-786
        • Pravdich-Neminsky W.
        Ein Versuch der Registrierung der Elektrischen Gehirnerscheinungen.
        Zentralbl Physiol. 1912; 27: 951-960
        • Boto E.
        • Holmes N.
        • Leggett J.
        • Roberts G.
        • Shah V.
        • Meyer S.S.
        • Brookes M.J.
        Moving brain imaging towards real-world applications using a wearable MEG system.
        Nature. 2018; : 17-19
        • Liao Y.
        • Huang X.
        • Wu Q.
        • Yang C.
        • Kuang W.
        • Du M.
        • et al.
        Is depression a disconnection syndrome? Meta-analysis of diffusion tensor imaging studies in patients with MDD.
        J Psychiatry Neurosci. 2013; 38: 49-56
        • Kaiser R.H.
        • Andrews-Hanna J.R.
        • Wager T.D.
        • Pizzagalli D.A.
        Large-Scale Network Dysfunction in Major Depressive Disorder: A Meta-analysis of Resting-State Functional Connectivity.
        JAMA Psychiatry. 2015; 72: 603-611
        • Li J.
        • Li N.
        • Shao X.
        • Chen J.
        • Hao Y.
        • Li X.
        • Hu B.
        Altered Brain Dynamics and Their Ability for Major Depression Detection using EEG Microstates Analysis.
        IEEE Transactions on Affective Computing. 2021; (1–1)
        • Murphy M.
        • Whitton A.E.
        • Deccy S.
        • Ironside M.L.
        • Rutherford A.
        • Beltzer M.
        • et al.
        Abnormalities in electroencephalographic microstates are state and trait markers of major depressive disorder.
        Neuropsychopharmacology. 2020; 45: 2030-2037
        • Xin Y.
        • Bai T.
        • Zhang T.
        • Chen Y.
        • Wang K.
        • Yu S.
        • et al.
        Electroconvulsive therapy modulates critical brain dynamics in major depressive disorder patients.
        Brain Stimul. 2022; 15: 214-225
        • McFadyen J.
        • Dolan R.J.
        • Garrido M.I.
        The influence of subcortical shortcuts on disordered sensory and cognitive processing.
        Nat Rev Neurosci. 2020; 21: 264-276
      3. Uludağ K, Roebroeck A (2014): General overview on the merits of multimodal neuroimaging data fusion. Neuroimage 102 Pt 1: 3–10.

        • Calhoun V.D.
        • Sui J.
        Multimodal Fusion of Brain Imaging Data: A Key to Finding the Missing Link(s) in Complex Mental Illness.
        Biological Psychiatry: Cognitive Neuroscience and Neuroimaging. 2016; 1: 230-244
        • Foti D.
        • Carlson J.M.
        • Sauder C.L.
        • Proudfit G.H.
        Reward dysfunction in major depression: multimodal neuroimaging evidence for refining the melancholic phenotype.
        Neuroimage. 2014; 101: 50-58
        • Keren H.
        • O’Callaghan G.
        • Vidal-Ribas P.
        • Buzzell G.A.
        • Brotman M.A.
        • Leibenluft E.
        • et al.
        Reward Processing in Depression: A Conceptual and Meta-Analytic Review Across fMRI and EEG Studies.
        Am J Psychiatry. 2018; 175: 1111-1120
        • Zhang Y.-D.
        • Dong Z.
        • Wang S.-H.
        • Yu X.
        • Yao X.
        • Zhou Q.
        • et al.
        Advances in multimodal data fusion in neuroimaging: Overview, challenges, and novel orientation.
        Inf Fusion. 2020; 64: 149-187
        • Stokes M.G.
        • Wolff M.J.
        • Spaak E.
        Decoding Rich Spatial Information with High Temporal Resolution.
        Trends Cogn Sci. 2015; 19: 636-638
        • Kamitani Y.
        • Tong F.
        Decoding the visual and subjective contents of the human brain.
        Nat Neurosci. 2005; 8: 679-685
        • Cichy R.M.
        • Ramirez F.M.
        • Pantazis D.
        Can visual information encoded in cortical columns be decoded from magnetoencephalography data in humans?.
        Neuroimage. 2015; 121: 193-204
        • Nasiotis K.
        • Clavagnier S.
        • Baillet S.
        • Pack C.C.
        High-resolution retinotopic maps estimated with magnetoencephalography.
        Neuroimage. 2017; 145: 107-117
        • Friston K.J.
        • Harrison L.
        • Penny W.
        Dynamic causal modelling.
        Neuroimage. 2003; 19: 1273-1302
        • Haxby J.V.
        • Gobbini M.I.
        • Furey M.L.
        • Ishai A.
        • Schouten J.L.
        • Pietrini P.
        Distributed and overlapping representations of faces and objects in ventral temporal cortex.
        Science. 2001; 293: 2425-2430
        • Liu Y.
        • Dolan R.J.
        • Higgins C.
        • Penagos H.
        • Woolrich M.W.
        • Freyja Ólafsdóttir H.
        • et al.
        Temporally delayed linear modelling (TDLM) measures replay in both animals and humans.
        eLife. 2021; 10https://doi.org/10.7554/elife.66917
        • deCharms R.C.
        • Zador A.
        Neural representation and the cortical code.
        Annu Rev Neurosci. 2000; 23: 613-647
        • Behrens T.E.J.
        • Muller T.H.
        • Whittington J.C.R.
        • Mark S.
        • Baram A.B.
        • Stachenfeld K.L.
        • Kurth-Nelson Z.
        What Is a Cognitive Map? Organizing Knowledge for Flexible Behavior.
        Neuron. 2018; 100: 490-509
        • Momennejad I.
        Learning Structures: Predictive Representations, Replay, and Generalization.
        Curr Opin Behav Sci. 2020; 32: 155-166
        • Sutton R.S.
        • Barto A.G.
        • Others
        Reinforcement learning.
        J Cogn Neurosci. 1999; 11: 126-134
        • Nair A.
        • Rutledge R.B.
        • Mason L.
        Under the Hood: Using Computational Psychiatry to Make Psychological Therapies More Mechanism-Focused.
        Front Psychiatry. 2020; 11: 140
        • Liu Y.
        • Nour M.M.
        • Schuck N.W.
        • Behrens T.E.J.
        • Dolan R.J.
        Decoding cognition from spontaneous neural activity.
        Nat Rev Neurosci. 2022; 23: 204-214
        • Skaggs W.E.
        • McNaughton B.L.
        Replay of neuronal firing sequences in rat hippocampus during sleep following spatial experience.
        Science. 1996; 271: 1870-1873
        • Nádasdy Z.
        • Hirase H.
        • Czurkó A.
        • Csicsvari J.
        • Buzsáki G.
        Replay and time compression of recurring spike sequences in the hippocampus.
        J Neurosci. 1999; 19: 9497-9507
        • Wilson M.A.
        • McNaughton B.L.
        Reactivation of hippocampal ensemble memories during sleep.
        Science. 1994; 265: 676-679
        • Jadhav S.P.
        • Kemere C.
        • German P.W.
        • Frank L.M.
        Awake hippocampal sharp-wave ripples support spatial memory.
        Science. 2012; 336: 1454-1458
        • Dupret D.
        • O’Neill J.
        • Pleydell-Bouverie B.
        • Csicsvari J.
        The reorganization and reactivation of hippocampal maps predict spatial memory performance.
        Nat Neurosci. 2010; 13: 995-1002
        • Ego-Stengel V.
        • Wilson M.A.
        Disruption of ripple-associated hippocampal activity during rest impairs spatial learning in the rat.
        Hippocampus. 2010; 20: 1-10
        • Girardeau G.
        • Benchenane K.
        • Wiener S.I.
        • Buzsáki G.
        • Zugaro M.B.
        Selective suppression of hippocampal ripples impairs spatial memory.
        Nat Neurosci. 2009; 12: 1222-1223
        • Diba K.
        • Buzsáki G.
        Forward and reverse hippocampal place-cell sequences during ripples.
        Nat Neurosci. 2007; 10: 1241-1242
        • Foster D.J.
        • Wilson M.A.
        Reverse replay of behavioural sequences in hippocampal place cells during the awake state.
        Nature. 2006; 440: 680-683
        • Singer A.C.
        • Frank L.M.
        Rewarded outcomes enhance reactivation of experience in the hippocampus.
        Neuron. 2009; 64: 910-921
        • Ambrose R.E.
        • Pfeiffer B.E.
        • Foster D.J.
        Reverse Replay of Hippocampal Place Cells Is Uniquely Modulated by Changing Reward.
        Neuron. 2016; 91: 1124-1136
        • Carey A.A.
        • Tanaka Y.
        • van der Meer M.A.A.
        Reward revaluation biases hippocampal replay content away from the preferred outcome.
        Nat Neurosci. 2019; 22: 1450-1459
        • Michon F.
        • Sun J.-J.
        • Kim C.Y.
        • Ciliberti D.
        • Kloosterman F.
        Post-learning Hippocampal Replay Selectively Reinforces Spatial Memory for Highly Rewarded Locations.
        Curr Biol. 2019; 29 (e5): 1436-1444
        • Mou X.
        • Pokhrel A.
        • Suresh P.
        • Ji D.
        Observational learning promotes hippocampal remote awake replay toward future reward locations.
        Neuron. 2021; https://doi.org/10.1016/j.neuron.2021.12.005
        • Gupta A.S.
        • van der Meer M.A.A.
        • Touretzky D.S.
        • Redish A.D.
        Hippocampal replay is not a simple function of experience.
        Neuron. 2010; 65: 695-705
        • Mattar M.G.
        • Daw N.D.
        Prioritized memory access explains planning and hippocampal replay.
        Nature Neuroscience. 2018; 21: 1609-1617
        • Ólafsdóttir H.F.
        • Bush D.
        • Barry C.
        The Role of Hippocampal Replay in Memory and Planning.
        Curr Biol. 2018; 28 (–R50): R37
        • Ólafsdóttir H.F.
        • Carpenter F.
        • Barry C.
        Task Demands Predict a Dynamic Switch in the Content of Awake Hippocampal Replay.
        Neuron. 2017; 96: 925-935.e6
        • Shin J.D.
        • Tang W.
        • Jadhav S.P.
        Dynamics of Awake Hippocampal-Prefrontal Replay for Spatial Learning and Memory-Guided Decision Making.
        Neuron. 2019; 104 (e7): 1110-1125
        • Kurth-Nelson Z.
        • Economides M.
        • Dolan R.J.
        • Dayan P.
        Fast Sequences of Non-spatial State Representations in Humans.
        Neuron. 2016; 91: 194-204
        • Chenani A.
        • Sabariego M.
        • Schlesiger M.I.
        • Leutgeb J.K.
        • Leutgeb S.
        • Leibold C.
        Hippocampal CA1 replay becomes less prominent but more rigid without inputs from medial entorhinal cortex.
        Nat Commun. 2019; 10: 1341
        • Vaz A.P.
        • Wittig Jr., J.H.
        • Inati S.K.
        • Zaghloul K.A.
        Replay of cortical spiking sequences during human memory retrieval.
        Science. 2020; 367: 1131-1134
        • Ji D.
        • Wilson M.A.
        Coordinated memory replay in the visual cortex and hippocampus during sleep.
        Nat Neurosci. 2007; 10: 100-107
        • Peyrache A.
        • Khamassi M.
        • Benchenane K.
        • Wiener S.I.
        • Battaglia F.P.
        Replay of rule-learning related neural patterns in the prefrontal cortex during sleep.
        Nat Neurosci. 2009; 12: 919-926
        • Kaefer K.
        • Nardin M.
        • Blahna K.
        • Csicsvari J.
        Replay of Behavioral Sequences in the Medial Prefrontal Cortex during Rule Switching.
        Neuron. 2020; 106: 154-165.e6
        • Yu J.Y.
        • Liu D.F.
        • Loback A.
        • Grossrubatscher I.
        • Frank L.M.
        Specific hippocampal representations are linked to generalized cortical representations in memory.
        Nat Commun. 2018; 9: 2209
        • Berners-Lee A.
        • Wu X.
        • Foster D.J.
        Prefrontal cortical neurons are selective for non-local hippocampal representations during replay and behavior.
        J Neurosci. 2021; https://doi.org/10.1523/JNEUROSCI.1158-20.2021
        • Tierney T.M.
        • Levy A.
        • Barry D.N.
        • Meyer S.S.
        • Shigihara Y.
        • Everatt M.
        • et al.
        Mouth magnetoencephalography: A unique perspective on the human hippocampus.
        Neuroimage. 2021; 225117443
        • Ruzich E.
        • Crespo-García M.
        • Dalal S.S.
        • Schneiderman J.F.
        Characterizing hippocampal dynamics with MEG: A systematic review and evidence-based guidelines.
        Hum Brain Mapp. 2019; 40: 1353-1375
        • Jafarpour A.
        • Fuentemilla L.
        • Horner A.J.
        • Penny W.
        • Duzel E.
        Replay of very early encoding representations during recollection.
        J Neurosci. 2014; 34: 242-248
        • Liu Y.
        • Mattar M.G.
        • Behrens T.E.J.
        • Daw N.D.
        • Dolan R.J.
        Experience replay is associated with efficient nonlocal learning.
        Science. 2021; 372https://doi.org/10.1126/science.abf1357
        • Wimmer G.E.
        • Liu Y.
        • Vehar N.
        • Behrens T.E.J.
        • Dolan R.J.
        Episodic memory retrieval success is associated with rapid replay of episode content.
        Nat Neurosci. 2020; 23: 1025-1033
        • Liu Y.
        • Dolan R.J.
        • Kurth-Nelson Z.
        • Behrens T.E.J.
        Human Replay Spontaneously Reorganizes Experience.
        Cell. 2019; 178 (e14): 640-652
        • Michelmann S.
        • Staresina B.P.
        • Bowman H.
        • Hanslmayr S.
        Speed of time-compressed forward replay flexibly changes in human episodic memory.
        Nat Hum Behav. 2019; 3: 143-154
        • Nour M.M.
        • Liu Y.
        • Arumuham A.
        • Kurth-Nelson Z.
        • Dolan R.J.
        Impaired neural replay of inferred relationships in schizophrenia.
        Cell. 2021; 184 (e17): 4315-4328
        • Eldar E.
        • Lièvre G.
        • Dayan P.
        • Dolan R.J.
        The roles of online and offline replay in planning.
        Elife. 2020; 9https://doi.org/10.7554/eLife.56911
        • McFadyen J.
        • Liu Y.
        • Dolan R.J.
        Differential replay for reward and punishment paths predicts approach and avoidance.
        bioRxiv. 2021, November 19; (11.18.468950): 2021
        • Mattar M.G.
        • Lengyel M.
        Planning in the brain.
        Neuron. 2022; 110: 914-934
        • Gagne C.
        • Dayan P.
        • Bishop S.J.
        When planning to survive goes wrong: predicting the future and replaying the past in anxiety and PTSD.
        Current Opinion in Behavioral Sciences. 2018; 24: 89-95
        • Huys Q.J.M.
        • Browning M.
        • Paulus M.P.
        • Frank M.J.
        Advances in the computational understanding of mental illness.
        Neuropsychopharmacology. 2021; 46: 3-19
        • Huys Q.J.M.
        • Maia T.V.
        • Frank M.J.
        Computational psychiatry as a bridge from neuroscience to clinical applications.
        Nat Neurosci. 2016; 19: 404-413
        • Groman S.M.
        • Massi B.
        • Mathias S.R.
        • Lee D.
        • Taylor J.R.
        Model-Free and Model-Based Influences in Addiction-Related Behaviors.
        Biol Psychiatry. 2019; 85: 936-945
        • Chen C.
        • Takahashi T.
        • Nakagawa S.
        • Inoue T.
        • Kusumi I.
        Reinforcement learning in depression: A review of computational research.
        Neurosci Biobehav Rev. 2015; 55: 247-267
        • Daw N.
        Model-Based and Model-Free Learning in Anorexia Nervosa and Other Disorders.
        Biol Psychiatry. 2020; 87: S20
        • Bishop S.J.
        • Gagne C.
        Anxiety, Depression, and Decision Making: A Computational Perspective.
        Annu Rev Neurosci. 2018; 41: 371-388
        • Voon V.
        • Reiter A.
        • Sebold M.
        • Groman S.
        Model-Based Control in Dimensional Psychiatry.
        Biol Psychiatry. 2017; 82: 391-400
        • Musa A.
        • Khan S.
        • Mujahid M.
        • El-Gaby M.
        The shallow cognitive map hypothesis: A hippocampal framework for thought disorder in schizophrenia.
        Schizophrenia. 2022; 8: 1-11
        • Nour M.M.
        • Dolan R.J.
        January 15): Synaptic Gain Abnormalities in Schizophrenia and the Potential Relevance for Cognition.
        Biological Psychiatry. 2022; 91: 167-169
        • Adams R.A.
        • Pinotsis D.
        • Tsirlis K.
        • Unruh L.
        • Mahajan A.
        • Horas A.M.
        • et al.
        Computational Modeling of Electroencephalography and Functional Magnetic Resonance Imaging Paradigms Indicates a Consistent Loss of Pyramidal Cell Synaptic Gain in Schizophrenia.
        Biol Psychiatry. 2022; 91: 202-215
        • Reed E.J.
        • Uddenberg S.
        • Suthaharan P.
        • Mathys C.D.
        • Taylor J.R.
        • Groman S.M.
        • Corlett P.R.
        Paranoia as a deficit in non-social belief updating.
        eLife. 2020; 9https://doi.org/10.7554/elife.56345
        • Suthaharan P.
        • Reed E.J.
        • Leptourgos P.
        • Kenney J.G.
        • Uddenberg S.
        • Mathys C.D.
        • et al.
        Paranoia and belief updating during the COVID-19 crisis.
        Nat Hum Behav. 2021; 5: 1190-1202
        • Adams R.A.
        • Vincent P.
        • Benrimoh D.
        • Friston K.J.
        • Parr T.
        Everything is connected: Inference and attractors in delusions.
        Schizophr Res. 2022; 245: 5-22
        • Sheffield J.M.
        • Suthaharan P.
        • Leptourgos P.
        • Corlett P.R.
        Belief Updating and Paranoia in Individuals with Schizophrenia.
        Biol Psychiatry Cogn Neurosci Neuroimaging. 2022; https://doi.org/10.1016/j.bpsc.2022.03.013
        • Katthagen T.
        • Fromm S.
        • Wieland L.
        • Schlagenhauf F.
        Models of Dynamic Belief Updating in Psychosis-A Review Across Different Computational Approaches.
        Front Psychiatry. 2022; 13814111
        • McKay D.
        • Sookman D.
        • Neziroglu F.
        • Wilhelm S.
        • Stein D.J.
        • Kyrios M.
        • et al.
        Efficacy of cognitive-behavioral therapy for obsessive–compulsive disorder.
        Psychiatry Res. 2015; 225: 236-246
        • Cuijpers P.
        • Berking M.
        • Andersson G.
        • Quigley L.
        • Kleiboer A.
        • Dobson K.S.
        A meta-analysis of cognitive-behavioural therapy for adult depression, alone and in comparison with other treatments.
        Can J Psychiatry. 2013; 58: 376-385
        • Fried E.I.
        • Koenders M.A.
        • Blom J.D.
        Bleuler revisited: on persecutory delusions and their resistance to therapy.
        Lancet Psychiatry. 2021; 8: 644-646
        • Sookman D.
        • Steketee G.
        Directions in Specialized Cognitive Behavior Therapy for Resistant Obsessive-Compulsive Disorder: Theory and Practice of Two Approaches.
        Cogn Behav Pract. 2007; 14: 1-17
        • Weiss A.
        • Chambon V.
        • Lee J.K.
        • Drugowitsch J.
        • Wyart V.
        Interacting with volatile environments stabilizes hidden-state inference and its brain signatures.
        Nat Commun. 2021; 12: 2228
        • Fradkin I.
        • Adams R.A.
        • Parr T.
        • Roiser J.P.
        • Huppert J.D.
        Searching for an anchor in an unpredictable world: A computational model of obsessive compulsive disorder.
        Psychol Rev. 2020; 127: 672-699
      4. Rouault M, Weiss A, Lee JK, Drugowitsch J, Chambon V, Wyart V (2021): Controllability reveals defining features of information seeking. Retrieved from https://europepmc.org/article/ppr/ppr260674

        • Pike A.C.
        • Robinson O.J.
        Reinforcement Learning in Patients With Mood and Anxiety Disorders vs Control Individuals: A Systematic Review and Meta-analysis.
        JAMA Psychiatry. 2022; 79: 313-322
        • FitzGerald T.H.B.
        • Dolan R.J.
        • Friston K.
        Dopamine, reward learning, and active inference.
        Front Comput Neurosci. 2015; 9: 136
      5. Liuzzi L, Chang KK, Zheng C, Keren H, Saha D, Nielson DM, Stringaris A (2021): Magnetoencephalographic Correlates of Mood and Reward Dynamics in Human Adolescents. Cereb Cortex. https://doi.org/10.1093/cercor/bhab417

        • Sambrook T.D.
        • Hardwick B.
        • Wills A.J.
        • Goslin J.
        Model-free and model-based reward prediction errors in EEG.
        Neuroimage. 2018; 178: 162-171
        • Kahnt T.
        A decade of decoding reward-related fMRI signals and where we go from here.
        NeuroImage. 2018; 180: 324-333
        • Queirazza F.
        • Fouragnan E.
        • Steele J.D.
        • Cavanagh J.
        • Philiastides M.G.
        Neural correlates of weighted reward prediction error during reinforcement learning classify response to cognitive behavioral therapy in depression.
        Sci Adv. 2019; 5eaav4962
        • Hitchcock P.
        • Forman E.
        • Rothstein N.
        • Zhang F.
        • Kounios J.
        • Niv Y.
        • Sims C.
        Rumination Derails Reinforcement Learning With Possible Implications for Ineffective Behavior.
        Clin Psychol Sci. 2021; 21677026211051324
        • Rich E.L.
        • Wallis J.D.
        Decoding subjective decisions from orbitofrontal cortex.
        Nat Neurosci. 2016; 19: 973-980
        • Zhou J.
        • Gardner M.P.H.
        • Stalnaker T.A.
        • Ramus S.J.
        • Wikenheiser A.M.
        • Niv Y.
        • Schoenbaum G.
        Rat Orbitofrontal Ensemble Activity Contains Multiplexed but Dissociable Representations of Value and Task Structure in an Odor Sequence Task.
        Curr Biol. 2019; 29 (e3): 897-907
        • Wang F.
        • Schoenbaum G.
        • Kahnt T.
        Interactions between human orbitofrontal cortex and hippocampus support model-based inference.
        PLoS Biol. 2020; 18e3000578
        • Elliott Wimmer G.
        • Büchel C.
        Learning of distant state predictions by the orbitofrontal cortex in humans.
        Nat Commun. 2019; 10: 2554
        • Alloy L.B.
        • Olino T.
        • Freed R.D.
        • Nusslock R.
        Role of Reward Sensitivity and Processing in Major Depressive and Bipolar Spectrum Disorders.
        Behav Ther. 2016; 47: 600-621
        • Eldar E.
        • Niv Y.
        Interaction between emotional state and learning underlies mood instability.
        Nat Commun. 2015; 6: 6149
        • Huys Q.J.
        • Pizzagalli D.A.
        • Bogdan R.
        • Dayan P.
        Mapping anhedonia onto reinforcement learning: a behavioural meta-analysis.
        Biol Mood Anxiety Disord. 2013; 3: 12
        • Eldar E.
        • Roth C.
        • Dayan P.
        • Dolan R.J.
        Decodability of Reward Learning Signals Predicts Mood Fluctuations.
        Curr Biol. 2018; 28 (e7): 1433-1439
        • Huys Q.J.M.
        • Daw N.D.
        • Dayan P.
        Depression: a decision-theoretic analysis.
        Annu Rev Neurosci. 2015; 38: 1-23
        • Dayan P.
        Improving Generalization for Temporal Difference Learning: The Successor Representation.
        Neural Comput. 1993; 5: 613-624
        • Russek E.M.
        • Momennejad I.
        • Botvinick M.M.
        • Gershman S.J.
        • Daw N.D.
        Neural evidence for the successor representation in choice evaluation.
        bioRxiv. 2021, August 31; (2021.08.29.458114)
      6. Russek E, Moran R, Liu Y, Dolan RJ, Huys QJM (2021, October 10): Selective outcome reinstatement during evaluation drives heuristics in risky choice. PsyArXiv. https://doi.org/10.31234/osf.io/kb6ew

        • Castegnetti G.
        • Tzovara A.
        • Khemka S.
        • Melinščak F.
        • Barnes G.R.
        • Dolan R.J.
        • Bach D.R.
        Representation of probabilistic outcomes during risky decision-making.
        Nat Commun. 2020; 11: 2419
        • Nour M.M.
        • Dahoun T.
        • Schwartenbeck P.
        • Adams R.A.
        • FitzGerald T.H.B.
        • Coello C.
        • et al.
        Dopaminergic basis for signaling belief updates, but not surprise, and the link to paranoia.
        Proc Natl Acad Sci U S A. 2018; 115: E10167-E10176
        • Wise T.
        • Liu Y.
        • Chowdhury F.
        • Dolan R.J.
        Model-based aversive learning in humans is supported by preferential task state reactivation.
        Sci Adv. 2021; 7https://doi.org/10.1126/sciadv.abf9616