Advertisement

Computational Psychosomatics and Computational Psychiatry: Toward a Joint Framework for Differential Diagnosis

  • Frederike H. Petzschner
    Affiliations
    Translational Neuromodeling Unit, Institute for Biomedical Engineering, University of Zurich and Swiss Federal Institute of Technology Zurich, Zurich, Switzerland
    Search for articles by this author
  • Lilian A.E. Weber
    Affiliations
    Translational Neuromodeling Unit, Institute for Biomedical Engineering, University of Zurich and Swiss Federal Institute of Technology Zurich, Zurich, Switzerland
    Search for articles by this author
  • Tim Gard
    Affiliations
    Translational Neuromodeling Unit, Institute for Biomedical Engineering, University of Zurich and Swiss Federal Institute of Technology Zurich, Zurich, Switzerland

    Center for Complementary and Integrative Medicine, University Hospital Zurich, Zurich, Switzerland
    Search for articles by this author
  • Klaas E. Stephan
    Correspondence
    Address correspondence to Klaas E. Stephan, M.D., Dr.med., Ph.D., Translational Neuromodeling Unit, Institute for Biomedical Engineering, University of Zurich and Swiss Federal Institute of Technology Zurich, Wilfriedstrasse 6, CH-8032 Zurich, Switzerland.
    Affiliations
    Translational Neuromodeling Unit, Institute for Biomedical Engineering, University of Zurich and Swiss Federal Institute of Technology Zurich, Zurich, Switzerland

    Max Planck Institute for Metabolism Research, Cologne, Germany

    Wellcome Trust Centre for Neuroimaging, University College London, London, United Kingdom
    Search for articles by this author
Open AccessPublished:May 24, 2017DOI:https://doi.org/10.1016/j.biopsych.2017.05.012

      Abstract

      This article outlines how a core concept from theories of homeostasis and cybernetics, the inference-control loop, may be used to guide differential diagnosis in computational psychiatry and computational psychosomatics. In particular, we discuss 1) how conceptualizing perception and action as inference-control loops yields a joint computational perspective on brain-world and brain-body interactions and 2) how the concrete formulation of this loop as a hierarchical Bayesian model points to key computational quantities that inform a taxonomy of potential disease mechanisms. We consider the utility of this perspective for differential diagnosis in concrete clinical applications.

      Keywords

      Psychiatry faces major challenges: its nosology is agnostic about mechanisms, lacks predictive validity, and leads to trial-and-error treatment (
      • Casey B.J.
      • Craddock N.
      • Cuthbert B.N.
      • Hyman S.E.
      • Lee F.S.
      • Ressler K.J.
      DSM-5 and RDoC: Progress in psychiatry research?.
      ,
      • Stephan K.E.
      • Bach D.R.
      • Fletcher P.C.
      • Flint J.
      • Frank M.J.
      • Friston K.J.
      • et al.
      Charting the landscape of priority problems in psychiatry, part 1: Classification and diagnosis.
      ). Strikingly, neuroscientific advances have hardly affected nosology or clinical practice (
      • Kapur S.
      • Phillips A.G.
      • Insel T.R.
      Why has it taken so long for biological psychiatry to develop clinical tests and what to do about it?.
      ). One response to this disconnect is computational psychiatry, with its emerging focus on clinical applications (
      • Maia T.V.
      • Frank M.J.
      From reinforcement learning models to psychiatric and neurological disorders.
      ,
      • Montague P.R.
      • Dolan R.J.
      • Friston K.J.
      • Dayan P.
      Computational psychiatry.
      ,
      • Stephan K.E.
      • Mathys C.
      Computational approaches to psychiatry.
      ,
      • Friston K.J.
      • Stephan K.E.
      • Montague R.
      • Dolan R.J.
      Computational psychiatry: The brain as a phantastic organ.
      ,
      • Wang X.J.
      • Krystal J.H.
      Computational psychiatry.
      ,
      • Huys Q.J.
      • Maia T.V.
      • Frank M.J.
      Computational psychiatry as a bridge from neuroscience to clinical applications.
      ,
      • Paulus M.P.
      • Huys Q.J.
      • Maia T.V.
      A roadmap for the development of applied computational psychiatry.
      ,
      • Corlett P.R.
      • Fletcher P.C.
      Computational psychiatry: A Rosetta Stone linking the brain to mental illness.
      ). One strategy for computational psychiatry is to learn from internal medicine, where mechanistic frameworks for differential diagnosis enable targeted treatment decisions for individual patients. Importantly, differential diagnosis does not necessarily require molecular mechanisms. Much coarser distinctions—inflammatory, infectious, vascular, neoplastic, autoimmunological, or hereditary causes of disease—can provide crucial guidance for treatment, as they disclose fundamentally distinct disease processes.
      This article outlines a framework for differential diagnosis that is motivated by a general computational perspective on brain function. While not the first attempt of its kind (
      • Maia T.V.
      • Frank M.J.
      From reinforcement learning models to psychiatric and neurological disorders.
      ,
      • Stephan K.E.
      • Friston K.J.
      • Frith C.D.
      Dysconnection in schizophrenia: From abnormal synaptic plasticity to failures of self-monitoring.
      ,
      • Huys Q.J.
      • Guitart-Masip M.
      • Dolan R.J.
      • Dayan P.
      Decision-theoretic psychiatry.
      ,
      • Mathys C.
      How could we get nosology from computation?.
      ), this article makes three contributions. First, we adopt a disease-independent motif—the inference-control loop as fundament of cybernetic theories (
      • Wiener N.
      Cybernetics.
      ,
      • Ashby W.R.
      An Introduction to Cybernetics.
      ,
      • Powers W.T.
      Feedback: Beyond behaviorism.
      ,
      • Conant R.C.
      • Ashby W.R.
      Every good regulator of a system must be a model of that system.
      )—and consider how this may help in systematizing computational perspectives on brain-world and brain-body interactions. Second, we consider a hierarchical Bayesian implementation that suggests three possible computational quantities (predictions, prediction errors [PEs], and their precisions) at five potential failure loci (sensation, perception, metacognition, forecasting, action). Third, we discuss the potential clinical utility of this taxonomy for differential diagnosis in computational psychiatry and psychosomatics [compare (
      • Mathys C.
      How could we get nosology from computation?.
      ,
      • Stephan K.E.
      • Manjaly Z.M.
      • Mathys C.
      • Weber L.A.E.
      • Paliwal S.
      • Gard T.
      • et al.
      Allostatic self-efficacy: A metacognitive theory of dyshomeostasis-induced fatigue and depression.
      ,
      • Haker H.
      • Schneebeli M.
      • Stephan K.E.
      Can Bayesian theories of autism spectrum disorder help improve clinical practice?.
      ,
      • Stephan K.E.
      • Baldeweg T.
      • Friston K.J.
      Synaptic plasticity and dysconnection in schizophrenia.
      )].

      Inference-Control Loops

      Different theories of adaptive behavior exist, but they share common themes. We focus on the closed loop between sensations and actions that is at the core of classical cybernetic theories (
      • Wiener N.
      Cybernetics.
      ,
      • Ashby W.R.
      An Introduction to Cybernetics.
      ) and homeostatic principles (
      • Cannon W.B.
      Organization for physiological homeostasis.
      ,
      • Modell H.
      • Cliff W.
      • Michael J.
      • McFarland J.
      • Wenderoth M.P.
      • Wright A.
      A physiologist’s view of homeostasis.
      ). We first summarize extended cybernetic/homeostatic theories (Figure 1) before considering one particular implementation as a foundation for a joint taxonomy of disease mechanisms in psychiatry and psychosomatics.
      Figure thumbnail gr1
      Figure 1(A) Simple example of a homeostatic reflex arc as described by classical cybernetics. Sensory inputs (sensations) about an environmental quantity “X” (e.g., current body temperature) are compared with a predefined set-point (e.g., ideal body temperature). Corrective actions occur as a function of the mismatch between input and set-point, such that “X” is moving closer to the set-point (e.g., heating or cooling the body). (B) Extension to an inference-control loop, where perception (inference of environmental states) under an individual’s generative model of the world updates beliefs that change the reflex arc’s set-point (e.g., allostatic control of bodily states); in other cases, actions might be chosen based on the perception rather than the sensation (not shown here). (C) Further extension of the inference-control loop to include forecasting and metacognition. We wish to emphasize that this plot is highly schematic and provides a core summary of different types of inference-control loops; it should not be misunderstood as a detailed circuit proposal.
      A useful starting point to reflect on adaptive behavior is the observation that it must be constrained by requirements of bodily homeostasis. In the simplest case, actions can be purely reactive. For instance, to maintain constant body temperature, sensor information can be compared with a predefined set-point (e.g., 37°C). Actions, such as heating or cooling the body, are then selected to bring sensory inputs closer to that set-point. This reflex arc—which implements the same feedback control as a simple thermostat—is illustrated in Figure 1A.
      If biological systems were like thermostats, with unambiguous sensory inputs and purely reactive in nature, simple feedback control would be sufficient. However, biological systems face three major challenges.
      First, sensations (inputs from sensory channels) (see Glossary in the Supplement) are noisy and often highly ambiguous because the world’s states (body or environment) that excite sensors can interact nonlinearly and/or hierarchically (
      • Kersten D.
      • Mamassian P.
      • Yuille A.
      Object perception as Bayesian inference.
      ,
      • Friston K.
      Learning and inference in the brain.
      ). It has long been recognized that the world’s true state is not directly accessible for the brain and needs to be inferred (
      • Helmholtz H.
      Handbuch der Physiologischen Optik (English trans., Southall JPC, Ed.).
      ,
      • Knill D.
      • Richards W.
      Perception as Bayesian Inference.
      ). This notion of perception as inference renders perception an interpretation of sensations, guided by prior beliefs and a model of the world (
      • Friston K.
      A theory of cortical responses.
      ,
      • Dayan P.
      • Hinton G.E.
      • Neal R.M.
      • Zemel R.S.
      The Helmholtz machine.
      ). Among many findings supporting this notion, illusions prominently illustrate how learned physical regularities can shape perception profoundly (Figure 2A) (
      • Adams W.J.
      • Graf E.W.
      • Ernst M.O.
      Experience can change the “light-from-above” prior.
      ,
      • Geisler W.S.
      • Kersten D.
      Illusions, perception and Bayes.
      ). When sensations are ambiguous, perception can expand the capacity for control, particularly when action selection requires information about hierarchically deep states of the world that relate nonlinearly to sensations. For example, in social interactions, inferring the nature of others’ acts that generated visual input may not be sufficient; instead, inference on deeper states, such as the intentions of others that generated their acts, may be required (
      • Diaconescu A.O.
      • Mathys C.
      • Weber L.A.
      • Daunizeau J.
      • Kasper L.
      • Lomakina E.I.
      • et al.
      Inferring on the intentions of others by hierarchical Bayesian learning.
      ,
      • Devaine M.
      • Hollard G.
      • Daunizeau J.
      The social Bayesian brain: Does mentalizing make a difference when we learn?.
      ).
      Figure thumbnail gr2
      Figure 2Two examples of perceptual inference. From left to right: prior belief, sensory data, resulting perception (posterior). (A) A classical example of a visual illusion. We perceive the surrounding objects in the image as concave and the center object as convex, even though the sensory data stem from a two-dimensional gray-scale image. The reason is that humans (likely resulting from experience) hold an implicit belief that light comes from above. If light comes from above, the shadow of a concave object should be located at the top, while the shadow of a convex object should be located at the bottom. The resulting percept is thus a reinterpretation of current sensory input based on an implicit a priori belief about lights and shadows. (B) Example of the placebo effect. Treatment with drugs that contain no therapeutic ingredient can alter the perception of a physical condition (e.g., reduce physical pain) and elicit autonomic reactions (e.g., an immune response). Again, the change in perception depends on a prior (implicit) belief—here, that the treatment will be effective. Notably, the placebo effect scales with the predicted efficacy of the intervention (for example, syringes are typically considered more potent than pills).
      Second, inference on current states of the world can only finesse reactive control. By contrast, prospective control requires predicting the world’s future states (forecasting), taking into account both the influence of possible actions (
      • Daw N.D.
      • Dayan P.
      The algorithmic anatomy of model-based evaluation.
      ,
      • Friston K.
      • Schwartenbeck P.
      • Fitzgerald T.
      • Moutoussis M.
      • Behrens T.
      • Dolan R.J.
      The anatomy of choice: Active inference and agency.
      ) and the world’s endogenous dynamics (Figure 1C) (
      • Penny W.
      • Stephan K.E.
      A dynamic Bayesian model of homeostatic control.
      ).
      Third, action selection and execution are influenced by beliefs about one’s abilities (
      • Chambon V.
      • Sidarus N.
      • Haggard P.
      From action intentions to action effects: How does the sense of agency come about?.
      ). This self-monitoring of one’s level of mastery in acting on the world is part of metacognition and can be seen as a high-level form of inference about one’s capacity for control (Figure 1C) (
      • Stephan K.E.
      • Manjaly Z.M.
      • Mathys C.
      • Weber L.A.E.
      • Paliwal S.
      • Gard T.
      • et al.
      Allostatic self-efficacy: A metacognitive theory of dyshomeostasis-induced fatigue and depression.
      ,
      • Fleming S.M.
      • Daw N.D.
      Self-evaluation of decision-making: A general Bayesian framework for metacognitive computation.
      ).
      Finally, given an inferred (or forecast) state of the world, actions can be selected to achieve a particular goal (optimize some objective function). This objective function can be defined differently—in terms of utility (
      • Ortega P.A.
      • Braun D.A.
      Thermodynamics as a theory of decision-making with information-processing costs.
      ), reward (
      • Daw N.D.
      • Dayan P.
      The algorithmic anatomy of model-based evaluation.
      ), cost (
      • Todorov E.
      Efficient computation of optimal actions.
      ), loss (
      • Kording K.P.
      • Wolpert D.M.
      The loss function of sensorimotor learning.
      ), or surprise (
      • Friston K.J.
      • Daunizeau J.
      • Kilner J.
      • Kiebel S.J.
      Action and behavior: A free-energy formulation.
      ). Figure 1C depicts a schematic illustration of the extended inference-control loop. Importantly, any given action alters the world, thus shaping future sensory input. In other words, sensation, perception, forecasting, and actions form a closed loop between the brain and its external world. For brevity, we refer to this entire cycle as inference-control loop. Its closed-loop nature is fundamentally important, as it creates problems of circular causality that are at the core of diagnostic challenges we examine below.

      Computational Modeling of Inference-Control Loops

      We now consider how inference-control loops can be formalized as concrete computational models. We adopt hierarchical Bayesian models (HBMs) here but emphasize that this is not the only possible perspective; for forecasting and control in particular, alternative (and arguably more established) modeling approaches exist [e.g., (
      • Daw N.D.
      • Dayan P.
      The algorithmic anatomy of model-based evaluation.
      ,
      • Huys Q.J.
      • Lally N.
      • Faulkner P.
      • Eshel N.
      • Seifritz E.
      • Gershman S.J.
      • et al.
      Interplay of approximate planning strategies.
      ,
      • Kaelbling L.P.
      • Littman M.L.
      • Cassandra A.R.
      Planning and acting in partially observable stochastic domains.
      ,
      • Keramati M.
      • Gutkin B.
      Homeostatic reinforcement learning for integrating reward collection and physiological stability.
      )]. We prefer a hierarchical Bayesian view for two main reasons. First, it uses the same formalism and quantities—precision-weighted predictions and precision-weighted PEs (pwPEs)—for implementing perception, forecasting, reactive/prospective control, and metacognition. This suggests a compact taxonomy of computational dysfunctions and their differential diagnosis [compare (
      • Stephan K.E.
      • Bach D.R.
      • Fletcher P.C.
      • Flint J.
      • Frank M.J.
      • Friston K.J.
      • et al.
      Charting the landscape of priority problems in psychiatry, part 1: Classification and diagnosis.
      ,
      • Mathys C.
      How could we get nosology from computation?.
      )]. Second, the formulation of control in HBMs is intimately connected to concepts of homeostatic (reactive) and allostatic (prospective) control, which are of central importance for psychosomatics.

      Bayesian Inference

      A widely adopted concept of perception is the Bayesian framework (
      • Knill D.
      • Richards W.
      Perception as Bayesian Inference.
      ,
      • Friston K.
      A theory of cortical responses.
      ,
      • Doya K.
      • Ishii S.
      • Pouget A.
      • Rao R.P.
      Bayesian Brain: Probabilistic Approaches to Neural Coding.
      ,
      • Friston K.
      The free-energy principle: A unified brain theory?.
      ). This casts perception as inference, where prior beliefs about hidden states of the world are updated in the light of sensory data to yield a posterior belief (Figure 3A) (
      • Kersten D.
      • Mamassian P.
      • Yuille A.
      Object perception as Bayesian inference.
      ,
      • Knill D.
      • Richards W.
      Perception as Bayesian Inference.
      ). A popular notion is that this computation rests on a (hierarchical) “generative model” of how sensory data are caused by hidden states of the world (Figure 3A) (
      • Friston K.
      A theory of cortical responses.
      ,
      • Dayan P.
      • Hinton G.E.
      • Neal R.M.
      • Zemel R.S.
      The Helmholtz machine.
      ,
      • Lee T.S.
      • Mumford D.
      Hierarchical Bayesian inference in the visual cortex.
      ,
      • Rao R.P.
      • Ballard D.H.
      Predictive coding in the visual cortex: A functional interpretation of some extra-classical receptive-field effects.
      ). Inverting this model under beliefs about the states’ a priori probability allows for inferring the causes of sensations.
      Figure thumbnail gr3
      Figure 3Schematic of inference-control in a Bayesian framework. (A) (Top panel) Illustration of Bayes’ rule using Gaussian distributions as an example. Bayes’ rule describes how different information sources—prior beliefs (predictions based on a model of the environment and the body within) and new sensory data (likelihood)—are combined to update the belief (posterior). The amount of belief update is proportional to the prediction error (PE)—the difference between predicted (prior) and actual (sensory) data—weighted by a precision ratio (π, inverse variance) of prior beliefs and sensory inputs (likelihood), respectively. Simply speaking, precise prior beliefs diminish and precise sensory data increase the impact of PEs on belief updates. (Bottom panel) Illustration of the concept of a generative model. A generative model infers hidden states of the world (environment or body) by inverting a probabilistic forward model from those states to possible sensory data (likelihood), under prior beliefs about the values of the hidden states. Inverting a generative model thus corresponds to the application of Bayes’ rule. Notably, the mapping from states to data can be mechanistically interpretable (e.g., biophysical models of neuronal responses) or descriptive, such as noisy fluctuations around a constant value or a periodic function (compare circadian rhythms of bodily states). (B) Example of an inference-control loop that is cast as a hierarchical Bayesian model. This figure is not meant to provide a detailed description, nor does it claim to represent the only possible layout. Briefly, the key premise here is that the brain represents and updates generative models (“model of the body/world”), with hierarchically structured beliefs. A low-level belief about a bodily/environmental state “x” (“prior”) is displayed separately from the rest of the model. The expected sensory inputs (under this prior) can be compared against actual sensations to yield a PE; this PE can be sent up the inference hierarchy and update the model. Switching from perception to action requires (temporarily) abolishing sensory precision [see (
      • Stephan K.E.
      • Manjaly Z.M.
      • Mathys C.
      • Weber L.A.E.
      • Paliwal S.
      • Gard T.
      • et al.
      Allostatic self-efficacy: A metacognitive theory of dyshomeostasis-induced fatigue and depression.
      ,
      • Friston K.J.
      • Daunizeau J.
      • Kilner J.
      • Kiebel S.J.
      Action and behavior: A free-energy formulation.
      ) for details]. Actions can then be implemented in two main ways. Homeostatic (reactive) control unfolds as a direct function of PE and serves to fulfill beliefs about sensory input [as encoded by the prior; this can be seen as a probabilistic set-point
      (
      • Stephan K.E.
      • Manjaly Z.M.
      • Mathys C.
      • Weber L.A.E.
      • Paliwal S.
      • Gard T.
      • et al.
      Allostatic self-efficacy: A metacognitive theory of dyshomeostasis-induced fatigue and depression.
      )
      ]. Allostatic (predictive) control prospectively shifts this probabilistic set-point to elicit actions; this requires predicting future states as a function of actions and bodily/environmental dynamics (“forecasting”). Finally, metacognition could be implemented as an additional layer in the model that holds (and updates) expectations with regard to the amount of PE at the top of the inference hierarchy
      (
      • Stephan K.E.
      • Manjaly Z.M.
      • Mathys C.
      • Weber L.A.E.
      • Paliwal S.
      • Gard T.
      • et al.
      Allostatic self-efficacy: A metacognitive theory of dyshomeostasis-induced fatigue and depression.
      )
      .
      Bayesian models explain phenomena across the spectrum of perception; for example, how humans combine multisensory information (
      • Ernst M.O.
      • Banks M.S.
      Humans integrate visual and haptic information in a statistically optimal fashion.
      ,
      • Angelaki D.E.
      • Gu Y.
      • DeAngelis G.C.
      Multisensory integration: Psychophysics, neurophysiology, and computation.
      ) and how biases and illusions result from prior beliefs and experience (
      • Kersten D.
      • Mamassian P.
      • Yuille A.
      Object perception as Bayesian inference.
      ,
      • Adams W.J.
      • Graf E.W.
      • Ernst M.O.
      Experience can change the “light-from-above” prior.
      ,
      • Petzschner F.H.
      • Glasauer S.
      • Stephan K.E.
      A Bayesian perspective on magnitude estimation.
      ,
      • Jazayeri M.
      • Shadlen M.N.
      Temporal context calibrates interval timing.
      ). A key point for this article is that Bayesian belief updates have, for most probability distributions, a generic form: the change in belief is proportional to PE—the difference between actual (sensory) data and predicted data (under the prior)—weighted by a precision ratio (
      • Mathys C.D.
      • Lomakina E.I.
      • Daunizeau J.
      • Iglesias S.
      • Brodersen K.H.
      • Friston K.J.
      • et al.
      Uncertainty in perception and the Hierarchical Gaussian Filter.
      ). The latter is critical, as it determines the relative influence of prior and sensory data: precise predictions (priors reduce, while precise sensory inputs increase, belief updates) (Figure 3A). Generally, abnormal computations and/or signaling of any of these three quantities—PEs, predictions, and precisions—could disrupt inference.

      Hierarchical Bayesian Models

      The hierarchical structure of the external world suggests an equivalent (mirrored) structure of the brain’s generative model (
      • Friston K.
      A theory of cortical responses.
      ,
      • Lee T.S.
      • Mumford D.
      Hierarchical Bayesian inference in the visual cortex.
      ). Anatomically, this “hierarchical Bayesian” idea is supported by structural hierarchies in cortex (
      • Felleman D.J.
      • Van Essen D.C.
      Distributed hierarchical processing in the primate cerebral cortex.
      ,
      • Mumford D.
      On the computational architecture of the neocortex. II. The role of cortico-cortical loops.
      ,
      • Hilgetag C.C.
      • Medalla M.
      • Beul S.F.
      • Barbas H.
      The primate connectome in context: Principles of connections of the cortical visual system.
      ). Popular HBMs include hierarchical filtering (
      • Mathys C.D.
      • Lomakina E.I.
      • Daunizeau J.
      • Iglesias S.
      • Brodersen K.H.
      • Friston K.J.
      • et al.
      Uncertainty in perception and the Hierarchical Gaussian Filter.
      ,
      • Browning M.
      • Behrens T.E.
      • Jocham G.
      • O’Reilly J.X.
      • Bishop S.J.
      Anxious individuals have difficulty learning the causal statistics of aversive environments.
      ) and predictive coding (
      • Friston K.
      A theory of cortical responses.
      ,
      • Rao R.P.
      • Ballard D.H.
      Predictive coding in the visual cortex: A functional interpretation of some extra-classical receptive-field effects.
      ). In these models, each level holds a belief (prediction) about the state of the level below (in predictive coding) or its rate of change (in hierarchical filtering). This prediction is signaled to the lower level, where it is compared against the actual state, resulting in a PE. This PE is sent back up the hierarchy to update the prediction—and thus reduce future PEs. Critically, again, this update is weighted by a precision ratio (Figure 3A): higher precision of bottom-up signals (sensory inputs or PEs) or lower precision of predictions leads to more pronounced belief updates. Neurobiologically, in cortex, predictions are likely signaled via N-methyl-D-aspartate receptors at descending connections, and PEs are likely signaled via alpha-amino-3-hydroxy-5-methyl-4-isoxazole propionic acid receptors (and possibly N-methyl-D-aspartate receptors) at ascending connections, while precision weighting depends on postsynaptic gain; this is determined by neuromodulators [e.g., dopamine, acetylcholine (
      • McCormick D.A.
      • Wang Z.
      • Huguenard J.
      Neurotransmitter control of neocortical neuronal activity and excitability.
      )] and gamma-aminobutyric acidergic inhibition [for reviews, see (
      • Corlett P.R.
      • Taylor J.R.
      • Wang X.J.
      • Fletcher P.C.
      • Krystal J.H.
      Toward a neurobiology of delusions.
      ,
      • Adams R.A.
      • Stephan K.E.
      • Brown H.R.
      • Frith C.D.
      • Friston K.J.
      The computational anatomy of psychosis.
      ,
      • Corlett P.R.
      • Honey G.D.
      • Krystal J.H.
      • Fletcher P.C.
      Glutamatergic model psychoses: Prediction error, learning, and inference.
      ,
      • Bastos A.M.
      • Usrey W.M.
      • Adams R.A.
      • Mangun G.R.
      • Fries P.
      • Friston K.J.
      Canonical microcircuits for predictive coding.
      ,
      • Friston K.
      • Kiebel S.
      Predictive coding under the free-energy principle.
      )].
      In an HBM context, the brain’s objective function can be seen as minimizing PEs (as a proxy to surprise) under its generative model (
      • Friston K.
      The free-energy principle: A rough guide to the brain?.
      ). Notably, PEs can be reduced not only by updating the generative model (as above) but also by changing the precision of sensory channels (attention) or by actions that fulfill predictions. The latter is “active inference” (
      • Friston K.
      • Schwartenbeck P.
      • Fitzgerald T.
      • Moutoussis M.
      • Behrens T.
      • Dolan R.J.
      The anatomy of choice: Active inference and agency.
      ,
      • Friston K.J.
      • Daunizeau J.
      • Kilner J.
      • Kiebel S.J.
      Action and behavior: A free-energy formulation.
      ,
      • Adams R.A.
      • Shipp S.
      • Friston K.J.
      Predictions not commands: Active inference in the motor system.
      ), a concept in line with the cybernetic notion that “… control systems control what they sense, not what they do” (
      • Powers W.T.
      Feedback: Beyond behaviorism.
      ).

      Forecasting, Action, and Metacognition in HBMs

      While HBMs are popular models of perceptual inference, they can also implement forecasting, action, and metacognition; again, this rests on pwPEs. Switching from inference to forecasting and actions requires “switching off” sensory precision (sensory attenuation) (
      • Adams R.A.
      • Shipp S.
      • Friston K.J.
      Predictions not commands: Active inference in the motor system.
      ,
      • Brown H.
      • Adams R.A.
      • Parees I.
      • Edwards M.
      • Friston K.
      Active inference, sensory attenuation and illusions.
      ,
      • Voss M.
      • Ingram J.N.
      • Wolpert D.M.
      • Haggard P.
      Mere expectation to move causes attenuation of sensory signals.
      ); this abolishes belief updates, while PEs are now used as simulation or action signals (
      • Stephan K.E.
      • Manjaly Z.M.
      • Mathys C.
      • Weber L.A.E.
      • Paliwal S.
      • Gard T.
      • et al.
      Allostatic self-efficacy: A metacognitive theory of dyshomeostasis-induced fatigue and depression.
      ,
      • Penny W.
      • Stephan K.E.
      A dynamic Bayesian model of homeostatic control.
      ,
      • Friston K.J.
      • Daunizeau J.
      • Kilner J.
      • Kiebel S.J.
      Action and behavior: A free-energy formulation.
      ).
      While different formalisms of forecasting exist (
      • Daw N.D.
      • Dayan P.
      The algorithmic anatomy of model-based evaluation.
      ,
      • Huys Q.J.
      • Lally N.
      • Faulkner P.
      • Eshel N.
      • Seifritz E.
      • Gershman S.J.
      • et al.
      Interplay of approximate planning strategies.
      ,
      • Kaelbling L.P.
      • Littman M.L.
      • Cassandra A.R.
      Planning and acting in partially observable stochastic domains.
      ,
      • Friston K.
      • Samothrakis S.
      • Montague R.
      Active inference and agency: Optimal control without cost functions.
      ), their common theme is a “forward simulation” under a given model. Bayesian implementations of forecasting include “planning by inference” (
      • Penny W.
      • Stephan K.E.
      A dynamic Bayesian model of homeostatic control.
      ,
      • Donnarumma F.
      • Maisto D.
      • Pezzulo G.
      Problem solving as probabilistic inference with subgoaling: Explaining human successes and pitfalls in the Tower of Hanoi.
      ) and inference on trajectories of states (generalized coordinates) (
      • Friston K.J.
      • Daunizeau J.
      • Kilner J.
      • Kiebel S.J.
      Action and behavior: A free-energy formulation.
      ,
      • Kiebel S.J.
      • Daunizeau J.
      • Friston K.J.
      A hierarchy of time-scales and the brain.
      ). One challenge for psychiatric and psychosomatic applications is that the model often needs to predict not only the effects of chosen actions but also the intrinsic dynamics of environment and body (
      • Penny W.
      • Stephan K.E.
      A dynamic Bayesian model of homeostatic control.
      ,
      • Keramati M.
      • Gutkin B.
      Homeostatic reinforcement learning for integrating reward collection and physiological stability.
      ).
      Turning to action, HBMs can implement both reactive and prospective control. The former occurs through a reflex arc at the bottom of the hierarchy (Figure 3B). Specifically, by replacing classical cybernetic set-points with beliefs about hidden states that cause sensory inputs, reactive control can be cast as a reflex where PEs elicit corrective actions that minimize surprise about sensory inputs (
      • Stephan K.E.
      • Manjaly Z.M.
      • Mathys C.
      • Weber L.A.E.
      • Paliwal S.
      • Gard T.
      • et al.
      Allostatic self-efficacy: A metacognitive theory of dyshomeostasis-induced fatigue and depression.
      ,
      • Friston K.J.
      • Daunizeau J.
      • Kilner J.
      • Kiebel S.J.
      Action and behavior: A free-energy formulation.
      ). Importantly, the precision of the belief determines the vigor of these actions (
      • Stephan K.E.
      • Manjaly Z.M.
      • Mathys C.
      • Weber L.A.E.
      • Paliwal S.
      • Gard T.
      • et al.
      Allostatic self-efficacy: A metacognitive theory of dyshomeostasis-induced fatigue and depression.
      )—a property that allows for new explanations of psychosomatic phenomena and placebo effects (see below). Prospective control can be implemented by dynamically adjusting this belief (e.g., its mean or precision) as a function of predicted future states (
      • Stephan K.E.
      • Manjaly Z.M.
      • Mathys C.
      • Weber L.A.E.
      • Paliwal S.
      • Gard T.
      • et al.
      Allostatic self-efficacy: A metacognitive theory of dyshomeostasis-induced fatigue and depression.
      ,
      • Perrinet L.U.
      • Adams R.A.
      • Friston K.J.
      Active inference, eye movements and oculomotor delays.
      ). These predictions could be signaled from higher levels in the HBM that implement forecasting.
      Action selection in HBMs could, in principle, proceed with respect to optimizing any chosen objective function, e.g., a subject-specific utility function (
      • Daunizeau J.
      • den Ouden H.E.
      • Pessiglione M.
      • Kiebel S.J.
      • Friston K.J.
      • Stephan K.E.
      Observing the observer (II): Deciding when to decide.
      ). We focus on active inference (
      • Friston K.
      • Schwartenbeck P.
      • Fitzgerald T.
      • Moutoussis M.
      • Behrens T.
      • Dolan R.J.
      The anatomy of choice: Active inference and agency.
      ,
      • Friston K.J.
      • Daunizeau J.
      • Kilner J.
      • Kiebel S.J.
      Action and behavior: A free-energy formulation.
      ,
      • Adams R.A.
      • Shipp S.
      • Friston K.J.
      Predictions not commands: Active inference in the motor system.
      ) as a specific proposal. Simply speaking, this postulates that actions serve to minimize PEs by changing the world (environment or body) to fulfill the brain’s expectation of sensory inputs. We focus on this idea because it is closely related to cybernetics [e.g., perceptual control theory (
      • Powers W.T.
      Feedback: Beyond behaviorism.
      )] and represents a probabilistic formulation of the core principle of homeostasis—that regulatory actions minimize discrepancies between expected and actual inputs. It thus provides a basis for formal models of brain-body interactions (
      • Stephan K.E.
      • Manjaly Z.M.
      • Mathys C.
      • Weber L.A.E.
      • Paliwal S.
      • Gard T.
      • et al.
      Allostatic self-efficacy: A metacognitive theory of dyshomeostasis-induced fatigue and depression.
      ) and a bridge to psychosomatics.
      Finally, metacognition could be incorporated into HBMs through an additional layer that holds expectations about the level of PEs throughout an inference hierarchy (Figure 1) (
      • Stephan K.E.
      • Manjaly Z.M.
      • Mathys C.
      • Weber L.A.E.
      • Paliwal S.
      • Gard T.
      • et al.
      Allostatic self-efficacy: A metacognitive theory of dyshomeostasis-induced fatigue and depression.
      ). This layer infers the performance of the inference-control loop as a whole, enabling a representation (and updating) of mastery or self-efficacy beliefs.

      Interoception and Homeostatic/Allostatic Control

      HBMs have been used for more than 2 decades to investigate perceptual inference on environmental states (exteroception) (
      • Lee T.S.
      • Mumford D.
      Hierarchical Bayesian inference in the visual cortex.
      ,
      • Rao R.P.
      • Ballard D.H.
      Predictive coding in the visual cortex: A functional interpretation of some extra-classical receptive-field effects.
      ,
      • Mathys C.D.
      • Lomakina E.I.
      • Daunizeau J.
      • Iglesias S.
      • Brodersen K.H.
      • Friston K.J.
      • et al.
      Uncertainty in perception and the Hierarchical Gaussian Filter.
      ,
      • Behrens T.E.
      • Woolrich M.W.
      • Walton M.E.
      • Rushworth M.F.
      Learning the value of information in an uncertain world.
      ,
      • Iglesias S.
      • Mathys C.
      • Brodersen K.H.
      • Kasper L.
      • Piccirelli M.
      • den Ouden H.E.
      • et al.
      Hierarchical prediction errors in midbrain and basal forebrain during sensory learning.
      ). However, the same inference challenge exists with regard to bodily states (interoception) (
      • Seth A.K.
      • Suzuki K.
      • Critchley H.D.
      An interoceptive predictive coding model of conscious presence.
      ,
      • Seth A.K.
      Interoceptive inference, emotion, and the embodied self.
      ,
      • Gu X.
      • Hof P.R.
      • Friston K.J.
      • Fan J.
      Anterior insular cortex and emotional awareness.
      ). Signals from bodily sensors (interosensations)—blood oxygenation and osmolality, temperature, pain, heart rate, or plasma concentrations of metabolites and hormones—reach the brain through various afferent pathways that converge on posterior and/or mid insula cortex (
      • Craig A.D.
      How do you feel? Interoception: The sense of the physiological condition of the body.
      ,
      • Craig A.D.
      Interoception: The sense of the physiological condition of the body.
      ,
      • Critchley H.D.
      • Harrison N.A.
      Visceral influences on brain and behavior.
      ), a region regarded as interoceptive cortex (
      • Cechetto D.F.
      • Saper C.B.
      Evidence for a viscerotopic sensory representation in the cortex and thalamus in the rat.
      ,
      • Evrard H.C.
      • Logothetis N.K.
      • Craig A.D.
      Modular architectonic organization of the insula in the macaque monkey.
      ). Several lines of evidence—in particular from pain and placebo research [for reviews, see (
      • Craig A.D.
      How do you feel? Interoception: The sense of the physiological condition of the body.
      ,
      • Wiech K.
      Deconstructing the sensation of pain: The influence of cognitive processes on pain perception.
      ,
      • Buchel C.
      • Geuter S.
      • Sprenger C.
      • Eippert F.
      Placebo analgesia: A predictive coding perspective.
      ,
      • Wager T.D.
      • Atlas L.Y.
      The neuroscience of placebo effects: Connecting context, learning and health.
      )]—indicate that interosensations are not processed “raw” but are shaped by prior beliefs (Figure 2B).
      Supported by anatomical and physiological findings [for reviews, see (
      • Seth A.K.
      Interoceptive inference, emotion, and the embodied self.
      ,
      • Feldman-Barrett L.F.
      • Simmons W.K.
      Interoceptive predictions in the brain.
      )], it has been proposed that perception and control of bodily states follow the same hierarchical Bayesian principles as for environmental states (
      • Seth A.K.
      • Suzuki K.
      • Critchley H.D.
      An interoceptive predictive coding model of conscious presence.
      ,
      • Seth A.K.
      Interoceptive inference, emotion, and the embodied self.
      ,
      • Gu X.
      • Hof P.R.
      • Friston K.J.
      • Fan J.
      Anterior insular cortex and emotional awareness.
      ). This implies a joint computational approach to characterizing disease mechanisms in exteroceptive (psychiatry) and interoceptive (psychosomatics) domains (Figure 4).
      Figure thumbnail gr4
      Figure 4Highly schematic illustration of the inference-control loop for interoception and exteroception. For exteroception, exterosensations (sensory inputs caused by states of the external environment) originate from receptors (e.g., mechanoreceptors, proprioceptors, photoreceptors) and are transmitted via the classical sensory channels (vision, audition, touch, taste, smell) to reach the brain’s primary sensory areas. From the perspective of perception as inference, exterosensations are combined with a priori beliefs, based on a model of the environment, resulting in a perception of the environment that is referred to as exteroception. For interoception, interosensations (sensory inputs caused by bodily states) originate from various bodily receptors (baroreceptors, chemoreceptors, thermoreceptors, etc.). Interosensations carry information about bodily states, such as temperature, pain, itch, blood oxygenation, intestinal tension, heart rate, hormonal concentration, etc., and reach the brain via two major afferent pathways: small-diameter, modality-specific afferent fibers in lamina 1 of the spinal cord that project to specific thalamocortical nuclei and the vagus and glossopharyngeal nerves projecting to the nucleus of the solitary tract. Both pathways converge on the posterior insula cortex. From the perspective of perception as inference, interosensations are combined with a priori beliefs, based on a model of the body, resulting in a perception of the body that is referred to as interoception. Interoception and exteroception combined yield the percept of the body within its environment that informs action selection with regard to both internally directed (autonomic) and externally directed (motor) actions.
      Notably, regulation of bodily states comes in two forms. Homeostatic control (
      • Cannon W.B.
      Organization for physiological homeostasis.
      ) is a form of reactive control (
      • Modell H.
      • Cliff W.
      • Michael J.
      • McFarland J.
      • Wenderoth M.P.
      • Wright A.
      A physiologist’s view of homeostasis.
      ) that is classically formalized as cybernetic feedback control (
      • Wiener N.
      Cybernetics.
      ). The more recent concept of allostasis (“stability through change”) (
      • Sterling P.
      Homeostasis vs allostasis: Implications for brain function and mental disorders.
      ) refers to prospective control, where actions are taken before homeostasis is violated. Put differently, allostasis is a self-initiated temporary change in homeostatic set-points to prepare for a predicted external perturbation (
      • Sterling P.
      Allostasis: A model of predictive regulation.
      ). When replacing classical set-points with homeostatic beliefs (expectations about bodily states), both can be cast formally as active inference (
      • Stephan K.E.
      • Manjaly Z.M.
      • Mathys C.
      • Weber L.A.E.
      • Paliwal S.
      • Gard T.
      • et al.
      Allostatic self-efficacy: A metacognitive theory of dyshomeostasis-induced fatigue and depression.
      ). Homeostatic control can then be understood as reflex-like emission of corrective actions that fulfill beliefs about bodily states, and allostatic control can be understood as changing homeostatic beliefs under guidance by higher beliefs or forecasts about future perturbations of bodily states (
      • Stephan K.E.
      • Manjaly Z.M.
      • Mathys C.
      • Weber L.A.E.
      • Paliwal S.
      • Gard T.
      • et al.
      Allostatic self-efficacy: A metacognitive theory of dyshomeostasis-induced fatigue and depression.
      ).
      Neuroanatomically concrete circuits for interoception and homeostatic/allostatic control have been suggested (
      • Stephan K.E.
      • Manjaly Z.M.
      • Mathys C.
      • Weber L.A.E.
      • Paliwal S.
      • Gard T.
      • et al.
      Allostatic self-efficacy: A metacognitive theory of dyshomeostasis-induced fatigue and depression.
      ,
      • Gu X.
      • Hof P.R.
      • Friston K.J.
      • Fan J.
      Anterior insular cortex and emotional awareness.
      ,
      • Craig A.D.
      Interoception: The sense of the physiological condition of the body.
      ,
      • Critchley H.D.
      • Harrison N.A.
      Visceral influences on brain and behavior.
      ,
      • Feldman-Barrett L.F.
      • Simmons W.K.
      Interoceptive predictions in the brain.
      ). Anterior insula (AI) and anterior cingulate cortex (ACC) play a central role in these proposals, as they are thought to represent current and predicted states of the body within the external world (
      • Craig A.D.
      Interoception: The sense of the physiological condition of the body.
      ,
      • Simmons W.K.
      • Avery J.A.
      • Barcalow J.C.
      • Bodurka J.
      • Drevets W.C.
      • Bellgowan P.
      Keeping the body in mind: Insula functional organization and functional connectivity integrate interoceptive, exteroceptive, and emotional awareness.
      ,
      • Nguyen V.T.
      • Breakspear M.
      • Hu X.
      • Guo C.C.
      The integration of the internal and external milieu in the insula during dynamic emotional experiences.
      ). Equipped with projections to regions with homeostatic reflex arcs (e.g., hypothalamus, brainstem), AI and ACC may signal the forecasts that guide allostatic control (
      • Stephan K.E.
      • Manjaly Z.M.
      • Mathys C.
      • Weber L.A.E.
      • Paliwal S.
      • Gard T.
      • et al.
      Allostatic self-efficacy: A metacognitive theory of dyshomeostasis-induced fatigue and depression.
      ,
      • Craig A.D.
      Interoception: The sense of the physiological condition of the body.
      ,
      • Feldman-Barrett L.F.
      • Simmons W.K.
      Interoceptive predictions in the brain.
      ). Furthermore, they likely interface interoceptive and exteroceptive systems and mediate their interactions, such as the influence of interoceptive signals on exteroceptive judgments (Figure 4) (
      • Allen M.
      • Frank D.
      • Schwarzkopf D.S.
      • Fardo F.
      • Winston J.S.
      • Hauser T.U.
      • et al.
      Unexpected arousal modulates the influence of sensory noise on confidence.
      ,
      • Garfinkel S.N.
      • Minati L.
      • Gray M.A.
      • Seth A.K.
      • Dolan R.J.
      • Critchley H.D.
      Fear from the heart: Sensitivity to fear stimuli depends on individual heartbeats.
      ,
      • Salomon R.
      • Ronchi R.
      • Donz J.
      • Bello-Ruiz J.
      • Herbelin B.
      • Martet R.
      • et al.
      The insula mediates access to awareness of visual stimuli presented synchronously to the heartbeat.
      ).

      Taxonomy of Failure Loci and Computational Dysfunctions

      Our general thesis is that conceptualizing adaptive behavior in terms of inference-control loops and their concrete implementation as HBMs systematizes potential failure loci and associated computational dysfunctions. The ensuing taxonomy of disease mechanisms could guide differential diagnosis, in analogous ways for computational psychiatry and computational psychosomatics. That is, in the general inference-control loop outlined above, maladaptive behavior could arise from primary disruptions at five major loci (Figure 3B): 1) sensory inputs (sensations), 2) inference (perception), 3) forecasting, 4) control (action), and 5) metacognition.
      Clearly, each of these processes could be conceptualized under different computational frameworks. In the specific case of HBMs, failures at any of these levels can arise from disturbances in a small set of computational quantities (Figure 3A): 1) bottom-up signals (sensory input or PEs), 2) top-down signals (expectations or predictions), and 3) their precision (inverse uncertainty).
      These two axes may lend useful overarching structure to pathogenetic considerations and provide a conceptual grid for classifying disease mechanisms in computational psychiatry and computational psychosomatics. However, this requires that the above levels and quantities can be inferred noninvasively in individual patients, using computational assays that can be applied to behavioral, (neuro)physiological, and neuroimaging data (
      • Stephan K.E.
      • Mathys C.
      Computational approaches to psychiatry.
      ,
      • Stephan K.E.
      • Schlagenhauf F.
      • Huys Q.J.
      • Raman S.
      • Aponte E.A.
      • Brodersen K.H.
      • et al.
      Computational neuroimaging strategies for single patient predictions.
      ). Suitable techniques for model-based inference on pwPE signaling in cortical hierarchies exist (
      • Stephan K.E.
      • Iglesias S.
      • Heinzle J.
      • Diaconescu A.O.
      Translational perspectives for computational neuroimaging.
      ); owing to space limitations, we discuss them in the Supplement.

      Computational Psychosomatics

      Psychosomatic medicine is concerned with somatic diseases that are caused or influenced by mental processes (
      • Fava G.A.
      • Sonino N.
      Psychosomatic medicine.
      ), for example, bodily symptoms caused by beliefs. Classic examples for the influence of beliefs on bodily states are placebo and nocebo effects (Figure 2B) (
      • Wiech K.
      Deconstructing the sensation of pain: The influence of cognitive processes on pain perception.
      ,
      • Buchel C.
      • Geuter S.
      • Sprenger C.
      • Eippert F.
      Placebo analgesia: A predictive coding perspective.
      ,
      • Wager T.D.
      • Atlas L.Y.
      The neuroscience of placebo effects: Connecting context, learning and health.
      ). In placebo and nocebo effects, expectations about the effects of an intervention trigger reactions that fulfill the expectation. Importantly, the strength of placebo is known to depend not only on beliefs about effect amplitude but also on the precision of this belief (
      • Buchel C.
      • Geuter S.
      • Sprenger C.
      • Eippert F.
      Placebo analgesia: A predictive coding perspective.
      ). Our framework offers a formal explanation for this empirical phenomenon because in HBM implementations of homeostatic control, the vigor of belief-fulfilling actions depends on the precision of the beliefs (
      • Stephan K.E.
      • Manjaly Z.M.
      • Mathys C.
      • Weber L.A.E.
      • Paliwal S.
      • Gard T.
      • et al.
      Allostatic self-efficacy: A metacognitive theory of dyshomeostasis-induced fatigue and depression.
      ).
      Computational treatments of psychosomatic disorders are rare [but see (
      • de Berker A.O.
      • Rutledge R.B.
      • Mathys C.
      • Marshall L.
      • Cross G.F.
      • Dolan R.J.
      • et al.
      Computations of uncertainty mediate acute stress responses in humans.
      ,
      • Fineberg S.K.
      • Steinfeld M.
      • Brewer J.A.
      • Corlett P.R.
      A computational account of borderline personality disorder: Impaired predictive learning about self and others through bodily simulation.
      )]. This may be due to the (perceived) lack of a comprehensive framework that formalizes interoception and homeostatic/allostatic control and makes them measurable in individual patients. In the following section, we consider one concrete problem of differential diagnosis and describe how the conceptual grid described above may guide the search for the locus of the primary (initial) abnormality.

      Example: Depression and Somatic Symptoms

      Many patients with depression have somatic abnormalities, including cardiac (
      • Joynt K.E.
      • Whellan D.J.
      • O’Connor C.M.
      Depression and cardiovascular disease: Mechanisms of interaction.
      ), immunological (
      • Dowlati Y.
      • Herrmann N.
      • Swardfager W.
      • Liu H.
      • Sham L.
      • Reim E.K.
      • et al.
      A meta-analysis of cytokines in major depression.
      ), and metabolic disturbances (
      • Renn B.N.
      • Feliciano L.
      • Segal D.L.
      The bidirectional relationship of depression and diabetes: A systematic review.
      ). One long-standing explanation of this association highlights maladaptive beliefs. For example, false high-level beliefs about volatility of the world could cause prolonged allostatic responses, with persistent sympathetic activation and ensuing damage to cardiovascular, immunological, and metabolic health (“allostatic load”) (
      • Sterling P.
      Allostasis: A model of predictive regulation.
      ,
      • McEwen B.S.
      Stress, adaptation, and disease. Allostasis and allostatic load.
      ). In our framework, the influence of high-level beliefs could be mediated via projections from allostatic control regions (e.g., AI, ACC) on sympathetic effector regions (e.g., hypothalamus, amygdala, or periaqueductal gray) where they elicit autonomic actions by altering homeostatic set-points. Notably, HBMs can infer fluctuations in beliefs about environmental volatility from behavioral and peripheral physiological measurements (
      • Diaconescu A.O.
      • Mathys C.
      • Weber L.A.
      • Daunizeau J.
      • Kasper L.
      • Lomakina E.I.
      • et al.
      Inferring on the intentions of others by hierarchical Bayesian learning.
      ,
      • Browning M.
      • Behrens T.E.
      • Jocham G.
      • O’Reilly J.X.
      • Bishop S.J.
      Anxious individuals have difficulty learning the causal statistics of aversive environments.
      ,
      • Iglesias S.
      • Mathys C.
      • Brodersen K.H.
      • Kasper L.
      • Piccirelli M.
      • den Ouden H.E.
      • et al.
      Hierarchical prediction errors in midbrain and basal forebrain during sensory learning.
      ,
      • de Berker A.O.
      • Rutledge R.B.
      • Mathys C.
      • Marshall L.
      • Cross G.F.
      • Dolan R.J.
      • et al.
      Computations of uncertainty mediate acute stress responses in humans.
      ). These belief trajectories could be integrated into physiological models [e.g., dynamic causal models (
      • Stephan K.E.
      • Iglesias S.
      • Heinzle J.
      • Diaconescu A.O.
      Translational perspectives for computational neuroimaging.
      ,
      • Friston K.J.
      • Harrison L.
      • Penny W.
      Dynamic causal modelling.
      )] of the above connection strengths and, by comparing models with and without modulatory effects of these beliefs, identify patients in whom bodily symptoms are possible consequences of beliefs. One might also hypothesize that these connection strengths correlate with peripheral indices of sympathetic activation (
      • Freeman R.
      • Chapleau M.W.
      Testing the autonomic nervous system.
      ).
      An opposite interpretation views depression as “reactive” to initial somatic disease. In our framework, this can be formalized as a metacognitive response to (real or perceived) chronic dyshomeostasis. One implementation of metacognition in HBMs is through a top-level layer that holds beliefs about the performance of the inference-control loop. In this “allostatic self-efficacy” (
      • Stephan K.E.
      • Manjaly Z.M.
      • Mathys C.
      • Weber L.A.E.
      • Paliwal S.
      • Gard T.
      • et al.
      Allostatic self-efficacy: A metacognitive theory of dyshomeostasis-induced fatigue and depression.
      ) concept, persistently elevated PEs decrease one’s beliefs of mastery over bodily states; this metacognitive “diagnosis” of lack of control may lead to depression as a form of learned helplessness. This proposal could be tested by correlating model-based indices of interoceptive PE signaling with questionnaire measures of self-efficacy and helplessness.
      Critically, our framework emphasizes that dyshomeostasis could be real or perceived and could exist independently from the brain or be caused by it:
      • 1.
        A real bodily source of dyshomeostasis (that evades cerebral attempts of regulation).
      • 2.
        Sensations—altered bodily receptors (“broken sensor”) [e.g., visceral hypersensitivity (
        • Holzer P.
        Acid-sensing ion channels in gastrointestinal function.
        )].
      • 3.
        Inference—illusionary dyshomeostasis, owing to impairments of the afferent branch of the inference-control loop, for example, atrophic (
        • Chatterjee S.S.
        • Mitra S.
        “I do not exist”—Cotard syndrome in insular cortex atrophy.
        ) or inflammatory (
        • Setiawan E.
        • Wilson A.A.
        • Mizrahi R.
        • Rusjan P.M.
        • Miler L.
        • Rajkowska G.
        • et al.
        Role of translocator protein density, a marker of neuroinflammation, in the brain during major depressive episodes.
        ) processes within the insula or functional pathologies of N-methyl-D-aspartate receptors and/or neuromodulators that alter the signaling of pwPEs [for reviews, see (
        • Corlett P.R.
        • Honey G.D.
        • Krystal J.H.
        • Fletcher P.C.
        Glutamatergic model psychoses: Prediction error, learning, and inference.
        ,
        • Stephan K.E.
        • Diaconescu A.O.
        • Iglesias S.
        Bayesian inference, dysconnectivity and neuromodulation in schizophrenia.
        )]. For example, abnormally high precision of beliefs about bodily states could render unremarkable events, such as normal sensory noise, meaningful; this is an interoceptive analog to “aberrant salience” (
        • Kapur S.
        Psychosis as a state of aberrant salience: A framework linking biology, phenomenology, and pharmacology in schizophrenia.
        ) in schizophrenia.
      • 4.
        Control—inadequate deployment of autonomic, endocrine, and immunological actions; for example, owing to inflammatory changes in allostatic control regions [AI, ACC (
        • Setiawan E.
        • Wilson A.A.
        • Mizrahi R.
        • Rusjan P.M.
        • Miler L.
        • Rajkowska G.
        • et al.
        Role of translocator protein density, a marker of neuroinflammation, in the brain during major depressive episodes.
        )], regions implementing homeostatic reflex arcs [e.g., hypothalamus (
        • Huitinga I.
        • Erkut Z.A.
        • van Beurden D.
        • Swaab D.F.
        Impaired hypothalamus-pituitary-adrenal axis activity and more severe multiple sclerosis with hypothalamic lesions.
        )], or their projections (
        • Hanken K.
        • Eling P.
        • Kastrup A.
        • Klein J.
        • Hildebrandt H.
        Integrity of hypothalamic fibers and cognitive fatigue in multiple sclerosis.
        ) or owing to inadequately shifted set-points as a result of false beliefs/forecasts (see above).
      Distinguishing these options is hard: the closed-loop nature of the inference-control cycle means that any primary disturbance will cause compensatory changes downstream. Inflammation-sensitive imaging (
      • Setiawan E.
      • Wilson A.A.
      • Mizrahi R.
      • Rusjan P.M.
      • Miler L.
      • Rajkowska G.
      • et al.
      Role of translocator protein density, a marker of neuroinflammation, in the brain during major depressive episodes.
      ,
      • Harrison N.A.
      • Cooper E.
      • Dowell N.G.
      • Keramida G.
      • Voon V.
      • Critchley H.D.
      • et al.
      Quantitative magnetization transfer imaging as a biomarker for effects of systemic inflammation on the brain.
      ) could help but covers only a few possible causes. Instead, we propose that model-based inference [from behavior and functional magnetic resonance imaging data (
      • Stephan K.E.
      • Iglesias S.
      • Heinzle J.
      • Diaconescu A.O.
      Translational perspectives for computational neuroimaging.
      )] (Supplement) on pwPE signaling in brainstem-hypothalamic-insular-cingulate circuitry could help identify a primary dysfunction. For example, under experimentally controlled perturbations of a (yet undisturbed) bodily state, pwPE signals in posterior and/or mid insula to predictable and unpredictable interosensations should differ, depending on whether the pathology is located at inference or control levels.

      Computational Psychiatry

      Bayesian perspectives of inference-control impairments feature frequently in computational concepts of depression (
      • Stephan K.E.
      • Manjaly Z.M.
      • Mathys C.
      • Weber L.A.E.
      • Paliwal S.
      • Gard T.
      • et al.
      Allostatic self-efficacy: A metacognitive theory of dyshomeostasis-induced fatigue and depression.
      ,
      • Feldman-Barrett L.F.
      • Simmons W.K.
      Interoceptive predictions in the brain.
      ), autism (
      • Haker H.
      • Schneebeli M.
      • Stephan K.E.
      Can Bayesian theories of autism spectrum disorder help improve clinical practice?.
      ,
      • Lawson R.P.
      • Rees G.
      • Friston K.J.
      An aberrant precision account of autism.
      ,
      • Pellicano E.
      • Burr D.
      When the world becomes “too real”: A Bayesian explanation of autistic perception.
      ,
      • Van de Cruys S.
      • de-Wit L.
      • Evers K.
      • Boets B.
      • Wagemans J.
      Weak priors versus overfitting of predictions in autism: Reply to Pellicano and Burr (TICS, 2012).
      ,
      • Austerweil J.L.
      Contradictory “heuristic” theories of autism spectrum disorders: The case for theoretical precision using computational models.
      ), schizophrenia (
      • Stephan K.E.
      • Friston K.J.
      • Frith C.D.
      Dysconnection in schizophrenia: From abnormal synaptic plasticity to failures of self-monitoring.
      ,
      • Stephan K.E.
      • Baldeweg T.
      • Friston K.J.
      Synaptic plasticity and dysconnection in schizophrenia.
      ,
      • Corlett P.R.
      • Taylor J.R.
      • Wang X.J.
      • Fletcher P.C.
      • Krystal J.H.
      Toward a neurobiology of delusions.
      ,
      • Adams R.A.
      • Stephan K.E.
      • Brown H.R.
      • Frith C.D.
      • Friston K.J.
      The computational anatomy of psychosis.
      ,
      • Fletcher P.C.
      • Frith C.D.
      Perceiving is believing: A Bayesian approach to explaining the positive symptoms of schizophrenia.
      ,
      • Jardri R.
      • Duverne S.
      • Litvinova A.S.
      • Deneve S.
      Experimental evidence for circular inference in schizophrenia.
      ), and anxiety (
      • Browning M.
      • Behrens T.E.
      • Jocham G.
      • O’Reilly J.X.
      • Bishop S.J.
      Anxious individuals have difficulty learning the causal statistics of aversive environments.
      ,
      • Paulus M.P.
      • Stein M.B.
      An insular view of anxiety.
      ,
      • Paulus M.P.
      • Stein M.B.
      Interoception in anxiety and depression.
      ). We briefly discuss one application of this framework to distinguish disease mechanisms in autism spectrum disorder (ASD).

      Example: ASD

      Hierarchical Bayesian theories of ASD revisit long-standing observations of perceptual anomalies in patients, including the excessive processing of irrelevant details and concomitant difficulties of abstraction. They suggest two competing explanations (
      • Lawson R.P.
      • Rees G.
      • Friston K.J.
      An aberrant precision account of autism.
      ,
      • Pellicano E.
      • Burr D.
      When the world becomes “too real”: A Bayesian explanation of autistic perception.
      ,
      • Van de Cruys S.
      • de-Wit L.
      • Evers K.
      • Boets B.
      • Wagemans J.
      Weak priors versus overfitting of predictions in autism: Reply to Pellicano and Burr (TICS, 2012).
      ,
      • Austerweil J.L.
      Contradictory “heuristic” theories of autism spectrum disorders: The case for theoretical precision using computational models.
      ): sensory inputs of overwhelming precision or higher-order beliefs that are too imprecise for providing generalizable predictions. In either case, a child with ASD would incessantly experience large PEs during perception (see equation in Figure 3A). Typical symptoms, such as repetitive behaviors and avoidance of complex and volatile situations (e.g., social interactions), can then be interpreted as coping mechanisms to reduce PEs [see (
      • Haker H.
      • Schneebeli M.
      • Stephan K.E.
      Can Bayesian theories of autism spectrum disorder help improve clinical practice?.
      ) for discussion].
      Additionally, individuals with ASD show various interoceptive disturbances (
      • Garfinkel S.N.
      • Tiley C.
      • O’Keeffe S.
      • Harrison N.A.
      • Seth A.K.
      • Critchley H.D.
      Discrepancies between dimensions of interoception in autism: Implications for emotion and anxiety.
      ,
      • Gu X.
      • Eilam-Stock T.
      • Zhou T.
      • Anagnostou E.
      • Kolevzon A.
      • Soorya L.
      • et al.
      Autonomic and brain responses associated with empathy deficits in autism spectrum disorder.
      ) that may equally result from an increase in sensory precision from the viscera or a failure to attenuate it (
      • Friston K.J.
      • Stephan K.E.
      • Montague R.
      • Dolan R.J.
      Computational psychiatry: The brain as a phantastic organ.
      ). Viscerosensory precision weighting has been linked to oxytocin; associated disturbances during development might compromise the construction of generative models that attribute self versus other agency to interoceptive experiences (
      • Friston K.J.
      • Stephan K.E.
      • Montague R.
      • Dolan R.J.
      Computational psychiatry: The brain as a phantastic organ.
      ,
      • Quattrocki E.
      • Friston K.
      Autism, oxytocin and interoception.
      ).
      The competing explanations of high sensory versus low belief precision (
      • Van de Cruys S.
      • de-Wit L.
      • Evers K.
      • Boets B.
      • Wagemans J.
      Weak priors versus overfitting of predictions in autism: Reply to Pellicano and Burr (TICS, 2012).
      ) could be disambiguated by psychophysical experiments in combination with Bayesian models of perception. These have previously been used to assess individual sensory processing (
      • Ernst M.O.
      • Banks M.S.
      Humans integrate visual and haptic information in a statistically optimal fashion.
      ,
      • Berniker M.
      • Voss M.
      • Kording K.
      Learning priors for Bayesian computations in the nervous system.
      ,
      • Vilares I.
      • Howard J.D.
      • Fernandes H.L.
      • Gottfried J.A.
      • Kording K.P.
      Differential representations of prior and likelihood uncertainty in the human brain.
      ) in healthy volunteers, as have been electroencephalography-based circuit models of precision weighting in auditory cortex (
      • Moran R.J.
      • Campo P.
      • Symmonds M.
      • Stephan K.E.
      • Dolan R.J.
      • Friston K.J.
      Free energy, precision and learning: The role of cholinergic neuromodulation.
      ). These models could be used in ASD to detect (sub)groups with exaggerated precision estimates of sensory inputs and insufficiently precise predictions, respectively (
      • Haker H.
      • Schneebeli M.
      • Stephan K.E.
      Can Bayesian theories of autism spectrum disorder help improve clinical practice?.
      ).

      Challenges and Opportunities

      Assessing the computational anatomy of circuit dysfunctions follows principles of homeostatic thinking, as is commonplace in medicine, and holds great diagnostic potential. However, clinical translation faces nontrivial challenges, particularly in application to psychosomatics.

      Chicken and Egg Problems

      The inference-control loop represents the conceptual heart of theories of homeostasis, allostasis, and cybernetics (Figure 1). Its closed-loop nature means that a dysfunction in one domain typically invokes a cascade of changes throughout the circuit, making it difficult to differentiate cause from consequence. However, different primary disturbances induce distinct patterns of change that might be discriminable statistically—as commonly done in fields familiar with compensatory changes throughout dyshomeostatic systems, such as internal medicine (compare differential diagnosis of hypothalamic, pituitary, and glandular disturbances in endocrinology). Computational psychiatry and psychosomatics could finesse this by statistical comparison of models embodying alternative disease processes (
      • Stephan K.E.
      • Schlagenhauf F.
      • Huys Q.J.
      • Raman S.
      • Aponte E.A.
      • Brodersen K.H.
      • et al.
      Computational neuroimaging strategies for single patient predictions.
      ). Additionally, in medicine, challenge (perturbation) approaches are often crucial for diagnosis. Combining designed perturbations with model selection and prospective assessments of disease trajectories (
      • Paulus M.P.
      • Huys Q.J.
      • Maia T.V.
      A roadmap for the development of applied computational psychiatry.
      ,
      • Stephan K.E.
      • Iglesias S.
      • Heinzle J.
      • Diaconescu A.O.
      Translational perspectives for computational neuroimaging.
      ) represents a promising approach to resolve ambiguity created by circular causality.
      One central challenge for computational psychosomatics concerns availability of somatic perturbation techniques. In contrast to computational psychiatry, where we can adopt methods for manipulating beliefs and precisions from psychology and psychophysics, manipulating the somatocerebral branch of psychosomatics has access to only a few techniques. These include cardiac challenges with short-acting sympathomimetics (
      • Khalsa S.S.
      • Craske M.G.
      • Li W.
      • Vangala S.
      • Strober M.
      • Feusner J.D.
      Altered interoceptive awareness in anorexia nervosa: Effects of meal anticipation, consumption and bodily arousal.
      ), manipulating inspiratory breathing load or air composition (
      • Pappens M.
      • Van den Bergh O.
      • Vansteenwegen D.
      • Ceunen E.
      • De Peuter S.
      • Van Diest I.
      Learning to fear obstructed breathing: Comparing interoceptive and exteroceptive cues.
      ,
      • Paulus M.P.
      • Flagan T.
      • Simmons A.N.
      • Gillis K.
      • Kotturi S.
      • Thom N.
      • et al.
      Subjecting elite athletes to inspiratory breathing load reveals behavioral and neural signatures of optimal performers in extreme environments.
      ), transcutaneous vagus nerve stimulation (
      • Brock C.
      • Brock B.
      • Aziz Q.
      • Moller H.J.
      • Pfeiffer Jensen M.
      • Drewes A.M.
      • et al.
      Transcutaneous cervical vagal nerve stimulation modulates cardiac vagal tone and tumor necrosis factor-alpha.
      ), baroreceptor stimulation (
      • Makovac E.
      • Garfinkel S.N.
      • Bassi A.
      • Basile B.
      • Macaluso E.
      • Cercignani M.
      • et al.
      Effect of parasympathetic stimulation on brain activity during appraisal of fearful expressions.
      ), acute induction of inflammation by vaccination (
      • Harrison N.A.
      • Brydon L.
      • Walker C.
      • Gray M.A.
      • Steptoe A.
      • Dolan R.J.
      • et al.
      Neural origins of human sickness in interoceptive responses to inflammation.
      ), or C-fiber stimulation under capsaicin (
      • Baumann T.K.
      • Simone D.A.
      • Shain C.N.
      • LaMotte R.H.
      Neurogenic hyperalgesia: The search for the primary cutaneous afferent fibers that contribute to capsaicin-induced pain and hyperalgesia.
      ). Developing further challenges that are noninvasive and provide temporal control should become a priority topic for computational psychosomatics.

      Universality versus Specificity

      The HBM framework suggests pwPEs as a central computational quantity for inference, forecasting, action, and metacognition. This generalizing view has pros and cons. On one hand, it suggests a conceptual grid for differential diagnosis and implies that computational differentiation of pwPE abnormalities could find broad diagnostic application. On the other hand, one may be concerned that we portray cortex as a “nonspecific hierarchical Bayesian machine” (as put by one of our reviewers) without neuroanatomical specificity. We do not wish to convey this impression. The inference problems the brain faces vary, for example, depending on the sensory channels involved and the depth of hierarchical coupling among environmental states. Different tasks require different types of (cortically represented) generative models and thus distinct circuits; compare proposed circuits for interoception/allostasis (
      • Stephan K.E.
      • Manjaly Z.M.
      • Mathys C.
      • Weber L.A.E.
      • Paliwal S.
      • Gard T.
      • et al.
      Allostatic self-efficacy: A metacognitive theory of dyshomeostasis-induced fatigue and depression.
      ,
      • Feldman-Barrett L.F.
      • Simmons W.K.
      Interoceptive predictions in the brain.
      ) and vision/oculomotor control (
      • Friston K.J.
      • Daunizeau J.
      • Kilner J.
      • Kiebel S.J.
      Action and behavior: A free-energy formulation.
      ,
      • Perrinet L.U.
      • Adams R.A.
      • Friston K.J.
      Active inference, eye movements and oculomotor delays.
      ). Empirically, in tasks using the same sensory modality but requiring inference on concrete versus abstract social quantities, pwPEs were reflected by activity in partially overlapping and partially distinct circuits (
      • Iglesias S.
      • Mathys C.
      • Brodersen K.H.
      • Kasper L.
      • Piccirelli M.
      • den Ouden H.E.
      • et al.
      Hierarchical prediction errors in midbrain and basal forebrain during sensory learning.
      ,
      • Diaconescu A.O.
      • Mathys C.
      • Weber L.A.
      • Kasper L.
      • Mauer J.
      • Stephan K.E.
      Hierarchical prediction errors in midbrain and septum during social learning.
      ).
      Furthermore, we do not claim that the framework presented covers all existing psychiatric and psychosomatic phenomena. Not all symptoms relate to perception, forecasting, action, or metacognition as the core components of our framework. However, where this relation exists, our framework may provide useful guidance in establishing analogous schemes for differential diagnostics in computational psychiatry and computational psychosomatics. Combined with models that can infer pwPE signaling in cortical hierarchies from neuroimaging or electrophysiological data (Supplement), this could allow for noninvasive readouts of circuit function that may support differentiation of potential failure loci. The promise and limitations of this approach require prospective patient studies that evaluate its predictive validity.

      Acknowledgments and Disclosures

      This work was supported by the René and Susanne Braginsky Foundation (KES), University of Zurich (FHP, KES), Deutsche Forschungsgemeinschaft Grant No. TR-SFB 134 (KES), and University of Zurich Clinical Research Priority Programs Multiple Sclerosis and Molecular Imaging Network Zurich (KES).
      The authors report no biomedical financial interests or potential conflicts of interest.

      Supplementary Material

      References

        • Casey B.J.
        • Craddock N.
        • Cuthbert B.N.
        • Hyman S.E.
        • Lee F.S.
        • Ressler K.J.
        DSM-5 and RDoC: Progress in psychiatry research?.
        Nat Rev Neurosci. 2013; 14: 810-814
        • Stephan K.E.
        • Bach D.R.
        • Fletcher P.C.
        • Flint J.
        • Frank M.J.
        • Friston K.J.
        • et al.
        Charting the landscape of priority problems in psychiatry, part 1: Classification and diagnosis.
        Lancet Psychiatry. 2016; 3: 77-83
        • Kapur S.
        • Phillips A.G.
        • Insel T.R.
        Why has it taken so long for biological psychiatry to develop clinical tests and what to do about it?.
        Mol Psychiatry. 2012; 17: 1174-1179
        • Maia T.V.
        • Frank M.J.
        From reinforcement learning models to psychiatric and neurological disorders.
        Nat Neurosci. 2011; 14: 154-162
        • Montague P.R.
        • Dolan R.J.
        • Friston K.J.
        • Dayan P.
        Computational psychiatry.
        Trends Cogn Sci. 2012; 16: 72-80
        • Stephan K.E.
        • Mathys C.
        Computational approaches to psychiatry.
        Curr Opin Neurobiol. 2014; 25: 85-92
        • Friston K.J.
        • Stephan K.E.
        • Montague R.
        • Dolan R.J.
        Computational psychiatry: The brain as a phantastic organ.
        Lancet Psychiatry. 2014; 1: 148-158
        • Wang X.J.
        • Krystal J.H.
        Computational psychiatry.
        Neuron. 2014; 84: 638-654
        • Huys Q.J.
        • Maia T.V.
        • Frank M.J.
        Computational psychiatry as a bridge from neuroscience to clinical applications.
        Nat Neurosci. 2016; 19: 404-413
        • Paulus M.P.
        • Huys Q.J.
        • Maia T.V.
        A roadmap for the development of applied computational psychiatry.
        Biol Psychiatry Cogn Neurosci Neuroimaging. 2016; 1: 386-392
        • Corlett P.R.
        • Fletcher P.C.
        Computational psychiatry: A Rosetta Stone linking the brain to mental illness.
        Lancet Psychiatry. 2014; 1: 399-402
        • Stephan K.E.
        • Friston K.J.
        • Frith C.D.
        Dysconnection in schizophrenia: From abnormal synaptic plasticity to failures of self-monitoring.
        Schizophr Bull. 2009; 35: 509-527
        • Huys Q.J.
        • Guitart-Masip M.
        • Dolan R.J.
        • Dayan P.
        Decision-theoretic psychiatry.
        Clin Psychol Sci. 2015; 3: 400-421
        • Mathys C.
        How could we get nosology from computation?.
        in: Redish A.D. Gordon J.A. Computational Psychiatry: What Can Theoretical Neuroscience and Psychiatry Teach Each Other? MIT Press, Cambridge, MA2016
        • Wiener N.
        Cybernetics.
        Wiley, New York1948
        • Ashby W.R.
        An Introduction to Cybernetics.
        Chapman & Hall, London1956
        • Powers W.T.
        Feedback: Beyond behaviorism.
        Science. 1973; 179: 351-356
        • Conant R.C.
        • Ashby W.R.
        Every good regulator of a system must be a model of that system.
        Int J Syst Sci. 1970; 1: 89-97
        • Stephan K.E.
        • Manjaly Z.M.
        • Mathys C.
        • Weber L.A.E.
        • Paliwal S.
        • Gard T.
        • et al.
        Allostatic self-efficacy: A metacognitive theory of dyshomeostasis-induced fatigue and depression.
        Front Hum Neurosci. 2016; 10: 550
        • Haker H.
        • Schneebeli M.
        • Stephan K.E.
        Can Bayesian theories of autism spectrum disorder help improve clinical practice?.
        Front Psychiatry. 2016; 7: 107
        • Stephan K.E.
        • Baldeweg T.
        • Friston K.J.
        Synaptic plasticity and dysconnection in schizophrenia.
        Biol Psychiatry. 2006; 59: 929-939
        • Cannon W.B.
        Organization for physiological homeostasis.
        Physiol Rev. 1929; 9: 399-431
        • Modell H.
        • Cliff W.
        • Michael J.
        • McFarland J.
        • Wenderoth M.P.
        • Wright A.
        A physiologist’s view of homeostasis.
        Adv Physiol Educ. 2015; 39: 259-266
        • Kersten D.
        • Mamassian P.
        • Yuille A.
        Object perception as Bayesian inference.
        Annu Rev Psychol. 2004; 55: 271-304
        • Friston K.
        Learning and inference in the brain.
        Neural Netw. 2003; 16: 1325-1352
        • Helmholtz H.
        Handbuch der Physiologischen Optik (English trans., Southall JPC, Ed.).
        Dover, New York1860/1962
        • Knill D.
        • Richards W.
        Perception as Bayesian Inference.
        Cambridge University Press, Cambridge1996
        • Friston K.
        A theory of cortical responses.
        Philos Trans R Soc Lond B Biol Sci. 2005; 360: 815-836
        • Dayan P.
        • Hinton G.E.
        • Neal R.M.
        • Zemel R.S.
        The Helmholtz machine.
        Neural Comput. 1995; 7: 889-904
        • Adams W.J.
        • Graf E.W.
        • Ernst M.O.
        Experience can change the “light-from-above” prior.
        Nat Neurosci. 2004; 7: 1057-1058
        • Geisler W.S.
        • Kersten D.
        Illusions, perception and Bayes.
        Nat Neurosci. 2002; 5: 508-510
        • Diaconescu A.O.
        • Mathys C.
        • Weber L.A.
        • Daunizeau J.
        • Kasper L.
        • Lomakina E.I.
        • et al.
        Inferring on the intentions of others by hierarchical Bayesian learning.
        PLoS Comput Biol. 2014; 10: e1003810
        • Devaine M.
        • Hollard G.
        • Daunizeau J.
        The social Bayesian brain: Does mentalizing make a difference when we learn?.
        PLoS Comput Biol. 2014; 10: e1003992
        • Daw N.D.
        • Dayan P.
        The algorithmic anatomy of model-based evaluation.
        Philos Trans R Soc Lond B Biol Sci. 2014; 369
        • Friston K.
        • Schwartenbeck P.
        • Fitzgerald T.
        • Moutoussis M.
        • Behrens T.
        • Dolan R.J.
        The anatomy of choice: Active inference and agency.
        Front Hum Neurosci. 2013; 7: 598
        • Penny W.
        • Stephan K.E.
        A dynamic Bayesian model of homeostatic control.
        Lecture Notes in Computer Science. 2014; 8779: 60-69
        • Chambon V.
        • Sidarus N.
        • Haggard P.
        From action intentions to action effects: How does the sense of agency come about?.
        Front Hum Neurosci. 2014; 8: 320
        • Fleming S.M.
        • Daw N.D.
        Self-evaluation of decision-making: A general Bayesian framework for metacognitive computation.
        Psychol Rev. 2017; 124: 91-114
        • Ortega P.A.
        • Braun D.A.
        Thermodynamics as a theory of decision-making with information-processing costs.
        Proc Roy Soc A. 2013; 469: 20120683
        • Todorov E.
        Efficient computation of optimal actions.
        Proc Natl Acad Sci U S A. 2009; 106: 11478-11483
        • Kording K.P.
        • Wolpert D.M.
        The loss function of sensorimotor learning.
        Proc Natl Acad Sci U S A. 2004; 101: 9839-9842
        • Friston K.J.
        • Daunizeau J.
        • Kilner J.
        • Kiebel S.J.
        Action and behavior: A free-energy formulation.
        Biol Cybern. 2010; 102: 227-260
        • Huys Q.J.
        • Lally N.
        • Faulkner P.
        • Eshel N.
        • Seifritz E.
        • Gershman S.J.
        • et al.
        Interplay of approximate planning strategies.
        Proc Natl Acad Sci U S A. 2015; 112: 3098-3103
        • Kaelbling L.P.
        • Littman M.L.
        • Cassandra A.R.
        Planning and acting in partially observable stochastic domains.
        Artif Intell. 1998; 101: 99-134
        • Keramati M.
        • Gutkin B.
        Homeostatic reinforcement learning for integrating reward collection and physiological stability.
        Elife. 2014; 3
        • Doya K.
        • Ishii S.
        • Pouget A.
        • Rao R.P.
        Bayesian Brain: Probabilistic Approaches to Neural Coding.
        MIT Press, Cambridge, MA2011
        • Friston K.
        The free-energy principle: A unified brain theory?.
        Nat Rev Neurosci. 2010; 11: 127-138
        • Lee T.S.
        • Mumford D.
        Hierarchical Bayesian inference in the visual cortex.
        J Opt Soc Am A Opt Image Sci Vis. 2003; 20: 1434-1448
        • Rao R.P.
        • Ballard D.H.
        Predictive coding in the visual cortex: A functional interpretation of some extra-classical receptive-field effects.
        Nat Neurosci. 1999; 2: 79-87
        • Ernst M.O.
        • Banks M.S.
        Humans integrate visual and haptic information in a statistically optimal fashion.
        Nature. 2002; 415: 429-433
        • Angelaki D.E.
        • Gu Y.
        • DeAngelis G.C.
        Multisensory integration: Psychophysics, neurophysiology, and computation.
        Curr Opin Neurobiol. 2009; 19: 452-458
        • Petzschner F.H.
        • Glasauer S.
        • Stephan K.E.
        A Bayesian perspective on magnitude estimation.
        Trends Cogn Sci. 2015; 19: 285-293
        • Jazayeri M.
        • Shadlen M.N.
        Temporal context calibrates interval timing.
        Nat Neurosci. 2010; 13: 1020-1026
        • Mathys C.D.
        • Lomakina E.I.
        • Daunizeau J.
        • Iglesias S.
        • Brodersen K.H.
        • Friston K.J.
        • et al.
        Uncertainty in perception and the Hierarchical Gaussian Filter.
        Front Hum Neurosci. 2014; 8: 825
        • Felleman D.J.
        • Van Essen D.C.
        Distributed hierarchical processing in the primate cerebral cortex.
        Cereb Cortex. 1991; 1: 1-47
        • Mumford D.
        On the computational architecture of the neocortex. II. The role of cortico-cortical loops.
        Biol Cybern. 1992; 66: 241-251
        • Hilgetag C.C.
        • Medalla M.
        • Beul S.F.
        • Barbas H.
        The primate connectome in context: Principles of connections of the cortical visual system.
        Neuroimage. 2016; 134: 685-702
        • Browning M.
        • Behrens T.E.
        • Jocham G.
        • O’Reilly J.X.
        • Bishop S.J.
        Anxious individuals have difficulty learning the causal statistics of aversive environments.
        Nat Neurosci. 2015; 18: 590-596
        • McCormick D.A.
        • Wang Z.
        • Huguenard J.
        Neurotransmitter control of neocortical neuronal activity and excitability.
        Cereb Cortex. 1993; 3: 387-398
        • Corlett P.R.
        • Taylor J.R.
        • Wang X.J.
        • Fletcher P.C.
        • Krystal J.H.
        Toward a neurobiology of delusions.
        Prog Neurobiol. 2010; 92: 345-369
        • Adams R.A.
        • Stephan K.E.
        • Brown H.R.
        • Frith C.D.
        • Friston K.J.
        The computational anatomy of psychosis.
        Front Psychiatry. 2013; 4: 47
        • Corlett P.R.
        • Honey G.D.
        • Krystal J.H.
        • Fletcher P.C.
        Glutamatergic model psychoses: Prediction error, learning, and inference.
        Neuropsychopharmacology. 2011; 36: 294-315
        • Bastos A.M.
        • Usrey W.M.
        • Adams R.A.
        • Mangun G.R.
        • Fries P.
        • Friston K.J.
        Canonical microcircuits for predictive coding.
        Neuron. 2012; 76: 695-711
        • Friston K.
        • Kiebel S.
        Predictive coding under the free-energy principle.
        Philos Trans R Soc Lond B Biol Sci. 2009; 364: 1211-1221
        • Friston K.
        The free-energy principle: A rough guide to the brain?.
        Trends Cogn Sci. 2009; 13: 293-301
        • Adams R.A.
        • Shipp S.
        • Friston K.J.
        Predictions not commands: Active inference in the motor system.
        Brain Struct Funct. 2013; 218: 611-643
        • Brown H.
        • Adams R.A.
        • Parees I.
        • Edwards M.
        • Friston K.
        Active inference, sensory attenuation and illusions.
        Cogn Process. 2013; 14: 411-427
        • Voss M.
        • Ingram J.N.
        • Wolpert D.M.
        • Haggard P.
        Mere expectation to move causes attenuation of sensory signals.
        PLoS One. 2008; 3: e2866
        • Friston K.
        • Samothrakis S.
        • Montague R.
        Active inference and agency: Optimal control without cost functions.
        Biol Cybern. 2012; 106: 523-541
        • Donnarumma F.
        • Maisto D.
        • Pezzulo G.
        Problem solving as probabilistic inference with subgoaling: Explaining human successes and pitfalls in the Tower of Hanoi.
        PLoS Comput Biol. 2016; 12: e1004864
        • Kiebel S.J.
        • Daunizeau J.
        • Friston K.J.
        A hierarchy of time-scales and the brain.
        PLoS Comput Biol. 2008; 4: e1000209
        • Perrinet L.U.
        • Adams R.A.
        • Friston K.J.
        Active inference, eye movements and oculomotor delays.
        Biol Cybern. 2014; 108: 777-801
        • Daunizeau J.
        • den Ouden H.E.
        • Pessiglione M.
        • Kiebel S.J.
        • Friston K.J.
        • Stephan K.E.
        Observing the observer (II): Deciding when to decide.
        PLoS One. 2010; 5: e15555
        • Behrens T.E.
        • Woolrich M.W.
        • Walton M.E.
        • Rushworth M.F.
        Learning the value of information in an uncertain world.
        Nat Neurosci. 2007; 10: 1214-1221
        • Iglesias S.
        • Mathys C.
        • Brodersen K.H.
        • Kasper L.
        • Piccirelli M.
        • den Ouden H.E.
        • et al.
        Hierarchical prediction errors in midbrain and basal forebrain during sensory learning.
        Neuron. 2013; 80: 519-530
        • Seth A.K.
        • Suzuki K.
        • Critchley H.D.
        An interoceptive predictive coding model of conscious presence.
        Front Psychol. 2011; 2: 395
        • Seth A.K.
        Interoceptive inference, emotion, and the embodied self.
        Trends Cogn Sci. 2013; 17: 565-573
        • Gu X.
        • Hof P.R.
        • Friston K.J.
        • Fan J.
        Anterior insular cortex and emotional awareness.
        J Comp Neurol. 2013; 521: 3371-3388
        • Craig A.D.
        How do you feel? Interoception: The sense of the physiological condition of the body.
        Nat Rev Neurosci. 2002; 3: 655-666
        • Craig A.D.
        Interoception: The sense of the physiological condition of the body.
        Curr Opin Neurobiol. 2003; 13: 500-505
        • Critchley H.D.
        • Harrison N.A.
        Visceral influences on brain and behavior.
        Neuron. 2013; 77: 624-638
        • Cechetto D.F.
        • Saper C.B.
        Evidence for a viscerotopic sensory representation in the cortex and thalamus in the rat.
        J Comp Neurol. 1987; 262: 27-45
        • Evrard H.C.
        • Logothetis N.K.
        • Craig A.D.
        Modular architectonic organization of the insula in the macaque monkey.
        J Comp Neurol. 2014; 522: 64-97
        • Wiech K.
        Deconstructing the sensation of pain: The influence of cognitive processes on pain perception.
        Science. 2016; 354: 584-587
        • Buchel C.
        • Geuter S.
        • Sprenger C.
        • Eippert F.
        Placebo analgesia: A predictive coding perspective.
        Neuron. 2014; 81: 1223-1239
        • Wager T.D.
        • Atlas L.Y.
        The neuroscience of placebo effects: Connecting context, learning and health.
        Nat Rev Neurosci. 2015; 16: 403-418
        • Feldman-Barrett L.F.
        • Simmons W.K.
        Interoceptive predictions in the brain.
        Nat Rev Neurosci. 2015; 16: 419-429
        • Sterling P.
        Homeostasis vs allostasis: Implications for brain function and mental disorders.
        JAMA Psychiatry. 2014; 71: 1192-1193
        • Sterling P.
        Allostasis: A model of predictive regulation.
        Physiol Behav. 2012; 106: 5-15
        • Simmons W.K.
        • Avery J.A.
        • Barcalow J.C.
        • Bodurka J.
        • Drevets W.C.
        • Bellgowan P.
        Keeping the body in mind: Insula functional organization and functional connectivity integrate interoceptive, exteroceptive, and emotional awareness.
        Hum Brain Mapp. 2013; 34: 2944-2958
        • Nguyen V.T.
        • Breakspear M.
        • Hu X.
        • Guo C.C.
        The integration of the internal and external milieu in the insula during dynamic emotional experiences.
        Neuroimage. 2016; 124: 455-463
        • Allen M.
        • Frank D.
        • Schwarzkopf D.S.
        • Fardo F.
        • Winston J.S.
        • Hauser T.U.
        • et al.
        Unexpected arousal modulates the influence of sensory noise on confidence.
        Elife. 2016; 5
        • Garfinkel S.N.
        • Minati L.
        • Gray M.A.
        • Seth A.K.
        • Dolan R.J.
        • Critchley H.D.
        Fear from the heart: Sensitivity to fear stimuli depends on individual heartbeats.
        J Neurosci. 2014; 34: 6573-6582
        • Salomon R.
        • Ronchi R.
        • Donz J.
        • Bello-Ruiz J.
        • Herbelin B.
        • Martet R.
        • et al.
        The insula mediates access to awareness of visual stimuli presented synchronously to the heartbeat.
        J Neurosci. 2016; 36: 5115-5127
        • Stephan K.E.
        • Schlagenhauf F.
        • Huys Q.J.
        • Raman S.
        • Aponte E.A.
        • Brodersen K.H.
        • et al.
        Computational neuroimaging strategies for single patient predictions.
        Neuroimage. 2017; 145: 180-199
        • Stephan K.E.
        • Iglesias S.
        • Heinzle J.
        • Diaconescu A.O.
        Translational perspectives for computational neuroimaging.
        Neuron. 2015; 87: 716-732
        • Fava G.A.
        • Sonino N.
        Psychosomatic medicine.
        Int J Clin Pract. 2010; 64: 1155-1161
        • de Berker A.O.
        • Rutledge R.B.
        • Mathys C.
        • Marshall L.
        • Cross G.F.
        • Dolan R.J.
        • et al.
        Computations of uncertainty mediate acute stress responses in humans.
        Nat Commun. 2016; 7: 10996
        • Fineberg S.K.
        • Steinfeld M.
        • Brewer J.A.
        • Corlett P.R.
        A computational account of borderline personality disorder: Impaired predictive learning about self and others through bodily simulation.
        Front Psychiatry. 2014; 5: 111
        • Joynt K.E.
        • Whellan D.J.
        • O’Connor C.M.
        Depression and cardiovascular disease: Mechanisms of interaction.
        Biol Psychiatry. 2003; 54: 248-261
        • Dowlati Y.
        • Herrmann N.
        • Swardfager W.
        • Liu H.
        • Sham L.
        • Reim E.K.
        • et al.
        A meta-analysis of cytokines in major depression.
        Biol Psychiatry. 2010; 67: 446-457
        • Renn B.N.
        • Feliciano L.
        • Segal D.L.
        The bidirectional relationship of depression and diabetes: A systematic review.
        Clin Psychol Rev. 2011; 31: 1239-1246
        • McEwen B.S.
        Stress, adaptation, and disease. Allostasis and allostatic load.
        Ann N Y Acad Sci. 1998; 840: 33-44
        • Friston K.J.
        • Harrison L.
        • Penny W.
        Dynamic causal modelling.
        Neuroimage. 2003; 19: 1273-1302
        • Freeman R.
        • Chapleau M.W.
        Testing the autonomic nervous system.
        Handb Clin Neurol. 2013; 115: 115-136
        • Holzer P.
        Acid-sensing ion channels in gastrointestinal function.
        Neuropharmacology. 2015; 94: 72-79
        • Chatterjee S.S.
        • Mitra S.
        “I do not exist”—Cotard syndrome in insular cortex atrophy.
        Biol Psychiatry. 2015; 77: e52-e53
        • Setiawan E.
        • Wilson A.A.
        • Mizrahi R.
        • Rusjan P.M.
        • Miler L.
        • Rajkowska G.
        • et al.
        Role of translocator protein density, a marker of neuroinflammation, in the brain during major depressive episodes.
        JAMA Psychiatry. 2015; 72: 268-275
        • Stephan K.E.
        • Diaconescu A.O.
        • Iglesias S.
        Bayesian inference, dysconnectivity and neuromodulation in schizophrenia.
        Brain. 2016; 139: 1874-1876
        • Kapur S.
        Psychosis as a state of aberrant salience: A framework linking biology, phenomenology, and pharmacology in schizophrenia.
        Am J Psychiatry. 2003; 160: 13-23
        • Huitinga I.
        • Erkut Z.A.
        • van Beurden D.
        • Swaab D.F.
        Impaired hypothalamus-pituitary-adrenal axis activity and more severe multiple sclerosis with hypothalamic lesions.
        Ann Neurol. 2004; 55: 37-45
        • Hanken K.
        • Eling P.
        • Kastrup A.
        • Klein J.
        • Hildebrandt H.
        Integrity of hypothalamic fibers and cognitive fatigue in multiple sclerosis.
        Mult Scler Relat Disord. 2015; 4: 39-46
        • Harrison N.A.
        • Cooper E.
        • Dowell N.G.
        • Keramida G.
        • Voon V.
        • Critchley H.D.
        • et al.
        Quantitative magnetization transfer imaging as a biomarker for effects of systemic inflammation on the brain.
        Biol Psychiatry. 2015; 78: 49-57
        • Lawson R.P.
        • Rees G.
        • Friston K.J.
        An aberrant precision account of autism.
        Front Hum Neurosci. 2014; 8: 302
        • Pellicano E.
        • Burr D.
        When the world becomes “too real”: A Bayesian explanation of autistic perception.
        Trends Cogn Sci. 2012; 16: 504-510
        • Van de Cruys S.
        • de-Wit L.
        • Evers K.
        • Boets B.
        • Wagemans J.
        Weak priors versus overfitting of predictions in autism: Reply to Pellicano and Burr (TICS, 2012).
        Iperception. 2013; 4: 95-97
        • Austerweil J.L.
        Contradictory “heuristic” theories of autism spectrum disorders: The case for theoretical precision using computational models.
        Autism. 2015; 19: 367-368
        • Fletcher P.C.
        • Frith C.D.
        Perceiving is believing: A Bayesian approach to explaining the positive symptoms of schizophrenia.
        Nat Rev Neurosci. 2009; 10: 48-58
        • Jardri R.
        • Duverne S.
        • Litvinova A.S.
        • Deneve S.
        Experimental evidence for circular inference in schizophrenia.
        Nat Commun. 2017; 8: 14218
        • Paulus M.P.
        • Stein M.B.
        An insular view of anxiety.
        Biol Psychiatry. 2006; 60: 383-387
        • Paulus M.P.
        • Stein M.B.
        Interoception in anxiety and depression.
        Brain Struct Funct. 2010; 214: 451-463
        • Garfinkel S.N.
        • Tiley C.
        • O’Keeffe S.
        • Harrison N.A.
        • Seth A.K.
        • Critchley H.D.
        Discrepancies between dimensions of interoception in autism: Implications for emotion and anxiety.
        Biol Psychol. 2016; 114: 117-126
        • Gu X.
        • Eilam-Stock T.
        • Zhou T.
        • Anagnostou E.
        • Kolevzon A.
        • Soorya L.
        • et al.
        Autonomic and brain responses associated with empathy deficits in autism spectrum disorder.
        Hum Brain Mapp. 2015; 36: 3323-3338
        • Quattrocki E.
        • Friston K.
        Autism, oxytocin and interoception.
        Neurosci Biobehav Rev. 2014; 47: 410-430
        • Berniker M.
        • Voss M.
        • Kording K.
        Learning priors for Bayesian computations in the nervous system.
        PLoS One. 2010; 5
        • Vilares I.
        • Howard J.D.
        • Fernandes H.L.
        • Gottfried J.A.
        • Kording K.P.
        Differential representations of prior and likelihood uncertainty in the human brain.
        Curr Biol. 2012; 22: 1641-1648
        • Moran R.J.
        • Campo P.
        • Symmonds M.
        • Stephan K.E.
        • Dolan R.J.
        • Friston K.J.
        Free energy, precision and learning: The role of cholinergic neuromodulation.
        J Neurosci. 2013; 33: 8227-8236
        • Khalsa S.S.
        • Craske M.G.
        • Li W.
        • Vangala S.
        • Strober M.
        • Feusner J.D.
        Altered interoceptive awareness in anorexia nervosa: Effects of meal anticipation, consumption and bodily arousal.
        Int J Eat Disord. 2015; 48: 889-897
        • Pappens M.
        • Van den Bergh O.
        • Vansteenwegen D.
        • Ceunen E.
        • De Peuter S.
        • Van Diest I.
        Learning to fear obstructed breathing: Comparing interoceptive and exteroceptive cues.
        Biol Psychol. 2013; 92: 36-42
        • Paulus M.P.
        • Flagan T.
        • Simmons A.N.
        • Gillis K.
        • Kotturi S.
        • Thom N.
        • et al.
        Subjecting elite athletes to inspiratory breathing load reveals behavioral and neural signatures of optimal performers in extreme environments.
        PLoS One. 2012; 7: e29394
        • Brock C.
        • Brock B.
        • Aziz Q.
        • Moller H.J.
        • Pfeiffer Jensen M.
        • Drewes A.M.
        • et al.
        Transcutaneous cervical vagal nerve stimulation modulates cardiac vagal tone and tumor necrosis factor-alpha.
        Neurogastroenterol Motil. 2017; 29
        • Makovac E.
        • Garfinkel S.N.
        • Bassi A.
        • Basile B.
        • Macaluso E.
        • Cercignani M.
        • et al.
        Effect of parasympathetic stimulation on brain activity during appraisal of fearful expressions.
        Neuropsychopharmacology. 2015; 40: 1649-1658
        • Harrison N.A.
        • Brydon L.
        • Walker C.
        • Gray M.A.
        • Steptoe A.
        • Dolan R.J.
        • et al.
        Neural origins of human sickness in interoceptive responses to inflammation.
        Biol Psychiatry. 2009; 66: 415-422
        • Baumann T.K.
        • Simone D.A.
        • Shain C.N.
        • LaMotte R.H.
        Neurogenic hyperalgesia: The search for the primary cutaneous afferent fibers that contribute to capsaicin-induced pain and hyperalgesia.
        J Neurophysiol. 1991; 66: 212-227
        • Diaconescu A.O.
        • Mathys C.
        • Weber L.A.
        • Kasper L.
        • Mauer J.
        • Stephan K.E.
        Hierarchical prediction errors in midbrain and septum during social learning.
        Soc Cogn Affect Neurosci. 2017; 12: 618-634