Abstract
Computational psychiatry aims to apply mathematical and computational techniques to help improve psychiatric care. To achieve this, the phenomena under scrutiny should be within the scope of formal methods. As emotions play an important role across many psychiatric disorders, such computational methods must encompass emotions. Here, we consider formal valuation accounts of emotions. We focus on the fact that the flexibility of emotional responses and the nature of appraisals suggest the need for a model-based valuation framework for emotions. However, resource limitations make plain model-based valuation impossible and require metareasoning strategies to apportion cognitive resources adaptively. We argue that emotions may implement such metareasoning approximations by restricting the range of behaviors and states considered. We consider the processes that guide the deployment of the approximations, discerning between innate, model-free, heuristic, and model-based controllers. A formal valuation and metareasoning framework may thus provide a principled approach to examining emotions.
Keywords
Computational psychiatry is a young field hoping to leverage advances in computational techniques to understand and improve mental health (
1
, 2
, 3
, 4
, 5
). It is motivated on the one hand by the necessity to bring novel statistical and machine-learning techniques to bear on the rapidly expanding complexity of novel datasets relevant to mental health, and on the other hand by the complexity of the problem itself as mental health relates to the most difficult tasks performed by the most complex of organs.Emotions are central to mental health, and emotional disorders contribute substantially to the burden of mental illnesses (
6
). The traditional dichotomization of emotion and reason might question the feasibility of applying computational techniques to the core issues of emotion. It is therefore imperative for computational psychiatry that we consider the ability of a computational and mathematical framework to address core emotional phenomena. Here, we argue that approaching emotion computationally requires the introduction of model-based valuation and metareasoning. Metareasoning considers optimal valuation in the face of resource constraints (7
, 8
, 9
). The proposal is that human emotions involve strategies to deal with the complexity of model-based or goal-directed decision making by focusing on particular aspects of the problem at hand.Research on human emotions is complicated as questions about their nature continue to divide the scientific community (
10
, 11
). Nevertheless, there is consensus on a number of key components that characterize emotions, and this review attempts to view them in a computational light. We first provide a description of important features of emotions, then introduce valuation and the metareasoning problem, then relate approximate metareasoning strategies to features of emotions, and finally describe the control of approximate metareasoning strategies.Ingredients of a Computational Approach to Emotions
Key features of human emotions that require accounting for and that are emphasized to various degrees in different conceptualizations are 1) correlated physiological, psychological and behavioral processes shaped by evolutionarily predefined neural circuitry; 2) interpretations or appraisals; and 3) conscious verbal self-report about emotions. Key problems in contemporary research on human emotions include to what extent the three feature domains are related (e.g., how conscious emotions in humans relate to evolutionarily predefined circuitry) and to what extent emotions are discrete entities.
Basic emotion theories suggest that there are a limited, relatively fixed, number of universal, evolutionarily shaped, culture-independent, and neurobiologically hard-coded emotional categories including happiness, surprise, sadness, disgust, anger, and fear (
11
, , 13
). For the present purpose, what is important is that these represent a set of innately interlinked physiological, behavioral, and psychological processes that are triggered in an inflexible manner by species-specific salient stimuli, akin to unconditioned responses. Animal research, in which specific responses to species-relevant stimuli are observable and readily quantifiable, has contributed to this view. However, behavioral responses in animals cannot be directly translated to emotional experiences in humans. Amygdalar and hippocampal damage, for instance, abolish physiological and autobiographical signatures of aversive conditioning, respectively, while leaving the other intact (14
). Furthermore, aversive conditioning can be performed subliminally and can evoke amygdala activity and physiological response, but can fail to result in any emotion of fear (15
, 16
), while amygdalar lesions can leave human fear unaffected (17
, 18
).Human emotional responses to stimuli are characterized by substantial within- and between-subject variability. Appraisal theory locates one explanation for this variability in the interpretation (be it conscious or unconscious) of a particular situation or stimulus as being relevant to the individual’s goals (). This interpretation depends on the goal and the individual’s beliefs in addition to the stimulus. A stimulus or situation being interpreted as increasing the changes of reaching one’s goals would, for instance, result in the emotion of joy or happiness (
20
, 21
, 22
). However, just like basic emotion theories, appraisal theories often view the expressed emotion itself as a “definable pattern of outputs that preexist within the individual” (10
). For instance, Scherer (23
) defined them as “episode[s] of interrelated, synchronized changes in the states of all or most […] organismic subsystems in response to the evaluation of an external or internal stimulus event as relevant to major concerns of the organism.”The evidence for discrete emotions is controversial. Autonomic responses, electroencephalographic features, and facial expressions do not permit simple categorization and show little evidence of the predicted correlations (
10
, 24
, 25
), though newer machine learning approaches have shown that categorical information can be extracted from physiological (26
) and neural (27
, 28
) data. The latter analyses have, however, clarified that there is no single underlying substrate for particular emotions. Rather, each emotional category depends on a distributed network of limbic but also cortical components that reflect the particular neurocognitive processes involved (29
).An alternative view is that the discreteness of emotions arises from the categorical labeling of internal events for the purpose of intra- or intersubject communication. Neuroimaging has provided some support for such a model, arguing that the ventrolateral prefrontal cortex is involved in categorical labeling of emotional states (
30
, 31
, 32
) evolving along the two major axes of valence (from good to bad) and arousal (from high to low). Indeed, factor analyses of a variety of measures of emotions including similarity ratings among words, facial expression, and autonomic measures reliably identify these two separate dimensions (33
). Neuroimaging has also been used to argue that while the amygdala tracks arousal, the orbitofrontal cortex tracks valence across emotions (34
).Valuation and Emotion
Basic and animal emotion research, with its grounding in evolutionarily shaped responses, emphasizes the importance of emotions in guiding behavior adaptively. A focus on adaptive responding is also present in appraisal theories, which suggest that emotions arise when events are judged to be relevant to the individual’s “needs, attachments, values, current goals and beliefs” (
35
). Computationally, inferring adaptive choices involves integrating not only immediate rewards, but also longer-term rewards, and for that reason requires consideration of the future course of events. This evaluation of the future is where the problem lies, as the further into the future one looks, the broader the range of potential events. Specifically, valuation involves summing over an exponentially expanding decision tree of future possibilities. Optimal valuation would search the entire tree, which is rarely feasible. Reinforcement learning is a thriving subfield of machine learning concerned with algorithmic solutions to this problem.Model-Free Accounts of Emotional Expression
A substantial body of work has related one such algorithmic solution to how emotional expressions change over time (
36
). In so-called model-free reinforcement learning, the stability of the world is exploited to replace integration over the future with actual past experiences. Clever bookkeeping allows the use of prediction errors to update values that, in the limit of extensive experience, are guaranteed to yield the true long-term values of states and behaviors (37
). Here, emotional responses are viewed as a type of high-level action, involving multiple biological and neural subsystems. One example of such an “action” is a freezing response, which has behavioral, attentional, and physiological components. These high-level actions are thought to be emitted either in an innate fashion (38
) in response to the appropriate species-specific unconditional stimulus (39
, 40
, 41
), or after learning in response to a conditioned stimulus. In the latter case, the expression of the action is proportional to the value attached to the conditioned stimulus, which in turn is a scalar measure of the average expected unconditional stimulus strength (42
, 43
, 44
). This has been applied to a wide variety of affective responses, including heart rate changes (45
), approach (46
), avoidance (47
, 48
), extinction (49
), vigor (50
, 51
), and others. Perhaps the most striking success of these models is their ability to capture how pavlovian affective responses can lead to maladaptive choices (43
, 52
).Model-free approaches are very valuable to understand how the expression of affect transfers between situations with experience. Although mostly restricted to individual laboratory sessions, the underlying model likely plays an important role in explaining how individual differences in the expression of (affective) behaviors emerge over (life)time, and potentially in response to behavioral psychotherapeutic interventions. Furthermore, a hierarchical version of model-free reinforcement learning has the capacity to explain how complex high-level actions consisting of multiple correlated processes might emerge (
52
, 53
, 54
), though this awaits application to the correlations among physiological, psychological, and behavioral aspects of emotions.Appraisals Require Model-Based Inference
Pure model-free accounts, however, fail to explain context effects on conditioning. For instance, the physiological response to a threat differs depending on whether the animal is restrained or freely moving (
56
) as well as whether a refuge or obstacle is present and at what distance (57
, 58
, 59
). In humans, framing the same movement as approach or withdrawal alters whether a pavlovian conditioned stimulus promotes or inhibits it (42
). Even startle reflexes are potentiated by fear induction (60
), enhanced by upregulating negative emotions (61
), and reduced by positive emotions (62
). More fundamentally, context determines what affective behavior is emitted (21
, 63
): the same emotion of anger may motivate harm not only through physical means in a boxing ring, but also through financial transactions in a boardroom setting.Similarly, scalar summaries of past experience cannot account for the impact of appraisals on human emotions. The appraisal of an event involves an assessment of what the event “means” (
35
). This interpretative process determines whether and which emotion results by inference of latent causes: a smile is pleasant if interpreted as emanating from kindness, but aversive if viewed as an expression of condescension. The fact that an event is meaningful can be inferred from changes in model-free values because these capture the expectation of future well-being in a relatively stable environment, and sudden changes in model-free expected values therefore indicate a meaningful event. This is reminiscent of the argument that changes in core affect invoke appraisals (10
, 33
). Certain aspects of meaning may also be precomputed and result in automatic appraisals (64
, 65
), but what an event means for well-being cannot be derived from model-free values. This assessment involves a series of variables such as goal congruence, controllability, and agency (35
, , 67
) that capture how the changed contingencies induced by the event and the behavior influence the controllable achievability of the goal (68
). Goal congruence, for instance, measures how events influence the ability and cost of achieving current goals and as such involves replanning a new path toward the goal and comparing the cost of this path to that of the previous plan.The computation of the meaning part of the appraisal requires the integration of a model capturing an individual’s beliefs about the consequences of choices, what reinforcements will be obtained in which states, how observations relate to hidden states, and how different states relate to each other (
37
, 65
, 69
). Inferring such values requires a model to be inverted or simulated. Mirroring the notion that some appraisals rely on rule-based processes, it suggests a role for nonautomatic components; it captures that appraisals generalize and change over time as new information is progressively integrated; it suggests how maladaptive beliefs influence emotions; and it suggests how a new understanding can alter emotions in explicit reappraisal (35
, 63
, , 70
, 71
).However, a measured and “rational” consideration of all possible outcomes is hardly a sufficient model of emotions (
72
). In fact, reasoning itself is profoundly affected by emotions (70
), as are perception, learning, and memory.Metareasoning
One factor that may be useful to consider is that model-based inference is mostly impossible due to the sheer size of most relevant model-based valuation problems. Figure 1A shows a simple planning task in a maze the solution of which corresponds to a binary decision tree (Figure 1B). The best action at the root of the tree is the one that leads to the path with the best total outcome, and this may not be the action with the best immediate reward. As in reality, there is usually more than one way to the goal, but the different paths have different intermediate outcomes rendering some better than others. Despite its simplicity, humans have difficulty solving the task fully, and employ strategies to avoid evaluating the entire tree even when there are only three or four choices to go (Figure 1C). Unless they are highly constrained, such as in feedforward motor control (
73
), optimal decisions in realistic situations are computationally extremely demanding.
Figure 1Model-based inference problems rapidly become too hard to fully solve. Consider a task (A) where subjects see six rectangles arranged hexagonally and are taught to navigate according to the underlying transition matrix indicated by the arrows, so that they can transition from each state to two successor states depending on which of two buttons they press. Every transition yields a reward or loss. Finding a path of a given length that maximizes total earnings corresponds to a tree-shaped decision problem, for instance the one in panel (B) for three transitions starting from state 3. Participants typically choose not to expend cognitive effort evaluating subtrees below a salient loss (here below transitions with –70 points), resulting in a cutting of or “pruning” of decision trees (B). (C) This in turn results in worse performance when the optimal path requires transitioning through a salient loss (red) than when it does not (orange). Black dots and gray lines show the effect in individual participants. (A) and (B) are modified with permission from Huys et al.
(
. (C) shows data replotted from Huys et al. 44
)(
.55
)The limited resources lead to the metareasoning problem, which concerns the optimal deployment of the available computational power (
7
, 8
, 74
, 75
). It is a decision problem about which of the various options to evaluate internally (Figure 2). Formally, the estimated value of performing a computation is the difference in expected utility between taking a choice without the additional computation, and taking a new alternative choice after having invested in the computation (8
, 76
). Although this decision problem is mathematically similar to the original problem, it is different from the original problem because simulations do not actually incur the costs of the real problem, and while taking real poor actions should be avoided to avoid incurring their loss, internally simulating poor actions can be useful (77
, 78
). Thus, the states in this metareasoning problem are all possible partial trees of the original tree, which is a far larger state space than in the original problem.
Figure 2Metareasoning. Given a state and a sequence of possible decisions, optimal action choice involves evaluating a decision tree (top left). An agent with constrained resources faces the challenge of deciding what to simulate, i.e., what to “think about.” For instance, it could choose to first simulate the action going left, and then continue down this branch (leftmost set of trees). Alternatively, it could superficially consider the right action, and then start examining the left action. The metareasoning problem hence is a decision problem where the states are knowledge states about the decision tree, and the task is to choose to think about the components of the problem at hand in the way that is most likely to yield a good final choice.
Emotions Implement Approximate Metareasoning Strategies
Model-based reasoning is hence faced with two profound challenges: the size of the problem and the even harder task of apportioning limited resources in an adaptive manner. These are fundamental problems and a strategy to deal with them is mandatory. The proposal here is that emotions can implement approximate solutions to these challenges. In particular, emotional states 1) come with a strong focus on particular behaviors and 2) induce a strong perceptual and processing focus such that evaluation is concentrated on a narrow set of states. Emotions thereby effectively function as approximate metareasoning strategies that prescribe how computational resources are allocated. To what extent these approximations are adaptive depends on how they are invoked. We first provide an outline for the implementation in terms of action tendencies and state observations, and then turn to the control issues.
Action Tendencies
One of the features of emotions about which there is more agreement is that they prioritize certain actions (
13
, 35
, , 80
, 81
). Constraining the action space can substantially simplify the valuation problem because the computational cost is exponential in the size of the action space.At an abstract level, emotional states are accompanied by distinct and richly experienced urges toward particular classes of actions. Frijda et al. (
20
) asked people to remember events of particular emotions and then to rate a list or 26 items about the kinds of behaviors they wanted to engage in, such as “I gave up,” “I wanted to protect myself from someone or something,” or “I wanted to help someone, to take care of someone.” From the ratings of these statements, the emotion characterizing the episode could be reliably recovered. Though very abstract, such rich descriptions are also important in psychotherapeutic settings. In dialectic behavior therapy, individuals are initially taught to recognize emotions by the action tendencies they feel (71
).Emotions also induce physiological and vegetative changes. However, physiological signatures of emotions do not appear to readily differentiate between categorically defined emotions, but rather provide a few classes of general action preparations [(,
82
, 83
, 84
), though see (26
)]. A preparatory increase in heart rate to compensate for the anticipated drop in peripheral resistance upon supplying blood to large muscle groups is required when running, be it for escape or fun. As such, these can be seen as a preparation toward a class of behaviors that share physiological requirements.State Observation
The complexity of model-based evaluation is also exponential in the range of states considered. There is ample evidence for emotion- or mood-congruent processing biases (
85
, 86
). For instance, Bradley et al. (87
) showed that exposure to sad music and recollection of sad memories produces an attentional bias toward sad words, and such biases arise from the emotional state rather than purely from the exposure to the emotional word (88
). By restricting attention to particular states and disregarding others, the problem could again be reduced in size (89
), for instance by pruning (44
) searches along branches of the decision tree that result in states outside the attentional focus.A further aspect is that there is usually uncertainty about the state. This profoundly complicates the computational task of valuation because policies for the various possible states have to be computed (
90
). By ascertaining the state, this complexity can be reduced. Introspection about the state of the body likely plays a particularly important role: the impact of a muscle’s activation depends on the joint position and the chances of success in a fight are reduced when already wounded.Controlling Metareasoning Strategies
If there are multiple approximate metareasoning strategies, then there must be some control over which is deployed when. The first source of control is likely evolutionary, where species-specific responses provide a (potentially very strong) bias toward evaluating particular actions, rather than toward emitting the action entirely inflexibly. This allows for the kinds of context effects on even innate responses mentioned above.
The second source of control could be, confusingly, model-free. Performers may learn from experience that a certain amount of catastrophizing improves their performance (
91
), without an understanding of why that is. Etkin et al. (92
) have recently argued for a model-free component in serial adaptations in the emotional conflict task, where individuals have to indicate the facial expression (fear or happy) of a face with either the matching or conflicting word superimposed over it (93
). Model-free learning has been argued to account for learning in strategy selection (94
): with repeated experience, individuals can slowly increase their frequency of using adaptive strategies for solving problems (95
). We have recently shown how the results of costly model-based evaluations are memorized and simply replayed upon repeated encounter of the same problem in a process called memoization (55
) that gives rise to decision-making biases that are characteristic of the individual but highly variable across the population.The third evaluative process for emotions allows for knowledge to be incorporated in the form of heuristics. Research on decisions about options with many attributes [for instance cars, with price, speed, size, brand, etc. (
96
)] have identified a host of different strategies. “Take the best” is appropriate in noncompensatory environments where one feature is most informative and can be used alone to rank options. In compensatory environments, humans spend more time and cognitive effort on examining multiple features and integrating the information, but only if they are not under time pressure (97
). This suggests that individuals can access approximate measures of how adaptive a particular cognitive strategy is, and use this to guide their choice (98
). In the affective domain, misguided beliefs or schemas () about the adaptiveness of strategies relate to a number of pathological emotion regulation phenomena. Pathological worry in generalized anxiety disorders (- Lieder F.
- Plunkett D.
- Hamrick J.B.
- Russell S.J.
- Hay N.
- Griffiths T.
Algorithm selection by rational metareasoning as a model of human strategy selection.
in: Ghahramani Z. Welling M. Cortes C. Lawrence N.D. Weinberger K.Q. Advances in Neural Information Processing Systems 27: 28th Annual Conference on Neural Information Processing Systems. Curran Associates,
Red Hook, NY2014: 2870-2878
100
, 101
) and rumination in depressive disorders (102
) are maintained by explicit beliefs about the usefulness of worry and rumination, respectively. People who dislike emotion regulation are more likely to respond with anger to provocation (103
). Depressed persons are not impaired at emotion regulation strategies such as positive imagery to improve their mood, but they have a reduced tendency to employ them (104
).The fourth evaluative process, again confusingly, could be model-based, where the precise consequences of particular emotions are examined and evaluated. This is rarely feasible and probably only commonly done in situational analyses in psychotherapy, where emotions, thoughts, behavior, and consequences are explicitly discussed (
71
, , 105
). This allows patients to learn to consciously and explicitly assess whether a particular emotion is appropriate and helpful in a given situation, and to adapt it by using reappraisal and other emotion regulation strategies if necessary.Discussion
We have attempted to sketch out a valuation framework for emotions. We have argued why appraisals point toward a model-based framework; how emotions may have a potentially important role in facilitating model-based decisions by functioning as internal strategies to allocate computational resources; how emotions’ adaptive nature depends on their deployment; and how a variety of different processes can lead to adaptive or maladaptive deployment of emotion strategies.
Three desiderata for a computational framework of emotions were put forth. The first was the at best partially correlated nature of physiological, psychological, and behavioral features. The flexibility the proposed framework allows for contrasts with the view of basic emotions as relatively fixed behavioral and physiological action packages. As such, it reflects the lack of identifiably discrete physiological or behavioral patterns or single neurobiological cause (
10
, 24
, 29
). Similar to other proposals, it emphasizes the importance of emotional processes in more complex decision-making settings (106
). Space constraints have prevented us from exploring the distinction between valence and arousal important to circumplex and core affect models, but this might naturally emerge from valuation in continuous-time settings, where the rate at which actions are emitted depends on the average reward rate in the environment, albeit in sometimes complex ways (36
, 50
, 51
, 107
). Notably, the current proposal allows for mixed emotions through a combination of metareasoning policies.The second desideratum was the ability to account for appraisal and contextual effects. The complexity of the model-based valuation required for this led to the notion of approximate metareasoning strategies. These approximate strategies are necessarily often suboptimal and may capture the prototypical adverse influences of emotion on cognition (
108
). The focus on valuation is compatible with models emphasizing prediction (109
), but distinct in that it suggests that the relevant predictions must be about long-term utility, and that emotions play a key role in facilitating such predictions, albeit approximately.The third desideratum concerned the nature of conscious qualia of emotions. Doing so fully awaits a theory of consciousness. However, two aspects are interesting to consider. First, Dehaene and Naccache’s (
110
) notion of a global workspace sits naturally with the notion of metareasoning. Situations with a high estimated value of computation should recruit neural resources more extensively, and hence be more likely to involve the brain-wide states postulated as representing the global workspace. Second, it has been suggested that the component processes in verbal self-report involve an interoceptive component followed by a classification process (30
). In our proposal, the metareasoning strategies would profoundly influence what information was processed, and as such may strongly determine future classification. Interestingly, the classification process has been suggested to involve the ventrolateral prefrontal cortex (31
, 32
), which is also known to mediate arbitration between valuation strategies (111
).The framework laid out here makes a number of testable predictions. First, the argument that appraisal involves model-based reasoning means that it should be influenced by cognitive, endocrine, and neuromodulatory variables known to influence model-based reasoning (
112
, 113
, 114
, 115
). Second, we emphasized the importance of the control of metareasoning strategies and suggested that it may be subject to substantial malleability. This in turn predicts that by training particular metareasoning strategies it should be possible to selectively facilitate certain emotions over others. Third, it suggests that emotions do not represent valuations themselves, but rather that they determine the process by which valuation occurs. This predicts an influence of emotion on search in a relevant model-based task rather than on values directly. The main challenge for the framework is that it points to the critical importance of understanding both the internal models of individuals and their strategies in searching them. Measuring these is a difficult scientific problem. Though this is as yet not feasible, recent advances including whole-brain mapping of semantic representations (116
, 117
) combined with active and passive sensing using mobile devices (118
, 119
) should open promising avenues.The emphasis on model-based processes was partially motivated by the finding that model-free measures of reward processing and learning are unimpaired in depression (
120
, 121
), and as such this work is an effort to start integrating cognitive phenomena of clinical importance like dysfunctional attitudes (122
), helplessness (68
), attributional/cognitive styles (123
, 124
), and appraisals (35
) into a valuation framework.Acknowledgments and Disclosures
This work was supported by Swiss National Science Foundation Grant No. 320030L_153449/1 (to QJMH).
We thank Peter Dayan for comments on an earlier version of this manuscript.
The authors report no biomedical financial interests or potential conflicts of interest.
References
- Are computational models of any use to psychiatry?.Neural Netw. 2011; 24: 544-551
- From reinforcement learning models to psychiatric and neurological disorders.Nat Neurosci. 2011; 14: 154-162
- Computational psychiatry.Trends Cogn Sci. 2012; 16: 72-80
- Computational approaches to psychiatry.Curr Opin Neurobiol. 2014; 25: 85-92
- Computational psychiatry as a bridge from neuroscience to clinical applications.Nat Neurosci. 2016; 19: 404-413
- Global burden of disease attributable to mental and substance use disorders: Findings from the global burden of disease study 2010.Lancet. 2013; 382: 1575-1586
- Rational choice and the structure of the environment.Psychol Rev. 1956; 63: 129-138
- Principles of metareasoning.Artif Intell. 1991; 49: 361-395
- The Adaptive Decision Maker.Cambridge University Press, New York1993
- Are emotions natural kinds?.Perspect Psychol Sci. 2006; 1: 28-58
- Neurologizing the psychology of affects: How appraisal-based constructivism and basic emotion theory can coexist.Perspect Psychol Sci. 2007; 2: 281-296
- An argument for basic emotions.Cogn Emot. 1992; 6: 169-200
- Basic emotions, natural kinds, emotion schemas, and a new paradigm.Perspect Psychol Sci. 2007; 2: 260-280
- Coming to terms with fear.Proc Natl Acad Sci U S A. 2014; 111: 2871-2878
- Nonconscious fear is quickly acquired but swiftly forgotten.Curr Biol. 2012; 22: R477-R479
- Neural substrates of explicit and implicit fear memory.Neuroimage. 2009; 45: 208-214
- Fear and panic in humans with bilateral amygdala damage.Nat Neurosci. 2013; 16: 270-272
- Panic anxiety in humans with bilateral amygdala lesions: Pharmacological induction via cardiorespiratory interoceptive pathways.J Neurosci. 2016; 36: 3559-3566
- Appraisal Theory.Oxford University Press, Oxford, UK2001
- Relations among emotion, appraisal, and emotional action readiness.J Pers Soc Psychol. 1989; 57: 212-228
Lazarus RS: Stress and Emotion: A New Synthesis. New York: Springer.
- Appraisal theory: Old and new questions.Emot Rev. 2013; 5: 125-131
- What are emotions? And how can they be measured?.Soc Sci Information. 2005; 44: 695-729
- The psychophysiology of emotion.in: Lewis R. Haviland-Jones J.M. Handbook of Emotions, 2nd ed. Guilford Press, New York2000: 173-191
- Facial expressions of emotion are not culturally universal.Proc Natl Acad Sci U S A. 2012; 109: 7241-7244
- Autonomic specificity of basic emotions: Evidence from pattern classification and cluster analysis.Biol Psychol. 2010; 84: 463-473
- Multivariate neural biomarkers of emotional states are categorically distinct.Soc Cogn Affect Neurosci. 2015; 10: 1437-1448
- Discrete neural signatures of basic emotions.Cereb Cortex. 2016; 26: 2563-2573
- A Bayesian model of category-specific emotional brain responses.PLoS Comput Biol. 2015; 11: e1004066
- Solving the emotion paradox: Categorization and the experience of emotion.Pers Soc Psychol Rev. 2006; 10: 20-46
- Putting feelings into words: Affect labeling disrupts amygdala activity in response to affective stimuli.Psychol Sci. 2007; 18: 421-428
- The functional neural architecture of self-reports of affective experience.Biol Psychiatry. 2013; 73: 631-638
- Core affect and the psychological construction of emotion.Psychol Rev. 2003; 110: 145-172
- Neural evidence that human emotions share core affective properties.Psychol Sci. 2013; 24: 947-956
- Appraisal theories of emotion: State of the art and future development.Emot Rev. 2013; 5: 119-124
- Algorithms for survival: A comparative perspective on emotions.Nat Rev Neurosci. 2017; 18: 311-319
- Reinforcement Learning: An Introduction.MIT Press, Cambridge, MA1998
- On the ability of prey to recognize predators.Z Tierpsychol. 1980; 54: 71-84
- Species-specific defense reactions and avoidance learning.Psychol Rev. 1970; 77: 32-48
- On the generality of the laws of learning.Psychol Rev. 1970; 77: 406-418
- Stimulus and response contingencies in the misbehavior of rats.J Exp Psychol Anim Behav Process. 1982; 8: 62-85
- Disentangling the roles of approach, activation and valence in instrumental and Pavlovian responding.PLoS Comput Biol. 2011; 7: e1002028
- The misbehavior of value and the discipline of the will.Neural Netw. 2006; 19: 1153-1160
- Bonsai trees in your head: How the Pavlovian system sculpts goal-directed choices by pruning decision trees.PLoS Comput Biol. 2012; 8: e1002410
- Opponent appetitive-aversive neural processes underlie predictive learning of pain relief.Nat Neurosci. 2005; 8: 1234-1240
- Modelling individual differences in the form of pavlovian conditioned approach responses: A dual learning systems approach with factored representations.PLoS Comput Biol. 2014; 10: e1003466
- A temporal difference account of avoidance learning.Network. 2008; 19: 137-160
- Two-factor theory, the actor-critic model, and conditioned avoidance.Learn Behav. 2010; 38: 50-67
- Context, learning, and extinction.Psychol Rev. 2010; 117: 197-209
- Tonic dopamine: Opportunity costs and the control of response vigor.Psychopharmacology (Berl). 2007; 191: 507-520
- Instrumental vigour in punishment and reward.Eur J Neurosci. 2012; 35: 1152-1168
- The misbehavior of organisms.Am Psychol. 1961; 16: 681-684
- Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learning.Artif Intell. 1999; 112: 181-211
- Hierarchically organized behavior and its neural foundations: A reinforcement learning perspective.Cognition. 2009; 113: 262-280
- Interplay of approximate planning strategies.Proc Natl Acad Sci U S A. 2015; 112: 3098-3103
- Dissociation of associative and nonassociative concomitants of classical fear conditioning in the freely behaving rat.Behav Neurosci. 1988; 102: 66-76
- Animal escapology II: Escape trajectory case studies.J Exp Biol. 2011; 214: 2474-2494
- Animal escapology I: Theoretical issues and emerging trends in escape trajectories.J Exp Biol. 2011; 214: 2463-2473
- Effect of refuge distance on escape behavior of side-blotched lizards (uta stansburiana).Can J Zoology. 2009; 87: 407-414
- Fear-potentiated startle conditioning in humans: explicit and contextual cue conditioning following paired versus unpaired training.Psychophysiology. 1997; 34: 451-458
- Suppression and enhancement of emotional responses to unpleasant pictures.Psychophysiology. 2000; 37: 515-522
- Emotion, attention, and the startle reflex.Psychol Rev. 1990; 97: 377-395
- Short-circuiting of threat by experimentally altering cognitive appraisal.J Abnorm Psychol. 1964; 69: 195-205
- Automatic constructive appraisal as a candidate cause of emotion.Emot Rev. 2010; 2: 139-156
- The algorithmic anatomy of model-based evaluation.Philos Trans R Soc Lond B Biol Sci. 2014; 369: 20130478
- Emotion and Adaptation.Oxford University Press, Oxford, UK1994
- On the nature and function of emotion: A component process approach.in: Scherer K.R. Ekman P. Approaches to Emotion. Erlbaum, Hillsdale, NJ1984: 293-317
- A Bayesian formulation of behavioral control.Cognition. 2009; 113: 314-328
- Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control.Nat Neurosci. 2005; 8: 1704-1711
- Depression: Clinical, Experimental and Theoretical Aspects.Harper & Row, New York1967
- Interactive Skill Training for Borderline Patients [German].Schattauer Verlag, Stuttgart, Germany2013
- Feeling and thinking: Preferences need no inferences.Am Psychol. 1980; 35: 151
- Internal models for motor control and trajectory planning.Curr Opin Neurobiol. 1999; 9: 718-727
- When time is money: Decision behavior under opportunity-cost time pressure.Organ Behav Hum Dec Process. 1996; 66: 131-152
- The anchoring bias reflects rational use of cognitive resources [published online ahead of print May 8].Psychon Bull Rev. 2017;
- Speed/accuracy trade-off between the habitual and the goal-directed processes.PLoS Comput Biol. 2011; 7: e1002055
- Pure exploration in finitely-armed and continuous-armed bandits.Theor Comput Sci. 2011; 412: 1832-1852
- Metareasoning for Monte Carlo tree search. Technical Report, Electrical Engineering and Computer Sciences.University of California at Berkeley, Berkeley, CA2011
- The Principles of Psychology.Dover, New York1980
- The circumplex model of affect: An integrative approach to affective neuroscience, cognitive development, and psychopathology.Dev Psychopathol. 2005; 17: 715-734
- Appetitive and defensive motivation: Goal-directed or goal-determined?.Emot Rev. 2013; 5: 230-234
- On the automaticity of emotion.in: Bargh J.A. Social Psychology and the Unconscious: The Automaticity of Higher Mental Processes. Psychology Press, Suffolk, UK2007: 173-217
- Feelings and the body: The Jamesian perspective on autonomic specificity of emotion.Biol Psychol. 2010; 84: 383-393
- Emotion and the motivational brain.Biol Psychol. 2010; 84: 437-450
- Attentional bias in emotional disorders.J Abnorm Psychol. 1986; 95: 15-20
- Cognitive vulnerability to emotional disorders.Annu Rev Clin Psychol. 2005; 1: 167-195
- Attentional biases for negative information in induced and naturally occurring dysphoria.Behav Res Ther. 1997; 35: 911-927
- Stroop interference following mood induction: Emotionality, mood congruence, and concern relevance.Cogn Therapy Res. 2000; 24: 491-502
- Planning under time constraints in stochastic domains.Artif Intell. 1995; 76: 35-74
- Planning and acting in partially observable stochastic domains.Artif Intell. 1998; 101: 99-134
- Correlates of adaptive and maladaptive musical performance anxiety.Med Prob Perform Artists. 1989; 4: 49-56
- The neural bases of emotion regulation.Nat Rev Neurosci. 2015; 16: 693-700
- Resolving emotional conflict: A role for the rostral anterior cingulate cortex in modulating activity in the amygdala.Neuron. 2006; 51: 871-882
- Strategy selection: An introduction to the modeling challenge.Wiley Interdiscip Rev Cogn Sci. 2014; 5: 39-59
- SSL: A theory of how people learn to select strategies.J Exp Psychol Gen. 2006; 135: 207-236
- Reasoning the fast and frugal way: Models of bounded rationality.Psychol Rev. 1996; 103: 650-669
- Take the best or look at the rest? Factors influencing “one-reason” decision making.J Exp Psychol Learn Mem Cogn. 2003; 29: 53-65
- Algorithm selection by rational metareasoning as a model of human strategy selection.in: Ghahramani Z. Welling M. Cortes C. Lawrence N.D. Weinberger K.Q. Advances in Neural Information Processing Systems 27: 28th Annual Conference on Neural Information Processing Systems. Curran Associates, Red Hook, NY2014: 2870-2878
- Cognitive models of depression.J Cog Psychotherapy Int Quart. 1987; 1: 5-37
- A metacognitive model and therapy for generalized anxiety disorder.Clin Psychol Psychother. 1999; 6: 86-95
- Worry: A cognitive phenomenon intimately linked to affective, physiological, and interpersonal behavioral processes.Cogn Therapy Res. 1998; 22: 561-576
- Rumination reconsidered: A psychometric analysis.Cogn Therapy Res. 2003; 27: 247-259
- How to bite your tongue without blowing your top: Implicit evaluation of emotion regulation predicts affective responding to anger provocation.Pers Soc Psychol Bull. 2006; 32: 589-602
- Emotion regulation and vulnerability to depression: Spontaneous versus instructed use of emotion suppression and reappraisal.Emotion. 2010; 10: 563-572
- Antecedent- and response-focused emotion regulation: Divergent consequences for experience, expression, and physiology.J Pers Soc Psychol. 1998; 74: 224-237
- Mood and judgment: The affect infusion model (aim).Psychol Bull. 1995; 117: 39-66
- Learning the opportunity cost of time in a patch-foraging task.Cogn Affect Behav Neurosci. 2015; 15: 837-853
- Out of control: Visceral influences on behavior.Org Behav Hum Decis Process. 1996; 65: 272-292
- The theory of constructed emotion: An active inference account of interoception and categorization.Soc Cogn Affect Neurosci. 2017; 12: 1-23
- Toward a cognitive neuroscience of consciousness: Basic evidence and a workspace framework.Cognition. 2001; 79: 1-37
- Neural computations underlying arbitration between model-based and model-free learning.Neuron. 2014; 81: 687-699
- The curse of planning: Dissecting multiple reinforcement-learning systems by taxing the central executive.Psychol Sci. 2013; 24: 751-761
- Working-memory capacity protects model-based learning from stress.Proc Natl Acad Sci U S A. 2013; 110: 20941-20946
- Ventral striatal dopamine reflects behavioral and neural signatures of model-based control during sequential decision making.Proc Natl Acad Sci U S A. 2015; 112: 1595-1600
- Dopamine enhances model-based over model-free choice behavior.Neuron. 2012; 75: 418-424
- Natural speech reveals the semantic maps that tile human cerebral cortex.Nature. 2016; 532: 453-458
- Decoding the semantic content of natural movies from human brain activity.Front Syst Neurosci. 2016; 10: 81
- New tools for new research in psychiatry: A scalable and customizable platform to empower data driven smartphone research.JMIR Ment Health. 2016; 3: e16
- 368-deep dynamic phenotyping: Neural changes underlying fluctuations in bipolar disorder over one year.Biol Psychiatry. 2017; 81: S150-S151
- Depression: A decision-theoretic account.Annu Rev Neurosci. 2015; 38: 1-23
- Association of neural and emotional impacts of reward prediction errors with major depression.JAMA Psychiatry. 2017; 74: 790-797
Weissman A, Beck A (1978): Development and validation of the Dysfunctional Attitude Scale. Presented at the Annual Meeting of the Association for the Advancement of Behavior Therapy, Chicago.
- The attributional Style Questionnaire.Cogn Therapy Res. 1982; 6: 287-299
- Suicidality and cognitive vulnerability to depression among college students: A prospective study.J Adolesc. 1998; 21: 473-487
Article Info
Publication History
Published online: July 11, 2017
Accepted:
July 1,
2017
Received in revised form:
June 30,
2017
Received:
November 5,
2016
Identification
Copyright
© 2017 Society of Biological Psychiatry.
User License
Creative Commons Attribution – NonCommercial – NoDerivs (CC BY-NC-ND 4.0) | How you can reuse 
Elsevier's open access license policy
Creative Commons Attribution – NonCommercial – NoDerivs (CC BY-NC-ND 4.0)
Permitted
For non-commercial purposes:
- Read, print & download
- Redistribute or republish the final article
- Text & data mine
- Translate the article (private use only, not for distribution)
- Reuse portions or extracts from the article in other works
Not Permitted
- Sell or re-use for commercial purposes
- Distribute translations or adaptations of the article
Elsevier's open access license policy
31773-0&id=gr1.jpg){kind=link}
31773-0&id=gr2.jpg){kind=link}