Editors' Note: Tim Meeker, a PhD student working with Joel Greenspan and David Seminowicz at the University of Maryland, Baltimore, US, recently led a Journal Club discussing work from Mathieu Roy and colleagues that was published in the November 2014 issue of Nature Neuroscience. PRF thanks Tim for sending along this summary and discussion of this interesting paper. Please consider continuing the conversation by leaving your own comment on this work! Log in to submit a comment below.
Most human neuroimaging studies of the neural correlates of pain processing focus on the intensity or unpleasantness of the stimuli or the combination of painful and other stimuli with the intent to modulate pain perception. Roy and colleagues, in a recent article in Nature Neuroscience, take the road less traveled by treating acute pain as an aversive stimulus which participants learn to avoid (Roy et al., 2014). In healthy individuals experiencing potentially an escapable aversive stimulus such as pain, this turns out to be the natural road to follow. The authors found that BOLD signal in the periaqueductal gray (PAG), but not the ventral striatum (VS), reflected recently developed axiomatic criteria for learning to avoid pain.
Learning to acquire rewarding or avoid aversive stimuli is thought to be best reflected in the surprise or prediction error (PE) a participant experiences at the outcome of a trial. Since correlation models are difficult, if not impossible, to rigorously falsify, the authors used an axiomatic model encapsulating the necessary and sufficient elements of a PE model (Rutledge et al., 2010). In Roy and colleagues' study, participants were instructed to select one of two visual stimuli which they thought would be followed with the lowest probability by a painful thermal stimulus. In reality, in order to maintain a stable learning rate for each participant, the probability of painful stimuli being delivered after a particular choice was determined by a random walk generated for each of the two possible choices.
The authors followed two approaches to analyze the resultant neuroimaging data. First, they modeled the BOLD response in the scanned brain regions with a temporal difference model which encapsulated the PE. PE is represented algebraically as the difference between the actual value of the reward or punishment of the upcoming trial and its predicted value. Expected value is calculated based on prior experience on previous trials. Since this model is highly correlated with the actual stimulus function, they compared the resultant spatial parametric map with a simple contrast map elucidating the areas where brain activity was modulated by painful stimuli and found that activity representing aversive PEs and BOLD modulation induced by painful stimuli greatly overlapped.
To counter this barrier to falsification, the authors employed an axiomatic model optimized to aversive, painful stimuli. This model tested BOLD responses in hypothesized brain regions during trials of various outcome-expectancy pairs against three axioms: 1) "consistent prize ordering: the outcome effect," where activity for pain outcomes should be higher than for no-stimulus outcomes; 2) "consistent lottery ordering: the expectancy effect," where activity should decrease with increasing probability of pain; and 3) "no surprise equivalence: expectancy and outcome effects have the correct relationship to each other," where completely predicted outcomes should generate equivalent responses. Using derivatives of these axioms, the authors conducted a whole-brain search for BOLD responses that supported each axiom.
The primary finding of the study is that while the BOLD response in the PAG fulfills all the axiomatic criteria, the VS, encompassing the nucleus accumbens, fails to fulfill any of the axioms, falsifying the hypothesis that this region encodes the aversive PE to painful stimuli. While pain responsive regions lacking modulation of BOLD signal in the face of increased expectancy of pain in the absence of pain, such as the anterior mid-cingulate cortex (aMCC), orbitofrontal cortex (OFC), and dorsomedial prefrontal cortex (dmPFC), fulfill axiom 1, they violate axiom 2 for no stimulus trials and axiom 3. The authors contend these regions are responsible for avoidance value updating and pain-specific PEs. Alternately, expectancy responsive regions which did not show greater modulation in the face of pain versus no stimuli, such as the putamen, hippocampus, and ventromedial prefrontal cortex (vmPFC) fulfilled axiom 2, but violated axioms 1 and 3. The authors demonstrate that these regions only support expectancy processing and not processing of aversive PEs.
As a last piece of the puzzle, the authors created a modular dynamic causal model of the neuronal interactions first among the PAG and expectancy responsive regions (putamen, hippocampus, and vmPFC) and then incorporating the interactions between the expectancy responsive regions and the avoidance value updating regions (OFC, aMCC, and dmPFC). The model with the best fit showed that pain triggers activity in the PAG, which combines this information with value-updating information from the putamen received via the vmPFC. This information summates to provide signals to the aMCC and dmPFC to allow pain PE signals on which the participant can act. Notably, this model arises from an acute, phasic pain stimulus.
This study is a significant leap forward in many ways. First, the axiomatic model employed allowed the authors to disambiguate the BOLD signal evoked in response to aversive PEs in the PAG and VS. Second, several brain regions involved in elements of the processing of aversive PEs were identified. Third, the authors discovered a model fitting the interactions of these brain regions and their modification by pain and valuation judgment. Note that PAG activity may be specific to learning to avoid noxious stimuli, particularly taking into account the neural circuitry specific to pain processing. PEs to other modalities of aversive stimuli, such as offensive odors, tastes, and shock stimuli, appear to be processed in the VS (Metereau and Dreher, 2013; Li et al., 2011).
Three groups could immediately benefit from studies similar to that conducted by Roy and colleagues. These include patients suffering from chronic pain, those with borderline personality disorder, and those with psychosis. This work will allow hypotheses addressing the modification of the neural processing of pain avoidance by the disease processes underlying pain chronification. Previous hypothesis- and data-driven studies have discovered chronic pain-driven alterations in the structure of many of the brain regions discovered to support learning from painful stimuli, such as the putamen, vmPFC, OFC, anterior cingulate cortex (ACC), and PAG (Davis and Moayedi, 2013). These modifications likely lead to abnormalities in the interactions of these regions in the processing of pain learning. An additional area where understanding disordered pain learning will be of great impact is in patients with borderline personality disorder (Schmahl et al., 2006). Another natural extension from understanding the processing of pain learning is evaluating the neural mechanisms underlying disordered processing of empathy to pain in psychopathic individuals (Decety et al., 2013). Disease-related alterations in nodes of the identified pain learning circuitry could be targeted by neuromodulatory methods such as transcranial direct current stimulation, repetitive transcranial magnetic stimulation, and transcranial pulsed ultrasound stimulation. Additionally, morphological modification in the nodes of the network could act as a biological proxy to track treatment progress, especially in disorders with subjective symptoms and in those patients prone to deception such as people with psychosis.
Additionally, the learning model employed may subsume and ultimately encompass activation of the PAG and related circuitry in attention to pain, anticipation of pain, and placebo effects (Bingel et al., 2006; Fairhurst et al., 2007; Valet et al., 2004; Roy et al., 2014). The reason for the subsumption and involvement of pain learning in these areas should be clear: attention to and anticipation of pain are inherently involved in aversive PE models in pain learning, while placebo responses are conceivably modified throughout a person’s life by repeated exposure to pain and its therapeutic relief.
There are a few limitations to the interpretation of this study. The optimized scanning method used forced the authors to ignore the involvement of somatosensory and parietal cortices. Another limitation related to the scanning methods employed in this study is that while the ROI extracted which supported all three axioms is centered on the PAG, another learning-related brainstem nucleus is close to the PAG—specifically, the ventral tegmental area (D'Ardenne et al., 2008). Future studies with modified and more advanced scanning technology can improve coverage and increase resolution up to the minimum point spread function of the BOLD signal (Satpute et al., 2013). Since modifications of reward circuitry have been implicated in the transition from acute to chronic pain, longitudinal evaluation of functional responses during learning of rewarding or aversive stimuli during this transition would be a valuable contribution (Baliki et al., 2012).
In summary, making use of a modified axiomatic approach to reward PE which allows falsification of three principal axioms defining PEs, Roy and colleagues dissociate aversive PE processing during pain avoidance in the PAG from PE processing in the VS. Additionally, the authors identified a dynamic neural network consistent with the BOLD activity observed during the learning of pain avoidance. These results can focus future research in pain and empathy to pain learning in patient populations.