Author: Mackenna Hill, Elizabeth Duval
Institution: [1] Department of Biomedical Engineering, University of Michigan, 2200 Bonisteel Blvd., Ann Arbor, MI 48109; [2] Health System Department of Psychiatry, University of Michigan, 4250 Plymouth Rd., Ann Arbor, MI 48109
doi:10.22186/jyi.31.3.9-14
ABSTRACT
Attention bias modification (ABM) has been shown to decrease self-reported symptom severity for those with social anxiety disorder (SAD). ABM may also decrease attention bias towards threat present in SAD. Currently, the most prominent form of ABM is a modified dot-probe paradigm that uses two affective/emotional faces to measure or train attention bias. Results are mixed in previous studies regarding the ability of ABM to alter attention bias. Carry-over effects from trial to trial may help determine the origin of these variations, as the presence of carry-over effects may alter the component of attention measured in each dot-probe trial. In this study, 85 subjects were randomly assigned to three ABM training conditions: attend positive, attend threat, and a control training. Before and after ABM, attention bias scores were measured. No carry-over effects for any group or ABM condition were found. In addition, attention bias scores did not differ between groups or training conditions at initial assessment and the ABM training did not alter attention bias scores. Although our findings do not support the role of carry-over effects, it is possible that carry-over effects may help elucidate the role of attention bias both in the dot-probe paradigm and in SAD. Thus, carry-over effects should be considered in dot-probe data analysis.
INTRODUCTION
Social Anxiety Disorder (SAD) is characterized by excessive worry and self-consciousness surrounding social situations due to significant discomfort, negative internal evaluations, or anticipated embarrassment (American Psychological Association, 2013). SAD is a debilitating disorder which leads to significant impairment in work and social situations. Annually, about 7% of the population meets DSM-V criteria for SAD and it accounts for 10% to 20% of people with anxiety disorders in outpatient settings (American Psychological Association, 2013; Ruscio et al., 2008). Not surprisingly, treatment for this disorder is in high demand.
In an effort to reduce anxiety symptoms, attention bias modification (ABM) has been explored as a potential computer-based treatment for SAD designed to alter how people process and pay attention to social information (Amir et al., 2008; Heeren, Lievens, & Philippot, 2011; Klumpp & Amir, 2010). ABM is a modified dot-probe paradigm using affective/emotional stimuli such as facial expressions or words. During the modified dot-probe task, two stimuli appear on the screen for less than 1000 milliseconds, after which a target (letter or symbol) replaces one of the stimuli. The participant is asked to respond to the target as quickly and accurately as possible. The trial is named for the type of stimulus being replaced by the target (i.e., if a threat stimulus is replaced, the trial is a threat trial).
In the literature, three ABM conditions have been tested: attend positive (positive stimulus is more often replaced by a target), attend threat (negative stimulus is more often replaced by a target), and a no-training control (stimulus types are replaced by a target with equal probability; Frewen, Dozois, Joanisse, & Neufeld, 2008; Klumpp & Amir, 2010). An attention bias, the tendency to attend to a certain type of information over other types of information, can be determined by calculating the difference in reaction times between threat trials and positive trials (Amir et al., 2009; Izetelny, 2006). This calculation is most commonly carried out using data from an assessment task, which is often a dot-probe task similar to the control training (Boettcher et al., 2013; White, Suway, Pine, Bar-Haim, & Fox, 2011).
Individuals with SAD often demonstrate an attention bias to threat in these tasks (Bogels & Mansell, 2004). ABM has been developed in an effort to reduce this attention bias. In the dot-probe paradigm, attention bias manifests itself in reaction times. For example, if a SAD subject has an attention bias towards threat, the subject is, on average, quicker to respond to threat trials than to positive trials. This occurs because the subject’s attention is immediately drawn to the threatening stimuli. When the stimulus is replaced with an arrow, the subject will then respond more quickly.
However, previous findings on the subject are mixed, with some studies reporting no bias among SAD subjects (Boettcher et al., 2013; Bradley et al., 1997) or even a bias away from threat (Pittig, Pawlikowski, Craske, & Alpers, 2014; Harrison & Turpin, 2003). This complicates the literature on ABM to date, as individual attention bias may interact with ABM tasks to produce inconsistent results.
A question in the literature that has not yet been adequately addressed is how ABM modifies attention bias. Studies have shown that it is possible to modify attention bias towards the direction of the training (Li, Tan, Qian, & Liu, 2008; Amir et al., 2009; White et al., 2011). In contrast, Heeren et al., (2011) and Boettcher et al., (2013) report that neither type of ABM, either towards or away from threat, was able to modify attention bias. With all of these conflicting studies, it is difficult to identify a clear mechanism underlying ABM effects.
One way to further explore the underlying mechanism of ABM is to investigate carry-over effects. Carry-over effect is the tendency of one stimulus to have an influence over the evaluation of the next stimulus (Nonyane & Theobald, 2007). Two important carry-over effects are assimilation and contrast. Assimilation occurs when the previous stimulus is congruent with the current stimulus (e.g., a threat trial follows another threat trial). Contrast occurs when the trials are incongruent (e.g., a threat trial follows a positive trial).
The majority of carry-over effects are studied in the emotional Stroop task due to its multiple formats (Holle, Neely, & Heimberg, 1997; Waters, Sayette, Franken, & Schwartz, 2005). Similar to the modified dot-probe task, the emotional Stroop task measures attention through reaction time. One type of Stroop task, blocked, involves presenting the same type of stimuli in a series or block and represents assimilation carry-over effects. The blocked emotional Stroop task has been linked to attention maintenance (maintaining focused attention) in SAD and healthy controls through assimilation effects (Holle et al., 1997; Jones-Chester, Monsell, & Cooper, 1998; Waters and Feyerabend, 2000; Waters et al., 2005). The second type of Stroop task, mixed, involves presenting different stimulus types in a random order and represents contrast carry-over effects. The mixed emotional Stroop task has been linked to orienting of attention (i.e., initially focusing one’s attention) in SAD and healthy controls through contrast effects (Holle et al., 1997; Jones-Chester et al., 1998; Waters and Feyerabend, 2000; Waters et al., 2005). This task may generalize to social experiences, in that a blocked format Stroop task represents a cue-rich environment such as a party, while a randomized format Stroop task represents a mix of social and non-social cues more similar to a single interpersonal interaction (Waters et al., 2005).
Although the dot-probe task uses a mixed format, assimilation and contrast effects may be measured. This is possible because carry-over effects may be simplified to a single previous trial rather than multiple previous trials (Aguirre, 2007). Thus, assimilation effects in the ABM paradigm occur on congruent trials, where the stimulus replaced by the target is the same emotion as the stimulus replaced during the previous trial. Contrast effects occur during incongruent trials, where the stimulus replaced by the target is different than that replaced in the previous trial. These effects have not previously been studied in the ABM paradigm. In addition, behavioral biases from carry-over effects are shown to be sensitive to task context, so a generalization between task and stimulus may not always be made (Aguirre, 2007). Accordingly carry over effects must be studied not only in the Stroop task but in ABM as well.
The purpose of this study was to investigate why ABM alters attention bias in some cases but not in others. This research will help determine the role of attention bias in the dot-probe paradigm and ABM training. It is possible that the modified dot-probe can be linked to the two components of attention in the same way as the emotional Stroop task, since both tasks contain the same carry-over effects. Klumpp and Amir (2010) and O’Toole and Dennis (2012) were among the first to introduce and test the idea that attention switching/disengaging from threat is the effective component of ABM. This same disengagement from threat stimuli in those with SAD is key to good performance, faster and more accurate responses, on the Stroop task as well (Cisler, Bacon, & Williams, 2009). Although blocked and mixed formats only exist in the Stroop task, dot-probe tasks simulate these with congruent and incongruent trials explained above. Therefore, we examined carry-over effects during dot-probe assessments, before and after ABM, to determine whether carry-over effects were linked to attention bias scores. If carry over effects are linked to attention bias scores the proceeding stimulus will affect the current stimulus and reaction times will differ for congruent or incongruent trial sequences ending in the same trial type (threat or positive).
METHODS
Participants
Participants (N=85) were recruited using flyers and online advertisements posted at a large university medical center and the surrounding community. Members of the surrounding community and patients seeking treatment at the university-affiliated outpatient anxiety clinic responded to the advertisements to express their interest in participating. The study protocol was approved by the Institutional Review Board at the University of Michigan Medical School. Participants with SAD and healthy controls (HC) were recruited through telephone and in-person diagnostic screenings with the Mini International Neuropsychiatric Interview (MINI; Sheehan et al., 1998), the Liebowitz Social Anxiety Scale (LSAS; Liebowitz, 1987), the Social Interaction Anxiety Scale (SIAS; Brown, Turvosky, Heimburg, Juster, Brown, & Barlow, 1997), and the Beck Depression Inventory (BDI-II; Beck, Steer, Ball, & Ranieri 1996). A licensed clinical psychologist performed all diagnostic assessments. All SAD participants had a current and primary SAD diagnosis based on the diagnostic assessment. HCwere free of current or past diagnosis of Axis I or II disorders. All participants were at least 18 years of age and provided written informed consent.
Participants with the following were excluded from the study: a) clinically significant medical or neurologic condition; b) primary psychiatric disorder other than SAD; c) life history of schizophrenia or bipolar disorder; d) current major depressive disorder; e) alcohol/drug abuse or dependence in the past year; f) current suicidal ideation; g) presence of an organic mental syndrome, mental retardation, or pervasive developmental disorder; h) unwilling/unable to sign the informed consent document; and i) current psychiatric medication use other than selective serotonin reuptake inhibitor (SSRI). SSRI medication at a stable dose for three months was permitted in the SAD group. Medications for minor conditions that would not directly affect measures related to the study (birth control, supplements, etc.) were also permitted. Beta blockers, stimulants, opioids, and other medications with known effects on sympathetic/parasympathetic functioning and/or attentional processing were not permitted. Current unipolar major depression or drug/alcohol abuse/dependence was an exclusion criterion for this study, but a past history of these conditions was not. Since these conditions are frequently present in SAD participants, exclusion of such participants would yield a biased and unrepresentative sample of the SAD population.
Procedure
After written informed consent was obtained, each participant was randomly assigned to one of three conditions (attend positive, attend threat, or control). Participants completed all tasks and self-report measures in the lab with researcher supervision over two visits. During the first visit, participants were screened for participation criteria and completed all self-report paper measures (LSAS, SIAS, BDI). During the second visit, participants completed three dot-probe tasks, including pre-assessment, ABM training, and post-assessment. Although not included in these analyses, self-report paper measures were repeated following ABM training.
Attention Bias Assessment
The attention bias assessment task occurred before and after ABM to assess attention bias. The assessment task was a modified dot-probe task consisting of 94 trials. Participants were presented with a fixation cross for 500ms followed by two affective faces, one positive (happy) and one negative (angry), for 500ms. After the faces disappeared, one face was replaced by an arrow. Participants were instructed to respond to the direction of the arrow (up or down), by pressing the arrow keys on a standard keyboard, as “quickly and accurately as possible”. The next trial began after a response was made. During this task, the positive and negative faces were replaced by the arrow with equal probability (50%). Reaction time and accuracy, used to determine the location of the participant’s attention, were recorded on each trial. Incorrect trials and trials where the reaction time was greater than three standard deviations from the mean of each participant were excluded.
Attention Bias Modification (ABM)
Prior to training, the participants were told researchers were testing a computer-based attention training program that could alter the way people process and pay attention to social information, which may help reduce SAD symptoms. ABM followed the same format as the assessment task. The only differences between the assessment and ABM was the face replacement ratio and the length. We employed three ABM training conditions: attend positive (arrow replaced the positive face 80% of the time), attend threat (arrow replaced the negative face 80% of the time), and control (both face types were replaced with equal probability). Each training session consisted of 744 trials. Note that the control ABM training was identical to the assessment task, excluding the number of trials, in order to control the impact of viewing faces and engaging in the task without manipulating attentional direction. All faces used in both ABM and the assessments were from the Pictures of Facial Affect (Ekman & Friesen, 1976), a widely used standardized face set.
Data Analysis
All analyses were conducted with International Business Machines Corporation’s (IBM) Statistical Package for the Social Sciences (SPSS v. 21). The threshold for significance was set at an alpha level of 0.05. Attention bias was calculated by subtracting the average reaction time on threat trials (arrow replaced the threat face) from the average reaction time on positive trials (arrow replaced the positive face). To examine carry-over effects, each trial was categorized based on not only the current trial but also the trial that preceded it. As seen in Figure 1, threat to positive occurred when the current trial is a positive trial and was preceded by a threat trial.
A 2 (time; pre-ABM, post-ABM) X 4 (trial type; positive to positive, threat to threat, positive to threat, threat to positive) X 2 (group; HC, SAD) X 3 (ABM condition; toward threat, toward positive, control) mixed ANOVA with time and trial type as within-subjects variables and group and condition as between-subjects variables was run with reaction time as the dependent variable. This analysis examined how the congruence of the arrow replacement on the previous trial affected reaction time on the next trial for different face types in HC and SAD.
To examine initial differences in HC and SAD groups, a t-test comparing mean pre-ABM bias scores in SAD versus HC was conducted. Finally, to determine if attention bias changed as a function of group or condition, we conducted a 2 (time; pre-ABM, post-ABM) X 2 (group; HC, SAD) X 3 (ABM condition; attend threat, attend positive, control) mixed ANOVA with time as a within-subject variable, group and condition as between-subject variables, and attention bias score as the dependent variable.
Fig. 1 ABM training and assessment images. An illustration of the ABM procedures, demonstrating an example of a threat to positive (incongruent) trial sequence (two trials) starting with screen 1 and ending with screen 6.
RESULTS
Self-Report Measures
Participants were similar across condition and group in age [F(2,79) = 0.859, p = .428] and gender [F(2,78) = 0.205, p = .815]. Participants were also similar within group across ABM conditions on all self-report measures: SIAS [F(2,79) = 0.282, p = .755], LSAS [F(2,79) = 2.105, p = .129], and BDI [F(2,79) = 0.142, p = .868]. There were significant differences between the HC and SAD groups on SIAS [F(1,79) = 356.4, p < .001], LSAS [F(1,79) = 365.5, p < .001], and BDI [F(1,79) = 49.76, p < .001], with SAD subjects reporting significantly more symptoms on all measures than HC . Demographics and self-report measures are summarized in Table 1.
Table 1. Group Demographics. Mean demographics and standard deviations (in parentheses) at the baseline assessment time point. A score of >38 on the SIAS and >55 on the LSAS indicate clinically significant SAD.
Attention Biases to Affective Faces
Mean biases and standard deviations for all groups and conditions are shown in Table 2. The average pre-ABM attention bias was 0.856ms (standard deviation: 32.7ms) for HC and -6.05ms (standard deviation: 32.4ms) for SAD. A negative bias indicates a bias towards threat, with a score of zero indicating no bias in either direction. One-sample t-tests confirmed that SAD and HC subjects’ bias scores did not differ significantly from zero, suggesting no attention bias prior to training in either group (p > .05). Attention bias did not change as a function of group or condition: no main effects or interactions were significant (p > .05). A one-sample t-test confirmed that SAD and HC subjects’ bias scores did not differ significantly from zero after ABM, suggesting no attentional bias prior to or after ABM, regardless of condition (p > .05).
Table 2. ABM Attention Bias Scores. Pre- and Post- ABM attention bias scores and standard deviations (in parentheses) for all groups and conditions are shown. Mean biases are in milliseconds and a negative bias indicates a bias towards threat.
The Effect of Carry-Over on Reaction Times
There was a significant effect of time [F(1,77) = 33.41, p < .001] but not of trial type [F(3,75) = .378, p = .798] on carry-over effects between groups and ABM conditions over time. There was also no interaction between time, trial type, and either group [F(3,75) = 0.876, p = .457] or condition [F(6,150) = 1.058, p = .391]. This demonstrates that participants’ responses became faster throughout the training independent of trial type, group, or condition. No other main effects or interactions were significant (p > .05).
DISCUSSION
The purpose of this study was to determine why ABM alters attention bias in some cases but not in others, with specific focus on carry-over effects, in an effort to better understand inconsistencies in the ABM literature. To our knowledge, this is the first study examining carry-over effects in a modified dot-probe paradigm. Our findings did not replicate previous reports that ABM alters attention bias (Li et al., 2008; Amir et al., 2009; White et al., 2011). However, there are numerous possible interpretations of these findings, including: 1) congruent and incongruent trials are measuring different components of attention; 2) two types of SAD biases exist, which impact response to training; and 3) trial order randomization may have impacted carry-over effects. Because of these possibilities, future studies should consider analyzing congruent and incongruent trials separately, as well as matching individual participants to different training protocols based on pre-training bias scores.
The majority of previous literature reports that participants with SAD have an attention bias towards threat, compared to positive faces, and HC show little to no attention bias either towards or away from threat faces (Bogels & Mansell, 2004). This study did not support these findings. The results of this study are consistent with the findings of Boettcher et al. (2013), who also report that SAD subjects’ bias scores did not differ significantly from zero and that neither type of ABM, either towards or away from threat, was able to modify attention bias. Despite the lack of significant ABM effects, we did obtain the expected effect of time, such that participants’ responses became faster from pre- to post- training. If carry-over effects were detected, we would have expected to see differences in the average reaction time for trial types ending in the same trial (e.g., threat to positive and positive to positive would differ). Since these differences were not present in our data, carry-over effects were not detected, indicating that carry-over effects were not present in this dot-probe paradigm. However, there may have been other contributing or conflicting factors.
One possible factor influencing our findings is that congruent and incongruent trials measure different components of attention. Carry-over effects in the emotional Stroop task allude to these two components of attention: orienting (to initially focus one’s attention) and maintenance (to maintain that focus; Waters et al., 2005). Similar to Stroop, it is possible that in the dot-probe task congruent trials measure maintenance of attention, while incongruent trials measure orienting attention. Dot-probe task data from subjects who display a bias in one direction should be analyzed in order to determine if different components of attention are being measured. This would also help determine how an initial attention bias in SAD affects the outcome of ABM and how initial bias can be utilized to improve current treatment. Future studies should aim to better understand how different components of attention impact performance on the dot-probe task.
A bias must exist in order to use carry-over effects to elucidate whether dot-probe measures two components or one component of attention. Because subjects in this study did not initially show a bias, their bias was not shifted as a result of ABM (O’Toole & Dennis, 2012). Thus, we were not able to further explore relationships between the different components of attention and trial types on the dot-probe task.
It is also possible that two types of SAD biases exist. There is some evidence in the literature that SAD subjects can be either “attenders” or “avoiders” of threat (O’Toole & Dennis, 2012). This means that some subjects with SAD have a bias toward threat while others have a bias away from threat. Most studies (Amir et al., 2008; White et al., 2011; Heeren et al., 2011) assume that all SAD subjects have the same bias. This would explain why some studies show a bias while others do not, resulting in significant variability within groups and making it difficult to detect effects of ABM. In our study, we did not have enough participants to properly investigate this theory, but it is possible that individual differences in attention bias interact with training type, resulting in certain people benefitting more from specific types of training. In order to optimize training effects, it is important to consider interactions between initial bias and training condition. Heeren et al. (2011) and Klumpp and Amir (2010) state that the process of disengaging attention may be the mechanism underlying ABM’s effectiveness. Therefore, to facilitate the most disengagement, it may be important to match initial bias with training condition. An attender (someone with a bias towards threat) may benefit most from the attention towards positive condition which requires them to disengage from the threat face, whereas an avoider (someone with a bias away from threat) may benefit most from training towards threat. Separating these two groups and training them differently may be the key to providing the ideal treatment for SAD.
The absence of carry-over effects could also be explained by the trial randomization procedure we used in the dot-probe paradigm, which was intended to eliminate carry-over effects.
Limitations
When interpreting these findings, there are a number of limitations to consider. First, this was a pilot study aiming to investigate effects of ABM on SAD symptoms. Therefore, the null findings may be due to lack of power. Statistical power analysis suggests that our sample size of approximately 14 participants per cell yields low to moderate power (65%) to detect differences between groups and ABM conditions. Another limitation stems from the intervention length. Although participants spent about 40 minutes completing 750 ABM trials, only a single session was completed. To properly simulate a future treatment, participants should engage in multiple sessions over a longer period of time. It is possible that additional sessions are necessary in order to produce alterations in attention bias (Li et al., 2008). Finally, a possible motivation limitation exists. Patients receiving a treatment, compared to paid participants, may be more motivated and vigilant during training and thus should produce more robust results. In order to ameliorate some of these limitations, future studies should aim to further investigate the role carry-over effects play in ABM in larger samples with additional assessment tools.
CONCLUSION
SAD is a debilitating disorder which leads to significant impairment in work and social situations. As a result of ongoing research in this field, computer-based ABM is a possible alternative to current SAD treatments. Although further research is needed to make this type of ABM effective, a great deal is being learned about attention and SAD through this paradigm. Ample evidence suggest that attention bias plays an integral role in SAD and its treatment (Amir et al., 2009; Frewen, Dozois, Joanisse, & Neufeld, 2008). Consequently, we can begin manipulating treatment to best serve patients. In an effort to further refine computer-based ABM protocols, this study examined carry-over effects as a possible explanation for inconsistent findings in the existing literature. Based on the findings of this and other studies, future studies should consider analyzing congruent and incongruent trials separately as well as grouping participants based on pre-training attention bias. Since ABM is more cost effective and easier to distribute than any current therapy for SAD, it has the potential to relieve many SAD patients of their symptoms.
ACKOWLEDGEMENTS
All of the data collection and analysis took place in Dr. Israel Liberzon’s Human Neuroimaging Lab at the University of Michigan. I would like to thank Elizabeth Duval, Ph.D. for allowing me to work on her study, use the results in my analysis, and for reviewing and editing this paper. Her time and effort in teaching me about ABM and how to analyze and discuss the data is greatly appreciated.
I would also like to thank those who assisted in this research and subsequent paper: Daniel Sheridan Ph.D., RN, FAAN for reviewing and editing this paper; Sonalee Joshi for reviewing and editing this paper, data collection, and scoring; and Kelsey Krus for data collection and scoring. This research and my time at the University of Michigan Psychiatry Department would not be possible without contributions from the Undergraduate Research Opportunities Program (UROP) and Biomedical & Life Sciences Summer Fellowship donors.
REFERENCES
Aguirre, G. K. (2007). Continuous carry-over designs for fMRI. NeuroImage, 35(4), 1480–1494. doi:10.1016/j.neuroimage.2007.02.005
Amir, N., Weber, G., Beard, C., Bomyea, J., & Taylor, C. T. (2008). The effect of a single-session attention modification program on response to a public-speaking challenge in socially anxious individuals. Journal of Abnormal Psychology, 117(4), 860-868. doi:10.1037/a0013445
Amir, N., Beard, C., Taylor, C. T., Klumpp, H., Elias, J., Burns, M., & Chen, X. (2009). Attention training in individuals with generalized social phobia: A randomized controlled trial. Journal of Consulting and Clinical Psychology, 77(5), 961–973. doi:10.1037/a0016685
American Psychiatric Association. (2013). Diagnostic and Statistical Manual of Mental Disorders, (5th ed.). Washington, DC: American Psychiatric Association.
Beck, A.T., Steer, R.A., Ball, R., & Ranieri, W.F. (1996). Comparison of Beck Depression Inventories -IA and -II in psychiatric outpatients. Journal of Personality Assessment, 67(3) 588-597. doi:10.1207/s15327752jpa6703_13
Boettcher, J., Leek, L., Matson, L., Holmes, E. A., Browning, M., MacLeod, C., … Carlbring, P. (2013). Internet-based attention bias modification for social anxiety: A randomised controlled comparison of training towards negative and training towards positive cues. PLoS ONE, 8(9). doi:10.1371/journal.pone.0071760
Bogels, S. M., & Mansell, W. (2004). Attention processes in the maintenance and treatment of social phobia: Hypervigilance, avoidance and self focused attention. Clinical Psychology Review, 24, 827-856. doi:10.1016/j.cpr.2004.06.005
Bradley, B. P., Mogg, K., Millar, N., Bonham-Carter, C., Fergusson, E., Jenkins, J., & Parr, M. (1997). Attentional biases for emotional faces. Cognition & Emotion, 11(1), 25–42. doi:10.1080/026999397380014
Brown, E. J., Turvosky, J., Heimburg, R. G., Juster, H. R., Brown, T. A., & Barlow, D. H. (1997). Validation of the social interaction anxiety scale and the social phobia scale across anxiety disorders. Psychological Assessment, 9(1), 21-27. doi:10.1037/1040-3590.9.1.21
Carlbring, P., Apelstrand, M., Sehlin, H., Amir, N., Rousseau, A., Hofmann, S. G., & Andersson, G. (2012). Internet-delivered attention bias modification training in individuals with social anxiety disorder - a double blind randomized controlled trial. BMC Psychiatry, 12(1), 66. doi:10.1186/1471-244X-12-66
Cisler, J. M., Bacon, A. K., & Williams, N. L. (2009). Phenomenological characteristics of attentional biases towards threat: A critical review. Cognitive Therapy and Research, 33(2), 221–234. doi:10.1007/s10608-007-9161-y
Ekman, P., & Friesen, W. V. (1976). Pictures of Facial Affect. Consulting Psychologists Press, Palo Alto, CA.
Frewen, P. A., Dozois, D. J. A., Joanisse, M. F., & Neufeld, R. W. J. (2008). Selective attention to threat versus reward: Meta-analysis and neural-network modeling of the dot-probe task. Clinical Psychology Review 28, 307–337. doi:10.1016/j.cpr.2007.05.006
Harrison, L. K., & Turpin, G. (2003). Implicit memory bias and trait anxiety: A psychophysiological analysis. Biological Psychology, 62(2), 97–114. doi:10.1016/S0301-0511(02)00129-1.
Heeren, A., Lievens, L., & Philippot, P. (2011). How does attention training work in social phobia: Disengagement from threat or re-engagement to non-threat? Journal of Anxiety Disorders, 25(8), 1108–1115. doi:10.1016/j.janxdis.2011.08.001
Holle, C., Neely, J., & Heimberg, R. (1997). Blocked versus random presentation and semantic relatedness of stimulus words on response. Cognitive Therapy and Research, 21(6), 681–697. doi:10.1023/A:1021860324879
Izetelny, Adam. (2006). Attentional biases for negative and positive attachment words associated with individual differences in adult attachment and defensiveness: Effects of stimulus content, intensity, and valence. Carbondale, IL: ProQuest.
Jones-Chester, M. H., Monsell, S., & Cooper, P. J. (1998). The disordered-salient Stroop effect as a measure of psychopathology in eating disorders. International Journal of Eating Disorders, 24, 65-82. doi:10.1002/(SICI)1098-108X(199807)24:1<65::AID-EAT6>3.0.CO;2-Z
Klumpp, H., & Amir, N. (2010). Preliminary study of attention training to threat and neutral faces on anxious reactivity to a social stressor in social anxiety. Cognitive Therapy and Research, 34(3), 263–271. doi:10.1007/s10608-009-9251-0
Li, S., Tan, J., Qian, M., & Liu, X. (2008). Continual training of attentional bias in social anxiety. Behaviour Research and Therapy, 46(8), 905–912. doi:10.1016/j.brat.2008.04.005
Liebowitz, M. R. (1987). Social phobia. Modern Problems in Pharmacopsychiatry, 22, 141-173.
Nonyane, B. A., & Theobald, C. M. (2007). Design sequences for sensory studies: achieving balance for carry-over and position effects. The British Journal of Mathematical and Statistical Psychology, 60(Pt 2), 339–349. doi:10.1348/000711006X114568
O’Toole, L., & Dennis, T. A. (2012). Attention training and the threat bias: An ERP study. Brain and Cognition, 78(1), 63–73. doi:10.1016/j.bandc.2011.10.007
Pittig, A., Pawlikowski, M., Craske, M. G., & Alpers, G. W. (2014). Avoidant decision making in social anxiety: The interaction of angry faces and emotional responses. Frontiers in Psychology, 5(September), 1050. doi:10.3389/fpsyg.2014.01050
Ruscio, A. M., Brown, T. A., Chiu, W. T., Sareen, J., Stein, M. B., & Kessler, R. C. (2008). Social fears and social phobia in the USA: Results from the National Comorbidity Survey Replication. Psychological Medicine, 38, 15-28. doi:10.1017/S0033291707001699
Sheehan, D., Lecrubier, Y., Harnett-Sheehan, K., Amoriam, P., Janavs, J., Weiller, E., et al. (1998). The mini international neuropsychiatric interview (MINI): The development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10. The Journal of Clinical Psychiatry, 59 Suppl 20, 22-33;quiz 34-57.
Waters, A. J., & Feyerabend, C. (2000). Determinants and effects of attentional bias in smokers. Psychology Of Addictive Behaviors, 14(2), 111-120. doi:10.1037/0893-164X.14.2.111
Waters, A. J., Sayette, M. a., Franken, I. H. a, & Schwartz, J. E. (2005). Generalizability of carry-over effects in the emotional Stroop task. Behaviour Research and Therapy, 43(6), 715–732. doi:10.1016/j.brat.2004.06.003
White, L. K., Suway, J. G., Pine, D. S., Bar-Haim, Y., & Fox, N. a. (2011). Cascading effects: The influence of attention bias to threat on the interpretation of ambiguous information. Behaviour Research and Therapy, 49(4), 244–251. doi:10.1016/j.brat.2011.01.004