Measuring Evaluation Fears in Adolescence: Psychometric Validation of the Portuguese Versions of the Fear of Positive Evaluation Scale and the Specific Fear of Negative Evaluation Scale

Modified measures of Fear of Negative Evaluation and Fear of Positive Evaluation were examined among Portuguese adolescents. These measures demonstrated replicable factor structure, internal consistency, and positive relationships with social anxiety and avoidance. Gender differences were found. Implications for evaluation and intervention are discussed.

confirmatory factor analysis; exploratory factor analysis; measurement; fear of negative evaluation; fear of positive evaluation

The fear of evaluation in social events refers to the subjective perception of being under the scrutiny of others and fearing the result of their critique. The fear of negative evaluation is commonly associated with psychopathology, particularly with social anxiety disorder ([1] ). Cognitive–behavioral models of social anxiety propose that this fear may be a risk factor for social anxiety, arising from the desire to convey a particular impression to others but believing oneself to be unable to do so ([12] ). Thus, fearing and believing oneself to be the recipient of undesirable evaluation by others, the individual faces social events with high levels of emotional arousal, and strives to avoid these situations or else remain unnoticed in them, by adopting various safety behaviors ([23] ).

The construct of fear of negative evaluation has been assessed using the Fear of Negative Evaluation scale (FNE; [35] ). Scores obtained by clinical samples on the brief version of the FNE scale ([20] ) have indicated adequate levels of reliability and validity ([3] ), but the results for its reverse-scored items show that responses to these items relate poorly to measures of social anxiety, are relatively insensitive to treatment effects, and are confounded by level of education ([30] ; [36] ). For this reason, two straightforward versions were proposed: a 12-item version in which the four reverse-scored items were reworded to become straightforward ([2] ), and an 8-item version, including only the straightforwardly worded items from the original 12-item set ([31] ). Comparison of the psychometric characteristics of the results for these two versions of the instrument supports the utility of the 8-item version ([1] ). Nevertheless, it may be the case that some of these items address a more general, rather than specific, fear of negative evaluation, and are, consequently in need of adjustment.

The construct of fear of positive evaluation has been proposed more recently by Weeks and colleagues ([37] ; [40] ). Working from a psycho-evolutionary model of social anxiety, the authors argue that fear of positive evaluation may derive from the preoccupation with creating a too-good impression, which may be interpreted by dominant members of the social group as a threat to the existing social hierarchy, leading the socially anxious individual into conflict with dominant others. Fear of positive evaluation may thus be highly correlated with fear of negative evaluation, but nevertheless be activated differently or in different individuals with social anxiety, as fear of negative evaluation may unfold from the concern of creating an impression of oneself as unworthy of social approval. Both fears question the position of the individual within the social group to which he or she wishes to belong, and so socially anxious individuals may exhibit fear of both types of evaluation.

The Fear of Positive Evaluation Scale (FPES) was developed to measure fear of positive evaluation, and the results of several studies have found that scores obtained on the FPES have demonstrated adequate internal consistency and convergent and discriminant validity in undergraduate ([39] ; [40] ) and clinical ([38] ) samples. Studies on the validity of this construct have produced results indicating that fear of positive evaluation contributed unique variance to the prediction of social anxiety, beyond the variance contributed by fear of negative evaluation ([37] ; [39] ; [40] ). Moreover, fear of positive evaluation (but not fear of negative evaluation) was associated with the discomfort felt when receiving positive feedback and with the perceived (in)accuracy of this feedback ([39] ).

The proposition that fears of both negative and positive evaluation underlie social anxiety has important implications for understanding social anxiety and for interventions with people who suffer with it, particularly in adolescence, a developmental period during which there is an increased risk for developing social anxiety ([19] ); it is only if we are able to measure constructs related to (and probably predisposing to) social anxiety that we may better understand the etiology and characteristics of social anxiety itself.

Nevertheless, measures for evaluating these constructs that may predispose a person to social anxiety are scarce in Portugal. There are a few instruments assessing the emotional, behavioral, and cognitive experiences of social anxiety, namely, the Social Anxiety and Avoidance Scale for Adolescents (SAASA; [5] ), the Social Anxiety Scale for Adolescents ([4] ), the Social Phobia and Anxiety Inventory–Brief Report ([34] ), or the Social Thoughts and Beliefs Scale ([32] ). They focus on the symptoms of social anxiety, such as the intensity of emotional arousal and avoidance of social events, or the thoughts activated in response to such events, but disregard the specific perception of being evaluated by others; fears of negative and positive evaluation may better help to explain how and why these symptoms of social anxiety are activated to begin with. Additionally, the experience of fearing evaluation from others has been little studied with adolescents. One of the few examples would be the recent work by [22] , who sought to examine the association between fear of positive evaluation and social anxiety in a clinic-referred sample of adolescents, and found that this fear accounted for significant variance in social anxiety and subtle avoidance, beyond that explained by sociodemographic variables or depressive symptoms.

The current work presents the process of translating and adapting a modified form for the Brief Fear of Negative Evaluation–Straightforward scale ([31] ) as well as translating the original FPES ([37] ) into Portuguese, and evaluating the psychometric properties of the results for these instruments obtained with an adolescent school-based Portuguese sample. Fear of negative and of positive evaluation have scarcely been previously studied using adolescent samples, even if social fears (and its associated correlates) have been found to increase from childhood to adolescence ([41] ). Adolescence, however, is a relatively long developmental period. During this period, changing developmental trajectories for social anxiety have been found ([21] ; [24] ), in addition to evidence for response patterns changing within the assessment of social anxiety and avoidance, possibly because one’s perspective on different fear stimuli varies with sociocognitive maturation ([41] ). Therefore, it seems important to explore the response patterns of persons from different age groups than those previously studied. Conclusions found based on the predominantly undergraduate student samples used previously to examine the psychometric characteristics of the results of the instruments under consideration (e.g., [31] ; [37] ) may be more directly generalizable to an age-group that immediately precedes the age-groups of those samples. For this reason, the present work focused on such an immediately antecedent age group; namely mid-adolescents 15 to 18 years old ([18] ).

Consequently, we address the following hypotheses based on a sample of Portuguese adolescents: (a) similarly to what was found by [37] , a two-factor measurement model will provide a good fit for the data, with one factor representing fear of negative evaluation and another factor representing fear of positive evaluation; (b) this measurement model will be invariant across distinct male and female adolescent samples and (c) across gender; (d) no gender differences will be found for fear of negative evaluation ([30] ) or for fear of positive evaluation ([37] ); (e) scores obtained from the translated/ modified measures of fear of negative evaluation and fear of positive evaluation will be approximately normally distributed ([30] ; [37] ), and (f) will produce very good internal consistency values ([31] ; [37] ); and, finally, (g) significant correlation and regression results will be found between measures of fear of negative and fear of positive evaluation and social anxiety ([37] ) and avoidance ([22] ).

Participants in the present research were 881 students from seven Portuguese public secondary schools ([1] ). Their ages varied between 15 and 18 years (M = 16.43; SD = 0.94). The majority of the sample was female (n = 556; 63.1%). Socioeconomic status was determined based on parents’ profession and considering the Portuguese classification of professions and standard incomes ([17] ). Examples of professions in the high socioeconomic status groups are judges, higher education teachers, or MDs; for the medium socioeconomic status group, examples include nurses, psychologists, or school teachers; for the low socioeconomic group examples are farmers, cleaning staff, or undifferentiated workers. Female and male students were uniformly distributed by socioeconomic status (χ2(2) = 4.93; p = .085) but not by school year (χ2(3) = 7.67; p = .022). Girls were predominant in the 12th grade, and boys were more prevalent in the 10th and 11th grades. The mean ages of boys (M = 16.35; SD = 0.95) and girls (M = 16.48; SD = 0.95) were not significantly different (t(879) = −1.94; p = .053).

-1 Note. Subsamples 1 and 2 resulted from randomly splitting the complete sample (N = 881). Subsample 3 refers to students who completed all study measures. BFNE-S = Brief Fear of Negative Evaluation–Straightforward; FPES = Fear of Positive Evaluation Scale.

-2 Contrasting with national statistics, a similar proportion of girls and boys are represented in the current sample (national statistics = 54.8% of secondary school students are female and 45.2% are male). Additionally, our sample presents a similar percentage of students attending the 10th grade (national statistics = 35.57%), slightly fewer students attending the 11th grade (national statistics = 32.52%) and slightly more students attending the 12th grade (national statistics = 32.26%). Also, the percentage of boys attending the 10th and 11th grade in our sample is higher than indicated by national statistics, in which girls were found to more frequently attend the 10th, 11th, and 12th grades (Direção-Geral de Estatísticas da Educação e Ciência, 2013).

To perform replication analyses on the factor structure of both instruments, this sample was randomly split into two subsamples. Subsamples 1 and 2 are presented in [1] . Students in the two subsamples were uniformly distributed by sex (χ2(1) = 0.363; p = .547), school year (χ2(3) = 0.404; p = .817), and socioeconomic status (χ2(2) = 0.992; p = .609). The mean ages of students in Samples 1 (M = 16.48, SD = 0.94) and 2 (M = 16.39, SD = 0.94) were not significantly different (t(879) = −1.33; p = .184).

Because of the differences in time availability at the participating schools, only 49.3% (n = 434; see [1] ) of the total sample had the opportunity to complete the measures of social anxiety and avoidance in addition to the measures of fear of negative and fear of positive evaluation. The majority of this third subsample who completed all study measures was also female (n = 289; 66.6%). Comparing students who only filled out the Brief Fear of Negative Evaluation Scale (BFNE) and the FPES with students who also filled out the measure of social anxiety and avoidance, we found that they were not similarly distributed by sex (χ2(1) = 4.45, p = .035; more boys than girls did not fill out the measure of social anxiety and avoidance), school year (χ2(2) = 14.89, p = .001; more 11th graders did not fill out the measure of social anxiety and avoidance), or socioeconomic status (χ2(2) = 9.65, p = .008; more students who came from the low and high socioeconomic groups than from the medium socioeconomic status did not fill out the measure of social anxiety and avoidance). Those who did fill out the measure of social anxiety and avoidance (M = 16.50 SD = 0.98) were significantly older than those who did not (M = 16.37, SD = 0.89; t(879) = −2.08, p = .038).

The Brief Fear of Negative Evaluation Scale–Straightforward items only (BFNE-S; [31] ) includes only the eight originally straightforwardly worded items from [20] BFNE scale. The respondent is asked to rate each item on a 5-point Likert-type scale, ranging from 1 (not at all like me) to 5 (very much like me), and the items’ scores are added to yield a final total score. The recommended use of only the eight straightforward items of the BFNE comes from previous works using confirmatory factor analysis and IRT analysis with a combined clinical and student sample ([31] ) and exploratory factor analysis (EFA) with a clinical sample ([36] ). Results from both works suggested that responses to the straightforward items of the BFNE load onto a single factor, that BFNE-S scores are only moderately associated with the scores of the reverse-scored items of the original BFNE, and that BFNE-S scores are more sensitive than the reverse-scored items to varying levels of the target construct (i.e., fear of negative evaluation).

The psychometric characteristics of scores from the BFNE-S have been well-ascertained. For example, responses to the BFNE-S items have demonstrated strong internal consistency (α ≥ .89 with community samples, [29] ; [31] ; α = .92 for clinical samples, [36] ), and BFNE-S scores have demonstrated strong convergent validity in relation to measures of social anxiety ([29] ; [31] ; [36] ), discrimant validity in relation to measures of anxiety sensitivity and depression, and sensitivity to cognitive–behavioral group treatment effects ([36] ). Significantly different scores on the BFNE-S have also been reported for individuals with or without a diagnosis of social anxiety disorder ([29] ; [36] ).

Nevertheless, a careful look at some of the BFNE-S items raises questions about their face validity. Items 1, 5, 6, and 7 refer to a general fear of being evaluated, rather than a specific fear of being negatively evaluated. Considering that the goal of this work was to prepare measures to evaluate and explicitly analyze the associations between fears of negative and positive evaluation and social anxiety and avoidance, the development of a version of the instrument with greater specificity to fear of negative evaluation was deemed necessary. The rewording of the items in question to make them specific to fear of negative evaluation is presented in [2] .

-3 Note. Items for the currently used version of eight items of the BFNE (Rodebaugh et al., 2004) were ordered 1 to 8, and therefore do not correspond to the item numbers of the 12-item version of the BFNE (Carleton et al., 2006).

The FPES ([37] ) consists of 10 items. Two reverse-scored items are included intending to potentially detect response biases but are not included in the FPES total score. The original (English) version of the FPES employed a 10-point Likert-type rating scale, ranging from 0 (not at all true) to 9 (very true). However, for the purposes of the present study, FPES items were administered with the same scale as the BFNE-S, that is, ranging from 1 (not at all like me) to 5 (very much like me). Results of confirmatory factor analysis with data from undergraduate students demonstrated that values for the eight straightforward FPES items load onto a single factor ([37] ). Responses to FPES items have also demonstrated good internal consistency (α = .85 in a clinical sample, [38] ; and α = .80 in an undergraduate sample, [37] ), and FPES scores have demonstrated strong convergent validity in relation to measures of social anxiety and fear of negative evaluation and discriminant validity in relation to generalized anxiety, worry, depression, and quality of life ([38] ; [37] ).

All 8 straightforward BFNE-S items (including the 4 modified items; see above and [2] ) and all 8 straightforward FPES items were adapted to the Portuguese language, by means of forward and backward translation ([11] ): the items were translated from English to Portuguese by a Portuguese researcher unrelated to this work, who had lived in England for 5 years. This version of the items was then subjected to thinking aloud analysis by a class of 23 eleventh graders, male and female, who were asked to verbalize any doubts or ambiguities they might find with the items and instructions, and to suggest adjustments if they so desire. This is a common procedure used to investigate test usability, in as much as it allows verbalizations of the cognitive processes of encoding and interpreting written language ([9] ), which seemed highly important given that the instruments had not been considered in an adolescents’ perspective before. After this analysis, suggested alterations were made. Subsequently, the Portuguese version was back-translated to English by a professor from an independent school of languages who was not familiar with the original versions of the instruments. The original and back-translated versions of the items were considered equivalent.

The SAASA ([5] ) was used to investigate the associations between fears of negative and positive evaluation and social anxiety and avoidance. It consists of 34 items that are each rated twice to form two separate subscales, one for social anxiety and one for avoidance of social events typical of adolescence. Based on the results of exploratory and confirmatory factor analyses with a sample of adolescents aged 12 to 18 years old ([5] ), and with an older adolescent sample aged 16 to 18 years old ([33] ), it has been proposed that each subscale is comprised of six dimensions: interaction with the opposite sex, assertive interaction, observation by others, interaction in new social situations, performance in formal social situations, and drinking and eating in public. Results from previous works with this instrument shown that responses to SAASA items demonstrate good-to-excellent internal consistency (α = .91 for the anxiety subscale and α = .87 for the avoidance subscale), and that SAASA scores demonstrated good 5-week test–retest reliability (r = .74, p < .01, for the anxiety subscale; and r = .71, p < .01, for the avoidance subscale), as well as convergent validity in relation to other social anxiety measures ([5] ; [33] ) and discriminant validity in relation to measures of generalized anxiety and depression ([5] ). A measurement model has been proposed specifically for adolescents aged 16 to 18 years old, and this being the case with the present sample, it was used for the present work ([33] ). Excellent internal consistency values were found for the responses to anxiety (α = .94) and avoidance (α = .91) subscales in the present sample.

This study was approved by the national committee for evaluation of ethics and procedures of studies conducted in school settings. Afterwards, authorization was sought and given by the participating schools and by the parents of participants below 18 years of age. Sampling followed a convenience and then random procedure: seven schools from the north of Portugal were selected for geographical convenience reasons pertaining to data collection, but were also selected based on their position in the national ranking of schools, which is based on students’ academic achievement (two schools presented below average results, three schools offered within average results, and two schools obtained above average results); participating classes in each school were randomly selected. One member of the research team went to each school and classroom to request the voluntary participation of the students, to whom the confidentiality of the data was guaranteed. No student refused to participate; furthermore, although some parents may not have provided informed consent for their children to participate in the present study, no details on this were available to the authors, as the schools preferred not to give such information to the research team, in order to protect the identity of students and their families. Participants completing only the BFNE-S and the FPES took between 5 to 10 minutes to complete the task, whereas participants who also completed the SAASA took between 20 to 25 minutes to complete the task.

A total of 15 students presented missing values in addition to random answers (e.g., diagonal answering); no student had only missing values. Missing values for the BFNE-S and the FPES were found for 1 to 2 items and missing values for the SAASA were found for 1 to 44 items. Of a total of 924 students evaluated, this represented only 1.67% of the sample. Therefore, a listwise deletion approach was applied to missing data, and these cases were not considered in any analysis or the participants section.

Data analysis was conducted using Mplus (v6.2; [25] ) and SPSS (v15.0). Mplus was first used for single confirmatory and EFA, in order to define the measurement model underlying the instruments under evaluation. Second, Mplus was used to test for factorial invariance across gender, meaning that the measurement model would be invariant and have the same meaning to boys and girls, so that valid comparisons between groups on the latent variables could be made. A forward approach to testing for factorial invariance was applied, following the guidelines provided by [7] : configural, then metric, and then scalar invariance were examined. At least partial scalar invariance is required if groups are to be compared on mean levels of latent constructs. Configural invariance indicates that the same basic factor structure is stable across groups; metric invariance determines that the item loadings on each factor are also identical across groups; and finally scalar invariance adds to this the imposition of equality of the variables’ intercepts, again, across groups.

For evaluating model fit for exploratory and confirmatory factor analysis, a 2-index criterion was considered ([14] ), which combines a value of standardized root mean square residual (SRMS) ≤ .08 with either a value of comparative fit index (CFI) or Tucker–Lewis index (TLI) ≥ .95, or a value of root mean square error of approximation (RMSEA) ≤ .06. For performing and comparing the exploratory factor solutions, the guidelines given by [10] were followed: an oblimin rotation was applied and fit indices were compared across different solutions; the selected solution denoted no better fit in comparison with its antecedent (e.g., overlap in the RMSEA value interval). Items were included if they had λ ≥ .32 on only one factor and concomitant loadings on all other factors < .32. Items either not matching these criteria or presenting λ < .32 for all factors were dropped. Total scores for each factor were computed by the sum of items with λ ≥ .32 in each factor; cross-loadings ≤ .32 were omitted.

For the replication of the EFA with a culturally and developmentally different sample from the ones previously used to study both instruments, internal (i.e., comparison of two subsamples) and external (i.e., comparison of the complete present sample with the sample of [37] ) replicability procedures were conducted. According to [27] , structural replicability shows that the same items represent the same constructs across samples, whereas factor loading replicability indicates that the loadings for the same items are of roughly equivalent magnitude across samples. In this last case, the squared difference of loadings for the same item in different samples should not exceed .04. Furthermore, to verify if the measurement model resulting from this EFA was not dependent on the characteristics of the study subsamples, configural, metric, and scalar invariance across Subsamples 1 and 2 were also examined.

SPSS was used for descriptive and internal consistency analysis, using the Cronbach Alpha (cutoff value of .70; [26] ) as well as correlation and regression analyses on the measures of fears of evaluation, social anxiety, and social avoidance.

Given that the two-factor structure of the combined pool of the 16 straightforwardly worded items of the BFNE-S and the FPES had been previously ascertained ([37] ) we began by performing a confirmatory factor analysis on this measurement model, using the total sample (N = 881). The fit indices for this two-factor solution did not satisfy the two-index criteria (CFI = .94, TLI = .93, RMSEA = .071, SRMR = .039). Given that an exploratory approach to the data is justified to identify latent constructs underlying measured variables ([10] ), and given that the current sample had specific characteristics (namely culture and age) that may have made it prone to different interpretation of the items, an EFA with direct oblimim rotation was subsequently employed, using a two-sample approach (n = 440 for Subsample 1 and n = 441 for Subsample 2); solutions for one to three factors were examined. Data were normally distributed (multivariate kurtosis = 1.23, no items with univariate skewness ≥2 or univariate kurtosis ≥ 7; for mean and standard deviation values see [1] ), and consequently the Maximum Likelihood estimation method was used when performing the EFA ([10] ).

Results for the EFA on Subsample 1 (n = 440) indicated that a three-factor solution was the only acceptable solution (CFI = .97, TLI = .96, RMSEA = .057, SRMR = .024). Item 4 from the BFNE-S (I am afraid that others will find fault in me) was nevertheless problematic, presenting λ ≥ 1 and concomitant negative residual variance. A second EFA was thus conducted excluding this item, and results for a two-factor solution were deemed acceptable (see [3] for fit indices and [4] for factor loadings).

-4 Note. χ2 = conventional chi-square fit statistics (under maximum likelihood estimation); df = degrees of freedom; CFI = comparative fit index; TLI = Tucker–Lewis index; RMSEA = root mean square error of approximation; CI = confidence interval; SRMR = standardized root mean square residual; EFA = exploratory factor analysis; M0 = baseline model (no invariance imposed); M1 = invariant factor loadings; M2 = invariant factor loadings and intercepts; M2P = invariant factor loadings and partially invariant intercepts (free intercept of Items 1, 3, 6, 12, and 13); CFA = confirmatory factor analysis. All chi-square are statistically significant (p < .001).

-5 Note. Short versions of the items are presented, and paraphrases are added in brackets; Item 4 for the FNE was omitted due to its loading value being superior to 1 and the associated residual variance being negative in the EFA performed for Subsample 1. EFA = exploratory factor analysis; CFA = confirmatory factor analysis.

Using the same item pool for Sample 2 (n = 441), results point to the same two-factor solution using either EFA or CFA (see [3] for fit indices and [4] for factor loadings), demonstrating the internal structural replicability of this fear of negative evaluation and fear of positive evaluation measurement model. Internal factor loading reliability was also demonstrated for all seven items included in the measure of fear of negative evaluation and all but one item assessing fear of positive evaluation ([4] ).

To verify that this two-factor measurement model comprised of 15 items was not dependable on the characteristics of the study subsamples (e.g., gender distribution by school year or socioeconomic level; see [1] ), we tested for configural and measurement invariance. Results for fit indexes for these analyses indicated a very good fit for the model for each group ([3] ), pointing to configural invariance of the measurement model. In addition, constraining the loadings to be equal across groups did not worsen the fit of the model (M1-M0: Δχ2 = 3.92, Δdf = 13, p > .995; ΔCFI = .00); neither did constraining both loadings and intercepts to be equal across groups (M2-M1: Δχ2 = 5.03, Δdf = 13, p > .20; ΔCFI = .00; [3] ). These results demonstrated strong measurement invariance, indicating that the items of the measures conveyed the same meaning to, and were responded to in the same way, by different respondents in Subsamples 1 and 2. The correlation between fear of negative and fear of positive evaluation scores were .72, p < .001, and .68, p < .001, for Subsample 1 and Subsample 2, respectively. The lowest loading value for both subsamples was found for Item 8 of the BFNE-S (λ = .77 for Subsample 1 and λ = .81 for Subsample 2) and for Item 2 of the FPES (λ = .72 for Subsample 1 and λ = .66 for Subsample 2). The highest loading value for both samples was found for Item 7 of the BFNE-S (λ = .96 for Subsample 1 and λ = 1.02 for Subsample 2) and for Item 6 for the FPES (λ = .91 for Subsample 1 and λ = .88 for Subsample 2).

The same 15-item pool was analyzed for the complete sample (N = 881), to evaluate external replicability for the FPES (external replicability for the BFNE-S measure was not possible given the difference in its constitution derived from the EFA solution in the present sample). Results from EFA and CFA demonstrated the suitability of the two-factor solution ([3] ) and the external structural replicability reliability of the results of the FPES only. External factor loading replicability was not demonstrated for fear of positive evaluation Items 3, 6, and 7 ([4] ). Given that these items nevertheless presented acceptable loading values in both subsamples, and that the total score for the FPES is obtained by the sum of the values given by each participant to the items, regardless of the loadings of each item, these different statistical weights were not considered when computing or analyzing the FPES total score. The correlation between the BFNE-S and the FPES for the complete sample was .478, p < .001. The lowest loading values found through CFA for the complete sample were .79 for item 8 of the BFNE-S and .69 for item 2 of the FPES; the highest loadings were achieved for Item 7 of the BFNE-S (λ = .99) and Item 6 of the FPES (λ = .89). Excellent internal consistency values were also obtained for both measures under study using the complete sample ([4] ).

To verify if the 15 item two-factor measurement model was invariant across gender, and therefore whether valid conclusions could be made regarding comparable levels of the latent variable between these groups, we tested for configural and measurement invariance, using male (n = 325) and female (n = 325) participants taken from the complete sample (N = 881). Results for the fit indexes indicated a very good fit for the model for each group ([3] ), indicating configural invariance of the measurement model. Results on constraining the loadings to be equal across groups did not significantly worsen the fit of the model (M1-M0: Δχ2 = 13.32, Δdf = 13, p > .20; ΔCFI = .00); however, results for constraining the intercepts to be equal across groups did (M2-M1: Δχ2 = 66.12, Δdf = 13, p = .001; ΔCFI = −.01). It was necessary to free the intercepts of Items 1, 3, 6 of the BFNE-S and Items 4 and 5 of the FPES one at a time to achieve a nonsignificant worsening of the fit of the model (M2P-M1: Δχ2 = 11.42, Δdf = 8, p > .10; ΔCFI = .00; [3] ), thus establishing strong but partial measurement invariance. The correlation between the fear of negative and fear of positive evaluation measures were .69, p < .001, and .68, p < .001, for male and female participants, respectively. The lowest loading value for male and female samples was found for Item 8 of the BFNE-S (λ = .78 for male and female samples) and for Item 2 of the FPES (λ = .73 for the male sample and λ = .64 for the female sample). The highest loading value for both male and female samples was found for Item 7 of the BFNE-S (λ = .92 for the male sample and λ = .98 for the female sample) and for Item 6 for the FPES (λ = .93 for the male sample and λ = .88 for the female sample).

Structured means analysis ([6] ) was applied to compare latent mean scores for boys and girls taken from the complete sample (N = 881). The male group was taken as the reference group, and the mean difference in comparison to the female group was .286, p < .001, Cohen d = 0.42 for fear of negative evaluation and .433, p < .001, Cohen d = 0.72 for fear of positive evaluation. Girls therefore obtained higher latent mean scores then boys for both types of fear, with medium effect size for fear of negative evaluation and close to large effect size for fear of positive evaluation.

Preliminary analyses were conducted to identify variables that needed to be controlled for when investigating the relation of fear of negative evaluation and fear of positive evaluation with social anxiety and social avoidance. Regarding sociodemographic variables, scores for fear of negative evaluation correlated with age (r = −.15, p < .001), and students in different grades presented different levels of this fear (F[2,880] = 5.52, p = .004), as did students of varying socioeconomic status (F[2,880] = 5.04, p = .007). Scores for fear of positive evaluation correlated with age (r = −.13, p < .001), and students in different grades presented different levels of this fear (F[2,880] = 3.09, p = .046). Therefore, the following correlational and regression analyses controlled for the possible confounding effects of these sociodemographic variables (in addition to gender), and were performed using Subsample 3 (n = 434), consisting of participants who completed the BFNE-S, the FPES, and the SAASA.

As expected, significant partial correlations were found between scores on fear of negative evaluation and social anxiety (r = .47, p < .001) and avoidance (r = .39, p < .001). Scores on fear of positive evaluation were also significantly associated with social anxiety (r = .43, p < .001) and avoidance (r = .40, p < .001).

To investigate if fear of positive evaluation would add to the prediction of scores on social anxiety and avoidance, above and beyond the contribution of fear of negative evaluation, a hierarchical regression was performed, entering sociodemographic variables in the first block, fear of negative evaluation scores in the second, and fear of positive evaluation scores in the third.

For the social anxiety subscale, sociodemographic variables were significant predictors (R2 = 0.052; F[4,413] = 5.61, p < .001, Cohen’s f = 0.055) of social anxiety. Scores for fear of negative evaluation were additional significant predictors of social anxiety (R2 = .264; R2Δ = .212; FΔ[1,408] = 117.33, p < .001, Cohen’s f = 0.36). Scores for fear of positive evaluation in turn added significantly to the model (R2 = .292; R2Δ = 0.028; FΔ [1,408] = 16.10, p < .001; Cohen’s f = 0.041; [5] ). For hierarchical Steps 2 and 3, sociodemographic variables were no longer significant predictors.

-6 Note. ns = nonsignificant.

-7 p < .001. **p < .01. *p < .05.

As for the avoidance subscale, sociodemographic variables were again significant predictors (R2 = 0.031; F[4,413] = 3.26, p = .01; Cohen’s f = 0.032). Scores for fear of negative evaluation were additional significant predictors of social avoidance (R2 = .177 R2Δ = 0.146; FΔ [1,408] = 72.33, p < .001; Cohen’s f = 0.22). Scores for fear of positive evaluation were also additional significant predictors of social avoidance (R2= .215; R2Δ = 0.038; FΔ [1,407] = 19.77, p < .001; Cohen’s f = 0.27; [5] ). For hierarchical Steps 2 and 3, only socioeconomic status remained a significant predictor (p = .021 and .022, respectively).

Results concerning the anxiety and avoidance dimensions are not presented in detail in the interest of brevity, but were consistent with the results reported for the full subscales.

This work set out to adapt and present psychometrically sound measures for fears of negative and positive evaluation. The growing evidence of the presence of a general fear of evaluation underlying social anxiety ([39] ; [40] ), as well as the cumulative evidence of the parallel and biased processing of negative and positive social information in social anxiety ([13] ; [15] ), justify investigation of the topic and require that adapted and valid measurement instruments become available. This is particularly important for adolescent samples, because even if social anxiety has been extensively studied in this developmental life period, fears of evaluation have not been. This is the first work evaluating the psychometric qualities of the scores obtained by instruments evaluating fears of negative and positive evaluation in an adolescent sample, and particularly adolescents in a school setting, who may be given to different interpretations of the relevant items comparing to adults. If this was the case, items might be organized into different meaningful factors than those previously found for undergraduate samples and thus address similar but not overlapping constructs.

Results obtained with this adolescent sample for the adapted Portuguese version of the BFNE-S ([31] ) and the Portuguese version of the FPES ([37] ) demonstrated good psychometric qualities. Thus, evidence was obtained confirming our hypothesis that: a two-factor model adequately fit the data from the complete sample, for distinct groups of (male and female) adolescents as well as for boys and girls separately; scores for indexing fear of negative and fear of positive evaluation approximate normal distribution and demonstrated very good internal consistency values; and scores indexing fear of negative and fear of positive evaluation correlated and acted as predictors of social anxiety and avoidance.

The overall factor structure for the results of the measures under study was best modeled, as expected, by two correlated factors (i.e., fear of negative evaluation and fear of positive evaluation). Results for this two-factor solution show acceptable fit of the model, which additionally cross-validated in two randomly split subsamples, confirming the stability and generalizability of the two fear of evaluation measurement model. One item from the fear of negative evaluation scale had to be dropped. This set of items had not been evaluated previously with adolescents, and this item in particular was possibly interpreted differently by this group, in comparison with the remaining items, given that the word fault may have multiple meanings in Portuguese (e.g., failure, shortcomings, or mistakes).

Results for factorial invariance proved this measurement model to be invariant across gender and thus adequate for validly comparing boys and girls. We found significant differences in fear of negative evaluation and fear of positive evaluation by sex, with moderate to large effect sizes, which was not in line with our hypothesis that no gender differences would be found ([30] ; [37] ). Adolescent girls in the current sample obtained higher latent mean scores than boys for both fear of negative and fear of positive evaluation. Concordantly with this findings, previous studies have found that Portuguese adolescent girls report higher levels of fear of negative evaluation as evaluated by the Social Anxiety Scale for Adolescent ([4] ) as did North American adolescent girls ([16] ). Together, these findings may express that the gender difference for fear of negative evaluation may lessen with age, as gender differences have been found for adolescent but not for adult samples. As for fear of positive evaluation, no gender differences had been previously found, in adult North American samples, drawn from either nonclinical ([37] ; [40] ) or clinical settings ([38] ). Evidence that women present higher levels of fear of positive evaluation had, however, been previously found for adults in a Portuguese nonclinical sample ([28] ). This particular finding concerning fear of positive evaluation may indicate some cultural specificity, warranting further investigation.

Validity evidence relating to external variables, specifically social anxiety and avoidance, was also obtained, in its general form and also pertaining to specific social contexts. Similar to the findings of Weeks and collaborators ([37] ; [39] ), scores on measures of fear of negative evaluation and fear of positive evaluation were highly correlated but independent, accounting for unique variance in the prediction of scores on a measure of social anxiety. Using the present sample, these same fears were good predictors of self-reported avoidant behavior, which had been previously found for clinic-referred adolescents ([22] ). These fears had also been associated with submissive behavior ([39] ; [40] ). It therefore seems that the same cognitive vulnerabilities represented in fears of evaluation may be underlying the emotionally anxious and the avoidant and submissive behavioral facets of social anxiety in adolescents.

Limitations to this work should be put forward, as they may sustain future research into aspects not considered in the current work. Particularly, the current findings are based on a specific age frame (15 to 18 years old) and relied solely on self-report questionnaires, using a cross-sectional design. Even if choosing this age frame was the optimal approach by which to make a priori predictions based on prior findings from samples of a immediately following age group (see the introduction section), the need for investigating the psychometric qualities of the results of the instruments under evaluation using samples with a broader ranges of age as well as socioeconomic and clinical characteristics is paramount. Jointly using self-report measures with parent and/or clinician reports of fears of evaluation, social anxiety, and avoidance may further support the validity and relevance of the constructs under examination and the conclusions drawn from the use of these measures in Portuguese samples.

The present work presents preliminary findings in support of the measurement of fears of evaluation in school-based samples of adolescents, a population that had not been evaluated regarding these constructs. Given the normative experience of social anxiety in adolescence ([19] ), these constructs may represent important etiological or explanatory markers of this problem in this type of sample. Distinguishing between these types of fears may, consequently, have important theoretical and clinical implications. Theoretically, they may help differentiate between dysfunctional behavioral patterns, which may interfere with adolescents’ healthy development ([42] ). Clinically, it may be necessary to review the intervention guidelines for social anxiety that promote the individual’s focus on positive evaluation, which may, according to our findings, be another important source of social anxiety.

Author BiographiesPaula Vagos is Lecturer in Psychology at the Univeristy of Aveiro, and researcher at the University of Coimbra. She mostly investigates social behaviors in adolescence, including social anxiety and avoidance, assertiveness and aggression. She has published mostly in psychometrics, including development of new instruments and cross-cultural validation of existing ones.Maria do Céu Salvador is a Clinician, a Lecturer and a Researcher in Psychology at the University of Coimbra. She is mostly interested in investigating in social phobia, especially in adolescence. She presented and published in assessment instruments and in several variables relevant for social phobia research.Daniel Rijo is Professor of Clinical and Forensic Psychology at the University of Coimbra. He has integrated several funded multidisciplinary projects, focusing on antisocial behavior, personality disorders and the rehabilitation of young offenders and students at risk of school dropout. He has published mostly on psychological evaluation and outcome research.Isabel M. Santos is Lecturer in Psychology and coordinator of the NeuroLab Unit of the Psylab, University of Aveiro, Portugal. Her research is mostly focused on cognitive neuroscience and the psychobiological bases of behavior. She also participates in various projects of adaptation of psychological assessment instruments to the Portuguese population.Justin W. Weeks, PhD, is Director of the Center for Evaluation and Treatment of Anxiety (CETA) at Ohio University. Dr. Weeks is well-known for his work on fear of positive evaluation as a core element of social anxiety disorder, as well as on the enhanced objective assessment of social anxiety.Richard G. Heimberg is Professor of Psychology and Director of the Adult Anxiety Clinic of Temple University.  He is Past President of the Association for Behavioral and Cognitive Therapies and the Society for a Science of Clinical Psychology. He is widely published in research on anxiety disorders and their treatment.