Department of Psychology, Rutgers University;
Brian C. Chu
Department of Clinical Psychology, Rutgers University
Acknowledgement: This research was supported by a Klingenstein Third Generation Foundation Young Investigator’s Award granted to Brian C. Chu. We thank the local school systems, their administration, and staff for their collaboration and support in this project, including Janet Barbin, Precious Acolaste, Frank DeVone, Paul Day, Roberta Mitchell, and Michael Rich. We also thank study coordinators Sofia Crocco, Petra Esseling, and Margaret Areizaga, and all research staff who made this project possible. We also express our gratitude to the youth who shared their experiences and time with us. Last, we would like to acknowledge Rens van de Schoot for his consultation on conducting measurement invariance analysis in Mplus.
Anxiety and depressive disorders are among the most prevalent psychiatric conditions affecting adolescents (
Available data comparing multiethnic and White youth on prevalence rates and symptom presentation of anxiety are limited. Several studies have suggested that African American or Black (hereafter referred to as Black) youths’ reported experience of somatic and panic symptoms may be greater, and levels of social and school anxiety may be lower compared with those of White youth (e.g.,
In order to accurately identify and interpret mean differences, measurement invariance of questionnaires needs to be examined to determine whether observed differences are based on psychometrically sound assessment tools or if they may be artifacts of measurement bias. When measurement invariance is not demonstrated, it suggests that groups are responding differently to items, and factor means cannot reasonably be compared across groups. There are several reasons why groups may systematically differ on a given measure. Different ethnic groups may conceptualize a construct differently, use varying symptoms to identify it, or may interpret the possible responses or response scale differently (
The Screen for Child Anxiety Related Emotional Disorders (SCARED;
The Center for Epidemiologic Studies–Depression Scale (CES–D;
In sum, only one study has investigated measurement invariance of the SCARED with multiethnic youth samples, and only a few have investigated invariance with the CES–D. The primary goal of the present study was to examine the cross-ethnic structure and validity of score interpretations of the SCARED and CES–D for White, Hispanic, Black, and Asian/Indian (hereafter referred to as Asian) youth in middle school. A secondary goal was to compare ethnic groups on factor means if invariance was established. This investigation adds to existing knowledge in several ways. First, ours is the first study to examine measurement invariance of the SCARED across several major ethnic groups and the second to do so for the CES–D. Second, we make methodological advances compared with previous studies by taking into account the categorical nature of the response scales. Third, we examined measures of anxiety and depression in the same multiethnic sample, whereas previous studies have tended to focus on one or the other.
Participants were 881 seventh-grade youth (45% female; age range = 11–14 years; M = 12.5, SD = 0.44) at a suburban/urban middle school in the northeastern United States. The middle school enrolls approximately 1,100 economically and ethnically diverse seventh- and eighth-grade students; 32% of youth are eligible for free or reduced-price lunch. Among the participating youth, 396 (44.9%) self-identified as Black, 185 (21%) Hispanic, 166 (18.8%) White, and 134 (15.2%) Asian/Pacific Islander/Indian. Ethnicity was collected as part of the school’s routine enrollment procedures; only broad ethnic categories were collected.
The SCARED (
The CES–D (
Youth participated in a grade-wide screening in two waves over 2 consecutive years aimed at identifying anxiety and depression symptoms. All English-speaking students in the seventh grade whose parents provided consent and who attended school the day of the screening were eligible. Students who were enrolled in English as a second language or who required significant academic assistance (i.e., receiving special education for the majority of their classes) due to learning disabilities were not eligible (22 and 18 youth were ineligible in the first and second waves, respectively). In Year 1 of the study, 427 of 497 eligible youth completed the screening (45 opted out, 25 agreed but were absent). In the second year, 454 of 524 eligible youth completed the screening (39 opted out, 31 agreed but were absent). Participants completed a 35-min battery of questionnaires, including the two for this study, during a gym or health class period.
Single and multiple group CFA models were estimated using Mplus software (
To test for measurement invariance, we followed the procedure as described by
For descriptive purposes, correlations between SCARED and CES–D total scores were calculated for each ethnic group. Correlations were positive and significant (p < .01) across all groups: White (r = .55), Black (r = .68), Hispanic (r = .66), and Asian (r = .62). Reliability estimates for SCARED and CES–D subscales are summarized in
CFA models were fit separately for each ethnic group. In all groups, the five-factor model showed adequate to good fit to the data (CFI and TLI ranged from .90 to .96, and RMSEA were all under .04; see Table S2 in the online supplemental materials). Further, all factor loadings were moderate to large, in the expected direction, and statistically significant for all four groups. Next we used multiple group CFAs to test for measurement invariance. Results are summarized in
These results support the five-factor structure of the SCARED across the four ethnic groups and suggest that youth across groups appear to interpret the individual items and the scoring scale the same. It is likely that SCARED scores represent the same construct in these groups, indicating that we can compare groups with confidence, making straightforward interpretations of latent means and correlations. These results differ somewhat from those of
Comparing latent factor means across ethnic groups (presented as z scores) revealed significant (p-values < .05) group differences in means of the SAD, SS, and Sch factors. Specifically, Hispanic youth reported higher levels of SAD than did White (ΔM = .38), Black (ΔM = .29), and Asian (ΔM = .35) youth. Hispanic youth also reported higher levels of Sch than White (ΔM = .27), Black (ΔM = .15), and Asian (ΔM = .36) youth. Black and Hispanic youth reported higher levels of SS than White youth (ΔM = .20; ΔM = .24, respectively), and Black youth reported higher Sch compared with Asian youth (ΔM = .20). These findings are consistent with previous literature suggesting that Hispanic youth are more likely to meet diagnostic criteria for SAD (
Separate CFA models for each group were estimated, and all fit CES–D data well. However, Item 7 (“I felt that everything I did was an effort”) failed to load significantly on its factor (SS) in all four ethnic groups. Following recommendations by
Models 1 and 2 were then compared with Model 4, the most restrictive mode. Model 4 showed the lowest AIC and BIC values, and a review of other fit indices also supported Model 4 (ΔCFI and ΔTLI were less than .005). These results indicated that Model 4 fit the data best, supporting strong invariance among White, Black, and Asian youth and partial strong invariance for the Hispanic group. Factor loadings and intercorrelations in Model 4 were positive and significant for all groups (See Table S5 in the supplemental online material for factor loadings).
These results provide support for the original four-factor structure of the CES–D across the four ethnic groups, similar to prior research (
Comparing factor means across ethnic groups revealed significant differences in means of the NA and SS factors, whereas there were no group differences in means for the PA and IP factors. Hispanic youth reported slightly higher levels of NA than did White (ΔM = .34), Black (ΔM = .25), and Asian (ΔM = .42) youth. Hispanic youth also reported higher levels of SS than White (ΔM = .27), Black (ΔM = .20), and Asian (ΔM = .45) youth. Asian youth reported slightly lower levels of SS than Black youth (ΔM = .25). The higher reported levels of NA and SS in Hispanic youth are consistent with previous epidemiological studies suggesting that Hispanic youth tend to have higher rates of depressive disorders than other ethnic groups in the United States (e.g.,
All results must be interpreted in light of methodological limitations. The present study relied on convenience samples. We did not have parent reports of youth symptoms and only included youth in the seventh grade. Relying on a single source limits the generalizability of these findings to overarching constructs of anxiety and depression. Results imply that diverse ethnic groups of youth are mostly invariant in how they perceive and report symptoms using two measures. We cannot account for parents or other reporters. The limited age range constrains the extent to which we can generalize our findings to older or younger youth. Perceptions of symptoms could vary across racial/ethnic groups at different ages and developmental stages. Future research should test for measurement invariance of these measures for parent-report and across a wider range of ages. Further, the sample was ethnically diverse but was limited to an English-speaking U.S. sample. Limiting the sample to English-speaking youth may have muted potential differences across different levels of acculturation. Further, recruiting participants from a single U.S. school may have constricted the diversity of cultural perspectives represented, even in this ethnically diverse school. Anecdotally, recent immigrants composed a noticeable proportion of the student body. However, this was not systematically assessed. In the current sample, we were also not able to discriminate ethnic groups beyond broad categories, potentially obscuring differences in factor structure and item bias among ethnic subgroups. Given previous research indicating potential differences in the structure of the CES–D in some ethnic subgroups, it is important for future research on these measures to identify youth beyond broad ethnic/racial categories.
This research supports the cross-ethnic measurement invariance of the SCARED and CES–D and their use as tools for assessing anxiety and depressive symptoms in English-speaking, ethnically diverse youth in a middle school in the United States. We have provided evidence of strong invariance of the SCARED and of full or partial strong invariance of a 19-item version of the CES–D across four ethnic groups. These results bolster confidence in previous and future research utilizing these measures in ethnically/racially diverse samples of U.S. youth and indicate that observed differences in mean anxiety and depressive symptoms are not likely due to differential measurement properties across ethnic groups. Results also indicate that certain groups of youth, Hispanic youth in particular, experience more anxiety and depression in certain domains than White, Black, and Asian youth. The mechanisms by which such differences evolve have yet to be identified. Cultural phenomena are one possibility. For instance, it has been proposed that strong family orientation in Hispanic culture may explain greater separation anxiety, and negative stigma for mental health problems may explain somatic expression of emotional distress (
Allen, L., & Mitchell, C. (1998). Racial and ethnic differences in patterns of problematic and adaptive development: An epidemiological review. In V. C.McLoyd & L.Steinberg (Eds.), Minority adolescents: Conceptual, methodological, and theoretical issues (pp. 29–54). Mahwah, NJ: Erlbaum.
American Psychiatric Association. (2000.) Diagnostic and statistical manual of mental disorders (4th ed., text rev.). Washington, DC: Author.
Birmaher, B., Brent, D. A., Chiappetta, L., Bridge, J., Monga, S., & Baugher, M. (1999). Psychometric properties of the Screen for Child Anxiety Related Emotional Disorders (SCARED): A replication study. Journal of the American Academy of Child & Adolescent Psychiatry, 38, 1230–1236. doi:10.1097/00004583-199910000-00011
Byrne, B. M. (2012). Structural equation modeling with Mplus: Basic concepts, applications, and programming. New York, NY: Routledge.
Byrne, B. M., Shavelson, R. J., & Muthén, B. (1989). Testing for the equivalence of factor covariance and mean structures: The issue of partial measurement invariance. Psychological Bulletin, 105, 456–466. doi:10.1037/0033-2909.105.3.456
Chen, F. F. (2007). Sensitivity of goodness of fit indexes to lack of measurement invariance. Structural Equation Modeling, 14, 464–504. doi:10.1080/10705510701301834
Cheung, G. W., & Rensvold, R. B. (2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling, 9, 233–255. doi:10.1207/S15328007SEM0902_5
Costello, E. J., Mustillo, S., Erkanli, A., Keeler, G., & Angold, A. (2003). Prevalence and development of psychiatric disorders in childhood and adolescence. Archives of General Psychiatry, 60, 837–844. doi:10.1001/archpsyc.60.8.837
Crockett, L. J., Randall, B. A., Shen, Y. L., Russell, S. T., & Driscoll, A. K. (2005). Measurement equivalence of the Center for Epidemiological Studies Depression Scale for Latino and Anglo adolescents: A national study. Journal of Consulting and Clinical Psychology, 73, 47–58. doi:10.1037/0022-006X.73.1.47
Ferrell, C. B., Beidel, D. C., & Turner, S. M. (2004). Assessment and treatment of socially phobic children: A cross-cultural comparison. Journal of Clinical Child and Adolescent Psychology, 33, 260–268. doi:10.1207/s15374424jccp3302_6
Ginsburg, G. S., & Silverman, W. K. (1996). Phobic and anxiety disorders in Hispanic and Caucasian youth. Journal of Anxiety Disorders, 10, 517–528. doi:10.1016/S0887-6185(96)00027-8
Gonzalez, A., Weersing, V. R., Warnick, E., Scahill, L., & Woolston, J. (2012). Cross-ethnic measurement equivalence of the SCARED in an outpatient sample of African American and non-Hispanic White youths and parents. Journal of Clinical Child and Adolescent Psychology, 41, 361–369. doi:10.1080/15374416.2012.654462
Hale, W. W., III, Crocetti, E., Raaijmakers, Q. A., & Meeus, W. H. (2011). A meta-analysis of the cross-cultural psychometric properties of the Screen for Child Anxiety Related Emotional Disorders (SCARED). Journal of Child Psychology and Psychiatry, 52, 80–90. doi:10.1111/j.1469-7610.2010.02285.x
Kingery, J. N., Ginsburg, G. S., & Alfano, C. A. (2007). Somatic symptoms and anxiety among African American adolescents. Journal of Black Psychology, 33, 363–378. doi:10.1177/0095798407307041
Motl, R. W., Dishman, R. K., Birnbaum, A. S., & Lytle, L. A. (2005). Longitudinal invariance of the Center for Epidemiologic Studies–Depression Scale among girls and boys in middle school. Educational and Psychological Measurement, 65, 90–108. doi:10.1177/0013164404266256
Muris, P., Merckelbach, H., Ollendick, T., King, N., & Bogie, N. (2002). Three traditional and three new childhood anxiety questionnaires: Their reliability and validity in a normal adolescent sample. Behaviour Research and Therapy, 40, 753–772. doi:10.1016/S0005-7967(01)00056-0
Muthén, L. K., & Muthén, B. O. (1998–2011). Mplus user’s guide, Sixth edition. Los Angeles, CA: Author.
Neal, A. M., & Ward Brown, B. J. (1994). Fears and anxiety disorders in African American children. In S.Friedman (Ed.), Anxiety disorders in African Americans (pp. 65–75). New York, NY: Springer.
Okazaki, S. (1997). Sources of ethnic differences between Asian American and White American college students on measures of depression and social anxiety. Journal of Abnormal Psychology, 106, 52–60. doi:10.1037/0021-843X.106.1.52
Perreira, K. M., Deeb-Sossa, N., Harris, K. M., & Bollen, K. (2005). What are we measuring? An evaluation of the CES–D across race/ethnicity and immigrant generation. Social Forces, 83, 1567–1601. doi:10.1353/sof.2005.0077
Radloff, L. S. (1977). The CES–D Scale: A self-report depression scale for research in the general population. Applied Psychological Measurement, 1, 385–401. doi:10.1177/014662167700100306
Radloff, L. S. (1991). The use of the Center for Epidemiological Studies Depression Scale in adolescents and young adults. Journal of Youth and Adolescence, 20, 149–166. doi:10.1007/BF01537606
Roberts, R. E., Andrews, J. A., Lewinsohn, P. M., & Hops, H. (1990). Assessment of depression in adolescents using the Center for Epidemiologic Studies Depression Scale. Psychological Assessment, 2, 122–128. doi:10.1037/1040-3590.2.2.122
Siegel, J. M., Aneshensel, C. S., Taub, B., Cantwell, D. P., & Driscoll, A. K. (1998). Adolescent depressed mood in a multiethnic sample. Journal of Youth and Adolescence, 27, 413–427. doi:10.1023/A:1022873601030
Stark, K. D., & Laurent, J. (2001). Joint factor analysis of the Children’s Depression Inventory and the Revised Children’s Manifest Anxiety Scale. Journal of Clinical Child and Adolescent Psychology, 30, 552–567. doi:10.1207/S15374424JCCP3004_11
Varela, R. E., Vernberg, E. M., Sanchez-Sosa, J. J., Riveros, A., Mitchell, M., & Mashunkashey, J. (2004). Parenting style of Mexican, Mexican American, and Caucasian-Non-Hispanic families: Social context and cultural influences. Journal of Family Psychology, 18, 651–657. doi:10.1037/0893-3200.18.4.651
Widaman, K. F., & Reise, S. P. (1997). Exploring the measurement invariance of psychological instruments: Applications in the substance use domain. In K. J.Bryant, M.Windle, & S. G.West (Eds.), The science of prevention: Methodological advances from alcohol and substance abuse research (pp. 281–324). Washington, DC: American Psychological Association. doi:10.1037/10222-009
Wren, F. J., Berg, E. A., Heiden, L. A., Kinnamon, C. J., Ohlson, L. A., Bridge, J. A., . . .Bernal, M. P. (2007). Childhood anxiety in a diverse primary care population: Parent–child reports, ethnicity, and SCARED factor structure. Journal of the American Academy of Child & Adolescent Psychiatry, 46, 332–340. doi:10.1097/chi.0b013e31802f1267
Submitted: November 21, 2012 Revised: September 3, 2013 Accepted: October 10, 2013