The impact of familial expressed emotion on clinical and personal recovery among patients with psychiatric disorders: The mediating roles of self-stigma content and process

Acknowledgement: The present study was funded by the Early Career Scheme of the Research Grants Council of Hong Kong (Project 28611515). We would like to express our sincere gratitude to the following nongovernmental organizations (in alphabetical order) for facilitating us in recruiting eligible participants from their service users: Baptist Oi Kwan Social Service, New Life Psychiatric Rehabilitation Association, The Mental Health Association of Hong Kong, and The Society of Rehabilitation and Crime Prevention.

Despite global efforts to avert psychiatric stigma at both societal and individual levels, discrimination continues to persist against patients with psychiatric disorders (K. K. S. Chan & Mak, 2014). Psychiatric patients may encounter, in addition to discrimination from the public, disapproval and criticism from their significant others (Chien, Yeung, & Chan, 2014). For example, many patients with psychiatric disorders report experiencing expressed emotion (EE) from their family members (Hooley, 2007). Such experiences may exacerbate the internalization of stigma among these patients and compromise their recovery (K. K. S. Chan & Mak, 2014, 2017; Chien, Lam, & Ng, 2015). The goal of the present study was to develop an integrated model that elucidates how EE may affect self-stigma and, in turn, recovery among psychiatric patients.

EE was first conceptualized in the 1960s by Brown and colleagues (Brown, Monck, Carstairs, & Wing, 1962; Brown & Rutter, 1966) to characterize the negative emotional climate of the interactions between patients with schizophrenia and their family members. In early studies, EE was documented as a significant predictor of relapses among patients with psychotic disorders (Brown, Birley, & Wing, 1972; Vaughn & Leff, 1976). More recent studies have further identified EE as a significant predictor of multiple negative outcomes among patients with a wide range of psychiatric disorders, including depressive (Meuwly, Bodenmann, & Coyne, 2012), bipolar (Kim & Miklowitz, 2004), anxiety (Renshaw, Chambless, & Steketee, 2006), substance-related (O’Farrell, Hooley, Fals-Stewart, & Cutter, 1998), eating (Hedlund, Fichter, Quadflieg, & Brandl, 2003), and neurocognitive (Vitaliano, Becker, Russo, Magana-Amato, & Maiuro, 1989) disorders.

EE is defined as the critical, hostile, and intrusive attitudes of family members toward patients with a psychiatric disorder (Brown et al., 1972; Vaughn & Leff, 1976). Family members with high EE may criticize not only patients’ specific, illness-related behaviors but also the patients as people (Hooley, 2007). They may, for example, feel that the disorder was controllable and that the patients were choosing not to get better (Hooley & Campbell, 2002). They may also believe that many problems of the family are caused by the patients, even when they are not (Amaresha & Venkatasubramanian, 2012). Further, they may perceive that the patients cannot engage in productive activities or even take care of themselves and thus become overprotective of them, which sometimes involves unnecessary acts of self-sacrifice (Chien & Chan, 2010).

Psychiatric patients and their family members may contribute to the development of EE in reciprocal ways (Hooley & Gotlib, 2000; Miklowitz, 2004). In the face of the patients’ clinical symptoms and problem behaviors, the family members—especially those with a poor understanding of psychiatric disorders and negative appraisals of their living situations and caregiving responsibilities—may experience chronically high levels of stress and resort to EE as means to cope with their negative emotions (Raune, Kuipers, & Bebbington, 2004). However, the resultant critical, hostile, and intrusive interactions may exacerbate the patients’ illness and hinder their recovery, causing even more clinical symptoms and problem behaviors (Kavanagh, 1992). Continued exposure to psychopathology, in turn, may intensify the family members’ EE over time, creating a vicious cycle (Hooley, 2007). Therefore, familial EE may be understood as a dynamic, bidirectional phenomenon, resulting from the family members’ reactions to the patients’ psychiatric disorders, as well as the family members’ own personal characteristics (Hooley, 2007).

EE affects clinical recovery among patients with psychiatric disorders (Butzlaff & Hooley, 1998; Hooley, 2007). Clinical recovery refers to the mitigation of psychiatric symptoms and psychosocial dysfunctions associated with mental illness (Roe, Mashiach-Eizenberg, & Lysaker, 2011). Cross-sectional studies have found that EE is linked to more relapses, longer hospitalizations, and more severe psychiatric symptoms and functional impairments among psychiatric patients (Cechnicki, Bielańska, Hanuszkiewicz, & Daren, 2013; Domínguez-Martínez, Medina-Pradas, Kwapil, & Barrantes-Vidal, 2014; Hooley, Orley, & Teasdale, 1986; Miklowitz, Goldstein, Nuechterlein, Snyder, & Mintz, 1988). Further, intervention studies have demonstrated that patients whose family members receive EE-related psychoeducation have better clinical outcomes (Leff, Kuipers, Berkowitz, Eberlein-Vries, & Sturgeon, 1982; Tomás et al., 2012).

EE harms not only clinical recovery but also personal recovery (Barrowclough et al., 2003; Chien et al., 2015). Personal recovery refers to the establishment of a satisfying, hopeful, and contributing life despite the limitations imposed by mental illness (Barber, 2012). It highlights the importance of reclaiming a positive sense of the self and finding the meaning of life (K. K. S. Chan & Mak, 2014; Song, 2017). EE may hinder personal recovery, because familial disapproval and criticism may leave psychiatric patients with a more negative sense of the self, which may further prevent them from engaging in meaningful activities (Chien et al., 2015). Studies have corroborated such views by documenting negative associations of EE with self-esteem, self-efficacy, and quality of life among patients with psychiatric disorders (Barrowclough et al., 2003; Chien et al., 2015; Ritsner et al., 2000, 2003).

In response to familial EE, patients may feel judged, humiliated, and marginalized, which may contribute to self-stigmatization (Chien et al., 2015). Self-stigma refers to patients’ acceptance of societal negative evaluations and assimilation of such views into their own value systems (Corrigan & Watson, 2002). It is important to note that because self-stigma involves the internalization of societal stigma and the development of negative self-thoughts, self-stigma may diminish self-esteem, self-efficacy, and psychological well-being (Livingston & Boyd, 2010).

Self-stigma has long been assessed as a unidimensional construct (Corrigan, Watson, & Barr, 2006), but recent research has suggested that self-stigma may be multifaceted (K. K. S. Chan & Mak, 2017). Patients with psychiatric disorders, for example, may vary in not only the extent to which they endorse their self-stigmatizing thoughts, referred to as self-stigma content, but also the extent to which their self-stigmatizing thoughts recur, referred to as self-stigma process (K. K. S. Chan & Mak, 2017). Thus, in the present study, we considered self-stigma content and process as distinct mediators that would independently account for the effects of EE on recovery.

Being aware of familial EE and stigmatizing thoughts against them, patients may endorse and internalize the content of such thoughts to various degrees (Corrigan, Rafacz, & Rüsch, 2011). If patients regard the criticism as fair and legitimate, they are likely to self-concur with familial stigmatizing thoughts and develop self-stigmatizing ideas (Watson, Corrigan, Larson, & Sells, 2007). Conversely, if patients consider the criticism as unfair and illegitimate, they are likely to feel indifferent or even angry in response to familial disapproval (Watson et al., 2007).

Patients who endorse and internalize the content of familial stigmatizing thoughts may develop negative evaluations of themselves and their stigmatized identities (Mak & Cheung, 2010). Specifically, they may perceive themselves as incompetent and inferior to others and suffer a decreased sense of self-worth (K. K. S. Chan & Mak, 2014). They may also feel ashamed of their stigmatized identities and begin to denigrate themselves and withdraw from others to conceal their stigmatized status (K. K. S. Chan & Mak, 2014).

In addition to the meaning patients attach to their stigmatized identities, EE may affect the recurrence of self-stigmatizing thoughts among patients. Specifically, the familial disapproval and criticism experienced by these patients may make their stigmatized status more salient (Chien et al., 2014). Such salience may continue to activate self-stigmatizing thoughts and prompt these patients to adopt a negative view of themselves (Chien et al., 2015). This may render these patients preoccupied with negative self-thoughts, particularly among those with strong tendencies to ruminate (Verplanken, Friborg, Wang, Trafimow, & Woolf, 2007).

When self-stigmatizing thinking occurs repetitively and persistently, it may become a dominant feature of the mind and develop into a mental habit (K. K. S. Chan & Mak, 2017). A mental habit refers to a thought that has acquired automaticity through repetition (Verplanken et al., 2007). For patients with mental habits of self-stigma, self-stigmatizing cognitions can become so automatic that they may emerge without conscious awareness or willful intent, demand little attention and cognitive resources, and grow increasingly difficult to control (K. K. S. Chan & Mak, 2017). As a result, the minds of these patients can be automatically occupied by self-stigmatizing thoughts without deliberation.

Self-stigma has important implications for clinical and personal recovery among patients with psychiatric disorders (R. C. H. Chan & Mak, 2016). With respect to clinical recovery, patients with self-stigma may be emotionally burdened by negative self-thoughts (Mak & Cheung, 2010). The potential cumulative effects are elevated distress and increased risks of mental health problems (Drapalski et al., 2013). With respect to personal recovery, patients with self-stigma may experience diminished self-esteem and self-efficacy, which undermines their hope of and optimism about pursuing important goals (Corrigan, Bink, Schmidt, Jones, & Rüsch, 2016). Consequently, they may find it harder to see the meaning of life and report lower levels of life satisfaction (K. K. S. Chan & Mak, 2014).

Despite the relevance of self-stigma to understanding the impact of EE on recovery, limited research has explored how EE, self-stigma, and recovery are associated among psychiatric patients (Chien et al., 2015). Existing studies have also focused on self-stigma content and clinical recovery. To the best of our knowledge, no study of EE has investigated self-stigma process and personal recovery. Simultaneously assessing the associations of self-stigma content and process with clinical and personal recovery is important, because it allows for a more comprehensive examination of the impact of self-stigma on recovery. Such an approach also addresses whether existing anti-self-stigma interventions and psychiatric rehabilitation services, which are geared toward tackling self-stigma content and promoting clinical recovery (K. K. S. Chan & Mak, 2017), should be extended to target self-stigma process and personal recovery as well.

The objectives of the present study were twofold. First, we examined the associations of EE with clinical and personal recovery and the potential mediating roles of self-stigma content and process in these associations. Using structural equation modeling (SEM), we tested a mediation model of EE, self-stigma, and recovery among patients with psychiatric disorders. We hypothesized that EE would be negatively associated with clinical recovery (as indicated by symptom remission and functional restoration) and personal recovery (as indicated by positive perceptions of recovery and life satisfaction). Moreover, we hypothesized that such associations would be mediated by self-stigma content and self-stigma process. Second, we examined whether the hypothesized associations and mechanisms would hold across patients with psychotic versus nonpsychotic disorders. Previous studies on the potential impact of EE were mostly based on patients with psychotic disorders, which are more chronically disabling and thus more easily stigmatized (Mueser & McGurk, 2004). The lack of multidiagnostic investigations has hindered the understanding of whether EE would uniformly, or differentially, affect recovery across patients with disorders of different degrees of severity and acuity, such as psychotic versus nonpsychotic disorders. The present study is the first to address this gap in the literature.

Participants were patients with psychiatric disorders. Based on multiple assumptions and criteria, including that there would be up to 13 indicators in each SEM model, that some SEM models would be tested across two subsamples, that the effect sizes of interest would range from small to medium, and that the power would be .80 (Wolf, Harrington, Clark, & Miller, 2013), the required sample size was estimated to be 260. To guard against other potential data collection issues, such as withdrawal from the study and refusal to report on parts of the questionnaire (Northouse et al., 2006), we set the target sample size to be 300.

Participants were recruited from community mental health service centers operated by four nongovernmental organizations in Hong Kong, China. Patients with psychiatric disorders were first informed of the study through flyers posted at the service centers and announcements during clinical groups and meetings. Patients with initial interest in joining the study provided contact information to our research assistants, who further explained the study to them and screened to determine their eligibility. Inclusion criteria were (a) being able to read and write in Chinese and (b) having been diagnosed with at least one Axis I disorder according to the Diagnostic and Statistical Manual of Mental Disorders (4th ed., text rev.; DSM–IV–TR; American Psychiatric Association, 2000) by a licensed clinician. Exclusion criteria were (a) having been diagnosed with intellectual disability or dementia by a licensed clinician, based on the DSM–IV–TR criteria, and (b) being clinically unstable (i.e., having been hospitalized in the preceding month). Diagnostic and clinical information was ascertained by social workers employed by the service centers, based on reviews of medical records. Eligible patients were given information sheets describing the objectives and procedures of the study and were asked to sign written consent forms. Of the 318 patients screened, 311 met the study criteria, provided written informed consent, and eventually participated.

The mean age of participants was 43.61 years (SD = 9.52). Most participants were female (65.9%), were not married (67.8%), and had received a high school education or more (88.4%). The median monthly household income was between HK$8,001 and HK$9,000 (between about US$1,026 and US$1,154), much lower than the median monthly household income of the larger population (HK$25,000, or about US$3,204; Census and Statistics Department, 2017). Most participants reported taking psychiatric medication (94.2%), and their primary diagnoses included psychotic (46.6%), depressive (41.2%), bipolar (5.8%), anxiety (4.5%), and other (1.9%) disorders. The mean duration of illness was 11.70 years (SD = 9.28).

The present study was approved by the Human Research Ethics Committee of The Education University of Hong Kong. The study, titled “The Content and Process of Self-Stigma in People With Serious Mental Illness,” was conducted between September 2015 and March 2017 and utilized a cross-sectional, quantitative research design. Each participant self-administered a questionnaire and received HK$200 (about US$26) as compensation.

All questionnaires were presented in Chinese. These questionnaires, originally developed in English, were forward- and back-translated to Chinese by two independent translators, who then compared their work, resolved the discrepancies, and finalized the items.

Expressed emotion

Familial EE was measured with the Level of Expressed Emotion (LEE) scale (Cole & Kazarian, 1988; Ng & Sun, 2011). A sample item was “My family members accuse me of exaggerating when I say I’m unwell.” Participants rated each item on a 4-point Likert scale ranging from 1 (not true) to 4 (true). The ratings were averaged, with higher scores indicating higher levels of familial EE. The LEE has been used to indicate familial EE among patients with psychiatric disorders, and its validity was evidenced by its significant correlations with theoretically related constructs (Chien et al., 2014). In the present study, the Cronbach’s alpha of the LEE was .89.

Self-stigma content

Self-stigma content was measured with the Internalized Stigma of Mental Illness (ISMI) scale (Boyd, Otilingam, & DeForge, 2014; Ritsher, Otilingam, & Grajales, 2003). A sample item was “I can’t contribute anything to society because I have a mental illness.” Participants rated each item on a 4-point Likert scale ranging from 1 (strongly disagree) to 4 (strongly agree). The ratings were averaged, with higher scores indicating more negatively valenced content of self-stigmatizing thoughts. The ISMI has been used to indicate self-stigma content among patients with psychiatric disorders, and its validity was evidenced by its significant correlations with theoretically related constructs (K. K. S. Chan, Lee, & Mak, 2018). In the present study, the Cronbach’s alpha of the ISMI was .86.

Self-stigma process

Self-stigma process was measured with the Self-Stigmatizing Thinking’s Automaticity and Repetition Scale (STARS; K. K. S. Chan & Mak, 2017). A sample item was “Thinking negatively about my identity as a person with mental illness is something I do every day.” Participants rated each item on a 5-point Likert scale ranging from 1 (strongly disagree) to 5 (strongly agree). The ratings were averaged, with higher scores indicating more habitual emergence of self-stigmatizing thoughts. The STARS has been used to indicate self-stigma process among patients with psychiatric disorders, and its validity was evidenced by its significant correlations with theoretically related constructs (K. K. S. Chan & Mak, 2017). In the present study, the Cronbach’s alpha of the STARS was .93.

Symptom remission

Symptom remission was measured with the Modified Colorado Symptom Index (MCSI; Conrad et al., 2001). A sample item was “In the past month, I have felt nervous, tense, worried, frustrated, or afraid.” Participants rated each item on a 5-point Likert scale ranging from 1 (not at all) to 5 (at least every day). The ratings were reverse-coded and then averaged, with higher scores indicating higher levels of symptom remission. The MCSI has been used to indicate symptom remission among patients with psychiatric disorders, and its validity was evidenced by its significant correlations with theoretically related constructs (Fukui, Davidson, Holter, & Rapp, 2010). In the present study, the Cronbach’s alpha of the MCSI was .92.

Functional restoration

Functional restoration was measured with the interpersonal relationships and work skills subscales of the Specific Level of Functioning (SLOF) scale (Schneider & Struening, 1983). Sample items were “I form and maintain friendships” and “I am able to sustain work efforts.” Participants rated each item on a 5-point Likert scale ranging from 1 (poorest function) to 5 (best function). The ratings were averaged, with higher scores indicating higher levels of functional restoration. The SLOF has been used to indicate social and occupational functioning among patients with psychiatric disorders, and its validity was evidenced by its significant correlations with theoretically related constructs (Bowie et al., 2007). In the present study, the Cronbach’s alpha of the SLOF was .92.

Recovery perceptions

Recovery perceptions were measured with the Recovery Assessment Scale (RAS; Corrigan, Salzer, Ralph, Sangster, & Keck, 2004; Roe, Mashiach-Eizenberg, & Corrigan, 2012). A sample item was “I’m hopeful about my future.” Participants rated each item on a 5-point Likert scale ranging from 1 (strongly disagree) to 5 (strongly agree). The ratings were averaged, with higher scores indicating more positive perceptions of recovery. The RAS has been used to indicate recovery perceptions among patients with psychiatric disorders, and its validity was evidenced by its significant correlations with theoretically related constructs (Hasson-Ohayon et al., 2014). In the present study, the Cronbach’s alpha of the RAS was .86.

Life satisfaction

Life satisfaction was measured with the Satisfaction With Life Scale (SWLS; Diener, Emmons, Larsen, & Griffin, 1985). A sample item was “I am satisfied with my life.” Participants rated each item on a 7-point Likert scale ranging from 1 (strongly disagree) to 7 (strongly agree). The ratings were averaged, with higher scores indicating higher levels of life satisfaction. The SWLS has been used to indicate life satisfaction among patients with psychiatric disorders, and its validity was evidenced by its significant correlations with theoretically related constructs (K. K. S. Chan & Mak, 2017). In the present study, the Cronbach’s alpha of the SWLS was .85.

Analyses were conducted in three steps using Mplus 7.4 (Muthén & Muthén, 2012). First, descriptive and correlation analyses were conducted for all variables. Second, SEM was conducted to evaluate the hypothesized model. To indicate the latent constructs, we grouped the LEE, ISMI, and STARS items into three parcels to represent EE, self-stigma content, and self-stigma process, respectively. The MCSI and SLOF scores were used as the indicators of clinical recovery, whereas the RAS and SWLS scores were used as the indicators of personal recovery. A measurement model was tested to examine how well the prescribed factor structure fit the data. A structural model was then tested to examine the associations among the latent constructs. Model fit was evaluated using chi-square, comparative fit index (CFI), Tucker−Lewis index (TLI), root-mean-square error of approximation (RMSEA), and standardized root-mean-square residual (SRMR). CFI and TLI values ≥.90 and RMSEA and SRMR values ≤.08 indicated a good fit (Hu & Bentler, 1999). Indirect effects in the structural model were estimated with the bootstrapping procedures suggested by Shrout and Bolger (2002). Bias-corrected confidence intervals were generated based on 1,000 bootstrapped samples from the data. The absence of 0 in the 95% confidence interval indicated a significant indirect effect. Finally, multigroup SEM was conducted to examine whether the hypothesized model would hold across patients with psychotic versus nonpsychotic disorders. Following the recommendations of Byrne (2012), we established configural invariance by fitting the model freely in the two subsamples. Factor loading invariance and structural parameter invariance were then established by adding relevant constraints (i.e., setting the factor loadings and structural parameters to be equal across the two subsamples) and by evaluating whether the addition of constraints would contribute to a significant chi-square change. A nonsignificant chi-square change indicated model invariance.

Table 1 presents the descriptive statistics of and intercorrelations among all variables. EE was positively correlated with self-stigma content and self-stigma process (ps < .001). All three variables were negatively correlated with symptom remission, functional restoration, recovery perceptions, and life satisfaction (ps < .04). The two indicators of clinical recovery (i.e., symptom remission and functional restoration) were positively correlated (p < .001). The two indicators of personal recovery (i.e., positive perceptions of recovery and life satisfaction) were also positively correlated (p < .001).

Table 2 presents the results of SEM. In the measurement model, all standardized factor loadings, ranging between .64 and .96, were significant (ps < .001). The measurement model fitted the data well, with all fit indices meeting the criteria of a good fit, χ2(55) = 122.11, p < .001, CFI = .97, TLI = .96, RMSEA = .06, SRMR = .04. In the structural model, all standardized path loadings, ranging between −.51 and .29, were significant (ps < .001). The structural model fitted the data well, with all fit indices meeting the criteria of a good fit, χ2(57) = 130.06, p < .001, CFI = .97, TLI = .96, RMSEA = .06, SRMR = .04.

Figure 1 shows the structural model. EE was positively related to self-stigma content and self-stigma process (ps < .001), which were in turn negatively related to clinical recovery and personal recovery (ps < .001). Table 3 presents the results of bootstrap analyses. EE had significant indirect effects on clinical recovery and personal recovery via self-stigma content and self-stigma process (ps < .003). The explained variances were 7%, 8%, 81%, and 48% for self-stigma content, self-stigma process, clinical recovery, and personal recovery, respectively.

Table 4 presents the results of multigroup SEM. To test for configural invariance, we freely estimated all parameters across patients with psychotic versus nonpsychotic disorders (Model 1). This model showed a good fit, indicating that the same model configuration held across the two subsamples. Next, to test for factorial invariance, we constrained all factor loadings to be equal across the two subsamples (Model 2). The chi-square change between Model 1 and Model 2 was nonsignificant, suggesting that the indicators loaded on the corresponding latent constructs in identical ways across the two subsamples. Finally, to test for structural invariance, we constrained all path coefficients and factor covariance among the latent constructs to be equal (Model 3). The chi-square change between Model 2 and Model 3 was nonsignificant, suggesting that the latent constructs were associated in identical ways across the two subsamples.

Consistent with our hypotheses, EE was positively related to self-stigma content and process, which were in turn negatively related to clinical and personal recovery among patients with psychiatric disorders. On a theoretical level, our findings documented the potential adverse effects of familial EE on psychiatric recovery and demonstrated how these negative effects could be mediated by the cognitive content and habitual process of self-stigma. On an applied level, our findings suggested that intervention programs targeting the criticism, hostility, and intrusiveness of family members (Öksüz, Karaca, Özaltın, & Ateş, 2017) should be extended to target the self-stigma and recovery of psychiatric patients.

Our study highlighted the potential implications of EE for recovery among patients with psychotic and nonpsychotic disorders. Considering the relevance of EE to recovery among patients with different psychiatric diagnoses, psychiatric rehabilitation service institutions should place emphasis on mitigating EE. Specifically, these institutions should incorporate into their services and programs effective initiatives that transform the familial emotional climate. To reduce familial disapproval and criticism, practitioners should develop psychoeducation that helps family members understand psychiatric disorders, particularly what is and what is not within the control of psychiatric patients, and adjust their expectations for and their communication with psychiatric patients accordingly (Öksüz et al., 2017). Further, practitioners should design interventions that improve problem-solving and coping skills among family members to ameliorate their caregiving stress and reduce their EE (Gleeson et al., 2013).

As indicated by the negative associations of EE with symptom remission and functional restoration, EE could hinder clinical recovery. This finding is consistent with those of previous studies showing that EE was positively related to the number of relapses and the duration of rehospitalizations among psychiatric patients (Cechnicki et al., 2013; Domínguez-Martínez et al., 2014; Hooley et al., 1986; Miklowitz et al., 1988) and that EE reduction programs were effective in improving clinical stability and functional competence among psychiatric patients (Leff et al., 1982; Tomás et al., 2012). Given the inhibitory effects of EE on clinical recovery, practitioners should engage patients from high-EE families in treatments that help mitigate their psychiatric symptoms and improve their adaptive functioning.

In keeping with prior studies indicating that EE was linked to lower levels of self-esteem, self-efficacy, and quality of life among patients with psychiatric disorders (Barrowclough et al., 2003; Chien et al., 2015; Ritsner et al., 2000, 2003), EE was associated with negative perceptions of recovery and reduced life satisfaction. Because EE may impede personal recovery, practitioners should provide person-centered, strength-based, and recovery-oriented interventions to patients from high-EE families to help them develop confidence, pursue aspirations, and lead a flourishing life. For example, strength-based recovery groups (Green, Janoff, Yarborough, & Paulson, 2013) and person-centered care planning (Adams & Grieder, 2005), whose efficacy in promoting personal recovery has been established (Stanhope, Ingoglia, Schmelter, & Marcus, 2013; Tse et al., 2016), should be integrated into the services tailored for these patients.

Building upon existing work on EE and recovery among patients with psychiatric disorders (Barrowclough et al., 2003; Chien et al., 2015), our study pointed to the potential negative effects of EE on clinical and personal recovery, as well as the potential mediating role of self-stigma in understanding these negative effects. Specifically, our findings suggested that experiences of EE, which often come with feelings of devaluation and inferiority, may prompt psychiatric patients to internalize the negative stereotypes and prejudicial attitudes toward their illness status. These patients may be more likely to be burdened by self-stigma, which may in turn inhibit their clinical and personal recovery.

Previous research has documented positive associations between EE and self-stigma content among psychiatric patients (Chien et al., 2015). The unique contribution of our study, however, lies in our discovery that EE was linked to both self-stigma content and process among these patients. In particular, our findings suggested that EE may increase the negative valence and heighten the salience of self-stigmatizing thoughts among these patients. More generally, these findings are consistent with those of previous studies showing that EE was linked to higher levels of perceived and internalized stigma among patients with psychiatric disorders (Chien et al., 2014, 2015).

Self-stigma content and process were negatively related to clinical recovery, likely because self-stigmatizing thoughts may exacerbate emotional distress and increase the risks of mental health problems. This finding resonates with past evidence showing that self-stigma is a reliable predictor for poorer prognosis among psychiatric patients (National Institute of Mental Health, 2008) and that patients with self-stigma are likely to not only experience higher levels of depressive, anxiety, and psychotic symptoms but also exhibit poorer social and occupational functioning (Cavelti, Rüsch, & Vauth, 2014; Drapalski et al., 2013; Lysaker, Davis, Warman, Strasburger, & Beattie, 2007).

The negative associations of self-stigma content and process with personal recovery indicated that self-stigma may reduce patients’ potential to attain a self-directing and fulfilling life. One plausible explanation of this finding is that patients with self-stigma may perceive themselves as incompetent and inferior to others and suffer a diminished sense of purpose and value, which may prevent them from playing meaningful roles in their lives (Corrigan et al., 2016). Such adverse effects of self-stigma on personal recovery are in line with results of previous studies showing that self-stigma was linked to lower levels of hope, self-esteem, self-efficacy, self-empowerment, and quality of life (K. K. S. Chan & Mak, 2014, 2017; Corrigan et al., 2006, 2011, 2012).

Consistent with past studies (K. K. S. Chan & Mak, 2017), the content and process of self-stigma were independent predictors of recovery, suggesting that the content of self-stigmatizing thoughts, as well as the process through which patients have these thoughts, are important factors to consider when designing anti-self-stigma interventions and recovery-oriented services. To reduce self-stigma content, practitioners may teach patients cognitive restructuring skills to challenge negative beliefs about themselves (K. K. S. Chan & Mak, 2017). Moreover, to alleviate self-stigma process, practitioners may empower patients to extend their self-definitions beyond their stigmatized status, which may help them think about their disadvantaged conditions less often (K. K. S. Chan & Mak, 2017).

Our study had several limitations. First, guided by the content–process theory of self-stigma (K. K. S. Chan & Mak, 2017), we tested EE as an exogenous variable and clinical and personal recovery as endogenous variables in our analytic model. However, other theories emphasize a bidirectional relation between EE and recovery, in which, for example, EE and symptom severity influence each other reciprocally over time (Hooley, 2007; Hooley & Gotlib, 2000; Miklowitz, 2004). Despite these alternative perspectives, our cross-sectional design did not allow us to disentangle the temporal order of the associations between EE and recovery. Future studies with longitudinal panel designs are needed to examine the utility of different theories in understanding the potential bidirectional associations of family members’ EE with patients’ symptomatic–functional and personal recovery. Second, the LEE assesses patients’ subjective perceptions of the amount of EE that they had received from family members rather than the amount of EE that family members had expressed. As advocated by Kavanagh (1992), EE may be best measured using multiple methods, including semistructured interviews, such as the Camberwell Family Interview (Brown et al., 1962; Vaughn & Leff, 1976), in which a family member is asked to talk freely about his or her feelings toward and relationship with the affected patient. Future studies should measure EE using both self-reports and semistructured interviews to retest our hypotheses. Third, our study was based on a convenience sample of patients with psychiatric disorders from Hong Kong, which limited the generalizability of our findings. Future studies should recruit representative samples of patients to test whether our findings are applicable to the larger population.

Despite these limitations, our study had important theoretical and practical implications. Theoretically, our findings indicated that EE may exacerbate the internalization of stigma and compromise clinical and personal recovery among patients with psychiatric disorders. Practically, our findings highlighted the utility of targeting familial EE and self-stigma in facilitating clinical and personal recovery among patients with psychiatric disorders. That being said, previous research has also pointed to the importance of supporting psychiatric patients by alleviating their symptoms, reducing the side effects of their medications, increasing their self-confidence, expanding their social circles, addressing their communication deviances and cognitive biases, providing them with behavioral training, and ensuring that they have appropriate housing and living conditions (Bellack, 2006; K. K. S. Chan & Mak, 2012, 2014; Ritsner et al., 2000; Silverstein & Bellack, 2008; Wang et al., 2000). Therefore, to truly promote recovery among psychiatric patients, practitioners should aim to provide them with multidimensional treatment and rehabilitation services that address their needs on medical, psychological, social, behavioral, and day-to-day levels.

Adams, N., & Grieder, D. M. (2005). Treatment planning for person-centered care: The road to mental health and addiction recovery. San Diego, CA: Elsevier Academic Press.

Amaresha, A. C., & Venkatasubramanian, G. (2012). Expressed emotion in schizophrenia: An overview. Indian Journal of Psychological Medicine, 34, 12–20. 10.4103/0253-7176.96149

American Psychiatric Association. (2000). Diagnostic and statistical manual of mental disorders (4th ed., text rev.). Washington, DC: Author.

Barber, M. E. (2012). Recovery as the new medical model for psychiatry. Psychiatric Services, 63, 277–279. 10.1176/appi.ps.201100248

Barrowclough, C., Tarrier, N., Humphreys, L., Ward, J., Gregg, L., & Andrews, B. (2003). Self-esteem in schizophrenia: Relationships between self-evaluation, family attitudes, and symptomatology. Journal of Abnormal Psychology, 112, 92–99. 10.1037/0021-843X.112.1.92

Bellack, A. S. (2006). Scientific and consumer models of recovery in schizophrenia: Concordance, contrasts, and implications. Schizophrenia Bulletin, 32, 432–442. 10.1093/schbul/sbj044

Bowie, C. R., Twamley, E. W., Anderson, H., Halpern, B., Patterson, T. L., & Harvey, P. D. (2007). Self-assessment of functional status in schizophrenia. Journal of Psychiatric Research, 41, 1012–1018. 10.1016/j.jpsychires.2006.08.003

Boyd, J. E., Otilingam, P. G., & DeForge, B. R. (2014). Brief version of the Internalized Stigma of Mental Illness (ISMI) scale: Psychometric properties and relationship to depression, self esteem, recovery orientation, empowerment, and perceived devaluation and discrimination. Psychiatric Rehabilitation Journal, 37, 17–23. 10.1037/prj0000035

Boyd Ritsher, J., Otilingam, P. G., & Grajales, M. (2003). Internalized stigma of mental illness: Psychometric properties of a new measure. Psychiatry Research, 121, 31–49. 10.1016/j.psychres.2003.08.008

Brown, G. W., Birley, J. L. T., & Wing, J. K. (1972). Influence of family life on the course of schizophrenic disorders: A replication. British Journal of Psychiatry, 121, 241–258. 10.1192/bjp.121.3.241

Brown, G. W., Monck, E. M., Carstairs, G. M., & Wing, J. K. (1962). Influence of family life on the course of schizophrenic illness. British Journal of Preventive & Social Medicine, 16, 55–68.

Brown, G. W., & Rutter, M. (1966). The measurement of family activities and relationships: A methodological study. Human Relations, 19, 241–263. 10.1177/001872676601900301

Butzlaff, R. L., & Hooley, J. M. (1998). Expressed emotion and psychiatric relapse: A meta-analysis. Archives of General Psychiatry, 55, 547–552. 10.1001/archpsyc.55.6.547

Byrne, B. M. (2012). Structural equation modeling with Mplus: Basic concepts, applications, and programming. New York, NY: Routledge.

Cavelti, M., Rüsch, N., & Vauth, R. (2014). Is living with psychosis demoralizing? Insight, self-stigma, and clinical outcome among people with schizophrenia across 1 year. Journal of Nervous and Mental Disease, 202, 521–529. 10.1097/NMD.0000000000000160

Cechnicki, A., Bielańska, A., Hanuszkiewicz, I., & Daren, A. (2013). The predictive validity of expressed emotions (EE) in schizophrenia. A 20-year prospective study. Journal of Psychiatric Research, 47, 208–214. 10.1016/j.jpsychires.2012.10.004

Census and Statistics Department. (2017). 2016 population by-census thematic report: Household income distribution in Hong Kong. Hong Kong, China: Author.

Chan, K. K. S., Lee, C. W. L., & Mak, W. W. S. (2018). Mindfulness model of stigma resistance among individuals with psychiatric disorders. Mindfulness. Advance online publication. 10.1007/s12671-018-0887-2

Chan, K. K. S., & Mak, W. W. S. (2012). Shared decision making in the recovery of people with schizophrenia: The role of metacognitive capacities in insight and pragmatic language use. Clinical Psychology Review, 32, 535–544. 10.1016/j.cpr.2012.06.001

Chan, K. K. S., & Mak, W. W. S. (2014). The mediating role of self-stigma and unmet needs on the recovery of people with schizophrenia living in the community. Quality of Life Research: An International Journal of Quality of Life Aspects of Treatment, Care and Rehabilitation, 23, 2559–2568. 10.1007/s11136-014-0695-7

Chan, K. K. S., & Mak, W. W. S. (2017). The content and process of self-stigma in people with mental illness. American Journal of Orthopsychiatry, 87, 34–43. 10.1037/ort0000127

Chan, R. C. H., & Mak, W. W. S. (2016). Common sense model of mental illness: Understanding the impact of cognitive and emotional representations of mental illness on recovery through the mediation of self-stigma. Psychiatry Research, 246, 16–24. 10.1016/j.psychres.2016.09.013

Chien, W.-T., & Chan, S. W. (2010). Further validation of the Chinese version of the Level of Expressed Emotion scale for research and clinical use. International Journal of Nursing Studies, 47, 190–204. 10.1016/j.ijnurstu.2009.05.019

Chien, W.-T., Lam, C. K., & Ng, B. F. (2015). Predictors of levels of functioning among Chinese people with severe mental illness: A 12-month prospective cohort study. Journal of Clinical Nursing, 24, 1860–1873. 10.1111/jocn.12843

Chien, W.-T., Yeung, F. K., & Chan, A. H. (2014). Perceived stigma of patients with severe mental illness in Hong Kong: Relationships with patients’ psychosocial conditions and attitudes of family caregivers and health professionals. Administration and Policy in Mental Health and Mental Health Services Research, 41, 237–251. 10.1007/s10488-012-0463-3

Cole, J. D., & Kazarian, S. S. (1988). The Level of Expressed Emotion scale: A new measure of expressed emotion. Journal of Clinical Psychology, 44, 392–397. 10.1002/1097-4679(198805)44:3<392::AID-JCLP2270440313>3.0.CO;2-3

Conrad, K. J., Yagelka, J. R., Matters, M. D., Rich, A. R., Williams, V., & Buchanan, M. (2001). Reliability and validity of a Modified Colorado Symptom Index in a national homeless sample. Mental Health Services Research, 3, 141–153. 10.1023/A:1011571531303

Corrigan, P. W., Bink, A. B., Schmidt, A., Jones, N., & Rüsch, N. (2016). What is the impact of self-stigma? Loss of self-respect and the “why try” effect. Journal of Mental Health, 25, 10–15. 10.3109/09638237.2015.1021902

Corrigan, P. W., Michaels, P. J., Vega, E., Gause, M., Watson, A. C., & Rüsch, N. (2012). Self-Stigma of Mental Illness Scale—Short Form: Reliability and validity. Psychiatry Research, 199, 65–69. 10.1016/j.psychres.2012.04.009

Corrigan, P. W., Rafacz, J., & Rüsch, N. (2011). Examining a progressive model of self-stigma and its impact on people with serious mental illness. Psychiatry Research, 189, 339–343. 10.1016/j.psychres.2011.05.024

Corrigan, P. W., Salzer, M., Ralph, R. O., Sangster, Y., & Keck, L. (2004). Examining the factor structure of the Recovery Assessment Scale. Schizophrenia Bulletin, 30, 1035–1041. 10.1093/oxfordjournals.schbul.a007118

Corrigan, P. W., & Watson, A. C. (2002). The paradox of self-stigma and mental illness. Clinical Psychology: Science and Practice, 9, 35–53. 10.1093/clipsy.9.1.35

Corrigan, P. W., Watson, A. C., & Barr, L. (2006). The self-stigma of mental illness: Implications for self-esteem and self-efficacy. Journal of Social and Clinical Psychology, 25, 875–884. 10.1521/jscp.2006.25.8.875

Diener, E., Emmons, R. A., Larsen, R. J., & Griffin, S. (1985). The Satisfaction With Life Scale. Journal of Personality Assessment, 49, 71–75. 10.1207/s15327752jpa4901_13

Domínguez-Martínez, T., Medina-Pradas, C., Kwapil, T. R., & Barrantes-Vidal, N. (2014). Relatives’ illness attributions mediate the association of expressed emotion with early psychosis symptoms and functioning. Psychiatry Research, 218, 48–53. 10.1016/j.psychres.2014.04.012

Drapalski, A. L., Lucksted, A., Perrin, P. B., Aakre, J. M., Brown, C. H., DeForge, B. R., & Boyd, J. E. (2013). A model of internalized stigma and its effects on people with mental illness. Psychiatric Services, 64, 264–269. 10.1176/appi.ps.001322012

Fukui, S., Davidson, L. J., Holter, M. C., & Rapp, C. A. (2010). Pathways to recovery (PTR): Impact of peer-led group participation on mental health recovery outcomes. Psychiatric Rehabilitation Journal, 34, 42–48. 10.2975/34.1.2010.42.48

Gleeson, J. F., Cotton, S. M., Alvarez-Jimenez, M., Wade, D., Gee, D., Crisp, K., . . .McGorry, P. D. (2013). A randomized controlled trial of relapse prevention therapy for first-episode psychosis patients: Outcome at 30-month follow-up. Schizophrenia Bulletin, 39, 436–448. 10.1093/schbul/sbr165

Green, C. A., Janoff, S. L., Yarborough, B. J. H., & Paulson, R. I. (2013). The Recovery Group Project: Development of an intervention led jointly by peer and professional counselors. Psychiatric Services, 64, 1211–1217. 10.1176/appi.ps.201200546

Hasson-Ohayon, I., Mashiach-Eizenberg, M., Elhasid, N., Yanos, P. T., Lysaker, P. H., & Roe, D. (2014). Between self-clarity and recovery in schizophrenia: Reducing the self-stigma and finding meaning. Comprehensive Psychiatry, 55, 675–680. 10.1016/j.comppsych.2013.11.009

Hedlund, S., Fichter, M. M., Quadflieg, N., & Brandl, C. (2003). Expressed emotion, family environment, and parental bonding in bulimia nervosa: A 6-year investigation. Eating and Weight Disorders, 8, 26–35. 10.1007/BF03324986

Hooley, J. M. (2007). Expressed emotion and relapse of psychopathology. Annual Review of Clinical Psychology, 3, 329–352. 10.1146/annurev.clinpsy.2.022305.095236

Hooley, J. M., & Campbell, C. (2002). Control and controllability: Beliefs and behaviour in high and low expressed emotion relatives. Psychological Medicine, 32, 1091–1099. 10.1017/S0033291702005779

Hooley, J. M., & Gotlib, I. H. (2000). A diathesis-stress conceptualization of expressed emotion and clinical outcome. Applied & Preventive Psychology, 9, 135–151. 10.1016/S0962-1849(05)80001-0

Hooley, J. M., Orley, J., & Teasdale, J. D. (1986). Levels of expressed emotion and relapse in depressed patients. British Journal of Psychiatry, 148, 642–647. 10.1192/bjp.148.6.642

Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1–55. 10.1080/10705519909540118

Kavanagh, D. J. (1992). Recent developments in expressed emotion and schizophrenia. British Journal of Psychiatry, 160, 601–620. 10.1192/bjp.160.5.601

Kim, E. Y., & Miklowitz, D. J. (2004). Expressed emotion as a predictor of outcome among bipolar patients undergoing family therapy. Journal of Affective Disorders, 82, 343–352.

Leff, J., Kuipers, L., Berkowitz, R., Eberlein-Vries, R., & Sturgeon, D. (1982). A controlled trial of social intervention in the families of schizophrenic patients. British Journal of Psychiatry, 141, 121–134. 10.1192/bjp.141.2.121

Livingston, J. D., & Boyd, J. E. (2010). Correlates and consequences of internalized stigma for people living with mental illness: A systematic review and meta-analysis. Social Science & Medicine, 71, 2150–2161. 10.1016/j.socscimed.2010.09.030

Lysaker, P. H., Davis, L. W., Warman, D. M., Strasburger, A., & Beattie, N. (2007). Stigma, social function and symptoms in schizophrenia and schizoaffective disorder: Associations across 6 months. Psychiatry Research, 149, 89–95. 10.1016/j.psychres.2006.03.007

Mak, W. W., & Cheung, R. Y. (2010). Self-stigma among concealable minorities in Hong Kong: Conceptualization and unified measurement. American Journal of Orthopsychiatry, 80, 267–281. 10.1111/j.1939-0025.2010.01030.x

Meuwly, N., Bodenmann, G., & Coyne, J. C. (2012). The association between partners’ expressed emotion and depression: Mediated by patients’ dysfunctional attitudes?Journal of Social and Clinical Psychology, 31, 690–706. 10.1521/jscp.2012.31.7.690

Miklowitz, D. J. (2004). The role of family systems in severe and recurrent psychiatric disorders: A developmental psychopathology view. Development and Psychopathology, 16, 667–688. 10.1017/S0954579404004729

Miklowitz, D. J., Goldstein, M. J., Nuechterlein, K. H., Snyder, K. S., & Mintz, J. (1988). Family factors and the course of bipolar affective disorder. Archives of General Psychiatry, 45, 225–231. 10.1001/archpsyc.1988.01800270033004

Mueser, K. T., & McGurk, S. R. (2004, June19). Schizophrenia. Lancet, 363, 2063–2072. 10.1016/S0140-6736(04)16458-1

Muthén, L. K., & Muthén, B. O. (2012). Mplus User’s Guide. Seventh Edition. Los Angeles, CA: Author.

National Institute of Mental Health. (2008). National Institute of Mental Health strategic plan. Bethesda, MD: U.S. Department of Health and Human Services.

Ng, S.-M., & Sun, Y.-S. (2011). Validation of the concise Chinese Level of Expressed Emotion scale. Social Work in Mental Health, 9, 473–484. 10.1080/15332985.2011.570653

Northouse, L. L., Rosset, T., Phillips, L., Mood, D., Schafenacker, A., & Kershaw, T. (2006). Research with families facing cancer: The challenges of accrual and retention. Research in Nursing & Health, 29, 199–211. 10.1002/nur.20128

O’Farrell, T. J., Hooley, J., Fals-Stewart, W., & Cutter, H. S. G. (1998). Expressed emotion and relapse in alcoholic patients. Journal of Consulting and Clinical Psychology, 66, 744–752. 10.1037/0022-006X.66.5.744

Öksüz, E., Karaca, S., Özaltın, G., & Ateş, M. A. (2017). The effects of psychoeducation on the expressed emotion and family functioning of the family members in first-episode schizophrenia. Community Mental Health Journal, 53, 464–473. 10.1007/s10597-017-0086-y

Raune, D., Kuipers, E., & Bebbington, P. E. (2004). Expressed emotion at first-episode psychosis: Investigating a career appraisal model. British Journal of Psychiatry, 184, 321–326. 10.1192/bjp.184.4.321

Renshaw, K. D., Chambless, D. L., & Steketee, G. (2006). The relationship of relatives’ attributions to their expressed emotion and to patients’ improvement in treatment for anxiety disorders. Behavior Therapy, 37, 159–169. 10.1016/j.beth.2005.11.001

Ritsner, M., Kurs, R., Gibel, A., Hirschmann, S., Shinkarenko, E., & Ratner, Y. (2003). Predictors of quality of life in major psychoses: A naturalistic follow-up study. Journal of Clinical Psychiatry, 64, 308–315. 10.4088/JCP.v64n0313

Ritsner, M., Modai, I., Endicott, J., Rivkin, O., Nechamkin, Y., Barak, P., . . .Ponizovsky, A. (2000). Differences in quality of life domains and psychopathologic and psychosocial factors in psychiatric patients. Journal of Clinical Psychiatry, 61, 880–889. 10.4088/JCP.v61n1113

Roe, D., Mashiach-Eizenberg, M., & Corrigan, P. (2012). Confirmatory factor analysis of the brief version of the Recovery Assessment Scale. Journal of Nervous and Mental Disease, 200, 847–851. 10.1097/NMD.0b013e31826b6d4a

Roe, D., Mashiach-Eizenberg, M., & Lysaker, P. H. (2011). The relation between objective and subjective domains of recovery among persons with schizophrenia-related disorders. Schizophrenia Research, 131, 133–138. 10.1016/j.schres.2011.05.023

Schneider, L. C., & Struening, E. L. (1983). SLOF: A behavioral rating scale for assessing the mentally ill. Social Work Research & Abstracts, 19, 9–21. 10.1093/swra/19.3.9

Shrout, P. E., & Bolger, N. (2002). Mediation in experimental and nonexperimental studies: New procedures and recommendations. Psychological Methods, 7, 422–445. 10.1037/1082-989X.7.4.422

Silverstein, S. M., & Bellack, A. S. (2008). A scientific agenda for the concept of recovery as it applies to schizophrenia. Clinical Psychology Review, 28, 1108–1124. 10.1016/j.cpr.2008.03.004

Song, L. Y. (2017). Predictors of personal recovery for persons with psychiatric disabilities: An examination of the unity model of recovery. Psychiatry Research, 250, 185–192. 10.1016/j.psychres.2017.01.088

Stanhope, V., Ingoglia, C., Schmelter, B., & Marcus, S. C. (2013). Impact of person-centered planning and collaborative documentation on treatment adherence. Psychiatric Services, 64, 76–79. 10.1176/appi.ps.201100489

Tomás, E. P., Hurtado, G., Noguer, S., Domènech, C., García, M., López, N., . . .Gallo, P. (2012). Effectiveness of family work interventions on schizophrenia: Evidence from a multicentre study in Catalonia. International Journal of Social Psychiatry, 58, 587–595. 10.1177/0020764011415595

Tse, S., Tsoi, E. W. S., Hamilton, B., O’Hagan, M., Shepherd, G., Slade, M., . . .Petrakis, M. (2016). Uses of strength-based interventions for people with serious mental illness: A critical review. International Journal of Social Psychiatry, 62, 281–291. 10.1177/0020764015623970

Vaughn, C., & Leff, J. (1976). The measurement of expressed emotion in the families of psychiatric patients. British Journal of Clinical Psychology, 15, 157–165. 10.1111/j.2044-8260.1976.tb00021.x

Verplanken, B., Friborg, O., Wang, C. E., Trafimow, D., & Woolf, K. (2007). Mental habits: Metacognitive reflection on negative self-thinking. Journal of Personality and Social Psychology, 92, 526–541. 10.1037/0022-3514.92.3.526

Vitaliano, P. P., Becker, J., Russo, J., Magana-Amato, A., & Maiuro, R. D. (1989). Expressed emotion in spouse caregivers of patients with Alzheimer’s disease. Journal of Applied Social Sciences, 13, 215–250.

Wang, P. S., Gilman, S. E., Guardino, M., Christiana, J. M., Morselli, P. L., Mickelson, K., & Kessler, R. C. (2000). Initiation of and adherence to treatment for mental disorders: Examination of patient advocate group members in 11 countries. Medical Care, 38, 926–936. 10.1097/00005650-200009000-00006

Watson, A. C., Corrigan, P., Larson, J. E., & Sells, M. (2007). Self-stigma in people with mental illness. Schizophrenia Bulletin, 33, 1312–1318. 10.1093/schbul/sbl076

Wolf, E. J., Harrington, K. M., Clark, S. L., & Miller, M. W. (2013). Sample size requirements for structural equation models: An evaluation of power, bias, and solution propriety. Educational and Psychological Measurement, 73, 913–934. 10.1177/0013164413495237