Validity of the K-10 in detecting DSM-IV-defined depression and anxiety disorders among HIV-infected individuals

It has been suggested that an HIV diagnosis may increase the likelihood of mental disorders among infected individuals and that the progression of HIV may be hastened by mental disorders like anxiety and depression. Therefore, a brief screening measure, with good sensitivity/specificity for psychiatric diagnoses that could be given to HIV-infected individuals would be useful. We assessed the validity of the K-10, using the MINI International Neuropsychiatric Interview as the gold standard, in a sample of 429 HIV-infected adults enrolled in HIV care and treatment services near Cape Town, South Africa. There was significant agreement between the K-10 and the MINI-defined depressive and anxiety disorders. A receiver operating characteristic (ROC) curve analysis indicated that the K-10 showed agreeable sensitivity and specificity in detecting depression (area under the ROC curve, 0.77), generalized anxiety disorder (0.78), and posttraumatic stress disorder (PTSD) (0.77). The K-10 may be a useful screening measure for detecting mood and anxiety disorders, including PTSD, in patients with HIV/AIDS.

Keywords: M.I.N.I; HIV/AIDS; anxiety; depression; Kessler 10

The adverse impact of HIV/AIDS and mental illness is pervasive and impacts individual quality of life, and country health expenditure and productivity. Despite the high prevalence of both HIV/AIDS and mental illness in shaping world population health, there are limited insights into the interaction between these illnesses in Africa (Myer et al., [17]). It has been suggested that an HIV diagnosis may present a significant stressor and thus increase the likelihood of mental disorders among infected individuals (Nott, Vedhara, & Spickett, [18]; Rabkin et al., [23]). Progression of HIV may also be hastened by mental disorders such as anxiety and depression (Farinpour et al., [7]; Leserman et al., [14]). In addition, research suggests that the burden of mental illnesses among HIV-infected individuals may be significant, predominantly substance, anxiety, and mood disorders, both globally and within South Africa (Demyttenaere et al., [5]; Evans et al., [6]; Lipsitz et al., [15]; Olley, Seedat, Neil, & Stein, 2004; Olley, Zeier, Seedat, & Stein, 2005; Olley et al., [19]). Murray and Lopez ([16]) suggested that mental illnesses among HIV-infected individuals are likely to intensify in coming decades. For example, the Global Burden of disease survey for 2001 estimated that both depression and HIV/AIDS are in the top 10 causes of the global disease burden (UNAIDS/WHO, [27]). In a study assessing common mental disorders among HIV-infected individuals in South Africa, Myer et al. ([17]) reported that the overall prevalence of depression in the previous 12 months, posttraumatic stress disorder (PTSD), and alcohol dependence/abuse was 14%, 5%, and 7%, respectively. Other studies in South Africa assessing mental disorders among HIV-infected individuals have shown similar results. For example, depression and PTSD were the most prevalent disorders at both baseline (34.9 and 14.8%) and six month follow-up (26 and 20%), respectively (Olley, Seedat, & Stein, 2006; Olley et al., [19]). Past depression was 18.1% at baseline and 15.4% at follow-up (Olley et al., [21]). The study conducted by Myer et al. ([17]) utilized brief assessment measures of mental health, which are less resource intensive than structured diagnostic instruments. To date, structured diagnostic instruments that correspond to the classification systems of Diagnostic and Statistical Manual of Mental disorders (DSM-IV) and International Classification of Diseases (ICD)-10 mental disorders have long been the "gold standard" for measuring mental health in clinical and epidemiological studies. While these instruments are able to provide a specific and differential psychiatric diagnosis, they are time consuming and require skill to administer relative to patient self-report scales, which may be a barrier to large-scale epidemiological research (Myer et al., [17]). It is for these reasons that in large international surveys, shorter scales of general mental health have been preferred to diagnostic instruments (Kessler et al., [13]).

The K-10, for example, is a shorter scale of mental health. This brief self-report scale consists of 10 items and is designed to measure the level of distress in population surveys (Furukawa, Kessler, Slade, & Andrews, 2003; Kessler et al., [13]). The measure has been widely implemented, including in the World Health Organization World Mental Health Survey, and as a clinical outcome measure. One of the scale's strengths is its brevity in screening for mood and anxiety disorders (Furukawa et al., [9]; Kessler et al., [13]). Recently, the K-10 was shown to be an excellent screening instrument for mental health disorders in two large-scale studies using structured clinical interviews as the criterion standard (Baillie, [2]; Cairney, Veldhuizen, Wade, Kurdyak, & Streiner, 2007). However, there is a paucity of studies which have validated the K-10 in HIV-infected populations, in low/middle-income countries, and in Africa in particular. Given the high levels of mental health problems in these populations, and the need for less resource intensive screening measures within clinical settings, the K-10 may potentially be a valuable measure to utilize. We therefore examined the K-10 as a brief rating scale of the most common mental disorders (depression and anxiety) among an HIV-infected population, using the MINI as the "gold standard" for clinical validation.

Data were drawn from a validation study of mental health measures in 536 HIV-positive adults seen at the three public sector clinics in South Africa (Myer et al., [17]). The total sample size used for the present study was 429 HIV-infected adults without significant cognitive impairment as determined by the HIV Dementia Scale (HDS) at a cut off score of nine, and the MINI Mental State Exam (MMSE) at a cut off score of 24. Participants were eligible for the study if they tested HIV-positive according to medical records, were between the ages of 18 and 65, and could provide written informed consent.

Participants were assessed for the presence of psychiatric disorders using the MINI International Neuropsychiatric Interview (M.I.N.I). The MINI is a short structured clinical interview which is used to detect mental illness. It is also translated into other languages. The MINI was designed as a brief structured interview across the major Axis I psychiatric disorders in the DSM-III-R, DSM-IV, and ICD-10 based on a standardized algorithm of questioning (Sheehan et al., [24]). The MINI has been validated against the Structured Clinical Interview for DSM-III-R (SCID), Composite International Diagnostic Interview (CIDI), Diagnostic Interview Schedule (DIS), and Present Status Examination (PSE) and has been used as the gold standard in various cross-cultural studies globally, including in HIV-infected patients in South Africa (Olley et al., [20], [22]). The results of studies validating the MINI have shown that the MINI has acceptably high validity and reliability scores (Sheehan et al., [24]).

All participants also completed a battery of self-report measures including the K-10. Receiver operating characteristic (ROC) curve analysis was used to determine the level of agreement between the K-10 and the MINI (mood and anxiety disorder modules).

Data were analyzed using the computer software package STATISTICA (version 8.0). A ROC curve analysis was performed in order to establish the K-10 cut off scores that would be appropriate for this sample. The area under the ROC curve (AUC) is an indicator of a particular scale's diagnostic ability to discriminate between those with and without the target diagnosis (Hanley & McNeil, [12]). Gill, Butterworth, Rodgers, and Mackinnon ([10]) stipulate that the AUC values range from 0.5 to 1.0, where a marker of 0.5 indicates that the scale is performing at a chance level, and 1.0 indicates perfect discrimination. There is no agreed standard for interpreting the significance of the AUC statistics. However, it has been suggested that values between 0.50 and 0.70 represent a scale with low accuracy, values between 0.70 and 0.90 are indicative of a useful screening scale and a value of 0.90 and above is indicative of a highly accurate screening scale with a perfect ability to identify those with the target diagnosis (Fischer, Bachmann, & Jaeschke, [8]; Swets, [26]). First, using the MINI as the "gold standard" for diagnosis of depressive and anxiety disorders (a measure that identifies those individuals who have or do not have a disorder); the sensitivity (true positive probability) and specificity (true negative probability) for various K-10 cut off scores was established. Second, AUC (and 95% confidence intervals) were calculated for each ROC curve using non-parametric methods. In addition, the utility of the measure was assessed with positive and negative likelihood ratios (LR+ and LR −), respectively. LRs are useful to estimate the likelihood that a screening test result (positive or negative) would occur in an individual with a given psychiatric disorder compared to the likelihood that that same result would be expected in a patient without that disorder.

A total of 429 participants with data for the K-10 screening test, HDS and MINI-defined mental illness were included. Three hundred and ten participants (72%) were black and 108 (25%) were of mixed race (Colored). There were 324 (76%) females and 104 (24%) males, of which 288 (67%) were Xhosa speaking, and 106 (25%) were Afrikaans speaking.

A Mann–Whitney U-test was performed to compare medians of the K-10 across categorical variables that is, gender and ethnicity. The results of these tests showed no significant differences between gender (p=0.66) and ethnicity (p=0.21) for the K-10 in detecting MINI-defined depression or anxiety. Moreover, Spearman correlations were calculated between the K-10, age, and education. No significant correlations were found for any of these variables. The Cronbach's α, a measure of internal consistency, found the K-10 to be reliable (α = 0.87).

Analysis of the ROC curve revealed that the K-10 performed adequately as a predictor of current major depressive episodes, with an AUC of 77%, respectively (LR + = 2.9; LR − = 0.3). When compared with past major depressive episodes, the predictive ability of the K-10 scale declined slightly, with an AUC of 75%, respectively (LR + = 2.8; LR − = 0.4). The K-10 also discriminated adequately between cases of mood disorders with psychotic features and non-cases, with an AUC of 84%, respectively (LR + = 4.4; LR − = 0.2).

Moreover, the results showed that the K-10 performed adequately as a predictor of anxiety disorders. The magnitude of the AUC for anxiety disorders was as follows: agoraphobia = 69% (LR + = 1.9; LR − = 0.5), generalized anxiety disorder (GAD) = 78% (LR + = 3.7; LR − = 0.3), social anxiety disorder (SAD) (social phobia) = 90% (LR + = 4.6; LR − = 0.2), and panic disorder = 77% (LR + = 3.1; LR − = 0.3). Furthermore, analysis of the ROC curve showed that the K-10 discriminated between PTSD caseness and non-caseness 77% of the time, respectively (LR + = 3.4; LR − = 0.3).

In examining the sensitivity and specificity values for each possible cut off of MINI-defined depressive and anxiety disorders, a score of ≤28 was chosen as the best screening cut off for both current (sensitivity, 0.67; specificity, 0.77; positive predictive value (PVP), 0.29; negative predictive value (PVN), 0.94) and past depression, respectively (sensitivity, 0.72; specificity, 0.75; PVP, 0.17; PVN, 0.97). In addition, a score of ≤30 was shown as the best screening cut off for mood disorders with psychotic features (sensitivity, 0.9; specificity, 0.79; PVP, 0.1; PVN, 0.99). A cut off of ≥30 was also chosen for GAD (sensitivity, 0.72; specificity, 0.80; PVP, 0.14; PVN, 0.99), and SAD (sensitivity, 0.92; specificity, 0.80; PVP, 0.12; PVN, 0.99). A cut off of ≥29 (sensitivity, 0.75; specificity, 0.78; PVP, 0.15; PVN, 0.99) was selected for PTSD, and a cutoff of ≥28 was identified for panic disorder (sensitivity, 0.76; specificity, 0.73; PVP, 0.10; PVN, 0.99), respectively. Lastly, the cut off score for agoraphobia was ≥26 (sensitivity, 0.65; specificity, 0.67; PVP, 0.08; PVN, 0.97). These cut off scores for anxiety and depressive disorders represent cases of more severe psychological distress (Andrews & Slade, [1]). The test characteristics are shown in Table 1 and the percentages of MINI-defined psychopathology are shown in Table 2.

Table 1. Results of receiver operating characteristic (ROC) curve analysis within 429 treatment seeking HIV-infected individuals.

Table 2. Percentage (%) of MINI-defined psychopathology within 429 treatment seeking HIV-infected individuals, overall and by home language.

We found the K-10 to be a useful diagnostic screening instrument for major depression (one month and 12-month estimates).The measure was able to correctly identify true cases of major depressive episodes one month and 12 months, respectively (67%; 70%), and correctly identify true non-cases 77%; 75% of the time. Furthermore, identification of MINI-defined anxiety disorders ranged from low to very good (0.63–0.92) for agoraphobia, GAD, PTSD, panic disorder, and SAD (social phobia). With the exception of agoraphobia, the sensitivity (0.72–0.92) and specificity (0.74–0.80) for anxiety disorders ranged from good to very good. The present study is one of the few studies assessing the validity of the K-10 in screening for psychopathology among HIV-infected individuals in Africa. In summary, we found that the K-10 was a useful measure of both depression and anxiety disorders. These findings were in keeping with prior international validation studies of the K-10 (e.g., Baillie, [2]; Cairney et al., [4]; Furukawa et al., [9]; Kessler et al., [13]).

A possible limitation of this study was the potential impact of cultural, education, and language factors on responses provided to self-report questions. In addition, gender differences in psychopathology are common, especially in depression with women more likely to report depressive symptoms compared to men (e.g., Hankin et al., [11]; Williams et al., [29]). Therefore, it is important to assess whether the differences in psychopathology are in fact an effect of socio-demographic variables and not a result of biased measurement. In keeping with previous studies (e.g., Baillie, [2]; Kessler et al., [13]), there was no significant difference in validity across gender, age, education, or ethnicity categories. Based on our significant findings, further research is needed to assess the extent to which the K-10 remains a valid measure of psychological distress in other cross-cultural populations.

There were differential prevalences of mental disorders among Afrikaans speakers compared to Xhosa speakers. Across all disorders, prevalence rates of psychopathology were higher for Afrikaans speakers. This may have been due to biases in interviewer conduct, resulting in an over detection in Afrikaans speakers or an under detection in Xhosa speakers. However, despite the fact that this has received little empirical attention in the context of HIV/AIDS, there is some suggestion that the interpretation of psychiatric symptoms may differ by linguistic group (Swartz, [25]). It may have been likely that differences in participant reporting were due to language group. Afrikaans speakers may have been more comfortable in discussing behavioral and emotional symptoms compared to Xhosa speaking individuals, due largely to the social differences in expressing emotions and feeling states (Myer et al., [17]).

The present study was limited to HIV-positive patients attending three public sector clinics in one South African province. However, the socio-demographic characteristics of this study sample are largely reflective of the socio-economic conditions across sub-Saharan Africa, and therefore, constitute an important group for research. In addition, there may have been response bias among the less literate patients interviewed in the present study. To some extent, the latter is inherent in using self-report rating scales in cross-cultural research and may not be overcome. Despite these limitations, the K-10 proved to be a useful screening measure that can be used to detect HIV related mood and anxiety disorders, including PTSD among patients who are HIV-positive. Figures 1, 2 and 3 show the ROC for the diagnoses of Agoraphobia, SAD, and MDD in this sample of HIV-infected individuals.

Graph: Figure 1. Receiver operating characteristic (ROC) curve for the diagnosis of agoraphobia within 429 treatment seeking HIV-infected individuals.

Graph: Figure 2. Receiver operating characteristic (ROC) curve for the diagnosis of social anxiety disorder (SAD) within 429 treatment seeking HIV-infected individuals.

Graph: Figure 3. Receiver operating characteristic (ROC) curve for the diagnosis of major depressive disorder (MDD) within 429 treatment seeking HIV-infected individuals.

In conclusion, the K-10, a screening measure with strong psychometric properties, is also useful in HIV research. There have been few published validation studies of brief assessment measures to explore mental illness in HIV-infected populations, in low/middle income countries, and in Africa in particular. The K-10 has proven to be a useful proxy measure of both anxiety and depression, both of which are common co-morbidities with HIV/AIDS. Moreover, the measure has demonstrated no significant differences in validity across relevant socio-demographic variables. The conciseness and ability to discriminate DSM-IV cases from non-cases make the K-10 an attractive option for use in general health surveys and contemporary psychiatry (Brooks, Beard, & Steel, [3]). The fact that the K-10 can be self-administered or interviewer administered in only 2–3 minutes also increases its favorability over clinician-administered diagnostic interviews. In addition, the K-10 is able to assess the severity of non-specific distress and might therefore be a useful proxy screening measure in clinical studies (Kessler et al., [13]). There are however, shortcomings to the use of the K-10 as a clinical outcome measure, such as its limited breadth of items which do not cover some important clinical domains (e.g., suicidality, major psychoses). Nevertheless, these brief screening tools may still be useful as an initial screening device in clinical practice to identify individuals who are at high risk of psychopathology and may require further evaluation (Myer et al., [17]).

This work is based upon research supported by the MRC Unit on Anxiety Disorders and the South African Research Chairs Initiative of the Department of Science and Technology and National Research Foundation. This research was funded by the Hendrik Vrouwens Scholarship.