The assessment and prevention of aggressive behavior are critical components of contemporary psychiatric inpatient care, treatment, and management. This prospective study compared the predictive validity of three dynamic violence risk assessment measures (i.e. Brøset Violence Checklist (BVC), Dynamic Appraisal of Situational Aggression (DASA), and HCR-20 Clinical scale) for imminent aggression (within the next 24 h). The DASA and BVC were developed specifically to assess imminent violence within psychiatric hospitals, whereas the HCR-20 is a 'general' violence risk assessment measure that can also be used for this purpose. Daily risk ratings were completed for 70 psychiatric inpatients; a total of 3449 ratings for each risk assessment measure were obtained. Results showed that the DASA and BVC were acceptable to outstanding predictive validity and were more accurate than the HCR-20 Clinical scale for predicting inpatient aggression. Actuarial and structured professional ratings were similar for the prediction of verbal threats, but actuarial ratings were more accurate for predicting interpersonal violence. Overall, these findings support the use of structured dynamic risk assessment measures to aid in the prediction of imminent aggression within inpatient psychiatric settings.
Keywords: actuarial; forensic psychiatric patients; gender; mental illness; structured professional judgment; violence risk assessment
In recent years, a number of studies have revealed an association between mental illness and violence; specifically, a small but clinically important relationship exists between psychosis and violence (Douglas, Guy, & Hart, [
Static and dynamic violence risk factors are differentiated in accordance with their relative changeability within specified time frames (Douglas & Skeem, [
Although the prediction of violence is fraught with conceptual difficulties and debates about the accuracy of different approaches (particularly, the relative predictive validity of structured vs. actuarial measures), research studies have overwhelmingly shown that structured violence risk assessment methods are not only more accurate than unstructured clinical judgment but are also preferred because of their increased transparency and reliability (Fazel, Singh, Doll, & Grann, [
In general, these dynamic risk assessment measures have been shown to be predictive of inpatient aggression in the short term (see Chu et al., [
There is also considerable debate about the utility and predictive accuracy of purely actuarial vs. Structured Professional Judgment (SPJ) ratings in the field of violence risk assessment (the latter incorporating clinical judgment based on a review of prescribed empirically and rationally derived risk factors). Although proponents of actuarial methods assert the predictive superiority of their approach and warn against structured methods primarily because these methods introduce (allegedly flawed) clinical judgments, spoiling the empirically derived calculation of risk probability (Quinsey et al., [
The present study sought to extend the comparisons conducted in Ogloff and Daffern's ([
Seventy patients (55 males and 15 females) who were present at the start of the study period (1 June 2002) or admitted into the acute units (two 15-bed male units and one 10-bed female unit) of the Thomas Embling Hospital (TEH) between 1 June 2002 and 31 October 2002 were included in the study. The TEH is a high-security forensic mental health hospital that provides psychiatric assessment and treatment for men and women in Victoria, Australia. More than three quarters of the sample (55/70, 78.6%) were Caucasian; 8.6% (6/70) were Asian, 5.7% (6/70) were of Aboriginal or Torres Straits Islander descent, 5.7% (6/70) were of Middle Eastern descent, and 1.4% (1/70) was Maori. During their hospitalizations at the TEH, the patients received involuntary pharmacological intervention as well as a range of other interventions including psychological, physiological, nursing, and social welfare support.
The BVC is a six-item violence risk assessment instrument that assesses changes in six behaviors (i.e. confusion, irritability, boisterousness, physical threats, verbal threats, and attacks on objects) that commonly precipitate inpatient violence (Almvik et al., [
The DASA is a seven-item violence risk assessment instrument that comprises strictly dynamic violence risk factors (Ogloff & Daffern, [
The HCR-20 is a 20-item structured risk assessment instrument that is used to assess the risk of violence in forensic and psychiatric settings (Webster et al., [
Ethical approval for the study was obtained from Monash University and permission to conduct the study was granted by Forensicare. Unit nursing staff completed the BVC, DASA, and HCR-20 Clinical scale for each patient at 1300 h daily (which was the end of the morning shift). Although inter-rater reliability analyses were not conducted, the second and third author trained the nurses in the use and rating of the BVC, DASA, and HCR-20.[
With regard to the order of coding for the measures/items, the nurses coded the HCR-20 items (with three-point scale) first, then the unique items on the DASA (i.e. sensitive to perceived provocation, easily angered when requests are denied, and unwillingness to follow directions – absence/present rating), and lastly, the BVC (absence/present rating). As mentioned earlier, the DASA has an overlap of two items with the BVC and an overlap of another two items with the HCR-20 Clinical scale. These overlap items were not rated again, rather they were incorporated into the tabulation of the total score for the DASA. For purpose of scoring the DASA, the rating scale for the overlap items on the HCR-20 Clinical scale was converted into a dichotomous scale of absence (0) and presence (
Aggressive behaviors were recorded on a modified version of the Overt Aggression Scale (OAS) (acts of self-harm that were not recorded, as this was not a topic of inquiry in this study and the main goal of this study was to examine violence perpetuated against other persons). Standard hospital untoward incident forms were also reviewed to identify any other acts of aggressive behavior that was not recorded on these untoward incident forms. The risk assessment ratings were then matched with the OAS (Yudofsky, Silver, Jackson, Endicott, & Williams, [
The potential for criterion contamination using this procedure was deemed to be low as the ratings were conducted at the end of the second shift (i.e. 1300 h), and the nurses conducting the ratings were not involved in preparing incident reports for aggressive incidents in at least two of the following three shifts during the follow-up 24 h. These ratings were documented daily and the nursing staff could not change these ratings subsequently even if they continued to work in the next shift. Incident reports were further reviewed by the authors to supplement the nursing staff's daily records of inpatient aggression to ensure that all the aggressive behaviors were recorded accurately. Lastly, the first author coded the patients' sociodemographic information and diagnoses from clinical case files.
The sample was first characterized using descriptive statistics, with categorical data reported as numbers and percentages, and continuous data presented in relation to the mean, median, standard deviation, and range. The predictive validity of the measures was assessed using the AUC of the Receiver Operating Characteristic (ROC); the ROC is less dependent on the base rate of violence than traditional measures of predictive accuracy (Douglas & Webster, [
The AUCs of the ROC Curve, which range from 0 (perfect negative prediction) to 1.0 (perfect positive prediction), are often considered indices of overall predictive validity. As a general rule, AUCs of more than.90 are considered as outstanding discrimination,.80–.89 are excellent, and.70–.79 are considered acceptable (Hosmer & Lemeshow, [
In addition, the AUCs for the different measures were compared using z-tests for dependent groups (Hanley & McNeil, [
The mean age of the overall sample at the point of assessment was 34.33 years (Mdn = 32.00; SD = 12.91; range = 17–83). With regard to the age at the first psychiatric hospitalization, the mean was 30.47 years (SD = 12.42). With its distribution being positively skewed, the median was 27.18 years, and the age at the first psychiatric hospitalization ranged from 13 to 76 years. The mean number of daily assessments conducted for each patient was 49.27 (Mdn = 46.50, SD = 33.47, range = 1–91).
More than a third of the sample (40%; 28/70) had multiple comorbid diagnoses recorded by their respective psychiatrists; notably, 80% (56/70) presented with psychotic disorders, 74.3% had substance abuse/dependence, 20% (14/70) had personality disorders, and 11.4% (8/70) had mood disorders. More than two-thirds (50/70, 71.4%) of the participants had an index violent offense, and more than half (39/70, 55.7%) were convicted of a past violent offense. However, 28.6% (20/70) did not have a past offense history. As shown in Table 1, assault, property damage, and theft/fraud were the most common forms of index offenses. In spite of the high prevalence rate of substance abuse/dependence diagnoses in this sample (52/70, 74.3%), comparatively fewer participants (8/70, 11.4%) were convicted of drug-related offenses.
Table 1. Type of past and index offenses.
Type of offense Past ( Index ( % % Arson 2 2.9 0 0.0 Assault 27 38.6 31 44.3 Breach of court orders 23 32.9 8 11.4 Burglary/criminal trespass 18 25.7 7 10.0 Murder/manslaughter 3 4.3 12 17.1 Possession/use of drugs 18 25.7 8 11.4 Possession/use of weapons 10 14.3 11 15.7 Property damage 17 24.3 13 18.6 Resist arrest 6 8.6 5 7.1 Robbery 10 14.3 8 11.4 Sexual-related 5 7.1 3 4.3 Stalking-related 1 1.4 2 2.9 Theft/fraud-related 34 48.6 18 25.7 Threats to kill 8 11.4 10 14.3 Traffic-related 16 22.9 8 11.4 Others 24 34.3 12 17.1 Note: Many participants had more than one type of offense; therefore, the numbers add up to >70. Although some of the patients were found not guilty by reason of mental illness for their offenses (i.e. the forensic patients), these offenses were included in this tabulation to illustrate the offense characteristics of the source sample. Examples of 'other' offenses included begging for alms, drunk and disorderly behavior, prostitution, and use of indecent language.
A total of 3449 daily risk ratings were completed for each risk assessment measure (i.e. the BVC, DASA, and HCR-20 Clinical scale). Mean scores for the BVC, DASA, and HCR-20 Clinical scale were 0.41 (SD = 1.00), 1.90 (SD = 2.21), and 3.55 (SD = 2.73), respectively. In addition, there were 2485 (72.0%) ratings of low risk on the DASA, 735 (21.3%) ratings of medium risk, and 198 (5.7%) ratings of high risk. Table 2 reveals that all three dynamic risk assessment measures were significant correlated with each other. There were a total of 90 incidents of inpatient aggression: 42 (46.7%) involved interpersonal violence and 73 (81.1%) involved verbal threat; moreover, 22.9% (16/70) of the sample engaged in interpersonal violence and 14.3% (10/70) in verbal threat.
Table 2. Correlation between the acute dynamic violence risk assessment measures.
Measure DASA Total Score BVC Total Score HCR-20 'C' Total Score DASA Total Score – .67∗∗∗ .73∗∗∗ BVC Total Score .67∗∗∗ – .43∗∗∗ HCR-20 'C' Total Score .73∗∗∗ .43∗∗∗ – ∗∗∗
Table 3 shows the predictive validity of the BVC, the DASA, and the HCR-20 Clinical scale for inpatient aggression (24-h follow-up). In general, the DASA and the BVC showed acceptable to excellent (see Hosmer & Lemeshow, [
Table 3. Predictive validity of BVC, DASA, and HCR-20 Clinical scale for inpatient aggression (24 h follow-up).
Inpatient aggression AUC ( 95% CI Any Inpatient Aggression HCR-20 'C' Scale score .68 (.03)∗∗∗ab .62–.73 DASA total score .76 (.03)∗∗∗a .71–.82 BVC total score .77 (.03)∗∗∗b .71–.82 Interpersonal Violence HCR-20 'C' Scale score .72 (.04)∗∗∗cd .65–.79 DASA total score .83 (.03)∗∗∗ce .77–.90 DASA clinical rating .75 (.04)∗∗∗ef .67–.83 BVC total score .83 (.04)∗∗∗df .76–.91 Verbal Threat HCR-20 'C' Scale score .68 (.03)∗∗∗ghi .62–.75 DASA total score .77 (.03)∗∗∗g .71–.83 DASA clinical rating .75 (.03)∗∗∗h .69–.82 BVC total score .77 (.03)∗∗∗i .70–.84 Note: ∗∗
Table 4 shows the classification accuracy indices of the three measures for inpatient aggression. The measures generally have high negative predictive values, but rather low positive predictive values across the range of cut-off scores. However, the low scores on the measures appear to have fairly high sensitivity indices. Table 5 shows the incremental validity for the DASA total score and clinical rating when used together. It was noted that the DASA clinical rating did not add incremental validity when used with the DASA total score to predict interpersonal violence. Moreover, the incremental validity was also mediocre when the DASA clinical rating was used with DASA total score to predict verbal threat.
Table 4. Classification accuracy indices of the cut-off scores for the BVC, DASA, and HCR-20 Clinical scale.
Cut-off Score Any inpatient aggression Interpersonal violence Verbal threat PPV NPV Sens. Spec. PPV NPV Sens. Spec. PPV NPV Sens. Spec. 1 .08 .99 .67 .80 .39 .05 1.00 .79 .80 .29 .07 .99 .68 .80 .38 2 .14 .99 .53 .91 .48 .09 1.00 .69 .91 .32 .11 .99 .52 .91 .49 3 .19 .98 .38 .96 .62 .12 .99 .52 .95 .48 .15 .99 .37 .95 .63 4 .21 .98 .24 .98 .76 .13 .99 .33 .97 .67 .13 .98 .22 .97 .78 5 .20 .98 .12 .99 .88 .12 .99 .12 .99 .88 .19 .98 .14 .99 .86 6 .03 .97 .03 .97 .97 .20 .99 .07 1.00 .93 .13 .98 .03 1.00 .97 1 .04 .99 .87 .42 .59 .02 1.00 .93 .42 .58 .03 .99 .86 .42 .60 2 .05 .99 .77 .59 .47 .02 1.00 .86 .59 .43 .04 .99 .78 .59 .47 3 .06 .99 .73 .69 .41 .03 1.00 .83 .69 .35 .05 .99 .75 .69 .40 4 .07 .99 .69 .76 .39 .04 1.00 .81 .76 .31 .06 .99 .70 .76 .38 5 .08 .99 .61 .76 .43 .05 1.00 .76 .82 .30 .07 .99 .60 .82 .44 6 .12 .98 .46 .91 .55 .07 .99 .57 .91 .44 .10 .99 .48 .91 .53 7 .15 .98 .23 .96 .77 .08 .99 .26 .96 .74 .13 .98 .25 .96 .75 1 .03 .99 .93 .18 .82 .01 1.00 .98 .18 .82 .02 .99 .92 .18 .82 2 .03 .99 .87 .29 .72 .02 1.00 .95 .29 .71 .03 .99 .85 .29 .73 3 .04 .99 .82 .41 .62 .02 1.00 .88 .41 .60 .03 .99 .82 .41 .62 4 .04 .99 .73 .52 .55 .02 1.00 .81 .52 .52 .03 .99 .74 .52 .55 5 .04 .98 .61 .64 .53 .02 .99 .67 .64 .49 .04 .99 .63 .64 .52 6 .05 .98 .51 .76 .55 .03 .99 .57 .76 .49 .05 .99 .55 .76 .51 7 .05 .98 .51 .82 .66 .03 .99 .36 .84 .66 .05 .98 .38 .85 .64 8 .05 .98 .26 .88 .75 .03 .99 .24 .90 .77 .06 .98 .27 .90 .74 9 .07 .98 .13 .95 .87 .03 .99 .12 .95 .88 .07 .98 .15 .95 .85 10 .06 .97 .04 .98 .96 .03 .99 .05 .98 .95 .05 .98 .04 .98 .96
Table 5. Incremental validity of DASA total score and clinical rating for interpersonal violence and verbal threat.
Predictors Wald Exp( 95% CI Δ Step 1 Total Score 0.57 .08 54.66 <.001 1.77 1.52–2.05 – Clinical Rating 1.26 .20 41.85 <.001 3.53 2.41–5.16 – Step 2 Total Score(1) – Clinical Rating(2) (1) 0.52 .09 31.16 <.001 1.68 1.40–2.01 (2) 0.25 .26 0.95 1.29 0.77–2.15 <.01 Clinical Rating(1) − Total Score(2) (1) 0.25 .26 0.95 1.29 0.77–2.15 (2) 0.52 .09 31.16 <.001 1.68 1.40–2.01 .08 Step 1 Total Score 0.46 .05 74.60 <.001 1.58 1.42–1.75 – Clinical Rating 1.43 .15 89.11 <.001 4.17 3.10–5.61 – Step 2 Total Score(1) – Clinical Rating(2) (1) 0.28 .07 15.63 <.001 1.33 1.15–1.53 (2) 0.79 .22 12.76 <.001 2.19 1.43–3.38 .02 Clinical Rating(1) − Total Score(2) (1) 0.79 .22 12.76 <.001 2.19 1.43–3.38 (2) 0.28 .07 15.63 <.001 1.33 1.15–1.53 .02
This study examined the relationship between three dynamic violence risk assessment measures (i.e. the BVC, the DASA, and the HCR-20 Clinical scale), compared their predictive validity within a forensic acute psychiatric inpatient setting, and tested actuarial vs. SPJ ratings for the DASA. An examination of the items in the BVC, the DASA, and the HCR-20 Clinical scale reveals much similarity; the DASA shares two items with the BVC and two with the HCR-20 Clinical scale. Therefore, it is unsurprising that the three measures' total scores were significantly correlated with each other. It is likely that each measures a similar construct, evidenced by the similarity of each measure's predictive validity (in terms of AUC) for the various types of inpatient aggression. Although the nature of this construct/state remains to be uncovered, emotional dysregulation could play a significant role in the perpetuation of violence in these forensic patients (e.g. Newhill, Eack, & Mulvey, [
Regarding actuarial vs. SPJ ratings, the predictive validity of the DASA actuarial rating for interpersonal violence was significantly better than the DASA SPJ ratings, and the former added more unique variance when used with the latter to predict any inpatient aggression. However, it should be noted that the additional unique variance was relatively modest. The difference in the predictive validity for verbal threats was statistically nonsignificant.
Further to the AUC analyses, a review of traditional indices of test accuracy provides important information on how these three instruments may be used in clinical practice. Across most cut-off scores, the sensitivity of the DASA appears slightly better than that of the BVC and HCR-20 Clinical Scale, particularly for interpersonal violence, but also for verbal threats and any inpatient aggression. The very low scores on the HCR-20 Clinical scale were the exception, although this corresponds with the very low specificity scores for these HCR-20 cut-off scores. The BVC showed high specificity scores for all types of aggression for all cut-off scores; for the HCR-20, the specificity scores were weak when low cut-off scores were employed although they increased, following a linear trend, as higher cut-off scores were used. Sensitivity decreased for all instruments as cut-off scores increased; for all scales, this decrease followed a linear trend.
The low PPVs and high sensitivity values associated with the lower cut-off scores across all instruments suggest that any score above the indicated cut-off is itself relatively poor at confirming imminent inpatient aggression (low PPV) and that further investigation must be undertaken (to eliminate the false positives, which in clinical terms may mean unnecessarily restrictive management). This conclusion is consistent with Fazel and colleagues' ([
Our review suggests that risk assessment tools in their current form can only be used to roughly classify individuals at the group level, and not to safely determine criminal prognosis in an individual case. This approach is mostly used in forensic psychiatry in the UK and other western countries, where they form part of a wider clinical assessment process. (p. 5)
Fazel and colleagues' ([
The low PPV and sensitivity is likely due to the fact that this study was conducted in the acute units of a high secure psychiatric setting where high-risk patients could well have been assertively managed by unit staff, rendering the high-risk score inaccurate, when in fact, if the patient had been left alone, they may have behaved aggressively. Though this is an issue with much violence risk assessment research, it is particularly important in inpatient settings where intervention following risk assessment is likely. Further, the low PPVs are likely a consequence of the small number of aggressive episodes in relation to the number of assessments undertaken during the study (2.6%). Finally, it is important to note that each instrument did correctly identify a large proportion of the aggressive incidents (high sensitivity), particularly for low cut-off scores.
Given that the ultimate goal of violence risk assessment is the prevention of violence (Douglas & Kropp, [
Finally, some diagnostic tests employ fixed cut-off scores to provide clinicians with guidance on when to intervene. Focusing on interpersonal violence, because this is the form of aggressive behavior that is of most interest to hospital staff, the present results suggest that acceptable sensitivity and specificity as well as least misclassification can be obtained with a cut-off score of 4/5 for the DASA, 1 for the BVC and 5/6 for the HCR-20 Clinical Scale. Finally, the relatively weak PPV for each instrument suggests that these cut-off scores should be used cautiously. There were certainly occasions when patients were assessed as being 'high-risk' (using these cut-off scores) when they did not proceed to behave aggressively. A high score on any of these instruments should not be used to justify any particular intervention, particularly when it restricts the patient's access to treatment opportunities. These instruments should be used to provide a systematic approach to daily violence risk considerations and to assist staff appraisals.
Inter-rater reliability indices were not obtained for the various measures in this study and this limitation should be addressed in future research. Moreover, we must caution that our sample is relatively small, even though the number of ratings was substantial. We recognized that the sample comprised a small portion of females, but it was not possible to separate the female subgroup for analyses given the small numbers. Future research could address this issue. An additional limitation is that each assessment was considered a separate unit of analysis. This was justified on the basis that patients' mental state changes rapidly within the context of acute symptomatology and environmental (hospital) demands. It must be noted, however, that the daily assessments are not purely discrete (i.e. a patient's behavior on one day may be affected by their experiences on previous days). Replication of this study with larger samples that allow for multi-level modeling analysis will give more confidence to these findings and the generalizability of these results.
Finally, like much violence risk assessment research, the predictive validity of the three measures may have been artificially lowered by hospital staff's identification and diffusion of instances of potentially violent behavior in high-risk state individuals via psychological (e.g. counseling or relaxation), biological (e.g. medication), and/or social (e.g. social or sporting activities) interventions. As part of the hospital's standard operating procedures, these strategies were likely to have reduced the frequency of the inpatient aggression that were exhibited by the participants; therefore, attenuating the predictive accuracy of the risk assessment instruments. Notwithstanding the authors' instructions, the nursing staff might have reacted to the risk assessment ratings and had subsequently implemented some of these strategies to prevent the occurrence of aggressive incidents. Future research should examine the predictive validity of these dynamic measures across gender and age, and with different (e.g. youth, elderly, and nonWestern) samples and in different settings (e.g. civil psychiatric). In addition, we must also identify the core features of the construct/state that contributes to violent behavior in these forensic patients and also whether there are important differences in these features across gender.
Overall, these findings reveal the predictive validity of the DASA, BVC, and HCR-20 Clinical Scale, thereby providing support for their use to assist identification of patients at risk of imminent aggression within forensic psychiatric inpatient settings. The instruments tested here are quick and easy to use; they provide staff with important information about each patient's risk state and propensity for aggression within the coming 24 h, allowing staff to implement biopsychosocial interventions and preventive strategies to avert aggression and to make decisions about care and management that are informed by violence risk state.
None.
By Chi Meng Chu; Michael Daffern and James R.P. Ogloff
Reported by Author; Author; Author