Measuring exposure to bullying and harassment at work: Validity, factor structure and psychometric properties of the Negative Acts Questionnaire-Revised

This study investigates the psychometric properties, factor structure and validity of the revised Negative Acts Questionnaire-Revised (NAQ-R), an instrument designed to measure exposure to bullying in the workplace. By reanalyzing data based on a heterogeneous sample of 5288 UK employees, the results show that the 22-item instrument has a high internal stability, with three underlying factors: personal bullying, work-related bullying and physically intimidating forms of bullying, although the instrument may also be used as a single factor measure. Criterion validity was explored by relating the scores on the NAQ-R to a single-item measure of perceived victimization from bullying, showing high correlations with both the total NAQ-R and scores on the three factors. Targets of bullying scored significantly higher on all 22 items compared to non-targets. The NAQ-R correlated as expected with measures of mental health, psychosocial work environment and leadership, indicating a good construct validity of the instrument. Furthermore, a latent class cluster (LCC) analysis showed that the instrument may be used to differentiate between groups of employees with different levels of exposure to bullying, ranging from infrequent exposure to incivility at work to severe victimization from bullying and harassment. The more commonly used operational criteria can also be used to detect targets of bullying. Hence, the NAQ-R is proposed as a standardized and valid instrument for the measurement of workplace bullying.

Keywords: harassment; victimization; survey; inventory; prevalence; Negative Acts Questionnaire; bullying

Workplace bullying (the persistent exposure to interpersonal aggression and mistreatment from colleagues, superiors or subordinates) is a prevalent problem in contemporary working life, with devastating effects on both targets and organizations (Einarsen, Hoel, Zapf, & Cooper, [13]; Rayner & Keashly, [47]). Much effort has, therefore, been put into the documentation of this serious problem by determining its frequency and risk groups within different occupational groups and organizations (see Zapf, Einarsen, Hoel, & Vartia, [55]), and a number of studies addressing the issue of prevalence of bullying within different countries and different occupational groups have been published (e.g. Mikkelsen & Einarsen, [40]; Niedhammer, David, & Degioanni, [42]). However, Keashly and Harvey ([27]) argue that this field of research has often been influenced by a desire to discover substantive issues at the expense of the development of sound methodology.

A range of self-report inventories and scales measuring exposure to bullying and harassment have been introduced in this field; however, most have been used only in one or a few studies (e.g. Dawn, Cowie, & Ananiadou, [8]), with the Negative Acts Questionnaire (Einarsen & Raknes, [15]) and the Leymann Inventory of Psychological Terror (LIPT; Leymann, [34]) as two notable exceptions. Many studies have also used instruments that are derivations, expansions or shortened versions of other original scales portraying unknown validity and reliability (e.g. Salin, [48]). Some scales are overly long and, consequently, difficult to use in standard organizational surveys. For instance, the WAR-Q (see Keashly & Neumann, [28]) contains 60 items in its original version, while the LIPT (Leymann, [34]) contains 48 items. While some instruments are based on the responses of undergraduate students (e.g. Keashly, Trott, & MacLean, [29]), others, such as the LIPT, are designed as diagnostic tools to identify victims of severe bullying likely to be traumatized from their experience, and hence are less suitable in surveys of the general working population. However, a main problem is that hardly any studies have been published in peer review journals explicitly outlining and describing a proposed instrument while simultaneously scrutinizing its validity and psychometric properties. Our paper addresses these shortcomings, proposing the Negative Acts Questionnaire-Revised as a valid, comprehensive, yet short scale to be used in organizational surveys of exposure to workplace bullying. In this paper we initially describe the theoretical foundation on which the NAQ-R rests, followed by a description of the instrument and its development. Subsequently, we provide empirical evidence on psychometric properties, factor structure and validity of the instrument, before we conclude by pointing out some of the limitations and strengths of the instrument and include its practical applications.

The concept of workplace bullying refers to situations where an employee is persistently exposed to negative and aggressive behaviours at work primarily of a psychological nature (Leymann, [33]), with the effect of humiliating, intimidating, frightening or punishing the target. Many of these single acts may be relatively common in the workplace (Leymann, [33]) and, when occurring in isolation, may be seen as signs of uncivil behaviour or "incivility" at work (Lim & Cortina, [35]). When persistently directed towards the same individual(s) over a longer period of time, they may turn into an extreme source of social stress (Zapf, [53]), capable of causing severe harm. Although the negative and unwanted nature of the behaviour involved is essential to the concept of bullying, the concept's core characteristic is not the nature of the behaviours per se, but rather the persistency of the experience (Einarsen et al., [13]). Thus, the emphasis is as much on the frequency and duration of what is done as it is on what and how it is done.

Hence, bullying constitutes evolving and often escalating hostile workplace relationships rather than discrete and disconnected events and is associated with repetition (frequency), duration (over a period of time) and patterning (of a variety of behaviours involved) as its most salient features (Einarsen et al., [13]). A simple distinction exists between direct actions, such as accusations, verbal abuse and public humiliation, on the one hand, and indirect acts of aggression, such as rumours, gossiping and social isolation, on the other (e.g. O'Moore, Seigne, McGuire, & Smith, [46]). With reference to the workplace setting, a further distinction has been made between work-related behaviours and person-related behaviours (Einarsen, [11]), evident in an early Finnish study (Vartia, [49]) where slander, social isolation and insinuation about someone's mental health may be seen as examples of person-related bullying, whilst giving a person too many, too few or too simple tasks, or persistently criticizing a person or their work, may be associated with work-related bullying. Although primarily concerned with negative behaviour of a psychological nature, studies involving targets of bullying have revealed that physically intimidating acts, and in some cases even physical violence or the threat of violence, form part of a wider repertoire of aggressive acts applied in bullying cases (Leymann, [32]).

Another feature of some definitions of bullying is the imbalance of the power relationships between the parties involved (Niedl, [43]). A pre-existing or evolved imbalance of power between the parties is considered central to the bullying experience, as this may limit targets' ability to retaliate or successfully defend themselves. In many cases the imbalance of power may simply mirror the formal power-structure of the organizational context in which the bullying scenario unfolds, as would be the case when someone is on the receiving end of negative acts from higher up in the organizational hierarchy (Hutchinson, Vickers, Jackson, & Wilkes, [23]). Alternatively, the source of power may be informal and related to factors such as knowledge and experience as well as access to social support (Einarsen et al., [13]). Most importantly, the nature of the bullying experience in terms of its frequency and long-term duration of exposure to negative acts tends to drain the coping resources of the target, thus in itself emphasizing the increasing powerlessness of targets (Leymann, [33]).

The distinction between subjective and objective experience has been essential when considering how best to operationalize the definition of bullying. According to Brodsky ([5]), subjective experience of bullying refers to targets' perceptions of their experience. By contrast, objective experience of bullying has to be validated or verified by third parties or observers, for example by means of peer nomination. However, Björkqvist and colleagues (Björkqvist, Österman, & Hjeltbäck, [4]) argued against the objectivity and, therefore, supposedly neutrality of such an approach, as economic dependency could effectively prevent people from being honest in their assessment. This would be particularly true when asked to assess one's superiors or people in formal positions of power. Nevertheless, according to Niedl ([43]), the "definitional core of bullying at work rests on the subjective perception made by the victim that these repeated acts are hostile, humiliating and intimidating and that they are directed at himself/herself" (p. 49). It follows that it is the subjective experience of the behaviours or the pattern of behaviours that will manifest themselves in mental and physical health problems.

The NAQ-R is based on the previous NAQ (Einarsen & Raknes, [14], [15]; Mikkelsen & Einarsen, [40]). Based on a review of the literature and a series of case studies, the original scale had 23 items describing negative acts of a personal as well as a work-related nature. Although this scale showed high internal consistency (as measured by Cronbach's alpha), containing items with good face validity and with evidence of good construct validity, the scale also had some serious shortcomings. Its items were overly influenced by the perspectives of severely affected targets, developed at a time when most bullying research and debate was confined to the Nordic countries. Its validity was only tested within a limited Scandinavian cultural context. When translated into English, the face validity of some items was questionable, with other items revealing a cultural bias. A further weakness was found in its factor structure, although a two-factor solution associated with work-related and a personal bullying respectively was used in some studies (see Einarsen & Raknes, [15]; Matthiesen & Einarsen, [39]). Hence, a revised scale for use in other national settings was clearly needed.

The NAQ-R was therefore created with the aim of establishing a reliable, valid, comprehensive, yet relatively short scale, tailor-made for use in a variety of occupational settings, and especially adapted to Anglo-American cultures. Items were developed and refined based on the original scale, conceptual reasoning and a focus group study employing 11 focus groups with 61 participants from a variety of UK occupations and positions across organizational hierarchies (see Hoel, Cooper, & Faragher, [20], for an overview). This resulted first in a 29-item new version of the NAQ (see Hoel, Cooper, & Faragher, [20], [21]). Later, on the basis of further analyses, a 22-item version was proposed (see Einarsen & Hoel, [12]; Nielsen et al., [44]). In the present paper we test the 22-item scale, which taps direct and indirect aspects of bullying and contains items that can be construed as work-related bullying, person-related bullying or physical intimidation respectively. All items are written in behavioural terms with no reference to the terms "bullying" or "harassment," following recommendations by Arvey and Cavanaugh ([1]) in relation to sexual harassment. Although based on self-report, such an approach is considered to provide a more objective estimate of exposure to bullying behaviours than self-labelling approaches, as respondents' need for cognitive and emotional processing of information would be reduced.

In the present study, the instrument's psychometric properties, factor structure and validity were scrutinized by utilizing and partly re-analyzing data from an existing large-scale survey of UK employees that focused on the prevalence, antecedents and consequences of workplace bullying (see also Hoel, [19]; Hoel, Cooper, & Faragher, 2001, 2004). Based on previous research on bullying (Einarsen, Hoel, Zapf, & Cooper, [13]; Rayner & Keashly, [47]; Zapf & Einarsen, [54]), we hypothesized that the NAQ-R would show negative correlations with measures on subjective health and well-being, and negative correlations with perceptions of the quality of the psychosocial work environment, including job satisfaction, commitment and relationships with superiors and colleagues. According to Leymann ([32]), bullying exists in organizations characterized by deficiencies in both work design and leadership and a negative social climate. Following [30] and colleagues (2004), "poor management" may be present in two forms: an active and abusive type of leadership, and an indirect and passive form where managers have abdicated their responsibilities. Where managers avoid taking charge or involving themselves with interpersonal conflicts and tensions, it is argued that there is a breeding ground for bullying (Leymann, [33]). In many cases, targets also report being bullied by their manager or supervisor (Zapf et al., [55]). Hence, we expect that those exposed to bullying will experience their immediate superior as an abusive and tyrannical leader. The NAQ-R should also be, albeit more moderately, associated with raised levels of sick-leave, reduced work performance and increased intention to leave. Furthermore, a strong positive association with subjective feelings of victimization from bullying at work should exist, as the NAQ-R should be able to differentiate clearly between targets and non-targets of bullying. Finally, building on a theoretical notion of bullying as a process and not an "either-or" phenomenon, NAQ-R should be able to identify different groups of respondents having unique experiences of bullying behaviours regarding the nature, frequency and severity of their experience (see also Notelaers, Einarsen, De Witte, & Vermunt, [45]).

In total, 12,350 questionnaires were distributed to employees in 70 organizations within the private, public and voluntary sectors across Great Britain, with a total of just under one million employees. A total of 5288 were returned providing a response rate of 42.8%, ranging from 27% to 57% in the different organizations (see also Hoel, [19]). A total of 2764 (52.4%) were males and 2508 (47.6%) females. The mean age for the total sample was 40.2 years (standard deviation = 9.8) and the median 40 years. A total of 84.9% of respondents worked full-time as opposed to 15.1% part time. Approximately two thirds of respondents had been in their present job for 4 years or more (65.8%), as opposed to 11.8% who took up their current job within the last 12 months.

The version of the NAQ-R tested in this study has 22 items, measuring exposure to bullying within the last 6 months, with the response alternatives: "Never," "Now and then," "Monthly," "Weekly" and "Daily". As the last category was seldom used, we collapsed the two latter categories in the statistical analysis. A single-item measuring self-labelled victimization from bullying during the last 6 months was then included after presenting the respondents with a global definition of bullying (see Einarsen & Skogstad, [16]; Salin, [48]). This was followed by a number of questions regarding the experience of bullying, such as frequency of encounters, duration of experience and who the main perpetrators were, etc. Participants were then given six alternatives: "no," "yes, very rarely," "yes, now and then," "yes, several times per month," "yes, several times per week" and "yes, almost daily." By being provided with a relatively unambiguous definition of bullying, respondents' tendency to use their own definitions when considering this question should be reduced.

Respondents were asked to state how many days they had been off work due to their own illness within the last 6 months, given the following options: no days off, 1–3 days, 4–10 days, 11–20 days and more than 20 days. For the variable "intention to leave," respondents were provided with five options (never, rarely, sometimes, quite often and very often) in response to the question: "Have you considered quitting your present job over the last 6 months?" Respondents were also asked to rate their present performance or productivity given five different options varying from 100% to less than 50% of normal capacity.

Mental health was measured by the 12-item version of the General Health Questionnaire (GHQ; Grayson, Bridges, Duncanjones, & Goldberg, [18]). Internal consistency as measured by Cronbach's alpha was 0.92 for these 12 items. Psychosomatic health complaints were measured by means of a 12-item scale taken from the Occupational Stress Indicator (Cooper, Sloan, & Marshall, [7]), with a high internal consistency in the present study (Cronbachs's alpha=.89).

Five subscales from the Pressure Management Indicator (PMI) were employed (Williams & Cooper, [52]): "Workload" included six items measuring the amount of work as well as possible problems encountered in the work situation (Cronbach's alpha=.86); "Relationships with colleagues" included eight items focusing on how well one interacts with colleagues and the support one may receive from them (Cronbach's alpha=.89); "Organizational climate" included four items measuring the "feel" or "atmosphere" within the workplace (Cronbach's alpha=.84); "Organizational satisfaction" included six items (Cronbachs's alpha=.88); while "Organizational commitment" measured the level of commitment a person feels to their organization and the contribution work makes to their quality of life, with five items (Cronbach's alpha=.78).

Destructive leadership by one's immediate superior was measured with two scales devised for the purpose of this study (see Hoel, [19], for more details). "Autocratic leadership," where there is no room for employee involvement in the decision-making process and where such involvement is considered unnecessary, was measured by five items taken from item pools produced by the focus group study (Cronbach's alpha=.76). Examples of the items were: "Keeps information to him/herself" and "Insists that a task should be undertaken in a particular way whether this is necessary or not." "Laissez-faire leadership" refers to indifference and lack of involvement in tasks as well as lack of interest in employees and was measured with three items (Cronbach's alpha=.77), e.g. "Turns a blind eye to conflicts and disputes among staff"; "Is never around to deal with problems when they arise."

In order to examine the dimensionality of the NAQ-R, a confirmatory approach employing LISREL 8.80 (Jöreskog, Sörbom, & du Toit, [26]) was conducted. Based on earlier research and theoretical notions we expected three underlying factors of bullying to exist (here referred to as sub-factors), tapping person-related bullying, work-related bullying and physical intimidation. Consequently, different measurement models were defined and tested. As the response categories of the NAQ-R strictly speaking are not interval measures, but rather may be conceived as ordinal data, weighted least squares estimators in LISREL 8.80 were used. Hence, next to the polychoric correlation matrix, an asymptotic covariance matrix was estimated to "correct" for non–normality. A non-significant or small chi-square value indicates that the model fits the data well. However, in large samples even small and substantively unimportant differences between the estimated model and the "true" underlying model will result in rejection of the model that is tested (Bentler & Chou, [3]). Therefore, we also considered other indices in judging the fit of our models. The root-mean square error of approximation or the RMSEA has been put forward as a measure for approximate fit (Jöreskog, [24]). A non-significant RMSEA indicates close fit. In addition we used more descriptive fit measures like the goodness-of-fit index (GFI), based on a ratio of the squared discrepancies to the observed variances (Jöreskog & Sorbom, [25]), the non-normed fit index (NNFI) which represents the increase in fit when comparing any hierarchical step-up comparison of two models (Bentler & Bonett, [2]), and the comparative fit index (CFI) levels of.90 or better for CFI, GFI and NNFI (Byrne, [6]).

A latent class cluster analysis (LCC) (Magidson & Vermunt, [37], [38]) using Latent GOLD (Vermunt & Magidson, [51]) was conducted in order to investigate whether different homogeneous groups of respondents exist, and which might differ according to the nature and degree of their exposure to bullying (Notelaers et al., [45]). LCC allows us to test empirically the number of different target groups as well as the prevalence of each group. This method takes the nature, frequency and severity of the different items into account (Vermunt, [50]), thus creating a more nuanced picture of what is measured, thereby facilitating the understanding of high correlations between the different dimensions (Notelaers et al., [45]). As with other cluster method, LCC classifies respondents into mutually exclusive groups, irrespective of the dimensionality of the measurement. Unlike other cluster methods, LCC does not rely on the strict assumption that responses are measured on an interval level and that data are normally distributed (Eid, Langeheine, & Diener, [10]). Hence, LCC can handle measures of bullying that necessarily produce highly skewed results with response alternatives, strictly speaking, being at an ordinal rather than an interval level. LCC classifies respondents into mutually exclusive groups with respect to a not directly observed (latent) trait (e.g. being a target of bullying), starting with the assumption that there is only one group, subsequently estimating more classes of respondents until a LCC model is found that statistically fits the data (Magidson & Vermunt, [37], [38]). To determine how many clusters are needed in order to explain the associations in the multiway table, the Bayesian Information Criterion (BIC) is used. In line with Magidson and Vermunt ([38]), the model with the lowest BIC was accepted. Next to BIC, and equally important, is the question whether the model fits to the data: given L2 with a given number of degrees of freedom and a p-value that has to exceed a value of.01 in large samples. However, for very sparse tables, it is likely that L2 does not follow a χ2 distribution. Therefore, Langeheine, Pannenkoek, and VandePol ([31]) suggest a bootstrapping procedure. Even though response categories have been recoded, such a bootstrapping procedure is still needed because bullying items are necessarily highly skewed and many combinations of behaviours are still very uncommon, which results in very sparse tables. Finally, the initial bivariate associations between the indicators should be sufficiently explained by the latent class model (Magidson & Vermunt, [38]).

Cronbach's alpha for the 22 items in the NAQ-R was.90, indicating excellent internal consistency whilst also suggesting that it may be a reliable instrument with an even fewer number of items. Yet, the "alpha if item deleted" analysis indicated that internal consistency would not improve if any of the 22 items were deleted. To test the proposed underlying dimensions of the NAQ-R we followed a confirmatory approach with three distinguishable measurement models. These were a one-dimension model with all items measuring the same latent variable, i.e. workplace bullying, a two-dimension model with items loading exclusively on either a person-related bullying dimension or a work-related bullying dimension, and a three-dimension model including physical intimidation with three items loading on this factor. The analysis showed that even though an overall workplace bullying measure (one dimension) fits the data (RMSEA=.051, p>.05; WLSχ2=3003.2 (209) p<.000) and is associated with acceptable descriptive fit measures, the two-dimension solution fits the data better since it is associated with a significantly decreasing χ2 (RMSEA=.051, p>.05; WLSχ2=2899.5 (208) p<.001). The three-dimension structure, however, that additionally distinguishes physically intimidating acts, has the best fit (RMSEA=.049, p>.05; WLSχ2=2741.4 (206) p<.001). The more descriptive statistics CFI (.91 for model 1 and.92 for models 2 and 3), NNFI (.91 for all models) and GFI (.98 for all models) also indicated that the fit is satisfactory for all three models. Table 1 presents the 22 items of the NAQ-R and their factor leadings on their respective sub-factor, along the correlations among the three sub-factors.

Table 1. Items, factors loadings and correlations among sub-factors of the NAQ-R.

All factor loadings exceed.70, with no cross loadings or error correlations. Yet, although the three-factor solution is associated with the best fit, the correlations between the factors or dimensions are very high:.96 between person-related and work-related bullying,.89 between work-related and physically intimidating bullying and.83 between person-related and physically intimidating bullying. Hence, several underlying dimensions of reported workplace bullying can be distinguished, yet they do not discriminate well between different types of bullying behaviours, suggesting co-occurrence of these different types of bullying.

Latent class cluster (LC) was computed using Latent Gold to investigate whether the NAQ-R may be used to identify and differentiate between groups of respondents sharing a certain probability regarding the nature and frequency of their exposure to bullying at work (see also Notelaers et al., [45]). As opposed to the factor models which identify one or more latent variables on a continuum which is then translated into a sum-score or a factor-score to be used for investigating relationships with other variables, this method estimates homogeneous groups of respondents regarding the nature and degree of their exposure to bullying. This method, therefore, allows us empirically to test the number and nature of different target groups as well as the prevalence of each group.

The fit statistics from the LCC-analysis showed that the Bayesian Information Criterion (BIC) dropped when adding an additional cluster up to the eighth cluster model. (For more details on the results of this analysis, please contact the first author.) BIC increased when a nine cluster solution was estimated. However, the difference in BIC between a seven and an eight cluster model (in which the latter solution only adds a new group that is not exposed to bullying) is much smaller than the differences between the other models. Next, the bivariate residual analysis of the seven and the eight cluster models showed that the association between items 3 and 4 was still not accounted for by the number of clusters. Since the association between items four and three was also larger than three in the eighth cluster model, the parsimonious principle urges us to free this association in the seventh cluster model. The BIC of a seven cluster model with an ('error') correlation between the third and the fourth indicator, was the lowest. Equally important, the seven cluster model fits the data well: next to L2 also χ2 that was bootstrapped is not significant (p=.25). Additionally, all bivariate residuals showed that the initial associations between the indicators in the one cluster model were sufficiently explained. Hence, we concluded that a model with seven clusters fits the data best.

The meaning of the seven clusters can be derived from the latent profile output (cf. Vermunt & Magidson, [51]). Table 2 summarizes this profile output. The first row denotes the size of the clusters. The other cells in the body of the table refer to the average conditional probability (CP) to respond "never," "now and then," "once a month" or "once a week/daily" to any of the 22 negative acts of the NAQ-R.

Table 2. The seven clusters, their prevalence and their average conditional probabilities across items (ranging from 0 to 1) for each response category.

Respondents from the first cluster were characterized by an average CP of.94 to report never having been subjected to any kind of negative acts during the last 6 months. Because of the high average conditional probability that these employees answer "never" having encountered any kind of negative acts measured by the NAQ-R, this cluster was labelled as a "No bullying" cluster. Twenty eight percent of the respondents belonged to this cluster.

The respondents of the second cluster are also characterized by a high probability to answer "Never." This time, however, the average CP dropped to.74. A closer inspection of the profile output showed some exposure to work-related negative acts of a frequency of "now and then" in this cluster. Other types of negative acts were hardly reported. Since this clusters reports relatively few negative behaviours and since those belonging to the cluster report limited exposure to almost exclusively work-related negative acts, we conclude that this cluster was not bullied, although they may be exposed to criticism of their work. This cluster is the second largest cluster and covers about 25% of the respondents in this sample, and was labelled "Some work-related criticism."

The average probability structure over items for the respondents in the third cluster was quite similar to that of the respondents in the second cluster, with work-related negative acts not reported more frequently than in the second cluster. Yet, in this cluster some person-related acts were reported more frequently, corresponding to the answer "now and then." However, some types of negative acts, e.g. social isolation and physical intimidation, are "never" being reported. Again we conclude that these employees, constituting some 15% of the sample, are probably not bullied. Yet, one may still say that they are experiencing some "occasional negative encounters."

The fourth cluster was characterized by an average CP over items of almost.50. In this cluster, participants reported having been subjected to a given negative act while at work "now and then." At the same time, the CP to report more frequent exposure was only.13. The dominant feature of this cluster was that, compared to the former clusters, the average probability to report exposure to other negative acts "now and then" concerning social isolation and other person-related bullying behaviour as well as physically intimidating behaviour doubles or even triples or quadruples. Yet, the CP of the reported negative acts referring to social isolation and physical intimidation remained relatively small. In sum, respondents in this cluster reported occasional but still clearly systematic exposure to workplace bullying, although most acts happened typically "now and them." Therefore, we label these respondents as "Occasional bullying." About 13% of the respondents belonged to this cluster.

The fifth cluster was not really characterized by its average probability structure. Although this cluster reported on average less workplace bullying (average CP for the "never" category equals.62), it is rather characterized by being more frequently exposed to work-related types of bullying. The CP to report being monthly or more frequently confronted with work-related bullying acts was particularly high, with the CP to be exposed on a "weekly" basis to work-related bullying being.24. Hence we label this cluster as the "Work-related bullying" cluster.

The sixth cluster was characterized by the highest average CP (.34) in that the respondents had been subjected to bullying behaviours on a weekly or more frequent basis. This is contrasted by the low average CP for the response alternative "never" (.22). Compared to the occasionally bullied cluster and the work-related bullied cluster, the CP for reporting exposure to bullying on a "monthly" or more frequent basis increased substantially. For work-related negative acts the CP doubled, whilst for personal related and/or physical intimidation, the CP increased nearly tenfold. As respondents in this cluster were clearly exposed to intense bullying, we label this as the "Severe bullying" cluster, comprising about 5% of the sample.

The seventh and last cluster showed a high average CP over items to report "never" to have been confronted with bullying behaviours. However, a closer inspection of the profile output shows that the CP to report to be exposed to some specific kinds of negative acts "weekly/daily" is much higher than among any of the "Severe bullying," the "Occasional bullied" and the "Work-related bullying" clusters. Looking at the nature of these specific negative acts, we conclude that these respondents are targets of persistent physical intimidation. We, therefore, label this cluster, which represents 3% of the respondents, as the "Physical intimidation" cluster.

Table 3 shows Pearson product–moment correlations between the total NAQ-R score and the three factors, on the one hand, and measures of health, sickness absenteeism, work performance and intention to leave, on the other. To correct for any skewed scales, we also calculated non-parametric correlations employing Spearman rank order correlations for all correlations in Tables 3 and 4. These correlations were either identical to or stronger than those reported except for sick leave, where the results provided.01 to.02 lower correlations with the NAQ-scales.

Table 3. Pearson product-moment correlations between total NAQ-R, the three sub-factors and measures of health, sick-leave, self-rated work performance and having considered leaving one's job during the last 6 months. Means and standard deviations for all scales.

Table 4. Pearson product-moment correlations between NAQ-R and its sub-factors and scales measuring: leadership style; work pressure from workload; climate and relationships with colleagues; and organizational satisfaction, commitment and security. Means and standard deviations for all scales.

Correlations between total NAQ-R and GHQ-12 and psychosomatic complaints respectively were statistically significant and moderately strong. Correlations between exposure to bullying and self-rated work performance, sick leave and intention to leave were also significant, yet moderate to weak, and in the expected direction. High scores on the NAQ-R were associated with more health complaints, reduced performance, raised sickness absenteeism and higher scores on greater inclination to leave one's job. As far as the three factors of the NAQ-R are concerned, the strongest correlations are found for work-related bullying (NAQ-R-work), with the weakest correlations found for physical intimidation (NAQ-R-Physical Intimidation).

As shown in Table 4, significant and fairly strong correlations exist between the scores on the NAQ-R and measures of work environment quality and ratings of the leadership style of one's immediate superior. All correlations are in the expected direction, indicating that respondents with high scores on the NAQ-R tended to rate their immediate superior as being high on autocratic leadership and/or high on laissez-faire leadership. High scores on NAQ-R were also related to high workload pressure, a negative organizational climate, negative relationships with colleagues, low scores on organizational satisfaction and low scores on commitment. It is worth noting that the strongest relationships were found between bullying and autocratic leadership, and similarly between bullying and experiencing a negative relationship with one's colleagues, while the weakest relationships were revealed as being between the NAQ-R and organizational commitment. Hence, reporting exposure to behaviours measured by the NAQ-R was more closely associated with relationships with colleagues and leaders than with one's relationship with the organization as such.

In total, 10.6% of respondents reported being a victim of workplace bullying during the last 6 months. Pearson's product–moment correlation between the total score on the NAQ-R and the perception of being a victim of workplace bullying was strong (r =.54, p <.001), indicating that having high scores on the NAQ-R was strongly associated with labelling oneself as a victim of bullying, yet also indicating that self-labelling may be related to factors other than mere exposure to bullying behaviours. Analysis of variance using one-way ANOVA revealed that self-reported victims of bullying scored significantly higher than non-victims on all 22 items (p <.001).

Lastly, we investigated the validity of the latent class clusters by relating them to a range of individual and work-related outcomes. Table 5 shows the mean standardized scores (z-scores) on eight outcome variables for all the seven latent clusters. A negative Z-score represents an unfavourable score on organizational commitment and satisfaction. For the other outcome variables, a positive Z-score represents an unfavourable position. After performing an analysis of variance, post-hoc multiple comparisons (pairwise Bonferroni tests) were administered. The one-way analysis of variance showed that there were significant differences among the clusters on all outcomes, although not all groups may be different from each other. The variation between the latent cluster categories regarding the outcome variables varied from about 1 to 1.80 standard deviations between the "severe bullying" cluster and the "not bullied at all" cluster, with the remaining clusters in between. It is worth noticing that the seventh cluster "Physical intimidation" was less problematic compared to the "severe bullying" cluster (see Table 5 for statistical details). Hence, overall, these results indicate good content and discriminatory validity for the latent class cluster solution.

Table 5. Comparisons of the mean scores of the seven latent clusters on eight individual and work-related outcomes using z-values. Identical superscripts (a, b or c) indicate that the pairs of means are not significantly different. Means with no identical superscripts differ significantly from each other (α=.05).

Based on the study presented, we conclude that the 22-item NAQ-R comprises a reliable and valid measure of exposure to workplace bullying, with three inter-related factors associated with person-related bullying, work-related bullying and physically intimidating bullying, respectively. Yet, the NAQ-R may also be used as a one-factor or even as a two-factor measurement of work-related and person-related bullying. Identifying stable and easily interpretable and meaningful factors in comprehensive measures of workplace aggression across samples has been difficult when a full spectrum of abusive behaviours is considered (Einarsen & Raknes, [15]; Keashly & Harvey, [27]), hence the instrument should also be tested in other cultural settings. Yet, the high internal consistency of the scale, as measured by Cronbach's alpha suggested that the number of items could be further reduced if used as a uniform one-factor measure of exposure to bullying at work. Furthermore, the latent class analysis revealed that the NAQ-R discriminates well between different target groups regarding the nature and the severity of their exposure, also revealing target groups that probably face some occasional aggression and incivility, rather than bullying in the strictest sense. The fact that at least 70% of the working population that we studied did not appear to be exposed to bullying, with some 5% of respondents being exposed to severe bullying, and with another 3% experiencing physical abuse or intimidation, also bears witness to the validity of the NAQ-R, as bullying theoretically, logically and empirically is considered to be an infrequent phenomenon, especially in its most severe forms.

Furthermore, six of the emerging clusters were identical to those identified by Notelaers and colleagues in a Belgian sample using the original NAQ scale (Notelaers et al., [45]), which in itself represents a further indication of good validity of the NAQ-R as well as generalizability of these result across cultures. In the present study, a seventh cluster not found in the latter study also emerged, associated with a high probability for exposure to physically intimidating behaviour only. However, this supports the results from the factor structure where this constituted a separate factor.

The latent class analysis is particularly well suited for instruments such as the NAQ-R, as this analysis has the advantage of handling both skewed data as well as data which could be considered as categorical in nature. Highly skewed responses with "never" as the most often chosen response pose a statistical problem in studies on interpersonal aggression in the workplace, and especially so when measuring bullying and harassment which by their very nature should be an infrequent phenomenon. Although this skewedness may create some statistical problems, such a result in itself also emphasizes the validity of the scale as it intends to measure experiences outside or at the outer limits of the normal range of social interaction at work. Therefore, the NAQ-R's items are indicators of inappropriate behaviours that one may experience, but normally should not experience at work, at least not in a regular or systematic way.

The present study builds on self-report data obtained by means of a cross-sectional design, which prevents us from drawing firm conclusions on the cause and effect directions of the observed relationships. Yet, this does not question the fact that NAQ-R showed the expected relationships with other relevant measures. Second, the relationships may be inflated due to common source of variance, thus the observed correlations between NAQ-R and the other related measures may be due to dispositional factors (Keashly et al., [29]). However, we have good reasons to believe that the latter is not the case. First, the strengths of the associations between NAQ-R and the other measures reported in this study varied considerably, from a correlation of.13 for sickness absenteeism to.58 for authoritarian leadership, seriously questioning such an explanation. In addition, the strongest associations were consistently found with other measures of social interaction at work, indicating the discriminatory power of these analyses. Supporting such an interpretation, the NAQ-R's items form separate factors with similar, but still unique patterns of relationships with other variables. Second, a range of earlier studies (some of which have used the original NAQ), have shown that dispositional and attitudinal factors do not explain away relationships between exposure to bullying behaviours, on the one hand, and ratings of health and other critical job-related variables on the other (e.g. Donovan, Drasgow, & Munson, [9]; Mikkelsen & Einarsen, [41]). Third, and as has previously been argued, the NAQ-R contains well specified observable behaviour with low requirements for participants' cognitive and emotional processing when responding, thus reducing the influence of attitudinal and dispositional factors. The use of well specified and observable behaviours in the items should also to some extent reduce the possibility of reporting artefacts in the scale, e.g. of under or over reporting, which may be a problem in sensitive topics such as bullying.

The NAQ-R was developed with the aim of creating a culturally anchored measurement of bullying in Anglo-American cultures. Previous research on bullying or similar concepts indicates a large degree of overlap of empirical findings across cultures (see Einarsen et al., [13]). Still, important cultural differences manifesting themselves in organizational behaviour and practices between different countries do remain, with implications for the meaning of individual instrument items, potentially both in the selection of items as well as in their wording. Consequently, studies must be conducted in order to test the applicability of the NAQ-R across cultural settings. For this purpose, the NAQ-R has recently been translated into a number of languages and is currently being tested in studies both within and outside the Anglo-American world (e.g. Lutgen-Sandvik, Tracy, & Alberts, [36]; Nielsen et al., [44]). Yet further work needs to be done regarding the applicability of the scale in different cultures, necessitating the need for systematic cross-cultural validation studies.

With a total of 22 short and easy-to-read items, thus easing the mental demands on respondents, the technical application of the NAQ-R appears to be fairly simple. As survey length is generally an issue in most organizational studies, the NAQ-R appears to be a comprehensible, yet short and valid instrument. The scale emphasizes experiences within the last 6 months, as recommended by Arvey and Cavanaugh ([1]) in their review of methodological problems in surveys of sexual harassment. Using such a relatively short time-frame ensures the measurement of repeated and on-going experiences, whilst simultaneously making responses less vulnerable to recall problems, memory biases and distortions (Hoel, Rayner, & Cooper, [22]). Measuring exposure to a range of repeated and systematic behaviours happening very often over a long time period of course has the downside of not tapping exactly who did what to whom in which situation, which may be possible in measurements with a focus on infrequent aggression happening over a shorter time period. Yet, this shortcoming may be overcome by also including a self-labelling approach where respondents are presented with a definition of bullying before being asked whether or not they perceive themselves as victims of bullying according to this definition, as was the case in the present study (see also Einarsen & Skogstad, [16]; Nielsen et al., [44]; Salin, [48]). While the approach used in the NAQ-R has the potential of measuring a wide range of experiences where bullying may be seen to exist on a continuum from "not exposed at all" to "highly exposed," the self-labelling approach, treating bullying as an "either-or" phenomenon, provides more conservative estimates of bullying generally limited to the more severe cases of perceived victimization (Zapf et al., [55]). Combining the two approaches, as recommended (Salin, [48]), opens many possibilities. First, following the self-labelling approach, one may include items on who are the main perpetrators, the total duration of the bullying situation, and so on (see also Einarsen & Skogstad, [16]; Mikkelsen & Einarsen, [40]), which the NAQ-R does not provide in itself. Second, combining the two approaches, the responses on the 22 items may be used to validate or qualify subjective claims of being a target by providing examples in behavioural terms of such experiences (Salin, [48]). When employing the 22 items of the NAQ-R in combination with a self-labelling measure the NAQ-R is construed in accordance with the views of researchers of sexual harassment (e.g. Arvey & Cavanaugh, [1]; Fitzgerald & Shullmann, [17]) who argue for separating the two methods in the questionnaire, allowing for participants to respond to the NAQ-R before being asked to label their experience as bullying or not. Thus, a notable strength of the NAQ-R is that it allows for the estimation of the prevalence of bullying without forcing respondents to label themselves as targets.

When estimating the prevalence of bullying from the responses to the 22 NAQ-R items, a method thought to provide a truer estimate of bullying than the self-labelling approach (Notelaers et al., [45]), one may either use a statistical approach employing latent class cluster analysis, as was done in this study, or the more commonly used operational criteria (see also Nielsen et al., [44], for a comparison). The latter approach builds on the work of Leymann ([32], [34]) who defined targets of bullying as those respondents who reported being subjected to at least one negative act specified in the Leymann Inventory of Psychological Terror (LIPT) on at least a weekly basis for a period of 6 months. Mikkelsen and Einarsen ([40]), applying the original NAQ, suggested using regular exposure to two negative acts as a criterion of bullying, first as some acts may be particularly prevalent in certain work settings and, second, as this criterion overlaps to a greater extent with the self-labelling method. A range of studies have used such a cut-off score to differentiate between targets and non-targets of bullying (Einarsen & Raknes, [15]; Lutgen-Sandvik, Tracy, & Alberts, [36]; Niedhammer, David, & Degioanni, [42]; Salin, [48]).Yet, in principle, both these criteria and the self-labelling approach reduce bullying to an "either-or" phenomenon. As bullying is considered to be a gradually escalating process, taking a snapshot of this ongoing process is always a reduction of the very nature of bullying as well as of its multi-dimensionality. The latent class cluster analysis employed in our study shows exposure to bullying to be a multifaceted phenomenon regarding both type and severity of exposure. Therefore, in this respect we will argue that the latent class cluster approach provides a more valid estimation (Nielsen et al., [44]; Notelaers et al., [45]). However, for practical purposes in organizational settings where the application of such an advanced approach is not feasible, the operational criterion approach may well be used.

To strengthen the applicability and practical value of the NAQ-R for researchers and practitioners alike, data from various studies using the scale ought to be collected and norms estimated accordingly. With access to normative data for the general population as well as for particular countries, for particular demographic groups, occupations and industries, the NAQ-R may become a useful tool in the identification of high-risk groups and in benchmarking exercises. With access to a valid, reliable and easy-to-use instrument, the NAQ-R could play an important role in problem identification and measurement of bullying, thus hopefully contributing to identifying, controlling and ultimately reducing the problem of workplace bullying.