Classroom Psychosocial Environment and Course Experiences in Pre-Service Teacher Education Courses at an Australian University

Research linking university students' perceptions of their classroom environment and course experiences was conducted in one Australian university. A sample of 495 students responded to the College and University Classroom Environment Inventory (CUCEI) and the Course Experience Questionnaire (CEQ). Multilevel regression analyses revealed that several CUCEI scales were significant predictors of CEQ scales. Overall, task orientation was the most potent predictor of all five CEQ scales: clear goals and standards, generic skills, good teaching, appropriate workload and appropriate assessment. Improvements in the classroom environment were linked to more positive course experiences which are being taken as indicators of institutional performance. It is recommended that more attention be paid to classroom environment in colleges and universities.

Keywords: classroom environment; Course Experience Questionnaire; pre-service courses

An issue that is gaining increased attention by university administrators and managers worldwide is how students perceive their experiences at university. While this could be due to a genuine interest in the well-being of students, it is much more likely that this is simply part of the accountability movement sweeping through universities, particularly in western countries. In Australia, this process is in full flight, with the performance of graduating students being quantified and used to compare universities and even departments within universities (see, for example, Department of Education, Science and Training [8]; Graduate Careers Council Australia [29]). Indeed, performance on the Course Experience Questionnaire (CEQ: Ramsden [53]; Wilson, Lizzio, and Ramsden [66]) – an instrument that is administered to all graduating students from all Australian universities – is assuming great importance today to university administrators. Despite well-acknowledged conceptual and measurement limitations (see, for example, Wilson, Lizzio, and Ramsden [66]), the CEQ's bandwagon status remains undiminished.

Another issue that should be of great importance to tertiary educators is the quality of the learning environment that students encounter in universities. This field of research focuses on the psychosocial dimensions of the environment – those aspects that focus on human behaviour in origin or outcome (Boy and Pine [2]). Accordingly, the concept of environment, as applied to educational settings, refers to the atmosphere, ambience, tone, or climate that pervades the particular setting (see Dorman [11]; Fraser [22]). The fundamental question for learning environment researchers is: 'What is it really like for students in this environment?' Importantly, research at all educational levels has shown that the quality of the learning environment is a potent predictor of student cognitive and affective outcomes (see, for example, Fraser [22]). The purpose of this article is to report research linking the field of learning environment research with course experience research. Before reporting the results of this study, a brief review of these two fields is provided.

Reviews of research literature on students' perceptions of their college or university experience have revealed a rapidly-developing research field. Despite these developments, Richardson's [54] review concluded that, while academic staff believe that student feedback in useful and informative, many academic staff and institutions do not take such feedback seriously. Administrative requirements like promotion applications have often been the driving force behind academic staff engaging in course and unit evaluations. Indeed, routine collection of evaluation data does not automatically lead to any improvement in teaching quality (Kember, Leung, and Kwan [36]) and only serious collegial deliberation by teaching staff on the results of evaluations will bring about changes in teaching. Pascarella and Terenzini [51] in the United States, and more recently McInnis [45] and McInnis and James [47] in Australia, have established that social interactions with staff and peers are the key determinants of intellectual and personal development outcomes. Yet, according to McInnis et al. [46], few instruments that purportedly evaluate college and university experiences truly assess these important areas of student life. One study that did assess personal development was reported by Lawless and Richardson [39] and Richardson [55]. They used the CEQ and the Personal and Educational Development Inventory (PEDI) to study experiences of distance education students at the Open University (UK). Correlational analyses revealed that the CEQ and PEDI had significant convergent validity, with over 65% of shared score variance.

One comprehensive instrument that has been widely used in North American research is the Students' Evaluations of Educational Quality (SEEQ: Marsh [43]). It assesses nine dimensions of effective teaching: learning/value, enthusiasm, organisation, group interaction, individual rapport, breadth of coverage, examinations/grading, assignments and workload/difficulty. According to Marsh's original instrument development work, scale internal consistency reliabilities (Cronbach coefficient alpha) ranged from 0.88 to 0.97. Marsh [44] described three older instruments which have well-defined factor structures: Frey, Leonard, and Beatty's ([27]) Endeavour instrument, which assesses presentation clarity, workload, personal attention, class discussion, organisation/planning, grading and student accomplishments; Hildebrand, Wilson, and Dienst's ([32]) Student Description of Teaching, which assesses analytic/synthetic approach, organisation/clarity, instructor–group interaction, instructor–individual interaction and dynamism/enthusiasm; and Warrington's [63] Student Instructional Rating System, which assesses instructor involvement, student interest and performance, student–instructor interaction, course demands and course organisation. While dated, these instruments have sound construct validity and demonstrate that students' evaluations can assess distinct components of teaching effectiveness (Marsh [44]).

As introduced earlier in this article, the focus of the research reported here is the Course Experience Questionnaire (CEQ), which is increasingly being taken as a key performance indicator of universities and departments within Australian universities. The CEQ is designed to assess students' perceptions of five dimensions of course experience: clear goals and standards, generic skills, good teaching, appropriate workload and appropriate assessment. There is also one item that assesses overall course satisfaction. Generic skills and satisfaction have been considered as outcomes in some studies (see, for example, Lizzio, Wilson, and Simons [40]).

Many studies in this field have used instruments in conjunction with the CEQ. Recently, Richardson ([56], [57]) and Price, Richardson, and Jelfs [52] used the Revised Approaches to Studying Inventory (RASI: Entwistle and Tait [17]; Entwistle, Tait, and McCune [18]) and the CEQ to investigate associations between CEQ scale scores and deep, surface and strategic approaches to studying in universities. Richardson's studies found that, whereas all significant correlations between CEQ scale scores and deep and strategic approaches to learning were positive, all correlations between CEQ scale scores and surface approaches to learning were negative. Jelfs and Richardson [34] reported only slight differences in course experiences and approaches to learning and studying by disabled and non-disabled students at the Open University.

A Norwegian study by Diseth et al. [9] developed a structural model in which course experiences predicted student to approaches to studying. More specifically, good teaching predicted deep, strategic and surface approaches; clear goals predicted strategic approaches and appropriate workload predicted deep and surface approaches to studying. In a similar earlier study, Lizzio, Wilson, and Simons [40] modelled deep and surface approaches to learning as possible mediators of the influence of good teaching on cognitive and affective outcomes. Brew and Ginns [3] investigated the relationship between Student Course Experience Questionnaire (SCEQ) results and an 'in-house' scholarship index. The SCEQ is very similar to the CEQ and the scholarship index was designed 'to provide recognition and reward for excellence in teaching and the communication of good practice' (536). Findings revealed a positive relationship between the scholarship of teaching and learning and student course experiences.

The CEQ has been subjected to substantial empirical scrutiny. For example, McInnis et al. [46] validated the items and scales of the CEQ and five additional scales using both Rasch modelling and classical test theory. Curtis and Keeves [6] analysed CEQ data using Rasch modelling and found that 8 of the 25 items did not fit the measurement model. Criticisms concerning the CEQ can be attributed to at least three areas: administration and feedback, methodological limitations and inadequate coverage of the learning experiences in universities. Davies et al. [7] note that the aggregated nature of data and the lag in CEQ feedback being received by institutions makes it difficult for institutions to engage in quality improvement activities. Furthermore, the CEQ is not designed to assess units within courses or the performance of individual academic staff. Yorke [68] and Eley [16] critiqued the design and form of the CEQ, including response style, acquiescence bias, item response formats, and the effect of item order on the questionnaire.

A final criticism of the CEQ is its inadequacy in tapping all relevant dimensions of the learning experiences in colleges and universities. Indeed, the CEQ does not adequately assess the social and psychological contexts in which student learning occurs. If teaching is a multidimensional skill, then obtaining a score on the good teaching scale will not be sufficiently fine-tuned to provide constructive information for teaching staff (Kember and Leung [35]). Another area of importance to tertiary education that is not assessed by the CEQ is the extent to which courses stimulate intellectual debate and challenge. That is, attempts by academic staff to engage students in higher order thinking are not assessed. In essence, within the CEQ, teaching is a technical act in which students are passive recipients and social environments are irrelevant to course experiences. In 1997, Wilson, Lizzio, and Ramsden noted that the quality of facilities (e.g. computing and library) and environment support services (e.g. enrolment advice) are clear omissions of the CEQ. Given the rapid shifts of the past decade towards a consumer-pays approach embedded in mandatory access to technology, this omission is simply unacceptable in any contemporary evaluation of course experiences. The CEQ is based on 1980s theory of curriculum and assessment and its conceptual narrowness is not surprising. What is surprising is the almost universal bandwagon status afforded to it by the Australian tertiary education sector in 2010. For example, the Institute for Teaching and Learning, University of Sydney ([33], 1) states: 'The CEQ has attracted many critics over the years, however it has stood the test of time. Numerous research studies have concluded that the factor scales and the survey items they derive from are valid across repeated administrations to different cohorts'.

Research on the psychosocial dimensions of classroom environments has made substantial progress during the past 40 years (see Dorman [11], [12]; Fraser [22]). This research has focused mainly on the atmosphere or climate that pervades the particular setting. The strong tradition of classroom environment research has been to conceptualise environments in terms of Murray's [49]beta press – the perceptions of the milieu inhabitants (i.e. students and teachers) – with instruments assessing particular dimensions of the environment (e.g. personalisation). Moos's conceptualisation of human environments as having relationship, personal growth, system maintenance and change dimensions has guided research in this field since the late 1960s (see Moos and Trickett [48]). Vivid descriptions and images of schools through powerful movies (e.g. To Sir, with Love) and less powerful dramatisations (e.g. Beverly Hills 90210) all attest to the centrality of the environment to the defining character of schools and classrooms.

Reviews of classroom environment research by Dorman [11] and Fraser [22] and edited books by Khine and Fisher [37] and Fisher and Khine [20] have reported research on the assessment, determinants and outcomes of learning environments. These studies have included comparisons of actual and preferred environments, the effects of determinants or antecedents on classroom environment (e.g. student gender, year, subject and school type), associations between classroom environment and outcomes, transition from primary to secondary school, evaluation of educational innovations, assessment of pre-service teacher education courses, differences between students' and teachers' perceptions of classrooms, and using environment instruments to alter classroom life. From a methodological perspective, articles by Dorman [12] and Dorman and Fraser [14] have demonstrated the effective use of structural equation modelling and multilevel analysis/hierarchical linear modelling when analysing learning environment data.

Studying associations between classroom environment and student outcomes has been the most prolific area of learning environment research. Studies have substantiated the clear link between the quality of the classroom environment and student cognitive and affective outcomes. In China, Wei, den Brok, and Zhou [65] used the Questionnaire on Teacher Interaction (Wubbels and Levy [67]) to establish links between teachers' interpersonal behaviour and student fluency in English in secondary schools. Another Asian study involving secondary school Singaporean students revealed links between the computer-mediated project-based classroom environment and attitudes toward project-based work (Seet and Quek [58]). Madu and Fraser [42] established positive associations between teachers' interpersonal behaviour, learning environments and student outcomes in five secondary schools in New York. Waldrip, Fisher, and Dorman [62] used assessments of the classroom environment to identify exemplary science teachers.

Over the past 40 years, many classroom environment instruments have been developed, validated and used in a range of educational settings. Some of the main instruments include the Classroom Environment Scale (CES: Moos and Trickett [48]), the Learning Environment Inventory (LEI: Fraser, Anderson, and Walberg [23]), the College and University Classroom Environment Inventory (CUCEI: Fraser [21]) and the What Is Happening In this Class (WIHIC: Fraser [22]). Additionally, context-specific instruments have been used in specific settings or for specific purposes. For example, Waxman et al. [64] recently developed a learning environment instrument to evaluate teachers' professional development.

In the 1980s, Fraser and Treagust [25] developed the CUCEI to assess the environment in classrooms in colleges and universities. This research broke new ground as previous university environment research had focused on the institutional-level environment. Fraser and Treagust reported the formulation of seven scales (personalisation, involvement, student cohesiveness, satisfaction, task orientation, innovation and individualisation) with very sound internal consistency reliabilities. Later, Fraser, Giddings, and McRobbie [24] researched the science laboratory environment in universities with a context-specific instrument called the Science Laboratory Environment Inventory. Despite these developments, no substantive research agenda on university classroom environment has been sustained during the past decade.

Some areas of recent classroom environment research include validating and using a modified Arabic translation of the WIHIC in the United Arab Emirates (MacLeod and Fraser [41]), studying the interpersonal behaviour styles of primary education teachers during science lessons (Fisher et al. [19]), using classroom psychosocial environment assessments in technologically-integrated pre-service teacher education courses in Singapore (Khine and Fraser [38]), investigating sports class learning environments (Dowdell, Tomson, and Davies [15]), and studying metacognitive aspects of the classroom environment in chemistry classes (Thomas and Anderson [61]). Classroom environment research has significant overlap with student engagement research (see Fredricks, Blumenfeld, and Paris [26]), and research by Cavanagh and Dellar [4] identified three major classroom environment influences on student engagement: relationships with classmates and the teacher, an orientation towards learning individually, and student confidence.

The research questions for this study were:

• Can scales to assess classroom environment be validated with a sample of pre-service teacher education students in an Australian multicampus university? and
• What is the relationship between students' perceptions of the classroom environment and their Course Experience Questionnaire scores?

The sample used in this study consisted of 495 students from three campuses of one Australian public university. As shown in Table 1, most students in the two primary school preservice Bachelor of Education courses were female. However, this gender imbalance was reversed in the Bachelor of Arts/Bachelor of Teaching secondary school course. The data are hierarchical, with students nested in courses within the university.

Table 1. Description of sample.

Students responded to the Course Experience Questionnaire (CEQ: Ramsden [53]) and a version of the College and University Classroom Environment Inventory (CUCEI: Fraser [21]) The CEQ assesses students' perceptions of five dimensions of course experience: clear goals and standards, generic skills, good teaching, appropriate workload and appropriate assessment. The version of the CEQ used in the present study had 25 items. Each item employs a 5-point Likert response format (strongly disagree = 1, disagree = 2, not sure = 3, agree = 4, strongly agree = 5) with item scores aggregated to form scale scores for each respondent. The CUCEI is an established classroom environment instrument which assesses personalisation, involvement, student cohesiveness, satisfaction, task orientation, innovation and individualisation in university classes. In the present study, five items from each CUCEI scale were chosen. As for the CEQ, a 5-point Likert response format is used with the CUCEI item scores aggregated to form scale scores for each respondent. Table 2 shows a description of each CEQ and CUCEI scale.

Table 2. Descriptive information and validation data for five CEQ and seven CUCEI scales.

Note: As CEQ documents do not explicitly state scale descriptions, these descriptions have been developed from scale items.

To validate the structure of the CUCEI and CEQ scales, exploratory factor analyses and reliability analyses were conducted. Multilevel regression analyses for each CEQ scale as a response variable and the set of classroom environment scales as explanatory variables were performed with MLwiN (Goldstein [28]). The purpose of these analyses was to examine which dimensions of the classroom environment influenced course experiences. Multilevel analyses were conducted because students are nested in courses. Student was the Level 1 variable and course was the Level 2 variable. Backward elimination of non-significant variables was conducted (p < .05). That is, all classroom environment scales were entered into regression equations and non-significant scales were removed. Parameter estimates and effect sizes for all significant explanatory variables were computed. Effect sizes provide an indication of the strength of the relationship between a classroom environment scale and a course experience scale. Correlations between explanatory and response variables were used to calculated effect sizes (ES) using Cohen's [5] formula r2 = d2/(4 + d2) where r is the Pearson correlation and d is the effect size.

A principal components factor analysis with varimax rotation revealed that 22 of the 25 CEQ items loaded on five factors, which accounted for 52.54% of variance in item scores. These factors corresponded to the five a priori scales of the CEQ. Factor loadings ranged from.49 to.78 (M = .66, SD = .08). All items had loadings of less than.30 with the remaining four factors. Despite this sound factor structure, some scales of the CEQ had marginal internal consistency reliability in the present study (see Table 2). For example, the Clear Goals and Standards scale had a Cronbach coefficient alpha of 0.59 – indicating only fair reliability, especially for a scale of such a well-regarded instrument. Table 2 also shows means and standard deviations for these scales.

Results confirmed the psychometric structure of the CUCEI. An exploratory factor analysis with varimax rotation indicated a seven-factor structure corresponding to the seven a priori scales of the CUCEI. The principal components analysis extracted 59.45% of variance in item scores. Factor loadings on a priori scales ranged from.34 to.88 (M = .76, SD = .15). Internal consistency reliability indices (Cronbach coefficient α) ranged from.75 for the Involvement scale to.90 for Student Cohesiveness (see Table 2). These data compare favourably with previous CUCEI validation data reported in Fraser [21] and Fraser and Treagust [25] and provide a sound basis for using CUCEI scales in subsequent analyses.

Variance components models for each of the five CEQ scales indicated that almost all of the variance in CEQ scores was at the student level. In fact, the proportion of total variance accounted for by course membership ranged from 0.00% for the generic skills scale to 2.67% for the good teaching scale. Despite these small proportions, multilevel analysis was the preferred approach to data analysis. Results of analyses conducted with MLwiN revealed significant associations between some dimensions of the classroom environment and students' scores on the CEQ. Table 3 shows these significant explanatory variables, their estimates in these multilevel analyses, and associated effect sizes. Clear Goals and Standards had four significant explanatory variables: personalisation, student cohesiveness, task orientation and individualisation. Effect sizes (ES) shown in Table 3 reveal that task orientation was by far the strongest predictor of clear goals and standards with a very large effect size (1.53). Large effect sizes were evident for personalisation and individualisation (0.77 and 0.78 respectively). As expected, the direction of these three effects was positive, indicating that increased levels of task orientation, personalisation and individualisation were associated with more positive perceptions of clear goals and standards.

Table 3. Results of multilevel models for CEQ scales as response variables and CUCEI scales as explanatory variables.

Note: Only explanatory variables statistically significant at p < .05 are listed.

The most potent predictors of Generic Skills were satisfaction and task orientation (ES = 1.01 and 0.92 respectively). Higher levels of satisfaction and task orientation were associated with improved generic skills. Personalisation was the strongest explanatory variable for the Good Teaching scale (ES = 1.42). Other variables that had a very large effect on good teaching were individualisation, task orientation and satisfaction. All four of these classroom environment variables had a significant positive effect on good teaching. Key explanatory variables of Appropriate Workload were satisfaction, task orientation and individualisation, with moderate effect sizes. Finally, task orientation was the only significant explanatory variable for Appropriate Assessment. A small effect size was evident (0.23). It is particularly noteworthy that task orientation was the only explanatory variable that featured in all multilevel models reported in Table 3. Additionally, the effects of task orientation on three of the five CEQ scales (namely, Clear Goals and Standards, Generic Skills, and Good Teaching) were very large.

Previous research linking classroom environment with performance on the CEQ in Australian universities has not been reported. There are at least three substantive implications to be drawn from this study. First, more positive classroom environments are associated with better CEQ scores. This is a plausible finding which is significant because the CEQ has been criticised for its narrow conceptualisation of course experiences. As noted earlier, the CEQ does not attempt to assess the psychosocial environment that students experience in university classrooms. This environment has been shown to be important to students' success at university (Pascarella and Terenzini [51]). Having shown that environment and CEQ scores are linked, this study provides greater confidence in CEQ scores as reflecting a broader picture of course experiences. This study suggests that university staff should pay closer attention to the quality of the psychosocial learning environment in classrooms if they wish to improve students' CEQ scores. Furthermore, university and faculty leaders should employ much more comprehensive assessment instruments than the CEQ when assessing course experiences. The CEQ should form one component of these assessments.

Second, relationship and personal growth dimensions were the key dimensions of the classroom environment that predicted CEQ scores. University staff need to recognise and respond to the importance of these dimensions. Further research on the interaction between staff and students in university environments is highly desirable. This research could be based on teacher–student interactions research work pioneered in the Netherlands 25 years ago and which has been conducted in many western countries. Much of this research has employed the Questionnaire on Teacher Interaction (QTI: Wubbels and Levy [67]) and its derivatives to provide an assessment of eight dimensions of interpersonal behaviour. These eight dimensions are conceptualised as having components on two orthogonal axes: proximity (cooperation–opposition) and influence (dominance–submission) (see Henderson and Fisher [31]; Madu and Fraser [42]; Telli, den Brok, and Cakiroglu [60]).

Third, task orientation was the key classroom environment scale in this study. It was the only scale that predicted all five CEQ scales. The importance of task orientation in predicting these outcomes scales is similar to findings of much environment–outcome research conducted in schools. Dorman's [10] classroom environment study involving 1055 Grade 8, 10 and 12 students in Australia found that, of 10 classroom environment scales, task orientation was the most potent predictor of student academic efficacy. Similarly, Allen and Fraser's ([1]) study in the United States revealed task orientation to be the only classroom environment scale to predict enjoyment of science lessons and final grade. Zandvliet and Fraser [69] found strong associations between task orientation and satisfaction with a sample of 1404 high school students in Canada and Australia. Of the eight classroom environment scales employed in their study, task orientation was the strongest predictor of satisfaction. In another environment–outcome study conducted by Ogbuehi and Fraser [50] in California, task orientation was the strongest predictor of two attitudinal scales: normality of mathematicians and enjoyment of mathematics. Overall, this present study's finding that task orientation predicts CEQ scores is consistent with much previous research. Clearly, a range of outcome measures (including CEQ scores in the present study) are responsive to changes in classroom task orientation – the extent to which class activities are clear and well organised.

There are four methodological issues relating to the research reported in this article. First, there are limitations to this study based on the student sample. As described earlier in this article, this study involved teacher education students in one Australian public university. As such, its results should not be generalised to all university students. Clearly, further research using a larger sample of students from a wider sample of courses in a sample of universities is needed. Cross-national research would also be desirable.

Second, one of the strengths of this study was the use of multilevel analysis to analyse data. This approach was adopted because students were nested in courses. Historically, classroom environment researchers have used either the individual student as the unit of analysis and ignored class membership or the class as the unit of analysis and ignored intraclass variation in scores. Both of these approaches have been shown to be problematic (see Goldstein [28]; Snijders and Bosker [59]). Dorman [13] demonstrated the effect of clustering on the results of statistical tests. Multilevel analysis in which the hierarchical nature of the data is preserved is the correct approach in most environment studies.

Third, it is clear that the CEQ falls well short of being a perfect measure of students' perceptions of their course experiences. While it has sound construct validity, its scales do not encapsulate course experiences in contemporary universities. Without scales to assess the psychosocial environment, it cannot be an adequate measure of university course experience in any era. An intuitive-rational approach to scale development which takes into account the subjective opinions of the researcher and stakeholders is more appropriate to the development of an authentic course experience questionnaire (see Hase and Goldberg [30]). This contrasts with the factor analytic approach, which relies entirely on the factor structure of the scores after field testing. Additionally, it could be argued that the CUCEI is not a complete measure of psychosocial environment. While both instruments have been used for over 20 years in college and university settings, this does not mean that their validity should be accepted without question.

Finally, this research employed an ex-post facto design with correlational analyses. As such, no causation can be implied. However, the body of correlational studies linking psychosocial environment in classrooms to cognitive and affective outcomes is substantial and this study's results add to our confidence in postulating a causal path from psychosocial environment to outcomes.

Research conducted over the past 40 years has shown the classroom psychosocial environment to be a strong predictor of cognitive and affective outcomes (see Fraser [22]). This study has broken new ground by reporting links between classroom environment and CEQ scores in universities. To date, no Australian research had been conducted in this area. Results suggest that improvements in the classroom environment can not only be beneficial to student learning, but can also improve institutional performance on accountability measures like the CEQ. It is recommended that academic staff in universities give more attention to classroom environment and in particular the relationship and personal growth dimensions of the environment. The findings of this study should be considered tentative, and to enhance the confidence in these associations between environment and external outcome measures, further studies involving a wider sample of students in other colleges and universities and in other countries should be conducted.