Zum Hauptinhalt springen

Investigating the Interaction between Teacher Mathematics Content Knowledge and Curriculum on Instructional Behaviors and Student Achievement

Sutherland, Marah ; Clarke, Ben ; et al.
In: Elementary School Journal, Jg. 123 (2022-12-01), Heft 2, S. 292-317
Online academicJournal

Investigating the Interaction between Teacher Mathematics Content Knowledge and Curriculum on Instructional Behaviors and Student Achievement 

Policy efforts in mathematics have focused on increasing teachers' mathematics content knowledge (MCK), with the goal of increasing teacher quality and in turn increasing student mathematics learning. An alternative approach to increasing student mathematics achievement is to investigate curricula that can be effectively used by teachers with a range of MCK. Drawing from a large-scale study of kindergarten students (n = 2,598) and their teachers (n = 130), the current study investigated the interaction between teacher MCK and curriculum (Early Learning in Mathematics core kindergarten curriculum vs. business-as-usual curricula) on (a) instructional behaviors and (b) student mathematics achievement gains. Results indicated differential significant interactions across instructional behaviors and a small but negative effect of teacher MCK on student mathematics achievement gains. Implications for future research, policy, and practice are discussed.

The importance of a strong foundation in mathematics is unlikely to be disputed by educational researchers, practitioners, and other stakeholders. However, the learning of mathematics is inherently complex, with many students failing to reach proficiency in understanding of whole number concepts in the elementary grades (National Assessment of Educational Progress, [27]). To successfully engage in mathematics, students must be able to participate in abstract reasoning, flexibly manipulate numbers, and apply previously learned strategies to novel concepts (Gersten et al., [13]). Teachers are central to addressing the challenge of students gaining proficiency in early mathematics and developing their mathematical understanding across grade levels.

Given the complexities of learning and teaching mathematics, teacher quality—or teachers' ability to effectively teach mathematics—has been a long-standing target for improving student outcomes in mathematics (e.g., Darling-Hammond & Berry, [7]). One approach from the early 2000s onward to increase teacher quality has been through teacher professional development (PD) programs and teacher training initiatives. A conservative estimate of the amount that K–12 schools spend annually on teacher PD lands in tens of billions of dollars (Kraft et al., [20]). PD initiatives have become not only commonplace but are often mandated by districts (Perez & Kumar, [31]). The vast majority of teacher PD programs in mathematics focus on deepening teachers' understanding of mathematics content as a means to improve teacher quality and lead to increased student learning (Garet et al., [12], [11]; Jacob et al., [19]).

An alternative approach to improve mathematics instruction is targeting the quality of core mathematics curricula and the inclusion of curricular supports to enable effective implementation for teachers across a range of skill sets (Stein et al., [40]). Lines of research have focused on identifying specific curricular features that may help teachers implement programs more effectively (Remillard et al., [33]; Stein & Kaufman, [39]). For example, curricula that provide scripting for teachers may allow for the use of precise and consistent mathematical language during instruction (Remillard & Reinke, [34]).

The current study investigated two targeted areas in mathematics reform (i.e., teachers' mathematics content knowledge [MCK] and quality of core mathematics curricula) and their associations with instructional behaviors and student mathematics achievement. The following section provides an overview of relations between teacher MCK and student mathematics achievement, as well as curricular features associated with greater student mathematics outcomes.

The Role of Teacher MCK in Research and Policy

Teacher MCK has been a construct of interest in the mathematics education literature for several decades and in turn has been defined and studied in a variety of ways. One approach is to measure proxies of teacher MCK, such as teachers' advanced mathematics courses, years of experience teaching, or scores on mathematics skills tests (Mullens et al., [25]). Although reviews of this literature suggest that these variables are associated with student mathematics achievement (Wayne & Youngs, [41]), a common critique is that there may be underlying factors contributing to these associations (Hill et al., [16]).

An alternative approach is to examine teachers' knowledge of how to effectively teach a given academic subject, often described as teachers' pedagogical content knowledge (Shulman, [37]). In mathematics, this can be conceptualized through teachers' use of mathematical representations to build conceptual understanding, responsiveness to student questions and errors, accurate analysis of student work, and adaption of instruction to support learning (Hill et al., [16]). Initial forays into examining associations between pedagogical MCK and student mathematics achievement have demonstrated that students of teachers with higher MCK tend to make greater gains across the course of the school year (Hill et al., [16]; Metzler & Woessmann, [24]).

Mathematical Knowledge for Teaching

Perhaps the most widely used assessment to measure teachers' MCK is the Mathematical Knowledge for Teaching (MKT; Hill et al., [17]), administered in a range of studies (e.g., Agodini & Harris, [1]; Campbell et al., [3]; Hill et al., [16]; Jacob et al., [19]; Stein & Kaufman, [39]). This measure is a multiple-choice teacher survey, designed to tap into both pedagogical MCK and more general mathematics knowledge. For example, on one assessment item, teachers are asked to examine hypothetical student work samples and identify which ones indicate a similar mathematical error for adding one- and two-digit numbers. On another item, teachers are asked to select the most instructionally relevant list of decimals to use during instruction to help students learn how to correctly order decimals from smallest to largest. Other items on the MKT assess for mathematics subject matter knowledge that would be known to someone with a background in mathematics but are not specific to teaching mathematics.

In the current study, we conceptualized MCK as the knowledge needed for effectively teaching mathematics in the elementary grades. We used the MKT measure as our proxy for MCK, which included items targeting pedagogical content knowledge (e.g., knowledge needed for effectively teaching mathematics) and more general content knowledge. Throughout this article, we use the term "MCK" to describe the construct that we were interested in studying, whereas the specific measure is referred to as "MKT."

Associations between Teacher MCK and Student Achievement

Examinations of teacher MCK have revealed a number of critical findings. Above and beyond other variables such as mathematics courses taken or teaching experience, teacher MCK significantly predicts student mathematics achievement gains across the academic year for students in the early and late elementary grades (Campbell et al., [3]; Hill et al., [16]). Researchers have also examined whether the relation between MCK and student mathematics achievement varies based on different levels of teacher MCK. Hill et al. ([16]) found that elementary students of teachers who scored in the lower third of the distribution of MCK made significantly less progress in mathematics compared with peers with more knowledgeable teachers; however, there was no association for teachers scoring above this threshold. Other researchers have found that differences in student outcomes occur when comparing teachers scoring at the mean with teachers scoring one standard deviation above the mean (Agodini & Harris, [1]).

Outcomes of Content-Focused Mathematics PD Evaluations

Given associations between teacher MCK and student mathematics achievement, systematic efforts have been carried out to investigate the impact of content-focused teacher PD programs on teachers' MCK, instructional behaviors, and student mathematics achievement (Garet et al., [12], [11]; Jacob et al., [19]; Sample McMeeking et al., [36]). These PD initiatives have spanned grade level and mathematical content, typically lasting for multiple days within a year, with some programs extending across multiple years. For example, Garet et al. ([12]) implemented a 2-year, 114-hour teacher PD program focused on building teachers' conceptual knowledge of rational numbers, including having teachers solve problems, explain concepts and procedures, and use visual representations during instruction. Garet et al. ([11]) investigated an 80-hour teacher PD program, Intel Math, focused on building teachers' knowledge of K–8 mathematics. Jacob et al. ([19]) implemented a weeklong summer PD program with follow-up in-service days for fourth- and fifth-grade teachers focused on building their MCK.

The majority of these programs led to increases in teacher MCK compared with a business-as-usual (BAU) control condition (for an exception, see Garet et al., [12]). In addition, studies that included a comparison of teachers' instructional behaviors following the PD found several improvements, such as teachers eliciting student thinking more frequently (Garet et al., [12]) and using a greater quantity and quality of mathematical explanations (Garet et al., [11]). Despite most content-focused teacher PD programs increasing teacher MCK and resulting in some improved instructional behaviors, the evidence that these PD programs effect change at the student level is extremely limited. For example, out of 32 mathematics PD evaluations with an acceptable research design, Gersten et al. ([14]) found that only two studies demonstrated a positive impact on student mathematics achievement. Of these two programs, only one focused on building teachers' MCK (i.e., Sample McMeeking et al., [36]), with the other study focused on teacher collaboration through lesson study groups.

The Role of High-Quality Curricula

Although limited research indicates that intervening on teacher MCK increases student mathematics achievement, the use of high-quality core curricula has a higher level of evidence for increasing mathematics achievement. For example, Agodini et al. ([2]) led a large-scale evaluation of four commonly used first-grade core curricula (Investigations, Math Expressions, Saxon, and Scott Foresman-Addison Wesley Mathematics [SFAW]) aligned with different pedagogical approaches. After randomly assigning schools within participating districts to one of four curricula, the researchers found that two of the curricula (Math Expressions and Saxon) resulted in greater student mathematics achievement gains across the school year compared with the other curricula.

Though teachers' use and implementation of curricula can vary, high-quality curricula may help teachers, even those with limited training in mathematics, employ effective instructional techniques (Stein et al., [40]). For example, curricula that provide guidance about how to enact lessons, intended use of materials, and specific language to use during instruction are associated with greater gains in student mathematics achievement compared with curricula that provide minimal guidance on teacher language and actions (Remillard et al., [33]). These more effective curricula are often fully or partially scripted, providing teachers with precise and consistent mathematical language to use across lessons (Remillard & Reinke, [34]).

MCK, Curriculum, and Student Mathematics Achievement

Curricula that provide a high level of instructional support for teachers, such as programs that include specific teacher language or explanations of how to effectively use mathematical representations, may reduce the impact of MCK on teaching behaviors and student outcomes. To date, two studies have investigated relations among teacher MCK and curriculum. The first study, conducted by Stein and Kaufman ([39]), examined two widely used mathematics curricula, Investigations and Everyday Mathematics. Upon examination of teacher supports in each curriculum, the researchers determined that Investigations provided greater implementation support due to it providing a rationale for how specific mathematics tasks should be taught and providing anticipation for how students might respond to mathematics tasks. The researchers conducted observations in two large districts that had each newly adopted one of the two programs and found that teachers using Investigations had a higher quality of implementation overall. Using the MKT, the researchers also examined correlations between teacher MCK and implementation quality. They found that teacher MCK was significantly associated with implementation quality for teachers using Everyday Mathematics but did not find significant associations for teachers using Investigations. They hypothesized this was due to Investigations providing a higher degree of teacher support, resulting in teachers' MCK being less related to their implementation of the curriculum.

The second study used data collected from the large-scale evaluation of four early elementary mathematics curricula (Investigations, Math Expressions, Saxon, and SFAW; Agodini et al., [2]). Agodini and Harris ([1]) investigated whether teachers' scores on the MKT moderated curriculum effects, comparing two curricula at a time. The researchers used hierarchical linear models (HLMs), with student mathematics achievement as the outcome variable, to account for nesting of the data. To detect moderation, the researchers examined curriculum effects for teachers at the average level of MCK and for teachers scoring one standard deviation above the mean. The researchers found that teacher MCK significantly moderated curriculum effects when comparing the two curricula previously demonstrated to produce the greatest gains in student mathematics achievement (Saxon and Math Expressions) to Investigations. Specifically, the effect of Investigations was similar to Saxon and Math Expressions for teachers with average MCK, but Investigations was less effective than the other programs for teachers with MCK one standard deviation above the mean, with effect sizes ranging from −.13 to −.16. The researchers interpreted this finding as Saxon and Math Expressions being more robust across the range of teacher MCK compared with Investigations. Although the researchers examined teacher MCK as a moderator of curriculum effects, they did not examine whether the relation between teacher MCK and student mathematics achievement varied depending on the curriculum used.

Study Purpose and Research Questions

Though mathematics PD programs frequently target teacher MCK, the vast majority of PD studies intervening on teacher MCK have found null effects on student mathematics achievement (Gersten et al., [14]). On the contrary, research indicates that the use of high-quality curricula is associated with higher student mathematics achievement (e.g., Agodini et al., [2]; Remillard et al., [33]). In the current study, we hypothesized that curricula with a high level of teacher support could lead teachers to effectively teach mathematics, regardless of their level of MCK. This hypothesis was supported by research indicating that curricula with greater teacher supports were more likely to lead to consistent student outcomes despite variations in teachers' MCK (Agodini & Harris, [1]; Stein & Kaufman, [39]). The current study builds on prior research to investigate whether the relation between teacher MCK and classroom outcomes (i.e., effective instructional behaviors and student mathematics achievement gains) varies based on the curriculum used.

This study draws from a large-scale efficacy study (Clarke et al., [4]) of Early Learning in Mathematics (ELM), a Tier 1 kindergarten curriculum that teachers used during the core mathematics block. In the efficacy trial, classrooms within schools were randomly assigned to teach ELM for the year, or to use district-approved, BAU curricula that they would have otherwise taught. ELM uses an explicit instructional design and provides built-in teacher supports within each lesson. It has been shown to be particularly effective for students at risk in mathematics (Clarke et al., [4]). When examining students in the ELM and control conditions classified as at risk, students receiving ELM made greater gains across their kindergarten year compared with students in the control condition and made greater gains toward catching up to their typically achieving peers. Using data from the ELM efficacy study, we investigated two related research questions (RQs).

RQ1: Does Curriculum (ELM vs. BAU) Moderate the Association between Teacher MCK and Instructi...

Research indicates that teacher MCK is positively associated with teaching behaviors (Garet et al., [12], [11]). Research also indicates that curricula adopting a teacher-directed or blended approach are associated with higher rates of effective instructional behaviors, such as teachers posing more response opportunities during instruction (Doabler et al., [8]). Thus, we hypothesized that teacher MCK would predict teachers' instructional behaviors for teachers using a BAU core curriculum that varied in the degree of teacher support provided. For teachers using ELM, we hypothesized that these associations would be weaker given that ELM includes built-in supports to guide teachers to enact effective instructional behaviors, regardless of their MCK. The specific instructional behaviors examined included teacher demonstrations, group response opportunities, and individual response opportunities. We selected these behaviors due to research indicating their association with student mathematics achievement (Doabler et al., [8]) and with effective instructional practices more broadly.

We examined this question using a continuous and binary indicator (lowest three quartiles vs. upper quartile) of teacher MCK, given research indicating that associations differ based on how MCK is defined or distributed (Agodini & Harris, [1]; Hill et al., [16]). Researchers have dichotomized MCK in various ways—by focusing on teachers scoring in the lower ranges of MCK (e.g., Hill et al., [16]) and those scoring in the upper ranges of MCK (Agodini & Harris, [1]). In the current study, we compared teachers across the lower three quartiles with teachers scoring in the upper quartile to determine if relations differed for the most knowledgeable teachers (e.g., those scoring at the 75th percentile and above on the MKT).

RQ2: Does Curriculum (ELM vs. BAU) Moderate the Association between Teacher MCK and Student M...

We hypothesized that in BAU classrooms, teachers' MCK would predict student mathematics achievement gains, given research indicating positive associations between these variables (e.g., Hill et al., [16]). We believed that these associations would be weaker in ELM classrooms given that ELM provides a high degree of support to teachers and therefore would reduce the effect of MCK on student outcomes. Consistent with the first research question, we examined this question using both continuous and binary indicators of teacher MCK.

Method

Design

The ELM large-scale efficacy trial (Clarke et al., [4]) was conducted in school districts in Oregon (2008–2009) and Texas (2009–2010; for detailed description of methods, see Clarke et al., [4]). This project utilized a group-randomized controlled trial (RCT) design (Murray, [26]) to investigate the efficacy of the ELM core kindergarten curriculum, with mathematics achievement data collected from individual students and teacher data (survey data and observations) collected from each classroom. Thus, the primary analysis framework was multilevel models that nested students within classrooms, the unit of randomization to condition. Kindergarten classrooms (n = 129) were randomly assigned to receive the ELM curriculum (n = 68) or BAU mathematics instruction (n = 61), blocking on school. The sample was approximately evenly split between Oregon (n = 64) and Texas (n = 65) classrooms. Classrooms were matched based on half-day (n = 17) or full-day (n = 112) schedules.

Participants

Schools and Teachers

The 46 participating schools in Oregon and Texas included public (n = 32), private (n = 11), and charter (n = 3) schools in urban and suburban areas. Participating kindergarten teachers (n = 130) taught the 129 classrooms (two teachers each taught a half-day in a single classroom). Of the 130 teachers, 127 (98%) were female. Teacher-reported demographics indicated that 92 (71%) were White, 22 (17%) were Hispanic, 11 (8%) were African American, 1 (<1%) was Native American, 1 (<1%) was Asian American, and 3 (2%) did not report demographic information. Regarding teacher-reported credentials and teaching background, 49 teachers (38%) held a master's degree, 30 (23%) completed three or more college mathematics courses (e.g., including mathematics methods courses and courses such as calculus), 68 (52%) completed college algebra, and 72 (55%) had taught kindergarten for 4 or more years. Twenty-four teachers (18%) reported spending 21–40 minutes per day on mathematics, 45 (35%) reported 41–60 minutes per day, and 37 (28%) reported 61 or more minutes per day.

Students

The full sample of students included 2,598 kindergarteners in ELM (n = 1,401) and control (n = 1,197) classrooms. Of the full sample of students, 120 (5%) were eligible for special education services, and 708 (27%) were identified as English learners (ELs). District-provided demographic data were only available for students who attended public schools, comprising 61% of the sample. Of the students with demographic data, approximately 76% of the student population qualified for free or reduced-price lunch programs. Across Oregon and Texas, 909 (57%) students were White (including students who were Hispanic), 265 (17%) were African American, 218 (14%) were American Indian/Alaska Native, 136 (9%) were Asian, 11 (<1%) were Native Hawaiian or Islander, and 43 (3%) were multiple races.

ELM Curriculum

ELM is a 120-lesson, core kindergarten curriculum, designed for whole-class instruction and focused on building foundational early mathematics concepts. Each lesson consists of a 15-minute daily calendar routine, as well as a 45-minute mathematics lesson. Each lesson includes four to five activities, allowing for lessons to be organized in tracks with skills introduced, built upon, and frequently reviewed over multiple lessons to allow for mastery and retention. Each ELM lesson includes a cumulative Math Practice worksheet with a "Note Home" in English and Spanish to encourage parent involvement and additional at-home practice. The curriculum was designed to support a wide range of learners, with a particular focus on supporting students entering kindergarten at risk in mathematics, through two key elements: (a) focusing on critical mathematics content and (b) using research-based instructional design principles.

Critical Content

The curriculum covers content across three math strands: numbers and operations, geometry, and measurement. Content was selected based on the National Council of Teachers of Mathematics Focal Points for kindergarten (NCTM, [28]) and aligns with the Common Core State Standards for Mathematics (National Governors Association Center for Best Practices & Council of Chief State School Officers, [29]). The program maintains a narrow focus to ensure mastery of critical mathematics skills, with the greatest emphasis placed on the development of whole number understanding (NCTM, [28]). Quarterly in-program assessments are built into the curriculum for teachers to assess the progress of individual students and the class as a whole.

Instructional Design Principles and Teacher Supports

ELM uses an explicit and systematic instructional design (Gersten et al., [13]) to create high-quality instructional interactions between teachers and students (Doabler et al., [8]). Embedded in the curriculum design are teacher supports to assist teachers in effectively delivering the curriculum. For example, each quarter of ELM includes a Teacher's Guide that outlines the scope and sequence of critical content taught across the quarter. A "Teacher Note" is included before each set of five lessons to outline lesson objectives and to alert teachers to potential difficulties students may encounter with new content. Detailed teacher instructions and suggested scripting are used to provide descriptions of lesson activities, provide consistent and precise mathematical language, and highlight key vocabulary words and definitions. ELM also uses teacher scripting to help teachers effectively scaffold instruction. For example, when introducing a new concept, teachers provide clear demonstrations while maintaining student attention by posing scripted questions or having students repeat a vocabulary word or describe a mathematical concept. Next, teachers guide students to practice mathematics skills as a group with a greater degree of student participation. Last, teachers have students practice mathematics skills independently while providing confirmative or corrective feedback. This sequence occurs within and across lessons, giving students repeated exposure to challenging concepts across time.

ELM also relies upon the concrete-representational-abstract (CRA) sequence to build deep understanding of complex mathematical concepts. The CRA sequence is outlined for teachers in lesson scripting and detailed information about program materials. Along with developing understanding of mathematical concepts, ELM places emphasis on developing procedural fluency and automaticity to help students master mathematics concepts and skills. Students receive ample practice on skills over time, leading to increased exposure to mathematical concepts and fostering automaticity. Additional student practice opportunities are built into the end of every lesson through student completion of the independent Math Practice worksheet.

BAU Curricula

The most commonly used curriculum by classrooms in the control condition was Texas Mathematics or Everyday Mathematics (n = 21). Other programs included Harcourt Math (n = 15), SFAW (n = 7), Progress in Mathematics (n = 3), and Investigations in Number, Data, and Space (n = 2). Four teachers indicated that they used teacher-made materials, and seven teachers did not respond to the survey item. The pedagogical approaches and level of teacher supports varied across the published curricula used in the control condition. For example, Everyday Mathematics has been described as having an "inquiry approach to mathematics, in which students are expected to develop mathematical thinking through the exploration and application of mathematical principles rather than through direct instruction" (Nelson et al., [30], p. 3). Everyday Mathematics has also been described as a "low-support curriculum" (Stein & Kaufman, [39], p. 667), due to limited support for anticipating student responses and for understanding lesson rationales. Similar to Everyday Mathematics, Investigations in Number, Data, and Space has been described as having a student-centered approach (Agodini et al., [2]; Remillard et al., [33]); however, it has been classified as a "high-support curriculum" (Stein & Kaufman, [39], p. 667).

Researchers have described Harcourt Math and SFAW as teacher directed, or explicit in nature (Agodini et al., [2]; Nelson et al., [30]). Both programs rely upon student worksheets as the primary mode of instruction. SFAW has been characterized as providing minimal support to teachers, without the provision of explicit scripts or guidance of teacher actions (Remillard et al., [33]). Harcourt Math largely relies upon whole-class instruction and practice in the form of student worksheets, with limited support for teachers to differentiate instruction for students beyond providing alternative worksheets (Nelson et al., [30]).

Measures

Test of Early Mathematics Ability, Third Edition (TEMA-3)

The TEMA-3 (Ginsburg & Baroody, [15]) is an individually administered, norm-referenced assessment, intended for use with children ages 3–8. It is designed to measure informal and formal mathematics skills, and it takes approximately 30–40 minutes to administer. The TEMA-3 includes a range of items sampling across skills in the domains of numbering, comparing numbers, number facts, calculation skills, and understanding of mathematical concepts. The TEMA-3 has good psychometric properties, with high internal reliability (coefficient alphas range from.94 to.96), high test-retest reliability (ranging from.82 to.93), and moderate criterion-related validity with other measures of early mathematics skills (.54–.91; Ginsburg & Baroody, [15]). TEMA-3 raw score gains (calculated by subtracting students' raw score from the fall administration from their raw score in the spring) were used as the measure of student mathematics achievement.

Mathematical Knowledge for Teaching

The MKT survey (Hill et al., [17]) was administered online to teachers in the ELM efficacy trial and consisted of 24 untimed multiple-choice items selected from Hill et al.'s ([17]) MKT measure. Completion time was approximately 20–30 minutes, with content covering mathematics from kindergarten to sixth grade and spanning number and operations, place value, geometry, and algebra. Items were designed to assess for teachers' subject matter knowledge and pedagogical content knowledge. For example, one subject matter knowledge item asked teachers to respond to the statement, "Multiplication makes numbers larger." Responses included: "True for all numbers," "Not always true," and "I'm not sure." An item designed to measure teachers' pedagogical content knowledge included viewing a hypothetical student's work ordering decimals and indicating error(s) made. Responses included: "They are ignoring place value," "They are ignoring the decimal place," "They are guessing," "They have forgotten their numbers between 0 and 1," or "They are making all of the above errors." Reliabilities of the MKT range from.86 to.95. The number and operations and algebra scales correlate at.76 (Hill et al., [17]). In the current study, teacher MKT scores were reported as the percentage correct out of the 24 multiple-choice items.

Classroom Observation of Student-Teacher Interactions–Mathematics

The Classroom Observation of Student-Teacher Interactions–Mathematics (COSTI-M; Doabler et al., [8]) observation system is a modified version of a reading instruction observation system (Smolkowski & Gunn, [38]) designed for mathematics. Observers collect frequency counts of teachers' instructional behaviors across an observation period, with the rate of instructional behaviors calculated by dividing the total number of instructional interactions by the total duration of the observation. In the current study, we analyzed the following instructional behaviors: (a) teacher demonstrations, (b) group response opportunities, and (c) individual response opportunities. We defined teacher demonstrations as instances that a teacher provided a clear mathematical explanation, verbalized mathematical thought processes, or employed a physical demonstration of a mathematical concept. We defined group response opportunities as math verbalizations from two or more students. We defined individual response opportunities as one student verbalizing or providing a physical demonstration of mathematical understanding. Previous investigations of the COSTI-M using the ELM data set have demonstrated stability intraclass correlation coefficients (ICCs) ranging from.26 to.44, with modest average stability of rates of instructional behaviors (.45–.65; Doabler et al., [9]). Using the ELM data set, Doabler et al. ([8]) found high interobserver agreement, with ICCs ranging from.67 to.95.

Study Procedures

Data Collection and Training

All participating teachers (ELM and control teachers) completed surveys, including teacher demographics and the MKT measure, in October 2009 (Oregon) and 2010 (Texas), prior to the implementation of ELM. Student mathematics achievement measures were collected in the fall prior to implementation of ELM, and in the spring after ELM classrooms completed the program. Trained staff administered student measures. These staff met a reliability standard of at least 0.85 prior to collecting data in schools and again during an in-school shadow-coding session. Data collector training ranged from 4 to 6 hours in the fall and again in the spring prior to posttesting. Observers underwent an additional 11+ hours of training focused on the COSTI-M and other observation measures that included lecture, video practice, a video reliability check, and a real-time reliability check in a classroom where observers were required to meet a reliability standard of.80. Trainers provided booster sessions prior to each additional observation. In each ELM and control classroom, an observation was conducted in the fall, winter, and spring, each approximately 6 weeks apart. Out of 379 scheduled observations, only 8 (2%) were not completed due to teacher absences or scheduling conflicts. Observations were conducted for the duration of core mathematics instruction, which varied between classrooms and lasted between 30 and 90 minutes.

Professional Development

ELM teachers participated in three 4-hour trainings conducted across the school year focused on curriculum implementation. The lead curriculum author and/or project coordinator led the trainings. Trainings focused on lesson content, classroom management, and implementing the ELM curriculum with fidelity. Teachers had the opportunity to practice teaching ELM lessons during the training and received feedback from trainers on teacher demonstrations, guided practice, and facilitating group and individual response opportunities.

Analytic Methods

For RQ1, the dependent variable was teacher instructional behaviors as measured by the COSTI, including (a) teacher demonstrations, (b) group response opportunities, and (c) individual response opportunities. For RQ2, the dependent variable was kindergarten student mathematics achievement gains on the TEMA-3. The independent variable for both RQ1 and RQ2 was teacher MCK, as measured by the MKT (Hill et al., [17]). Curriculum (ELM vs. BAU) was examined as a moderating variable for both RQs.

For tests of teacher-level moderation effects with a relatively small teacher sample size, the trade-off between Type I and Type II errors represents a delicate balance. False conclusions about significant interaction effects (Type I errors) are problematic, as is failing to detect interaction effects (Type II errors). To balance the likelihood of the two types of errors, Cohen ([6]) and Rosnow and Rosenthal ([35]) recommend an adjustment to alpha, the Type I error rate. Thus, we set alpha to 0.10 for tests of main effects, interaction effects, and for follow-up subgroup analyses in the presence of a significant interaction. A power analysis conducted using G*Power (Faul et al., [10]) revealed that, given a sample of approximately 64 teachers per condition, there was power (.80) to detect correlations >.31 between MCK and teacher outcomes within the ELM and BAU conditions. Prior to conducting analyses, we examined assumptions of linear regression and HLMs and found them tenable across all models. We describe analysis procedures specific to each RQ below.

RQ1

The purpose of RQ1 was to investigate whether the association between teacher MCK and teacher instructional behaviors varied between teachers of ELM and teachers of BAU curricula. We estimated Pearson's r bivariate correlations among teacher MCK, teacher characteristics, and instructional behaviors. Multiple regression models were specified to regress instructional behaviors on Teacher MCK, Condition, and the MCK × Condition interaction, as specified by the following equation,

Y=b0+b1MCK+b2Condition+b3(MCK×Condition)+e,

where Y = the raw score on the teaching behavior variable and b0, b1, b2, and b3 are the intercept and regression coefficients associated with Teacher MCK, Condition, and the MCK × Condition interaction. We centered teacher MCK to allow for interpretation of the intercept as the average instructional behavior for a teacher with average MCK (MCK = 0) in the BAU condition (Condition = 0). Separate models were analyzed for each instructional behavior, with predictor variables entered into the regression models simultaneously. Specifically, Model 1 included the effect of Teacher MCK, Condition, and the interaction between the two. If the interaction term was not significant, then Model 2 was respecified to exclude the interaction term. If the interaction was significant, then simple slope analyses were performed within each study condition. A significant interaction indicated that the effect of Teacher MCK on a given instructional behavior (i.e., rate of teacher demonstrations, group response opportunities, or individual response opportunities) varied by condition. After examining Teacher MCK continuously, we dichotomized Teacher MCK by comparing teachers in the lowest three quartiles of MCK with teachers in the upper quartile of teacher MCK, in alignment with previous research on the interaction between teacher MCK and curriculum (e.g., Agodini & Harris, [1]). We computed all regression models using SPSS (IBM Corp, [18]).

RQ2

The purpose of RQ2 was to investigate whether the association between teacher MCK and student mathematics achievement varied between teachers of ELM and teachers of BAU curricula. Given the nested nature of the student mathematics achievement data, with individual students receiving instruction within a given classroom, we used two-level HLMs to investigate the interaction of teacher MCK and condition on student mathematics achievement. We omitted school-level data as a third level of nesting because the primary foci of the study included teacher and classroom variables without taking school effects into account. The multilevel models regressed student mathematics achievement gains at Level 1 on classroom characteristics (Teacher MCK, Condition, and Teacher MCK × Condition) at Level 2:

Level‐1 Model:MATHGAINSij=β0j+rijLevel‐2 Model:β0j=γ00+γ01×(MCKj)+γ02×(Conditionj)+γ03×(MCK×Conditionj)+u0jMixed Model:MATHGAINSij=γ00+γ01×MCKnj+γ02×Conditionj+γ03×MCK×Conditionj+u0j+rij

We used full information maximum likelihood estimation for all analyses and reported robust standard errors. Similar to RQ1 analyses, we evaluated teacher MCK as a continuous, centered variable and as a dichotomous, uncentered variable (lower three quartiles vs. upper quartile) in separate analyses. Specifically, Model 1 included the effect of Teacher MCK, Condition (0 = BAU, 1 = ELM), and their interaction. Significant interaction terms indicated that the effect of Teacher MCK on student mathematics achievement gains varied by condition. For models without any evidence of moderation (i.e., the p value of the interaction term was ≥.10), we respecified Model 2 to exclude the interaction term and analyzed the main effects model. We computed all HLMs using HLM 8.0 (Raudenbush et al., [32]).

Baseline Equivalency Analyses

We conducted baseline equivalency analyses to ensure that random assignment of classrooms to the ELM and control conditions resulted in equivalent groups on student and teacher pretest characteristics. We used independent samples t-tests and chi square analyses to examine data for baseline differences between the ELM and control conditions on demographic and key study variables. We did not observe any differences across conditions on any student and teacher demographics, with the exception of a higher proportion of ELs in the ELM (n = 407) compared with BAU (n = 301) condition, (X2(1) = 4.96, p <.05). In addition, we did not observe differences on TEMA-3 pretest scores or teacher variables including MKT scores, the number of college math courses taken, and the number of years teaching kindergarten.

Missing Data

Mathematics achievement gain scores on the TEMA-3 were available for 1,972 students (approximately 76% of the total sample). Given the proportion of missing data, chi square and independent samples t-tests were conducted to determine whether key variables differed between students with and without available TEMA-3 gains scores. We did not find differences in regard to gender (X2(1) > 0.009, p =.922) or the number of half- and full-day absences (t(588.10) = −1.12, p =.26, 95% CI [−1.24, 0.34]). However, missing TEMA-3 gains scores were systematically related to classification as an EL (X2(1) = 82.96, p <.001), non-White ethnicity (X2(5) = 118.58, p <.001), and eligibility for special education services (X2(1) = 24.79, p <.001). In addition, TEMA-3 pretest scores of students with missing data at the spring administration (M = 15.28, SD = 8.76) were significantly lower than students without missing data (M = 20.32, SD = 10.30; t(250.22) = 7.54, p <.001, 95% CI [3.72, 6.36]). Last, two Oregon teachers (1.5%) in the control condition who did not complete any of the teacher survey measures were excluded from the analyses. Due to the nested nature of the data and the centrality of these variables to the RQs, their students were excluded from analyses as well.

Treatment of Missing Data

Because group comparisons indicated that missing TEMA-3 gain score data were systematically related to other variables, we used multiple imputation to estimate missing values. Predictors of student-level missing data included TEMA-3 pretest scores and dummy-coded demographic characteristics: EL status (0 = English speaker, 1 = English learner), ethnicity (0 = White, 1 = non-White), and special education eligibility (0 = not eligible, 1 = eligible). For each HLM, we imputed 100 data sets, and their average was used to generate model parameters.

Results

RQ1: Does Curriculum (ELM vs. BAU) Moderate the Association between Teacher MCK and Instructi...

Bivariate correlations

We report descriptive statistics for study variables by condition in Table 1, along with Pearson's r bivariate correlations among teacher characteristics and instructional behaviors. Correlations ranged from −0.11 to 0.47. Teacher MCK was significantly correlated with the number of college math courses (r =.24, p =.007), teacher demonstrations (r =.19, p = 0.34), and group response opportunities (r =.29, p =.001). Teacher MCK was not significantly correlated with individual response opportunities (r =.13, p =.154). Correlations were highest between teacher demonstrations and group response opportunities (r =.47, p <.001). Correlations were not significant between the number of college math courses taken and any of the instructional behaviors. The number of years teaching was not significantly correlated with any other variable.

Table 1. Descriptive Statistics by Condition and Correlations among Key Variables (n = 127)

(1)(2)(3)(4)(5)(6)ELMControl
M (SD)M (SD)
Teacher variables:
1. Teacher MCK (% correct)51.4147.39
(15.41)(17.15)
2. Years teaching−.023.763.85
(1.35)(1.31)
3. Number of college math courses.24**.071.941.81
(1.02)(1.03)
4. Teacher demonstration rate/min.19*−.08−.10.58.52
(.30)(.31)
5. Group response rate/min.29**−.03.06.47**1.34.77
(.63)(.62)
6. Individual response rate/min.13.02−.11.08.19*.65.46
(.38)(.27)
Student variables:
TEMA-3 pretest raw score19.7020.06
(10.46)(10.04)
TEMA-3 gain score14.3613.38
(.21)(.21)

Graph

1 Note. ELM = Early Learning in Mathematics; MCK = mathematics content knowledge; TEMA-3 = Test of Early Mathematics Achievement, Third Edition.

  • 2 * p <.05.
  • 3 ** p <.01.
Multiple regression analyses

We first examined teacher MCK as a continuous predictor, with regression model results displayed in Table 2. The Model 1 columns in Table 2 show that the MCK-by-condition interaction was not significant for two of the instructional behavior outcomes: teacher demonstrations (p =.136) and group response opportunities (p =.793). We dropped the interaction term from these models to examine the main effect of teacher MCK with condition retained as a covariate (see Table 2, Model 2).

Table 2. Results of Regressing Teachers' Instructional Behaviors on Teacher MCK as a Continuous Variable, Condition, and the Teacher MCK-by-Condition Interaction

Teacher Demonstration (rate/min)Group Response (rate/min)Individual Response (rate/min)
Model 1Model 2Model 1Model 2Model 1
Intercept.509**.505**.761**.759**.449**
(.37)(.037)(.078)(.078)(.043)
Teacher MCK.005*.003*.011**.010**−.001
(.002)(.002)(.005)(.003)(.003)
Condition.071.071.558**.558**.191**
(.051)(.051)(.107)(.107)(.059)
Teacher MCK × Condition−.005−.002.006+
(.003)(.007)(.004)
R2.067*.050*.251**.251**.113**
F2.95*3.27*13.76**20.76**5.22**

Graph

  • 4 Note. Model 1 presents results from the moderation analyses. When interaction terms were not significant at the p ≤.10 level, the interaction term was excluded from the model to examine main effects of teacher MCK and condition (see Model 2). Standard errors are reported in parentheses. MCK = mathematics content knowledge.
  • 5 + p <.10.
  • 6 * p <.05.
  • 7 ** p <.01.

For teacher demonstrations, there was a significant positive linear relationship between teacher MCK and the rate of teacher demonstrations after controlling for condition (b = 0.003, SE = 0.002, β =.17, p =.053). Specifically, for a 10% increase in teacher MCK, the teacher demonstration rate increased by 0.03 demonstrations per minute. ELM teachers did not have significantly higher rates of teacher demonstrations than teachers of BAU curricula after controlling for MCK (b = 0.071, SE = 0.051, β =.12, p =.167). For group response opportunities, there was a significant positive linear relationship between teacher MCK and the outcome after controlling for condition (b = 0.010, SE = 0.003, β =.24, p =.003). For a 10% increase in teacher MCK, the rate of group response opportunities increased by 0.10 responses per minute. ELM teachers had significantly higher rates of group response opportunities than teachers of BAU curricula after controlling for MCK (b = 0.558, SE = 0.107, β =.41, p <.001), eliciting an additional 0.56 group responses per minute.

The interaction term for the rate of individual response opportunities was significant (b = 0.006, SE = 0.004, p =.086), indicating that the association between teacher MCK and the rate of individual responses varied by condition. Simple slopes analyses revealed that for teachers using ELM, the relationship between teacher MCK and individual response opportunities was positive and significant (b = 0.005, SE = 0.003, β =.25, p =.047). Specifically, for every 10% increase in teacher MCK, ELM teachers elicited an additional 0.05 individual response opportunities per minute. For teachers of BAU curricula, the relationship between teacher MCK and individual response opportunities was not significant (b = −0.001, SE = 0.003, β = −.05, p =.674).

We next examined teacher MCK as a dichotomous variable, comparing teachers who scored in the lower three quartiles with teachers who scored in the upper quartile (0 = lower, 1 = upper; see Table 3). The interaction term for teacher demonstrations was significant (b = −0.250, SE = 0.126, p =.049), indicating that the association varied by condition between the lower three quartiles and upper quartile of teacher MCK and teacher demonstrations. Decomposing the significant interaction revealed that for ELM teachers, the difference in teacher demonstration rate between the lower three quartiles and the upper quartile of teacher MCK was not significant (b = −0.036, SE = 0.077, d = −.12, p =.639). For teachers using BAU curricula, there was a significant positive difference in teacher demonstration rate between the lower three quartiles and the upper quartile of teacher MCK (b = 0.214, SE = 0.099, d =.71, p =.033). Thus, the effect of being a control teacher in the upper quartile of teacher MCK resulted in a 0.21 increase in the rate of teacher demonstrations compared with a control teacher in the lower three quartiles.

Table 3. Results of Regressing Teachers' Instructional Behaviors on Teacher MCK as a Dichotomous Variable, Condition, and the Teacher MCK-by-Condition Interaction

Teacher Demonstration (rate/min)Group Response (rate/min)Individual Response (rate/min)
Model 1Model 1Model 2Model 1Model 2
Intercept.462.644**.682**.449**.430**
(.041)**(.086)(.081)(.047)(.045)
Teacher MCK.214*.548+.327*.015.129+
(.099)(.208)(.128)(.114)(.070)
Condition.130*.638**.563**.146*.185**
(.058)(.121)(.108)(.066)(.059)
Teacher MCK × Condition−.250*−.356.183
(.126)(.264)(.145)
R2.058+.245**.233**.118*.107**
F2.52+13.28**18.88**5.51*7.42**

Graph

  • 8 Note. Model 1 presents results from the moderation analyses. When interaction terms were not significant at the p ≤.10 level, the interaction term was dropped from the model to examine main effects of teacher MCK and condition (see Model 2). Standard errors are reported in parentheses. Teacher MCK was coded as 0 = lower three quartiles, 1 = upper quartile. MCK = mathematics content knowledge.
  • 9 + p <.10.
  • 10 * p <.05.
  • 11 ** p <.01.

The interaction term was not significant for group response opportunities (p =.179), or individual response opportunities (p =.208; see Table 3, Model 1), and thus we dropped the interaction term from the models to examine main effects. For group response opportunities, the effect of being a teacher in the upper quartile of MCK compared with the lower three quartiles resulted in a 0.33 increase in the rate of group response opportunities. ELM teachers had significantly higher rates of group response opportunities than BAU teachers, after controlling for level of teacher MCK (b = 0.563, SE = 0.108, d =.89, p <.001). Specifically, ELM teachers elicited an additional 0.56 group response opportunities per minute.

For individual response opportunities, there was a significant positive difference in the rate of individual response opportunities between the lower three quartiles and upper quartile of teacher MCK after controlling for condition (b = 0.129, SE = 0.070, d =.34, p =.069), with teachers in the upper quartile eliciting an additional 0.13 individual response opportunities per minute. ELM teachers had significantly higher rates of individual response opportunities compared with control teachers after controlling for level of teacher MCK (b = 0.185, SE = 0.059, d =.49, p =.002), eliciting an additional 0.19 opportunities per minute.

RQ2: Does Curriculum (ELM vs. BAU) Moderate the Association between Teacher MCK and Student M...

Table 4 presents the results of the HLMs regressing student gains on the TEMA-3 across kindergarten on (a) teacher MCK as both a continuous and dichotomous predictor (lower three quartiles vs. upper quartile), (b) condition (ELM vs. BAU), and (c) the interaction between the two. For both definitions of teacher MCK, the interaction term was not significant (p =.847 when examining MCK continuously, and p =.973 when examining MCK dichotomously; see Table 4, Model 1) and was therefore excluded from Model 2. Thus, interpretation focuses on Model 2, regressing student mathematics achievement gains on teacher MCK and condition.

Table 4. Results of Hierarchical Linear Models Regressing Student Mathematics Achievement Gains on Teacher MCK as a Continuous and Dichotomous Variable, Condition, and the Teacher MCK-by-Condition Interaction

MCK as Continuous PredictorMCK as Dichotomous Predictor
Model 1Model 2Model 1Model 2
Fixed effects:
Intercept15.34**15.41**15.81**15.82**
(.51)(.52)(.60)(.56)
Teacher MCK−.06*−.05*−1.72−1.79*
(.03)(.02)(1.04)(.80)
Condition1.64*1.54*1.52+1.52*
(.71)(.71)(.83)(.72)
Teacher MCK × Condition.01−.05
(.04)(1.51)
Variances:
Between classrooms8.57**8.63**8.79**8.72**
(2.93)(2.94)(2.96)(2.95)
Between students39.9239.899.6639.93
(6.32)(6.32)(6.30)(6.32)

Graph

  • 12 Note. Model 1 presents results from the moderation analyses. In both analyses, the interaction term was not significant and was excluded from the model to examine the main effects of teacher MCK and condition (see Model 2). Standard errors are reported in parentheses. When examining Teacher MCK as a dichotomous predictor, the variable was coded as 0 = lower three quartiles, 1 = upper quartile. MCK = mathematics content knowledge.
  • 13 + p <.10.
  • 14 * p <.05.
  • 15 ** p <.01.

First, examining teacher MCK as a continuous predictor, for a 1 percentage point increase in teachers' MCK score, student gain scores decreased by −0.05 points (SE = 0.02, p =.033) after controlling for intervention condition. Compared with students of control teachers, students of ELM teachers had higher gain scores by 1.54 points (SE = 0.71, p =.038) after controlling for teacher MCK. Variance components indicated that 21.6% of the variance in student mathematics occurred between classrooms. Significant between-classroom variation remained after accounting for teacher MCK and condition (p <.001). We next examined teacher MCK as a dichotomous predictor of student mathematics achievement. Having a teacher who scored in the upper quartile of MCK compared with the lower three quartiles of MCK resulted in a decrease in student mathematics achievement gains by 1.79 points (SE = 0.80, p =.026) after controlling for condition. Students of ELM teachers benefited from a 1.52 increase in student mathematics achievement gains (SE = 0.72, p =.036) after controlling for teacher MCK.

Discussion

The purpose of the current study was to investigate the interaction between teacher MCK and curriculum (ELM vs. BAU) on teacher instructional behaviors (RQ1) and student mathematics achievement (RQ2). With heightened focus from policy makers and PD leaders on increasing teachers' MCK, along with research indicating that teacher MCK is associated with teaching behaviors and student outcomes, the results of this study contribute to the extant literature by examining MCK in the context of a research-based curriculum (Clarke et al., [4]) compared with BAU curricula. This approach provides a more nuanced look into the effect of teacher MCK on teacher and student outcomes, including how these relationships may vary depending on the curriculum used.

RQ1: Instructional Behaviors

Although our hypotheses did not specify different patterns of findings for the instructional behaviors examined (i.e., teacher demonstrations, group response opportunities, and individual response opportunities), this emerged in the analyses. Significant interactions also differed depending on how teacher MCK was examined (i.e., continuously or dichotomously). Below, we summarize results and interpretations separately for each instructional behavior.

Teacher demonstrations

Examining teacher MCK continuously, no significant interaction emerged between teacher MCK and condition on teacher demonstrations, though the interaction term was trending toward significance with a p value of.136. Examining teacher MCK dichotomously, there was a significant interaction between the lower three quartiles and upper quartile of teacher MCK and condition on teacher demonstration rate. This interaction revealed that for ELM teachers, the rate of teacher demonstrations was not significantly different depending on level of teacher MCK (rates were 0.592 and 0.556 for the lower three and upper quartiles, respectively). In a 60-minute mathematics lesson, this would translate to ELM teachers providing about 33–36 teacher demonstrations, regardless of teacher MCK level. For control teachers, there was a higher rate of teacher demonstrations for teachers scoring in the upper quartile of MCK compared with those scoring in the lower three quartiles. Specifically, teachers in the lower three quartiles of MCK had a demonstration rate of 0.462 (about 28 demonstrations in a 60-minute lesson) compared with 0.676 (about 41 demonstrations in a 60-minute lesson) for teachers in the upper quartile of MCK.

The nature of this interaction aligned with our original hypothesis that a positive relationship between teacher MCK and instructional behaviors would emerge for control teachers, but that the instructional behaviors of ELM teachers would be relatively consistent across different levels of teacher MCK. Given that ELM is fully scripted and provides highly specified teacher models, it is not surprising that the rate of teacher demonstrations was consistent across levels of teacher MCK for ELM teachers. Control teachers used curricula that varied in the degree of built-in teacher supports to facilitate demonstrations, such as specific language to model mathematics concepts. Several teachers in the control condition used teacher-developed curriculum materials, which likely did not specify demonstrations of mathematical concepts. It follows that teacher MCK would play a greater role in determining the rate of teacher demonstrations for teachers of BAU curricula.

Group response opportunities

Examining teacher MCK both continuously and dichotomously, no significant interactions emerged between teacher MCK and condition on the rate of teacher-elicited group responses. Main effects models revealed a positive relationship between teacher MCK and the rate of teacher-elicited group responses. In addition, ELM teachers provided a higher rate of group response opportunities overall. Examining teacher MCK continuously, ELM teachers with average MCK had a group response rate of 1.317, eliciting about 79 group responses in a 60-minute mathematics lesson. Control teachers with average MCK had a group response rate of 0.759, eliciting about 46 group responses in the same time frame.

Our findings were contrary to our hypotheses, with the results of the current study suggesting that, regardless of curriculum, the rate of teacher-elicited group response opportunities was positively associated with teacher MCK. In addition, across the continuum of MCK, ELM teachers provided higher rates of group response opportunities. These findings are noteworthy for several reasons. First, although ELM is a fully scripted curriculum, teachers' use of group responses varied depending on their knowledge for teaching mathematics. This suggests that even with highly specified group response opportunities, teachers with varying levels of MCK made use of the curriculum in different ways. Another point of interest is the finding that ELM teachers provided higher rates of group response opportunities across levels of MCK. To further contextualize this, a control teacher with average MCK would need to increase their MKT score by about 50% to equal a teacher with average MCK in the ELM condition. Perhaps the specificity of group response opportunities written into ELM supported teachers with lower MCK to use a higher base rate of group response opportunities compared with BAU teachers. It is also possible that ELM teachers with higher MCK went above and beyond, making use of the group response opportunities built into the curriculum but also incorporating more opportunities for students to participate as needed throughout instruction. The fact that curriculum played a role in the rate of teacher-facilitated group response opportunities is in line with previous research that curriculum can influence teacher behaviors and student learning opportunities (Remillard & Reinke, [34]; Remillard et al., [33]).

Individual response opportunities

We detected a significant interaction between teacher MCK and condition on individual response opportunities when examining teacher MCK continuously, though not dichotomously (p =.208). The pattern of significant findings indicated that for ELM teachers, as teacher MCK increased, the rate of individual response opportunities increased, whereas for control teachers there was no relationship between teacher MCK and the rate of individual response opportunities. Specifically, a 10% increase in teacher MCK for ELM teachers resulted in an additional 0.05 individual response opportunities per minute, translating to about three additional individual response opportunities in a 60-minute lesson.

The nature of the significant interaction was contrary to our hypothesis. We found a positive relationship between teacher MCK and individual response opportunities for ELM teachers, and no relationship for control teachers. This finding could be explained by the way that individual response opportunities are presented in the ELM curriculum. Typically, directions for eliciting individual responses are less specific and include statements such as "As time allows, call on several children to choose a card and perform an action," or "Have individual children show you groups of 2 from the chant." Given that the number of individual responses to provide was left open to teacher interpretation, it is possible that teachers with higher MCK differentiated instruction to a greater degree or made better use of instructional time overall, allowing for more individual practice. In the BAU condition, teachers provided a similar rate of individual response opportunities regardless of MCK. Though purely speculation, it is possible that control teachers with higher MCK prioritized different instructional behaviors during their instructional time, such as group response opportunities. It would follow that control teachers with lower MCK were more likely to provide individual response opportunities, and teachers with higher MCK minimized their use of individual response opportunities, resulting in relatively equal rates of individual response opportunities across MCK levels.

RQ2: Student Mathematics Achievement

We hypothesized that there would be a significant interaction between teacher MCK and curriculum on student mathematics achievement gains, such that for ELM teachers the association would be weaker between teacher MCK and student mathematics achievement compared with teachers of BAU curricula. Contrary to these hypotheses, the results of the current study suggest that there was not a significant interaction between teacher MCK and curriculum on student mathematics achievement. Main effects models revealed a similar pattern when examining teacher MCK continuously and dichotomously; therefore, only the continuous results are discussed here. There was a negative effect of teacher MCK on student mathematics achievement after controlling for condition, such that a 1 percentage point increase in teacher MCK resulted in a decrease in student mathematics achievement gain scores by 0.05 points. By these same parameters, a 10% increase in teacher MCK would result in a 0.50-point decrease in student mathematics achievement gain scores. There was a positive effect of ELM on student mathematics achievement, resulting in an increase in student mathematics achievement gains by 1.54 points after controlling for MCK. The finding that teacher MCK had a negative effect on student mathematics achievement gains is surprising given other research indicating a positive relationship between the two variables (e.g., Hill et al., [17], [16]). Given that the results of this study are limited to our specific sample across two states, we use caution when interpreting this finding. In addition, the clinical significance of a 0.50-point decrease in student mathematics achievement gains following a 10% increase in teacher MCK is debatable.

Nevertheless, the negative effect of teacher MCK is interesting and warrants discussion. One consideration with the current sample of students is that a large number of students were identified as at risk (defined as ≤40th percentile on the TEMA-3, including 66% of the sample in Clarke et al. [[5]] and 50% of students in Clarke et al. [[4]]). Teacher MCK has largely been investigated across students with a broader range of mathematical skill, and so it is possible that it may have differential associations with mathematics achievement for a sample of students with a higher degree of risk. A second consideration is the role that higher mathematics education plays in both teacher MCK and pedagogical practices. In the current sample, teacher MCK was positively correlated with the number of mathematics courses taken (r =.24, p <.01). Preservice and in-service general education teachers are increasingly being encouraged to adopt constructivist practices (Liang & Akiba, [23]), the "dominant pedagogical theory in contemporary educational circles" (Krahenbuhl, [21], p. 98). Constructivist practices are often "student centered" in nature, with more student independence and less teacher guidance (Krahenbuhl, [21]). Although this approach may be beneficial for students with a strong foundation in mathematics, students at risk for mathematics difficulties may struggle with the absence of clear teacher modeling and feedback. Though purely speculative, it is possible that teachers with higher MCK were also more likely to engage in constructivist teaching practices, which could have a negative impact on student mathematics achievement for an at-risk sample. Kutaka et al. ([22]) implemented an 18-credit graduate mathematics education program, Primarily Math, and found that participating teachers increased their MCK within number and operations and also their beliefs supporting constructivist practices relative to teachers in a comparison condition. There may accordingly be some association of increased teacher MCK with constructivist teaching approaches. Future research should investigate relations among teacher MCK and student mathematics achievement across varying levels of student skill to see whether associations differ for students at risk compared with those on track in mathematics. In addition, researchers should investigate the role of different pedagogical approaches on these relations.

Limitations and Future Research Directions

The findings of the current study must be considered in light of several limitations. First, given that the documented effect of teacher MCK is small (e.g., Agodini & Harris, [1]) and that there were only ∼64 teachers per condition, there was limited power to detect interaction effects. This was countered by expanding the definition of statistical significance to ≤.10, though it should be noted that the majority of significant findings fell within the more traditional range of <.05. Second, previous research using the ELM data set indicates that the reliability of the COSTI-M in the current data set was variable, with stability ICCs ranging from.26 to.44 (Doabler et al., [9]). Perhaps stronger relationships between teacher instructional behaviors and other variables would be observed with more accurate measurement of instructional behaviors, including an increased number of observations. In addition, although the COSTI-M provides rate data, it does not include a measure of instructional quality. It is possible that variables outside of the scope of the COSTI-M, such as the quality of instructional behaviors, would shed light on relationships with other variables above and beyond the rate of instructional behaviors.

Third, although the specific curricula used in the BAU condition were documented for most teachers, it would be useful to have more information about instruction in the control condition, including what curricula was used (if any) by the seven teachers who did not respond to the item. In addition, because teachers in the control condition used a variety of different curricula, we are unable to pinpoint specific curricular features that may have resulted in interaction effects on teacher instructional behaviors. An examination of the most commonly used BAU curricula revealed that around 70% of control teachers were using programs that have been described in the research as providing minimal or low teacher support. Nonetheless, one can only draw conclusions about features of ELM compared with a range of BAU curricula in the control condition that may or may not have included similar features. Last, given that ELM was implemented in the context of a larger RCT, ELM teachers received three 4-hour PD sessions throughout the academic year. Whether control teachers received similar levels of support is unknown. Although we consider it unlikely given the research demonstrating that PD alone typically does not have an impact on teacher behavior (e.g., Garet et al., [12]), it is possible that the PD influenced teacher behaviors or interacted in such a way that ELM teachers were more likely to make better use of the curriculum compared with teachers of control curricula.

Final Thoughts and Conclusions

Given the national focus on increasing student mathematics achievement through teacher variables such as teachers' MCK, the current investigation presents a timely exploration of possible mechanisms to move the dial on student mathematics achievement. Several findings are particularly worthy of revisiting. The results of the current study indicate that teacher MCK did influence teachers' implementation of ELM across two of the three instructional behaviors examined. ELM teachers with higher MCK provided higher rates of both group and individual response opportunities. This supports prior research indicating that different teachers make use of curricula in different ways (Remillard et al., [33]) and that teacher MCK plays a role in curriculum effects (Agodini & Harris, [1]).

These findings have implications for curriculum developers, PD leaders, and districts selecting core mathematics curricula, in regard to how teachers with varying backgrounds might make use of different curricular programs and features. The results of the current study indicate that both teacher MCK and curricular features may contribute to teachers engaging in different instructional behaviors. In addition, although districts should consider that the training and MCK of their teachers may affect implementation of curricula, the results of the current study also suggest that curricula with a high degree of implementation support have the power to shape the instructional behaviors of teachers above and beyond MCK.

Of equal interest is the need for further exploration of the role that teacher MCK and other teaching variables have on student mathematics achievement, particularly for at-risk students. The results of the current study suggest that teacher MCK had a small but negative effect on student mathematics achievement gains, whereas ELM had a positive effect. Given that the effect of teacher MCK was somewhat negligible from a clinical standpoint, targeting curricula with a high degree of implementation support that can be used by teachers with a range of skill sets may be a more practical and effective way to increase student mathematics achievement. Although this study begins to unpack for whom and in what contexts curricula can be effectively implemented, future investigations of teacher and student characteristics are warranted. Developments in this research may help increase our knowledge base of how to best support teachers across the continuum of MCK to effectively implement curricula and ultimately lead to increased student outcomes across the board.

Notes 1 This investigation was supported by the Institute of Education Sciences, US Department of Education, through grants R305K040081 and R305A080699 awarded to the Center on Teaching and Learning at the University of Oregon and Pacific Institutes for Research. The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: Drs. Ben Clarke, Scott K. Baker, and Hank Fien are eligible to receive a portion of royalties from the University of Oregon's distribution and licensing of certain ELM-based works. Potential conflicts of interest are managed through the University of Oregon's Research Compliance Services. Dr. Marah Sutherland is a research associate at the Center on Teaching and Learning at the University of Oregon; Dr. Ben Clarke is a professor of school psychology and the director of the Center on Teaching and Learning at the University of Oregon; Dr. Derek B. Kosty is an associate scientist at the Oregon Research Institute and an assistant research professor in the Prevention Science Institute at the University of Oregon; Dr. Scott K. Baker is a research professor at Wheelock College of Education and Human Development, Boston University; Dr. Christian T. Doabler is an associate professor of special education at The University of Texas at Austin; Dr. Keith Smolkowski is a senior research scientist at the Oregon Research Institute; Dr. Hank Fien is the director of the National Center on Improving Literacy and is a professor in the Teaching & Learning Department at Wheelock College of Education and Human Development, Boston University; Dr. Joanna Goode is the Sommerville Knight Professor of education at the University of Oregon. Correspondence may be sent to Marah Sutherland at marahs@uoregon.edu. References Agodini, R., & Harris, B. (2016). How teacher and classroom characteristics moderate the effects of four elementary math curricula. Elementary School Journal, 117(2), 216–236. https://doi.org/10.1086/688927 2 Agodini, R., Harris, B., Atkins-Burnett, S., Heaviside, S., Novak, T., & Murphy, R. (2009). Achievement effects of four early elementary school math curricula: Findings from first graders in 39 schools (NCEE 2009-4052). National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, US Department of Education. 3 Campbell, P. F., Nishio, M., Smith, T. M., Clark, L. M., Conant, D., Rust, A. H., DePiper, J. N., Frank, T. J., Griffin, M J., & Choi, Y. (2014). The relationship between teachers' mathematical content and pedagogical knowledge, teachers' perceptions, and student achievement. Journal for Research in Mathematics Education, 45(4), 419–459. https://doi.org/10.5951/jresematheduc.45.4.0419 4 Clarke, B., Baker, S., Smolkowski, K., Doabler, C. T., Strand Cary, M., & Fien, H. (2015). Investigating the efficacy of a core kindergarten mathematics curriculum to improve student mathematics learning outcomes. Journal of Research on Educational Effectiveness, 8, 303–324. https://doi.org/10.1080/19345747.2014.980021 5 Clarke, B., Smolkowski, K., Baker, S. K., Fien, H., Doabler, C. T., & Chard, D. J. (2011). The impact of a comprehensive Tier I core kindergarten program on the achievement of students at risk in mathematics. Elementary School Journal, 111(4), 561–584. 6 Cohen, J. (1990). Things I have learned (so far). American Psychologist, 45(12), 1304–1312. https://doi.org/10.1037/0003-066X.45.12.1304 7 Darling-Hammond, L., & Berry, B. (2006). Highly qualified teachers for all. Educational Leadership, 64(3), 14–20. 8 Doabler, C. T., Baker, S. K., Kosty, D., Smolkowski, K., Clarke, B., Miller, S. J., & Fien, H. (2015). Examining the association between explicit mathematics instruction and student mathematics achievement. Elementary School Journal, 115, 303–333. https://doi.org/10.1086/679969 9 Doabler, C. T., Stoolmiller, M., Kennedy, P. C., Nelson, N. J., Clarke, B., Gearin, B., Fien, H., Smolkowski, K., & Baker, S. K. (2018). Do components of explicit instruction explain the differential effectiveness of a core mathematics program for kindergarten students with mathematics difficulties? A mediated moderation analysis. Assessment for Effective Intervention, 44(3), 197–211. https://doi.org/10.1177/1534508418758364 Faul, F., Erdfelder, E., Lang, A.-G., & Buchner, A. (2007). G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods, 39, 175–191. https://doi.org/10.3758/BF03193146 Garet, M. S., Heppen, J. B., Walters, K., Parkinson, J., Smith, T. M., Song, M., Garrett, R., Yang, R., & Borman, G. D. (2016). Focusing on mathematical knowledge: The impact of content-intensive teacher professional development (NCEE 2016–4010). US Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance. Garet, M. S., Wayne, A., Stancavage, F., Taylor, J., Eaton, M., Walters, K., Song, M., Brown, S., Hurlburt, S., Zhu, P., Sepanik, S., & Doolittle, F. (2011). Middle school mathematics professional development impact study: Findings after the second year of implementation (NCEE 2011-4024). National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, US Department of Education. Gersten, R., Chard, D. J., Jayanthi, M., Baker, S. K., Morphy, P., & Flojo, J. (2009). Mathematics instruction for students with learning disabilities: A meta-analysis of instructional components. Review of Educational Research, 79(3), 1202–1242. https://doi.org/10.3102/0034654309334431 Gersten, R., Taylor, M., Keys, T. D., Rolfhus, E., & Newman-Gonchar, R. (2014). Summary of research on the effectiveness of math professional development approaches (REL 2014–010). US Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, Regional Educational Laboratory Southeast. http://ies.ed.gov/ncee/edlabs Ginsburg, H., & Baroody, A. (2003). Test of early mathematics ability, third edition. Pro-Ed. Hill, H. C., Rowan, B., & Ball, D. L. (2005). Effects of teachers' mathematical knowledge for teaching on student achievement. American Educational Research Journal, 42(2), 371–406. https://doi.org/10.3102/00028312042002371 Hill, H. C., Schilling, S. G., & Ball, D. L. (2004). Developing measures of teachers' mathematics knowledge for teaching. Elementary School Journal, 105, 11–30. https://doi.org/10.1086/428763 IBM Corp. (2016). IBM SPSS Statistics for Windows (Version 24.0) [Computer software]. IBM Corp. Jacob, R., Hill, H., & Corey, D. (2017). The impact of a professional development program on teachers' mathematical knowledge for teaching, instruction, and student achievement. Journal of Research on Educational Effectiveness, 10(2), 379–407. https://doi.org/10.1080/19345747.2016.1273411 Kraft, M. A., Blazar, D., & Hogan, D. (2018). The effect of teacher coaching on instruction and achievement: A meta-analysis of the causal evidence. Review of Educational Research, 88(4), 547–588. https://doi.org/10.3102/0034654318759268 Krahenbuhl, K. S. (2016). Student-centered education and constructivism: Challenges, concerns, and clarity for teachers. Clearing House: A Journal of Educational Strategies, Issues and Ideas, 89(3), 97–105. https://doi.org/10.1080/00098655.2016.1191311 Kutaka, T. S., Smith, W. M., Albano, A. D., Edwards, C. P., Ren, L., Beattie, H. L., Lewis, W. J., Heaton, R. M., & Stroup, W. W. (2017). Connecting teacher professional development and student mathematics achievement: A 4-year study of an elementary mathematics specialist program. Journal of Teacher Education, 68(2), 140–154. https://doi.org/10.1177/0022487116687551 Liang, G., & Akiba, M. (2015). Teacher evaluation, performance-related pay, and constructivist instruction. Educational Policy, 29(2), 375–401. https://doi.org/10.1177/0895904813492379 Metzler, J., & Woessmann, L. (2012). The impact of teacher subject knowledge on student achievement: Evidence from within-teacher within-student variation. Journal of Development Economics, 99(2), 486–496. https://doi.org/10.1016/j.jdeveco.2012.06.002 Mullens, J. E., Murnane, R. J., & Willett, J. B. (1996). The contribution of training and subject matter knowledge to teaching effectiveness: A multilevel analysis of longitudinal evidence from Belize. Comparative Education Review, 40(2), 139–157. https://doi.org/10.1086/447369 Murray, D. M. (1998). Design and analysis of group-randomized trials. Oxford University Press. National Assessment of Educational Progress. (2019). The nation's report card. Mathematics 2019: National Assessment of Educational Progress at grades 4 and 8. National Center for Education Statistics, Institute of Education Sciences, US Department of Education. National Council of Teachers of Mathematics. (2006). Curriculum focal points for prekindergarten through grade 8 mathematics: A quest for coherence. National Council of Teachers of Mathematics. National Governors Association Center for Best Practices & Council of Chief State School Officers. (2010). Common core state standards for mathematics. http://www.corestandards.org/assets/CCSSI%5fMath%20Standards.pdf Nelson, C., Kaufman, J., Booker, K., & Hill, B. (2007). Elementary-grade math programs in the Pittsburgh public schools: A comparison of Everyday Mathematics and Harcourt Math. Mathematica Policy Research. Perez, K. D., & Kumar, D. D. (2018). STEM professional development policies in the United States: Trends and issues. In M. Shelley & S. A. Kiray (Eds.), Research highlights in STEM education. ISRES. Raudenbush, S. W., Bryk, A. S, Cheong, Y. F., & Congdon, R. (2019). HLM 8 for Windows [Computer software]. Scientific Software International. Remillard, J. T., Harris, B., & Agodini, R. (2014). The influence of curriculum material design on opportunities for student learning. ZDM, 46(5), 735–749. https://doi.org/10.1007/s11858-014-0585-z Remillard, J. T., & Reinke, L. T. (2012, April). Complicating scripted curriculum: Can scripts be educative for teachers? [Paper presentation]. Annual Meeting of the American Educational Research Association, Vancouver, BC. Rosnow, R. L., & Rosenthal, R. (1989). Statistical procedures and the justification of knowledge in psychological science. American Psychologist, 44(10), 1276–1284. https://doi.org/10.1037/0003-066X.44.10.1276 Sample McMeeking, L., Orsi, R., & Cobb, R. B. (2012). Effects of a teacher professional development program on the mathematics achievement of middle school students. Journal for Research in Mathematics Education, 43(2), 159–181. https://doi.org/10.5951/jresematheduc.43.2.0159 Shulman, L. S. (1986). Those who understand: Knowledge growth in teaching. Educational Researcher, 15(2), 4–14. https://doi.org/10.3102/0013189X015002004 Smolkowski, K., & Gunn, B. (2012). Reliability and validity of the Classroom Observations of Student–Teacher Interactions (COSTI) for kindergarten reading instruction. Early Childhood Research Quarterly, 27(2), 316–328. https://doi.org/10.1016/j.ecresq.2011.09.004 Stein, M. K., & Kaufman, J. H. (2010). Selecting and supporting the use of mathematics curricula at scale. American Educational Research Journal, 47(3), 663–693. https://doi.org/10.3102/0002831209361210 Stein, M. K., Remillard, J., & Smith, M. S. (2007). How curriculum influences student learning. In F. K. Lester Jr. (Ed.), Second handbook of research on mathematics teaching and learning (pp. 319–370). Information Age. Wayne, A. J., & Youngs, P. (2003). Teacher characteristics and student achievement gains: A review. Review of Educational Research, 73(1), 89–122. https://doi.org/10.3102/00346543073001089

By Marah Sutherland; Ben Clarke; Derek B. Kosty; Scott K. Baker; Christian T. Doabler; Keith Smolkowski; Hank Fien and Joanna Goode

Reported by Author; Author; Author; Author; Author; Author; Author; Author

Titel:
Investigating the Interaction between Teacher Mathematics Content Knowledge and Curriculum on Instructional Behaviors and Student Achievement
Autor/in / Beteiligte Person: Sutherland, Marah ; Clarke, Ben ; Kosty, Derek B. ; Baker, Scott K. ; Doabler, Christian T. ; Smolkowski, Keith ; Fien, Hank ; Goode, Joanna
Link:
Zeitschrift: Elementary School Journal, Jg. 123 (2022-12-01), Heft 2, S. 292-317
Veröffentlichung: 2022
Medientyp: academicJournal
ISSN: 0013-5984 (print) ; 1554-8279 (electronic)
DOI: 10.1086/721877
Schlagwort:
  • Descriptors: Mathematics Education Mathematics Curriculum Pedagogical Content Knowledge Teacher Behavior Teaching Methods Mathematics Achievement Kindergarten Elementary School Teachers
Sonstiges:
  • Nachgewiesen in: ERIC
  • Sprachen: English
  • Language: English
  • Peer Reviewed: Y
  • Page Count: 26
  • Document Type: Journal Articles ; Reports - Research
  • Education Level: Early Childhood Education ; Elementary Education ; Kindergarten ; Primary Education
  • Abstractor: As Provided
  • Entry Date: 2022

Klicken Sie ein Format an und speichern Sie dann die Daten oder geben Sie eine Empfänger-Adresse ein und lassen Sie sich per Email zusenden.

oder
oder

Wählen Sie das für Sie passende Zitationsformat und kopieren Sie es dann in die Zwischenablage, lassen es sich per Mail zusenden oder speichern es als PDF-Datei.

oder
oder

Bitte prüfen Sie, ob die Zitation formal korrekt ist, bevor Sie sie in einer Arbeit verwenden. Benutzen Sie gegebenenfalls den "Exportieren"-Dialog, wenn Sie ein Literaturverwaltungsprogramm verwenden und die Zitat-Angaben selbst formatieren wollen.

xs 0 - 576
sm 576 - 768
md 768 - 992
lg 992 - 1200
xl 1200 - 1366
xxl 1366 -