Validating the Early Numeracy Teacher Rating Scale for Preschoolers (TRS-EN)

The aim of this study was to validate a teacher rating scale (TRS-EN) for assessing early numeracy skills and for identifying mathematically low-performing 3- and 4-year-old children. Fifty-six preschool teachers assessed children's (3-year-olds = 127; 4-year-olds = 145) numerical relational, counting, and basic arithmetic skills. Construct validity evidence through confirmatory factor analysis supported both unidimensional and three-factor (i.e. numerical relational, counting and basic arithmetic skills) models. Also, TRS-EN demonstrated excellent internal consistency in both age groups. The teacher assessment scores corresponded well with standardized direct early numeracy measures, and predicted children's early numeracy skills six months later. The results clearly demonstrate that the TRS-EN can be used to assist teachers in assessing early numeracy skills, and help with identifying mathematically low-performing children.

Keywords: Assessment; early childhood education; early numeracy; low-performing; psychometrics; teacher rating scale

Early numeracy (EN) skills are crucial for later academic performance. Individual differences in EN skills are visible already during the early years and these differences grow even larger throughout the years in education (Zhang et al. [45]). To recognize low-performing children early on to provide sufficient support in EN skills, researchers, as well as educators, need adequate and valid assessment tools (Drost [9]). Traditionally employed direct assessments (i.e. face-to-face administered interviews) may be perceived as time-consuming and challenging (Purpura et al. [34]) and, particularly in preschool-aged children, inattentiveness and distractibility may likely pose some challenges when administering direct assessment tools (Reid et al. [35]). Teacher rating scales provide a tempting alternative to bypass such challenges. Previous research has exhibited mixed results concerning the validity of preschooler's EN teacher rating scales (Baker et al. [4]; Lambert, Kim, and Burts [25]), and these have included several limitations, such as relatively narrow target groups, inadequate concurrent measures and including rather few items in the scales (Reid et al. [35]). Thus, this study aimed to address these limitations, and validate a teacher rating scale that enables teachers to assess 3- to 4-year-old children's EN skills and identify low-performing children at an early stage, without administering direct assessments.

Core EN skills refer to a group of numerical skills that are considered to be foundational for later mathematical development (Aunio and Räsänen [3]). These core EN skills consist of symbolic and non-symbolic number sense, numerical relational, counting and basic arithmetic skills that are known to predict later mathematical skills well (Aunio and Räsänen [3]).

Symbolic and non-symbolic number sense refers to the ability to recognize differences in numerosities (i.e. approximate number system; vanMarle [40]). Children start presenting such abilities already at infancy (vanMarle [40]) which is assumed to lay the foundation for later learning of mathematics (Aunio and Räsänen [3]).

Counting skills emerge around the age of 2, when children start to produce small number words and connect them into number word sequences (i.e. verbal counting), first by mixing the order (Gelman and Gallistel [18]). After learning to say number words, children start enumerating objects with the corresponding number words (i.e. one-to-one correspondence in counting; Wynn [42]). Prior to the age of 5 children start to recognize that the number word pronounced last, marks how many objects there are altogether (i.e. cardinality; Purpura et al. [34]; Wynn [42]). Further, children become aware that each number word represents a unique quantity and that numbers following one another add one to the prior number (i.e. understanding of cardinal principles; Chu et al. [6]; Gelman and Gallistel [18]; Wynn [43]).

Children must also understand how different numerals and quantities are related to one another (Wynn [44]). Understanding numerical relations requires skills of ordering (e.g. seriation, classifying) and comparing numerals and quantities, which children learn at an early age (Bryant [5]). Starting from numbers below 10, children start learning to connect number words into numerals and quantities into numerals (Wynn [42]) thus learning to identify for example that number 10 corresponds with 10 objects (Litkowski et al. [27]).

The knowledge of counting and numerical relations builds up into the knowledge of basic arithmetical skills (Starkey, Klein, and Wakeley [37]). Children are not expected to perform basic arithmetic operations in a formal manner in preschool, although many children start developing an understanding of them during the preschool years (Starkey, Klein, and Wakeley [37]). However, prior to age 5, children start to present non-symbolic addition skills (i.e. two dogs and three cats equals five animals) and symbolic addition skills (i.e. 2 + 3 = 5; Li et al. [26]). They also begin to recognize that the outcome in addition tasks is larger than either one of the addends (Gelman [17]).

All aforementioned EN skills are known to predict later mathematics learning and furthermore, precede the understanding of more complex mathematical skills (Chu et al. [6]; Geary et al. [16], [15]). Low performance in EN skills have been found to predict later mathematical learning difficulties in the early school years (Zhang et al. [45]). Although cutoffs for low-performing children vary in the literature from 15th to 30th percentiles (Hellstrand [20]), it appears important to identify low-performing children as early as possible. In order to identify children in need of support in EN skills, preschool teachers require adequate, reliable and valid tools for assessing children's skills (Drost [9]).

Direct assessment (i.e. standardized interview-based tests) requires thorough delving into the material, practicing and time for administering and scoring (Reid et al. [35]) and can thus be perceived as challenging and time-consuming (Purpura et al. [34]). Also, young children's developmental limitations such as inattentiveness and distractibility may pose significant constraints to administering direct assessment (Reid et al. [35]). Teachers hold valuable information about their development, which may easily get overlooked when administering direct assessments (Kowalski et al. [24]). Teacher rating scales, assessment tools that teachers can administer without direct assessment, can serve as a worthy alternative and an additional tool for assessing preschooler's EN skills. Teacher rating scales require little assessment training, are relatively easy to administer, low in expenses and efficient (Reid et al. [35]) as for assessing EN skills, teachers need to be familiar with the child and, also, capable of evaluating such skills.

Previous research on teacher rating scales on K-12 children's academic skills (i.e. language and mathematics) has demonstrated small to moderate correlations between direct assessments (i.e. mean correlation.62, range -.03 to.80; Südkamp, Kaiser, and Möller [38]). Only six teacher rating scales have focused on preschooler's EN skills (see Appendix 1). Results on whether teacher ratings correspond with direct assessment scores in preschooler's EN performance have been rather mixed (see Appendix 1; Concurrent validity). In addition, correlations between teacher rating scales and direct assessments have generally been smaller for preschooler's than for older students' (Südkamp, Kaiser, and Möller [38]). Prior studies have also suggested that teachers, studied in the United States, hold many misconceptions concerning preschool children's EN skills and that teachers are not able to differentiate between high- and low-performing children (Kilday et al. [22]).

Prior studies have had several limitations such as investigating the validity of the teacher rating scale on a relatively narrow population (e.g. low-income sample; Furnari et al. [13]), including limited contents of EN skills in the scales or a variety of early mathematical skills (Reid et al. [35]) or contrasted teacher rating scale scores to a direct measure with noncorresponding contents (Baker et al. [4]). Further, as participants in earlier studies have been exclusively from the US, and as both teacher education and preschool practices vary widely between countries, more research from other countries is needed (Niemi and Jakku-Sihvonen [31]). In addition, although children between the ages of 3 and 4 develop rapidly, exhibiting large individual differences, previous studies have not analyzed age groups separately (Litkowski et al. [27]). Some previous scales have also included up to 68 items per skills set (Kowalski et al. [24]) which suggests that administering these scales is neither easy nor time efficient. In addition, the extent to which the teacher rating scale scores predict later performance has only been investigated with rather a small subsample (i.e. predictive validity; Reid et al. [35]) and previous studies have not addressed which sub-skills of EN the teacher rating scale measures (i.e. construct validity). There is also a lack of research on the extent to which preschool teachers are able to identify children with low EN performance by using teacher rating scales.

By validating a teacher rating scale for assessing preschoolers' EN skills (TRS-EN), this study aimed to address the need for a valid and reliable but also an easy-to-use assessment tool for teachers to evaluate preschooler's EN skills and to identify low-performing children. Items for TRS-EN were developed in a group of specialists in special education and educational assessment from one Finnish university. Prior research on numeracy assessment tools (VanLuit, Van de Rijt, and Aunio [39]; Hellstrand et al. [21]; Lopez-Pedersen et al. [28]) formed a basis for the developed items.

Previous teacher rating scales have included a variety of early mathematical skills (see Appendix 1). As TRS-EN was also aimed to identify low-performing children, the items should reflect skills that best predict later mathematical performance (i.e. core EN skills). As the framework of Aunio and Räsänen ([3]) is focused on 5- to 8-year-olds, using a similar strategy, we searched for the EN skills that were seen as the most important for 3- to 4-year-old children. The development started with reviewing existing EN measures (Table 1) for the age groups in question.

Table 1. Existing Direct EN Measures Covering Age Groups 3–4.

Counting skills, understanding of numerical relations, basic arithmetic skills seemed to form the most important EN skills in 3- to 4-year-olds. This finding was contrasted to the results of longitudinal studies investigating which EN skills are the strongest predictors of later mathematical performance in 3- to 4-year-old children. The reviewed longitudinal studies that covered both 3- and 4-year-olds suggested that counting (Chu et al. [6]; Geary et al. [16]) and understanding numerical relations (Purpura et al. [33]; Scalise and Ramani [36]) were the strongest predictors of later mathematical performance in these age groups. As basic arithmetical performance is highly associated with counting skills and as research shows that children start presenting non-symbolic arithmetic skills prior to the age of 5, basic arithmetic skills were also included in the scale (Li et al. [26]; Litkowski et al. [27]). By this decision, a possible ceiling effect was also accounted for.

Based on these findings, 22 items concerning counting (13 items), numerical relational (7 items) and basic arithmetic skills (2 items) were formed. Before collecting the main data, two preschool teachers were asked to give feedback on TRS-EN. The two preschool teachers were asked whether the scale included all necessary aspects of 3- to 4-year-old's EN development and whether they thought that the items were comprehensible and could be relatively easily assessed if the child was familiar to the teacher. As these preschool teachers did not point out any details that would have to be changed, the original TRS-EN was used for the current sample.

The aim of this study was to validate a teacher rating scale for assessing EN skills and identifying low-performing 3- to 4-year-old children. Specifically, this study aimed to (1) investigate the construct validity (2) internal consistency, and (3) criterion-related validity in terms of the concurrent and predictive validity of TRS-EN. In relation to concurrent validity, families' socioeconomic statuses effect on teacher ratings was examined. The selected validity and reliability criteria used in this study were based on standards for educational and psychological testing (AERA, APA, and NCME [1]).

Individual differences in early numeracy start emerging already at preschool age and only widen throughout the school years. To provide support as early as possible, it is crucial to identify these low-performing children. This study contributes to the understanding of whether TRS-EN is a valid and reliable tool for teachers to assess children's EN skills and identify low-performing children. Furthermore, this study adds novel information concerning the ability of teacher rating scales to predict later EN performance. Also, this study adds knowledge of the capacity of TRS-EN to differentiate specific EN sub-skills among 3- and 4-year-old children. Results are examined independently for both age groups as preschool-aged children develop rapidly and present large individual differences (Litkowski et al. [27]).

This study was a part of the Active Early Numeracy project, investigating the associations between preschooler's motor skills, physical activity and academic skills. In total, 16 preschools participated in the project in 2020. A research permit was obtained from one large municipality, preschools as well as from the ethical board of the university. The municipality suggested preschools from diverse demographic areas in Southern-Finland. The participants were then recruited from the preschools that permitted the research. Permission forms were sent to children's parents specifying the project's aims and procedure. Parents were asked to discuss with their children whether they were willing to participate. Children were able to withdraw from participating at any time if they or their parents expressed such desire. Direct assessments were not conducted if the child seemed too tired or could not concentrate during that time.

Preschool teachers who were involved in the project were asked whether they thought that they were able to fill in teacher rating scales at the time being. Thus, participants in this study were 56 preschool teachers and 273 children (3-year-olds: n = 127; MAge = 3.37, SD = 0.29; 4-year-olds: n = 145; MAge = 4.36, SD = 0.27). Demographic characteristics of the sample are presented in Table 2. The participating children's home language and family's mean net income and parents' employment status represent the distribution of demographic features of the municipality in question well. However, guardians with higher education degrees were slightly overrepresented, whereas guardians with low educational degrees were slightly underrepresented in our sample.

Table 2. Socioeconomic Background of the Participants.

TRS-EN included 22 items measuring counting (13 items) for example 'is able to count forward from 1 to 10 correctly', numerical relational (7 items) for example 'is able to order objects according to a given criteria (e.g. size: from the biggest to the smallest)', and basic arithmetical skills (2 items) for example 'is able to add and subtract with numbers below 10 using objects (e.g. When playing, notices that there are six toy animals in total when there are 3 cats and 3 dogs)'. Counting and numerical relational skills were emphasized in test development (see Table 1). Two items concerning basic arithmetic skills were included to cover for large individual differences in EN skills and to avoid ceiling effects. Every item was assessed on a 3-point scale with the options being: never, often and always. The teacher's were also asked to fill in the preschool's name, playgroup, teacher's name, date and how long the teacher had worked with the child.

The Early Numeracy Test (ENT; VanLuit, Van de Rijt, and Aunio [39]) was used as a concurrent measure for measuring children's EN skills. ENT comprises 40 items that measure children's numerical relational and counting skills. The test is standardized for children between the ages of 4.5 and 7.5 years (VanLuit, Van de Rijt, and Aunio [39]). Aunio and colleagues ([2]) have recently used ENT with 3–4 year-olds successfully, in 3 year-olds numerical relational skills, and in 4 year-olds numerical relational and counting skills can be measured in a reliable way. For the current sample, ENT's internal consistency was good for 3-year-olds (i.e. numerical relational skills sub-scale), α =.743 and 4-year-olds, α =.891 (Table 3).

Table 3. Descriptive Statistics and Internal Consistency for the Assessments.

Participating children's parents were asked to report their employment status (i.e. unemployed, student, part-time and full-time), educational level (i.e. comprehensive school or none, upper secondary or vocational school, bachelor's degree and graduate degree), home language (i.e. Finnish or other) and income level (i.e. below 19 999€/year, 20 000–39 999 €/year; 40 000–59 999€/year; 60 000–79 999€/year; above 80 000 €/year) on a form that was sent to children's homes. Family educational level and employment status was determined by using the higher education and employment status if two distinct levels of statuses were reported. Parent's higher educational level and employment status was coded to indicate family educational level and employment status.

The scales were organized into envelopes according to playgroups. Two members of the research group delivered and collected the envelopes in June 2020 as it was not possible to visit preschools from March to July 2020 due to the COVID-19 outbreak. Children's names were written on the scales beforehand. A cover letter including instructions for how to fill in the scale was also handed out to the teachers. The teachers were asked to fill in the scales based on their experience of working with the child. Completing TRS-EN took approximately 10 to 15 minutes per child. Teachers were encouraged to contact the research group if they had any questions concerning the scale.

Direct assessments were administered individually in a quiet location in the children's own preschools during regular preschool hours. All of the assessments were administered by trained research assistants from November 2019 to March 2020 and for predictive validity from November 2020 to June 2021.

First, the distribution and outliers of the data were examined and it was confirmed that there were no significant outliers in the data. To examine the construct validity of TRS-EN, confirmatory factor analysis (CFA) was conducted by using the weighted least square mean and variance adjusted (WLSMV) estimator in Mplus version 8, to consider the non-normally distributed items in the data. Cluster sampling due to teacher evaluation was considered by using the type = complex command, which takes into account the non-independence of observations (Muthén and Muthén BO [30]-[30]). Comparative fit index (CFI), Tucker-Lewis index (TLI) and Root mean square error of approximation (RMSEA) were used to examine the goodness of model fit. CFI/TLI of >.90 and RMSEA <.08 are considered as representing adequate model fit (Marsh et al. [29]). Internal consistency was evaluated using Cronbach's alpha. Alpha values above.70 are considered acceptable (Kline [23]).

Criterion-related validity was measured in terms of concurrent and predictive validity. Concurrent validity was measured using correlation analyses between TRS-EN and direct assessments. Correlation coefficients below.40 are considered small, above.40 moderate and above.60 strong (Evans [11]). Concurrent validity and internal consistency analyses were conducted using SPSS version 26. For predictive validity, a path analysis was conducted to indicate whether teacher ratings at the first time point predicted teacher ratings and direct EN measure scores at the second time point. Teacher ratings from the first time point were set as an independent variable and direct EN measure scores and teacher ratings from the second time point were set as dependent variables in the same model. The maximum likelihood (ML) estimation in Mplus version 8 was used.

In order to test whether TRS-EN would identify the same low-performing children as the direct EN measure, a configural frequency analysis (von Eye [41]) was conducted. The children were grouped into low-performing and typically performing groups in TRS-EN and direct EN measure based on a 20th percentile cutoff derived from prior research (Geary [14]; Hellstrand [20]; Lopez-Pedersen et al. [28]). For 3-year-olds, a numerical relational skills sub-scale was used to categorize children into low-performing and typically performing groups. By means of configural frequency analysis, observed frequencies are compared to the expected frequencies by a crosstabulation. This analysis allows us to examine whether observed frequencies are larger or smaller than could be expected by a base model (von Eye [41]). First-order configural frequency analysis base model was adopted to focus on children's individual stability (i.e. whether the same children identified as low-performing or typically performing with the TRS-EN, were also identified as low-performing or typically performing in the direct EN measure) and individual change (i.e. whether changes across the groups that cannot be explained by random fluctuation occur). Bonferroni correction was applied to the significance testing (.05/4 =.013) to account for the increased risk of Type-I error.

By means of multivariate regression, it was tested whether teacher ratings were affected by the socioeconomic status (SES) of the family (i.e. parents' education and employment status, first language and income level; see Table 2). Maximum likelihood estimation with robust standard errors (MLR) in Mplus version 8 was applied due to skewed SES variables. Family SES variables and children's direct EN measure scores were set as independent variables and TRS-EN scores as dependent variables. Independent variables were allowed to correlate by using with-statements.

Descriptive statistics as well as Cronbach's alphas for TRS-EN and direct EN measure concurrently and subsequently are presented in Table 3. As expected, 4-year-olds presented higher mean scores in the TRS-EN overall, as well as in all sub-skills, than 3-year-olds.

Items that were developed for the TRS-EN were hypothesized to represent a three-factor model (i.e. counting, numerical relational, and basic arithmetic skills) which fitted the data well (see Table 4). Factor loadings were significant (p <.001) and above.561 among the 3-year-olds and above.669 among the 4-year-olds' (see Appendix 3). However, in both age groups, the latent factors were highly correlated and, therefore, a unidimensional model was also tested, which resulted in a good model fit (see Table 4). In the one-factor model, the factor loadings were significant (p <.001) and above.535 for 3-year-olds and above.664 for 4-year-olds (Appendix 3). Thus, both the one-factor and three-factor models presented an acceptable model fit. The significance of the differences between the two models was tested with the DIFFTEST-option in Mplus. Results revealed that the three-factor model fitted the data significantly better compared to the one-factor model in both 3-year-olds, DIFFTEST (3) 26.06, p<.001 and in 4-year-olds, DIFFTEST (3) 16.00, p =.0011.

Table 4. Model Fit Indices for One- and Three-factor Models for Both Age Groups.

In both models, one item (item 6), measuring backwards rote counting from 20 to 1 was deleted due to its difficulty (M = 0.16, SD = 0.456) and one item (item 14) measuring children's ability to classify objects according to different principles due to its easiness (M = 1.73, SD = 0.460).

Cronbach's alphas for both factor solutions of the TRS-EN are presented in Table 3. Excellent internal consistency was obtained for the TRS-EN in both age groups. Counting skills factor presented the highest internal consistency and basic arithmetical skills the lowest.

Correlation coefficients between TRS-EN and the direct EN measure are presented in Appendix 2. TRS-EN in total exhibited moderate to strong correlations with the direct EN measure. Moderate correlations were also obtained for 4-year-olds between the TRS-EN counting and numerical relational skills factor and the corresponding factors in the direct EN measure. Also, 3-year-olds' TRS-EN numerical relational skills factor correlated moderately with the corresponding factor of the direct EN measure.

To test whether the TRS-EN would identify the same low-performing children as the direct EN measure, the children were grouped into low-performing and typically performing children in both tests using the 20th percentile as a cutoff. This grouping resulted in four configurations (typical/typical; typical/low; low/typical; and low/low). By means of configural frequency analysis, the observed frequencies were compared to the expected frequencies (see Table 5).

Table 5. Change and Stability in Performance Groups Across TRS-EN and Direct EN Measure.

1 Note: ENT = Early Numeracy Test. For 3-year-olds, numerical relational skills sub-scale was used to categorize children into low-performing and typically performing groups.

For 3-year-olds, the configural frequency analysis identified two stable configurations and two non-significant configurations. Of the 18 children who were identified as low-performing with TRS-EN, 12 (66%) were also identified as low-performing with the direct EN measure. Also, of the 78 children who were identified as typically performing with TRS-EN, 60 (77%) were also identified as typically performing with the direct EN measure. These configurations were occurring more than expected by chance, indicating stability in the classification of low-performing and typically performing status.

For 4-year-olds the configural frequency analysis identified two stable configurations and two non-significant configurations. Of the 19 children who were identified as low-performing with TRS-EN, 9 (47%) were also identified as low-performing with the direct EN measure. Also, of the 84 children who were identified as typically performing with TRS-EN, 71 (85%) were also identified as typically performing with the direct EN measure.

Descriptive statistics for predictive validity are presented in Table 3. Repeated measures ANOVA indicated that TRS-EN scores increased from the first measurement to the second among 3-year-olds, F(1,32) 109.96, p <.001, ηp2 =.775 and 4-year-olds, F(1,23) 33.80, p <.001, ηp2 =.595. Results from the path analysis indicated that among 3-year-olds teacher ratings in the first time point predicted significantly both direct EN measure scores (β = 684, p <.001) and teacher ratings (β = 615, p <.001) in the second time point. Teacher ratings in the first time point explained a significant amount of variance in the second time point direct EN measure scores (R2 =.532, p <.001) and teacher ratings (R2 =.622, p <.001).

Also, among 4-year-olds teacher ratings in the first time point predicted significantly both direct EN measure scores (β = 625, p <.001) and teacher ratings (β = 715, p <.001) in the second time point. Teacher ratings in the first time point explained a significant amount of variance in the second time point direct EN measure scores (R2 =.390, p <.001) and teacher ratings (R2 =.512, p <.001).

To check whether teacher ratings may be affected by family SES, a multivariate regression analysis was applied, where both family SES and direct EN skills were included as independent variables. Both models were saturated, as the number of observed variables was equal to the number of the estimated parameters. To examine model fit indices, we ran the model for both age groups with only numerical relational skills (3-year-olds) or direct EN measure scores (4-year-olds) predicting teacher ratings. Model fit indices were excellent for both, 3-year-olds, CFI = 1.00, TLI = 1.00 and RMSEA <.001 and 4-year-olds, CFI =.977, TLI =.913 and RMSEA =.075. Results confirmed (see Table 6) that the direct EN measure was the only significant predictor of teacher ratings at the first time point, indicating that teacher ratings were affected by the child's actual EN skills measured by the direct measure, rather than the family SES indicators. The model explained 36% of the variance in 3-year-olds and 44% of 4-year-olds teacher ratings. ENT correlated significantly with all other SES variables than parents' employment status.

Table 6. Summary of ENT and SES variables predicting teacher ratings.

2 Note: ENT = Early Numeracy Test.

The aim of this study was to validate a teacher rating scale for assessing 3- and 4-year-old children's EN skills and to be able to identify mathematically low-performing children. The results indicated that the TRS-EN can be used to measure EN skills both as a composite, and also, as three separate constructs measuring counting, numerical relational, and basic arithmetic skills. Although the three-factor solution fitted the data slightly better for both 3- and 4-year-old children, the TRS-EN may well be used to measure children's EN skills as a composite as well, as it allows preschool teachers to efficiently evaluate children's overall skills contrasted to other children. However, by considering the three factors separately, the TRS-EN offers teachers the possibility to examine sub-skills of EN in greater detail to target individualized support at a specific sub-skill area. As prior studies have not examined EN teacher assessments separately for different age groups, the results of this study add novel information on how teacher rating scales can be appropriately used for assessing 3- and 4-year-olds EN skills.

The TRS-EN presented excellent internal consistency parallel to prior findings (Baker et al. [4]; Lambert, Kim, and Burts [25]), indicating that the items are consistent with one another inside the constructs. However, there were some differences in the extent of internal consistency between counting, numerical relational, and basic arithmetic skills items, with counting presenting the highest internal consistency. This finding is supported by the fact that a larger number of items generally contributes to higher internal consistency (Duhachek, Coughlan, and Iacobucci [10]).

Further, the results also revealed that teacher ratings corresponded well with the direct EN scores and predicted children's later EN skills, confirming that the TRS-EN scores reflect the ones of the widely used EN measure (see AERA, APA, and NCME [1]). Although previous research has shown similar results, teacher ratings in this study seemed to correspond more strongly with the direct EN measure scores when compared to previous studies (e.g. Baker et al. [4]; Reid et al. [35]). Also, children who were identified as low-performing with the TRS-EN were also likely to perform in the lowest 20th percentile measured with the direct EN measure. Similarly, children who were identified as typically performing with the TRS-EN, were likely to present typical performance also measured with the direct EN measure. This finding indicates that TRS-EN through the observations of the teachers was able to differentiate between low-performing and typically performing children quite well.

Prior studies from the US have suggested that teachers may have challenges in identifying individual differences in early numeracy (Kilday et al. [22]). Conversely, the results of this study indicate that teachers are relatively well aware of children's skills, and also, that teacher's assessments do not seem to be affected by family SES, when controlling for direct EN skills. This may have resulted from preschool teachers in Finland receiving their bachelor's degrees in early childhood education from universities, and have thus received instruction on children's EN skills development, assessment and how to recognize children with lower numeracy (Niemi and Jakku-Sihvonen [31]).

Despite the promising results, our study has some limitations. One limitation concerning concurrent validity is that teachers assessed children's academic skills a few months after the direct assessments were administered. This may have led to small differences between the results as children develop rapidly at this age (Litkowski et al. [27]). Also, it should be recognized that though children were likely to belong to the same performance group within both measures, there are children that were placed in different performance groups with the teacher rating scale and direct EN measure. We recognize that especially psychometric measures contain an error. Early childhood educators should evaluate children's EN skills using different measures and by working within a groups of professionals and parents if a concern is raised on a child's development. Thus, TRS-EN should be seen as an additional tool for evaluating children's EN skills, not an alternative. TRS-EN may be of especial use in the beginning of the process for early childhood educators when a concern on a child's EN development is raised. It is notable, that only 36% (3-year-olds) and 44% (4-year-olds) of variance in teacher ratings was explained by our direct EN measure and SES variables. One possible explanation of the low explained variance in 3-year-olds could be only using numerical relational skills as an indicator of early numeracy skills though TRS-EN represented numerical relational, counting and basic arithmetic skills. However, as previous research has indicated that teacher ratings may be affected largely by children's background as opposed to directly measured EN skills, our findings are somewhat encouraging, as children's SES did not appear to influence teacher ratings (Furnari et al. [13]). There was also quite a high dropout rate from the first time point into the second time point, which may well have affected the results of the predictive validity. Another limitation is that although the preschools were from diverse areas, our sample did not fully represent the demographic features of the municipality in question as parents with high educational degrees were slightly overrepresented, and parents with low educational degrees were somewhat underrepresented. When addressing the dimensionality of the TRS-EN, two items had to be removed, one due to its difficulty (i.e. backwards counting from 20 to 1) and one due to its easiness (i.e. categorizing objects into groups). Therefore, the age appropriateness of the items should be carefully considered when developing EN measures for certain age groups.

Future studies should consider the possibility of using teacher rating scales as assessment tools for assessing intervention effects. For example, in single case studies where multiple baseline assessments are recommended, teacher rating scales could provide a cost- and time-efficient instrument for monitoring children's progress and intervention effects.

As concluding remarks, the results of this study suggest that the TRS-EN is a viable tool for teachers to evaluate children's early numeracy skills and identifying low-performing children. As individual differences in early numeracy become visible during the early years and only widen throughout the school years, TRS-EN can provide information on which EN skills should be further addressed in the instruction. Furthermore, TRS-EN can help to identify low-performing children at an early stage.

The authors would like to thank all contributors involved in collecting the data for the current manuscript. In addition, the authors thank all participants in the research group's seminars for their unwavering support through commenting.

No potential conflict of interest was reported by the author(s).

Summary of Previous Studies on Preschoolers' EN Skills Teacher Rating Scales.

3 Note. SES = Socioeconomic status.

Correlations Between EN Assessments for 3- and 4-year-old Children Separately.

• 4 Note. The results for the 4-year-olds are shown above the diagonal. The results for the 3-year-olds are shown below the diagonal.
• 5 ENT = Early Numeracy Test.
• 6 * p <.05. ** p <.001

Factor Loadings, R2 and Factor Correlations in the One- and Three-factor Models.