Neurobehavioral Factors Associated with Referral for Learning Problems in a Community Sample: Evidence for an Adaptational Model for Learning Disorders.

We evaluated community general education (CGE; n = 178), community special education (CSE; n = 30) and hospital-referred (HR, n = 145) children (ages 7-6 to 11-11) prospectively over a 2-year period. During this period, 17 CGE children were referred for evaluation (community referred; CR). Prior to referral, CR children performed more poorly than community-nonreferred (CNR) children on cognitive ability, academic achievement, attention problems, and information processing. CR group performance was equivalent to that of CSE and HR groups, but HR children showed poorer academic achievement. Referred children performed more poorly on all measures than nonreferred, whether they met formal diagnostic criteria for a learning disorder or not. Learning disorders may be better conceptualized as a context-dependent problem of functional adaptation than as a disability analogous to physical disabilities, raising questions about the validity of using psychometric test scores as the criterion for identification.

Identification remains one of the central dilemmas in the clinical management of children with learning disabilities (LD). Most observers would subscribe to some version of the premise that learning disabilities are distinguished by unexplained school difficulties in the context of an otherwise cognitively typical child. It is also presumed that this academic difficulty is biologically based (Kirk, 1962). In most states, these assumptions are translated into a categorical standard for eligibility, requiring that a child demonstrate a statistically reliable discrepancy of some specified magnitude between ability and achievement on standardized testing in order to be classified as having LD (Frankenberger & Fronzaglio, 1991).

Although the discrepancy criterion is generally regarded as a gold standard, it has also been the source of considerable controversy and dissatisfaction. Cogent arguments have been leveled on a psychometric basis against the validity of the discrepancy criterion. Prominent researchers have pointed out that discrepant readers and garden-variety poor readers are differentiated largely by IQ (the former by mathematical necessity, having higher IQ scores) but share a core deficit in phonological processing (Fletcher et al., 1994; Shaywitz, Fletcher, Holahan & Shaywitz, 1992; Stanovich & Siegel, 1994; Stuebing et al., 2002).

The ecological validity of these formal criteria, moreover, is problematic. Studies that have assessed the correspondence between the standard definitions and the actual school practices in eligibility determinations have documented a wide gap between the psychometric definitions of LD, which in theory define the disorder, and the actual decisions of school assessment teams (Bocian, Beebe, MacMillan, & Gresham, 1999; Gresham, MacMillan, & Bocian, 1998; Kavale & Reese, 1992; MacMillan, Gresham, & Bocian, 1998). MacMillan et al. (1998) found that only 30% of their total sample of referred children in public schools in Southern California met the formal criteria for LD, yet 54% were formally identified as having LD by school assessment teams. Kavale and Reese (1992) reported similar statistics in Iowa.

At first blush, the school-based decisions might appear arbitrary or ill informed, resulting in the inappropriate overidentification of children as having LD. Bocian et al. (1999), however, studied these referred children further, operationalizing the criteria associated with the three main phases of the identification process: teacher referral, psychoeducational testing, and the team determination of special education eligibility. It is not surprising that they found only modest consistency across the three paradigms. Despite this lack of agreement, their data did reveal a fair degree of systematic behavior. Teachers and schools appeared to factor achievement and observed classroom behavior together in reaching their decisions. The ultimate determination was further influenced by important contextual factors, such as the normative performance of children at the local level, program availability, and the relative skill of particular classroom and special education teachers within the school itself, in addition to other cultural and socioeconomic considerations. "Schools," Bocian et al. commented, "seek flexibility and the opportunity to exercise professional judgment rather than a rigid code of precise formulas" (p. 12). They noted further that "teachers may be 'imperfect tests,' but in terms of classroom relevance, their perceptions outrank student performance on isolated tasks in ideal, pristine conditions" (p. 12).

These findings, based on studies focused at the level of the individual child, echo patterns at the state level. Lester and Kelman (1997) examined the predictors of interstate disparities in the identification of children with LD and contrasted them with the predictors of identification for children with the most common categories of physical disabilities. Whereas demographic and sociopolitical factors reliably predicted interstate disparities for the diagnosis of LD, the same contextual factors contributed little to the identification of children with physical disabilities. These data argue further that the identification of children as having LD is a complex process in which contextual factors play a far greater role than is commonly acknowledged.

The inability of psychometrically constructed definitional systems to adequately address the LD identification problem, therefore, may not be a function of the particular algorithm used (e.g., discrepancy vs. low achievement definitions). Because these criteria can be insensitive to context, they may by their very nature fail to capture the functional impact of the broader repertoire of cognitive and behavioral problems that bring children to clinical attention for learning difficulties in diverse educational settings. Although this repertoire often includes deficits in discrete academic skills, it is rarely limited to these deficits (Aloyzy-Zera, 2001; Morris et al., 1998; Stanovich & Siegel, 1994). Moreover, for children with LD, behavioral characteristics outside the domain of discrete skills, such as task orientation and inattention, can be significant prognostic indicators of ultimate academic success (McKinney, Osborne, & Schulte, 1993). The extent to which fixed, psychometrically referenced criteria capture the phenomena that actually bring children to clinical attention or interfere with their academic success may thus be problematic.

The gap between the rigorous psychometric definitions and actual school practices has implications extending beyond decisions about eligibility for services. MacMillan et al. (1998) raised concerns about the vast LD research literature, most of which adheres to psychometric definitions to determine participants' eligibility for research studies. "It would seem," they wrote, "that school identification practices and research enterprises are on divergent paths.... As the gap widens, re search findings from studies examining research-identified samples apply less and less to the population of children being served in the public schools as LD" (p. 324).

Like the rest of the LD literature, research on the neurobehavioral aspects of learning disabilities has relied almost exclusively on psychometric definitions to identify study samples who may not be representative of the children who are actually being identified by schools and families. The obvious concern is that this research has limited clinical relevance, as it is based on only a subset of the children with whom schools, and especially evaluators, actually deal.

Given this dilemma, those who wish to do research on neurobehavioral processes in children with learning problems have several options. First, they could continue to respect the strict legal definitions of LD and hope that these definitions will be adhered to in practice, thereby conferring validity on their data. Second, they could finesse the issue of LD by focusing on specific skill areas. The latter is the prevalent strategy among researchers who are interested in the neural underpinnings of LD, who have focused almost without exception on reading skill. A third option, which has not yet been pursued, is to study neurobehavioral processes among the children who are actually referred by schools and parents, treating their diagnostic status, vis-à-vis standard categorical criteria, as one of a number of outcome variables. This strategy allows for the investigation of neurobehavioral factors that may be of sufficient ecological significance to trigger a referral.

The present study employs this third approach. Its obvious disadvantage is that the reasons for referral typically vary and are often subjective. Characterizing the referred group can therefore be difficult, and replication is a significant problem. It is also difficult to generalize from a specific referred population to the larger population of children. The advantage, however, is that it provides data on the broader range of children who are actually experiencing learning problems in school, not just those children who meet an a priori set of psychometric criteria. It can potentially have greater ecological validity, therefore, than the approaches that stress fixed criteria.

This study was conceptualized within the systemic model of neuropsychological development (Bernstein, 2000; Bernstein & Waber, 1997; Holmes-Bernstein & Waber, 1990). This model, which was formulated in the tradition of Luria and Vygotsky, places primary emphasis on the transaction between the child and the environmental and experiential context. The primary focus, therefore, is not a fixed disability or defect within the child, but the dynamic, developmental interaction between the child and the schooling process. Within this framework, the learning problem is not regarded as analogous to a physical disability; rather, it is a problem of adaptation. Children with learning problems come to be identified (referred) because their complement of skills (biological risk) interferes with their ability to adapt successfully to the demands and expectations of their particular educational context. These skills can include, but are by no means limited to, discrete academic skills. The essential problem to be diagnosed, then, is not a disability, but a lack of fit between the child's complement of skills (broadly defined) and the socially determined expectations and demands of that child's setting. Identification and diagnosis thus become a joint function of the child and the setting.

This contextually referenced model of neuropsychological development is compatible with the educational model laid out by Bocian et al. (1999), who examined the consistency of LD identification across paradigms. Referral, they suggested, is guided by relativity --that is, the teacher's perception that outside assistance is necessary to help narrow the gap between a child's academic function and that child's particular peer group. Psychometric testing, they argued, is intended to determine acceptability--to "detect or document the existence of a within-child problem," (p. 2) relative to legally mandated, objective criteria that are not referenced to local norms and expectations. Finally, the team decision is motivated by profitability --that is, whether available special education services would benefit that particular child. The identification process, therefore, is in practice based on decision criteria that are not fixed but highly contextually bound. Moreover, referral and decision--the phases of the process most relevant to clinical management--are the more subjective components of the process.

We undertook a multidisciplinary research program to investigate the neurodevelopmental basis of learning problems in school-age children. In line with our theoretical perspective, children were eligible for inclusion in our study based on the fact that they had been referred for evaluation of a suspected learning problem. Thus, referral for evaluation was regarded as the primary indicator of the criterial lack of fit between the child's competencies and the demands of that child's educational setting.

To date, this research program has yielded findings that are generally compatible with the model. In one study, for example, we focused on the neuropsychological characteristics of children who were referred for learning problems but whose performance on standardized tests of academic achievement was at least average for their age. We compared a group of referred children with low academic achievement to a group of referred children who demonstrated adequate achievement (Morgan, Harris, Bernstein, & Waber, 2000). The adequate-and low-achieving referred children demonstrated remarkably similar neuropsychological profiles, with prominent difficulties in organization and fluency of output, on-line activities that can be expected to influence their effectiveness in the classroom. Although both groups' profiles were similar, the scores of the adequate-achieving referred children were generally higher. Even so, the adequate achievers demonstrated diminished organization and output efficiency relative both to their IQ levels and to the normative expectation for their age.

In a companion study (Singer-Harris, Forbes, Weiler, Bellinger, & Waber, 2001), the performance of adequate-achieving referred children was compared to that of nonreferred children matched for single-word reading ability on measures of low-level information processing. Although both groups' reading scores were identical, the referred group demonstrated poorer performance on the information processing tasks, more like that of the low-achieving referred children.

In a third study (Waber, Wolff, Forbes, & Weiler, 2000), we demonstrated that diminished visual naming speed (measured by the Rapid Automatized Naming test; Denckla & Rudel, 1976), generally viewed as a risk factor for reading impairment (Wolf & Bowers, 1999), is an excellent predictor of referral for evaluation in general. Even referred children who were adequate readers demonstrated slower naming speed than their nonreferred peers. Among the referred children, however, naming speed was not an accurate predictor of the presence of a reading impairment.

Thus, children who were referred for evaluation of learning problems exhibited processing inefficiencies whether they exhibited frank deficits on isolated measures of academic skill or not. These inefficiencies could contribute to their struggles in their real-world school environments, resulting eventually in referral for evaluation. Consistent with the systemic model, these findings support the assertion that children referred for evaluation have neuropsychological characteristics that put them at risk for failure to adapt in the academic setting, whether they meet standard psychometric criteria for a diagnosis of LD or low achievement or not.

In the present study, we further evaluated the model, this time using a prospective design. We focused on two key premises of the model. The first premise is that although academic skill deficits are important elements of the process whereby a child is referred for evaluation, other neurobehaviorally based processes can enhance or diminish the risk of adaptive problems in the academic setting. A second premise is that the identification process is highly context dependent and will not easily conform to fixed diagnostic criteria. Which children come to be identified can be a function of local norms and expectations and, thus, could theoretically vary from community to community. This context dependence is not limited to skills that require instruction, such as reading or mathematics, but can also be demonstrated in terms of more basic neurobehavioral risk factors.

This study involved a large, community-based sample of elementary school-age children, none of whom had a prior referral for evaluation of learning problems when they were recruited to the study. The children were evaluated twice over a 2-year period. During the interim between the first and second evaluations, some of these children were referred for evaluation of learning problems.

This design provided documentation of function in four key areas: cognitive ability, academic achievement, attentional behavior, and low-level information processing. The information processing tasks were novel and nonlinguistic, with outcomes measured in terms of relatively subtle differences in timing of input or output. They could thus be interpreted as indicative of the integrity of neurobehavioral function. We compared the performance on these measures of the newly referred children to that of the nonreferred children from the same community. We also compared their performance to that of previously referred children from the same community, who were already receiving special education services for learning problems at the time of their initial recruitment to the study, and to that of a group of children who had been referred to a hospital-based clinic for evaluation. Referred children from different settings were included in order to evaluate the extent to which the community-referred children were representative of referred children more generally.

On the basis of the two key premises cited earlier, the following specific predictions were made:

Community General Education. Children between the ages of 7 years 6 months and 11 years 11 months were recruited from a public school system in the metropolitan Boston area. They were included in the sample if their estimated IQ scores based on the Kaufman Brief Intelligence Test (K - BIT; Kaufman & Kaufman, 1990) were greater than 80; they were not on psychoactive medication; there was no evidence of neurological impairment based on a medical history obtained from a parent questionnaire; English was their primary language; and they were not enrolled in special education.

Children who were included also obtained a score below 6 on the Hyperactivity scale for either the parent or teacher version of the Diagnostic Rating Scale (DRS; Weiler et al., 1999; Weiler, Bellinger, et al., 2000). Children who met criteria for attention-deficit/ hyperactivity disorder (ADHD), hyperactive or combined type, according to the Diagnostic and Statistical Manual of Mental Disorders, fourth edition (DSM-IV; American Psychiatric Association, 1994), were excluded from the larger project because of concerns about the complicating effects of stimulant medication and the ability of the children to comply with the lengthy behavioral protocol. Children who met DSM-IV criteria for ADHD, inattentive subtype, were not excluded unless they were on stimulant medication at the time of recruitment. The decision was made to include these children because compliance would not be a problem for them and because some investigators have suggested that this subtype may be closer to a learning disorder (Barkley, 1998). As we have reported elsewhere, this subgroup of children did not have difficulty sustaining attention throughout the tasks (Weiler, Bernstein, Bellinger, & Waber, 2002). This group included 201 children.

Community Special Education. These children were recruited from the same school system as the community general education group and met virtually all the same inclusion and exclusion criteria, except that they were already enrolled in special education at the time of the initial recruitment. School records were reviewed to ascertain the reason for the special education designation. Children were included in this group only if the records indicated that the child was receiving special education for learning problems. No child carried a diagnosis of ADHD, combined or hyperactive type, and none had a diagnosis of primary behavioral or emotional problems. This group included 32 children.

Hospital Referred. Children were recruited from outpatient programs for the evaluation of children with learning problems at Children's Hospital, Boston. These children met the same inclusion criteria as the community groups, with the following differences: IQ estimates were based on the Wechsler Intelligence Scale for Children, third edition (WISC-III; Wechsler, 1991), Full Scale IQ; neurological impairment was documented by examination in addition to history; the presence of significant behavior disturbance was evaluated based on the Behavioral Symptoms index of the Behavioral Assessment System for Children, parent version (BASC; Reynolds & Kamphaus, 1992). This group included 203 children.

All children were evaluated at the time of recruitment and again 2 years later. Figure 1 shows the number of children who participated in the baseline (Time 1) and follow-up (Time 2) testing. As part of the follow-up evaluation, the parents of the children in the community general education group were asked whether their child had been referred for evaluation of a learning problem since the previous visit and, if so, to describe the problem that prompted the referral. This process identified 17 children (8% of the total sample) who had been referred for evaluation of learning or attention problems. These children's performance on the cognitive ability, attention, information processing, and achievement measures was then compared with that of the rest of the community general education group as well as with that of the community special education group and the hospital-referred group. The present study is based only on those children who participated in both the baseline and the follow-up testing.

Table 1 displays age, gender, and demographic characteristics of the children in the four resulting groups: community referred (CR); community nonreferred (CNR); community special education (CSE); and hospital referred (HR). There was a higher proportion of girls in the CR group than in the other two referred groups. Also, maternal occupational status was higher in the HR group than in the CSE group. Otherwise, there were no significant differences.

Fourteen (82%) of the 17 children in the CR group were enrolled in a formal special education program at the second evaluation. Children in the CR group tended to be among the younger children in the sample; 71% were in the second or third grade (as opposed to the fourth or fifth) at the time of initial recruitment.

Measures were selected to assess four key areas: cognitive ability, academic achievement, attention, and low-level information processing. An ancillary set of measures related to reading was also administered to provide a more detailed description of the different groups.

Cognitive Ability. Cognitive ability was assessed by the Kaufman Brief Intelligence Test (K-BIT; Kaufman & Kaufman, 1990), which includes a Verbal scale, a Matrices scale, and a Composite scale.

Academic Achievement. The Wechsler Individual Achievement Test (WIAT; Wechsler, 1992) was used to measure academic achievement. The WIAT includes three subtests: Basic Reading, a measure of single-word reading; Spelling, a measure of dictated spelling; and Calculation, a measure of computational skills.

Attention Problems. The prevalence of attention problems was measured by the Diagnostic Rating Scale (DRS), a DSM-IV-referenced questionnaire that has a parent and a teacher version (Weiler et al., 1999; Weiler, Bellinger, et al., 2000). The scales are an adaptation of the ADHD Teacher Scale (Wolraich, Hannah, Baumgartel, & Feurer, 1998). The original scale included questions representing the 18 DSM-IV criteria for the predominantly inattentive and the predominantly hyperactive-impulsive ADHD subtypes, the 8 DSM-IV criteria for oppositional-defiant disorder, and 7 of the 15 DSM-IV criteria for conduct disorder, as well as 7 questions to screen for anxiety and depression (Wolraich et al., 1998). The DRS includes the following modifications to the original scale: the term ADHD was eliminated from the form; the language was changed so that the questions exactly matched the wording of the DSM-IV; the questions were reordered so that items pertaining to diagnostic categories were not all grouped together; and comparable parent and teacher versions were developed. For the present study, we evaluated the Hyperactivity and Inattention scales.

Low-Level Information Processing. The Low-Level Information Processing (LLIP) battery is an integrated, computer-based test battery designed to collect information on basic aspects of information processing in school-age children. It includes four tasks presented in the context of a space adventure, where the child's mission is to deliver medical supplies to a dying planet. A second version, an undersea adventure, was developed for the follow-up testing to maintain the children's interest and motivation. Although the adventure contexts were different for the two versions, the information processing tasks were identical, and they were administered in the same order. All the tasks were novel and nonverbal and measured basic aspects of information processing. Two tasks were chosen because prior studies had demonstrated that they were associated with reading skill (Rapid Auditory Processing : Tallal, 1980; and Motor Timing Control : Wolff, Michel, Ovrut, & Drake, 1990). The other two tasks were chosen because they were likely to be sensitive to information processing problems in children with LD. The Serial Reaction Time task was included to measure the sensitivity to implicit sequential order. The Visual Filtering task measured the efficiency of visual serial search. For the purposes of the present study, a single dependent variable was selected for each task, based on prior analyses that indicated the variable that best discriminated children with and without LD (Waber et al., 2002; Waber, Weiler, Singer-Harris, et al., 2000; Waber et al., 2001; Weiler et al., 2000).

The test battery was presented by a Hewlett-Packard Vectra VL 5/133 MT Series 4™ computer on a Mitsubishi Diamond Scan 15 HX (Model SD5700C)™ monitor with a 35-cm diagonal viewable image. The children were seated approximately 75 cm from the screen. Responses were recorded on one of several custom response boxes, depending on the particular task. The software enabled response time (RT) to be measured accurately to within 1 millisecond.

Rapid auditory processing (RAP). This task (Waber et al., 2001) was modeled after that of Tallal (1980). The child was to indicate by a button press whether pairs of rapidly presented, brief complex-tone stimuli were the same or different. There were 72 trials (combining three stimulus durations and three interstimulus intervals), and the dependent variable was the total number of errors.

Motor timing control (MTC). The child was trained to finger-tap to the beat of a metronome and instructed to maintain the rhythm after the metronome was turned off (Waber, Weiler, et al., 2000). The child tapped with his or her index finger on a pressure-sensitive plate in time to a metronome signal delivered by the computer through headphones. There were three symmetric alternation trials (2 Hz, 3 Hz, 4 Hz) and three asymmetric (two taps on one hand for every one tap on the other) alternation trials for each hand. For each trial, the child listened to the metronome for 10 seconds, tapped in unison with the beat for 10 seconds, and then continued tapping at the same rate for 20 seconds with the metronome signal turned off. The data from the 20-second metronome-off interval were used for analysis. The dependent variable was the stability of the child's tapping rate, measured as the coefficient of variation of the intertap intervals for each bimanual condition. These were combined to yield a composite score that represented performance across all the bimanual conditions. The dependent variable was this composite score.

Serial reaction time (SRT). As described in detail in Waber et al. (2002), the child was trained to press one of three buttons corresponding to the location of a target (asterisk) appearing in one of three boxes arrayed in a row on the screen (see Figure 2). There were seven blocks of trials; in Blocks 1 and 6, the stimuli were presented in a random order; in Blocks 2 through 5 and in Block 7, the stimuli were presented in a repeating sequence. Typically, in this paradigm, the response times (RT) and errors decrease across the blocks of sequenced trials, even though the participants have no explicit awareness of a repeating sequence (Curran, 1998). All children responded with their right hand.

Each block consisted of 60 trials: 10 repetitions of a 6-element sequence for the sequenced blocks, and 60 randomly sequenced trials for the random blocks. Children were instructed to respond as quickly as possible without making errors, and the dependent variable was the number of errors. The number of errors was used rather than the pattern of response times because it better discriminated between the groups.

Visual filtering (VF). This was a visual serial search task, in which a target (X) was presented among an array of distractor lines arranged in different degrees of deviation from parallel, with four different levels of distraction present (0%, 40%, 80%, and 100% of distractor lines displaced, as illustrated in Figure 3; see also Weiler, Singer-Harris, et al., 2000, for a detailed description). Targets and distractors were superimposed on a random field of gray and white squares; 120 stimuli were presented in random order. The child signaled by a button press to indicate whether the target was on or off a gray area. For each trial, the computer recorded both the child's response and RT from stimulus onset until button press. The dependent variable was the mean of the median RTs for the four levels of distraction.

Ancillary Measures of Reading. The Woodcock-Johnson Test of Reading Ability, Word Attack Subtest (Woodcock & Johnson, 1989) was used to evaluate pseudoword decoding. The Test of Word Reading Efficiency (TOWRE; Torgesen, Wagner, & Rashotte, 1999) was used to evaluate the efficiency of presenting graded lists of real words (Sight Word Efficiency) and pseudowords (Phonological Coding Efficiency). The child was asked to read as many words as possible within 45 seconds. The NEPSY Phonological Processing subtest (Korkman, Kirk, & Kemp, 1997) included items requiring elision and sound substitution to measure the child's ability to segment the sound stream. In the Rapid Automatized Naming Letters and Numbers subtests (RAN; Denckla & Rudel, 1976), the child was asked to name a repeating series of five letters or digits presented randomly in five rows of 10 items, each as quickly as possible. The outcome variable was the number of seconds needed to read each array.

Children in the community groups were tested in a laboratory in their school. Children in the hospital-referred group were tested in a laboratory in the hospital. Table 2 displays the measures that were administered and their distribution across the two time points.

ANOVA was used to compare the demographic characteristics of the four groups. A chi-square test was used to test for differences in gender composition between groups. Within the hospital-referred and community groups, separate unpaired t tests were used to compare the children who returned for the 2-year follow-up with those who did not return. Pearson correlations were used to assess the test-retest reliability of the outcome measures. Linear regression, adjusting for age and gender, was used to evaluate group differences on continously distributed outcomes. Bonferroni methods were used within a single analysis to adjust for multiple pairwise group comparisons. Matched-pair t tests were used to compare matched pairs of referred and nonreferred children from the community sample.

Children were also classified as meeting the criteria for diagnosis of a learning or attention disorder. A child was classified as showing low achievement if his or her score on any WIAT subtest was less than 90. A child was classified as having ability-achievement discrepancy if his or her score on any WIAT subtest was more than or equal to 15 points below the regressed K-BIT Composite score. A child was classified as having attention-deficit/hyperactivity disorder, inattentive subtype (ADHD/IA), if his or her score was 6 or more on the Inattention scale for both parent and teacher versions of the DRS. Chi-square analyses were used to evaluate the distribution of these diagnoses across groups.

The following summary variables were created as indicators of the four major functional domains: ability (K-BIT composite), achievement (average of 3 WIAT subtests), information processing (average of 4 LLIP measures), and inattention (average of parent and teacher DRS Inattention scales). Regression was used to evaluate the contribution of referral and diagnosis status to these four functional domains.

Furthermore, logistic regression was used to evaluate how well each of the four functional domains (ability, achievement, information processing, and inattention) predicted referral status (referred versus nonreferred) and diagnostic status (diagnosis versus no diagnosis). These analyses were carried out using individual domains as predictors, pairs of domains as predictors, and all four domains together as predictors.

Prior to the main analyses, we compared the demographic and basic cognitive profiles of the children who participated in the 2-year follow-up with those of the ones who did not. These analyses were carried out separately for the community general education group (N = 201), the community special education group (N = 32), and the hospital-referred group (N = 203). We compared age, parent education and occupation, K-BIT Composite score, and the scores on the three WIAT achievement measures. There were no differences for the community groups. For the HR group, differences emerged for mother's education and occupation, both of which were lower for the children who did not return for follow-up testing (p < .05).

Group comparisons were carried out to evaluate the first two predictions. Two basic questions were asked:

Because it was of the greatest interest to ascertain factors that might predict referral in the community general education sample, group differences for the four key domains were evaluated for the variables measured at Time 1, prior to referral for the children in the CR group. Age and gender were included in the model as covariates. Analyses carried out for variables measured at both time points (i.e., achievement and information processing), although not reported here to conserve space, yielded equivalent results.

Group membership was a highly significant predictor (p < .001) for every outcome variable, except the DRS Teacher Hyperactivity scale (p < .05), the latter presumably because children with high scores on this scale were excluded from the study. Tables 3 and 4 display the adjusted means by group for each outcome variable as well as between-group comparisons. Group comparisons relevant to the primary questions posed at the outset are detailed in the following sections.

Community Referred Versus Community Nonreferred. The first prediction was that the CR group would differ from the CNR group not only on ability and achievement but also on attention and information processing. In fact, the CR group demonstrated not only lower general cognitive ability and achievement but also more attention problems and poorer information processing (see Table 3). In terms of the ancillary reading measures (see Table 4), the CR group exhibited poorer phonological awareness skill and reading fluency and slower performance on the RAN, although this last result did not reach statistical significance. Interestingly, nonword decoding skills (WJ-R Word Attack) were comparable for both groups and above average for age, which was unexpected in light of the CR group's poorer performance on the phoneme elision task. Thus, the relative deficits of the CR group were not limited to academics or even to cognitive ability but extended to broader domains of behavior and information processing.

As a stronger test of the prediction, the 17 children in the CR group were matched for age, gender, and academic achievement (at Time 1) with 17 children selected from the CNR group. For each child in the CR group, the child from the CNR group who best matched on those criteria was selected for comparison on ability, information processing, and attention (see Table 5). There were no differences for age, gender, or achievement, indicating that the matching had been successful. The groups did differ, however, on ability (p = .02), information processing (p = .01), and inattention symptoms (p = .01). Thus, even though the two groups had the same achievement levels, children who were subsequently referred for evaluation of learning problems demonstrated consistently greater difficulty in other aspects of behavior and information processing that may have enhanced their difficulties in the classroom and, hence, their likelihood of referral.

CR Versus Other Referred. The second prediction was that the CR group would be comparable to the other referred groups on the four key measures, and that all the referred groups would differ from the nonreferred group. In fact, the CR group differed minimally from the other two referred groups (see Table 3). There were no differences among the three referred groups in general ability level. The three groups also performed comparably on the information processing and attention measures. The only domain in which differences emerged was academic achievement. The HR group performed more poorly than the CR group, with the CSE group midway between and not different from the other two groups. As predicted, moreover, the CNR group differed from the three referred groups in all four functional domains.

In terms of the ancillary reading measures, the referred groups did not differ from one another on any measure except Word Attack, and in general, all the referred groups differed from the nonreferred group (see Table 4). The only exception was that the CR and CNR groups did not differ on RAN and Word Attack, as described in the previous section.

The analyses of the inattention symptoms were repeated, eliminating those children who could be classified as meeting DSM-IV criteria for ADHD/IA. The results were unchanged; the three referred groups did not differ from one another, but all three referred groups differed from the nonreferred group. Thus, the differences observed for attention were not a function of the inclusion of a subset of children with ADHD/IA.

CR Achievement Relative to CNR and Population Norms. The third prediction was that the CR group would demonstrate poor performance relative to peers from their community, even though they might not deviate from the norm for the population at large. Although the mean ability and achievement scores for the CR group were at the anticipated population mean for their age, those of the CNR group were consistently higher (see Table 3). Thus, within the local context, the children in the CR group were performing relatively more poorly even though their scores as a group were psychometrically coincident with the norm for the general population.

Relationship of Referral and Diagnosis to Function. The fourth prediction was that children referred for evaluation would exhibit a higher prevalence of inattention problems and poorer information processing than nonreferred children whether they met the formal criteria for a diagnosis or not.

Prior to evaluating this prediction, we examined the distribution of individuals who met criteria for low achievement, ability-achievement discrepancy, or ADHD/IA for each group (see Table 6). Of particular interest was the comparison between the CNR and CR groups. Only half the children in the CR group met any criterion for a diagnosis, most of these exhibiting a score below 90 on one WIAT achievement test (low achievement criterion) but not an ability-achievement discrepancy. Moreover, 14% of the children in the CNR group met some criterion for a diagnosis but had not been referred. The characteristics of this group are explored further in following analyses.

The CR group had proportionally more children with low achievement than did the CNR group, although it had fewer such children than the CSE and HR groups. Similarly, there were more children in the CR group than in the CNR group who met criteria for ADHD/IA, but fewer than in the other two referred groups. In contrast, very few children in the CR or CNR groups demonstrated a discrepancy, and the proportions were actually equivalent for the two groups (11.8%).

We next evaluated the relationship of referral and diagnostic status to the four key domains of function. Because there were so few differences among the referred groups, they were combined for the purpose of these analyses. Figure 4 illustrates the means for the primary indicators. After adjusting for age and gender, there was an effect of referral status (all p s < .0001) for all four variables. There was also a main effect of disorder status for achievement (p < .0001), attention (p < .05), and information processing (p < .0001), as well as a trend for cognitive ability level (p < .1). Children in the referred groups performed more poorly than those who were not referred, and children with a diagnosis performed more poorly than those who did not have a diagnosis. None of the two-way interactions reached statistical significance.

As a stronger test of the model, planned comparisons were carried out to compare the performance of nonreferred children who did meet criteria for a disorder (false negative) and referred children who did not meet criteria for disorder (false positive), in order to further evaluate the relative importance of referral and disorder status. Referred children who did not meet criteria for a disorder in fact showed poorer cognitive ability level (p = .01), exhibited more inattention problems (p < .0001), and tended to exhibit poorer academic achievement (p = .09) than nonreferred children who did meet criteria for a disorder. Moreover, the groups did not differ in terms of information processing (p = .3).

Predictors of Referral and Diagnostic Status. Finally, the fifth prediction was that information processing and attention would be better predictors of referral than cognitive ability and achievement, but that the reverse would be true for diagnosis. Logistic regression was applied to determine how well the four summary variables, individually and in combination, predicted referral and diagnosis status. Table 7 shows the results of these analyses for each of the four variables individually and in combination. The odds ratios for these continuous predictors are interpreted as follows: Using the inattention variable to predict referral, for example, an increase of 1 point on the DRS Inattention scale is associated with 2.34 times the likelihood of being in the referred group. An increase of 2 points is associated with a 2.34² = 5.5 times the risk of being in the referred group. Odds ratios for information processing and inattention are greater than 1 because the high scores on those tests are associated with referral--higher is worse. Because higher scores are better on the achievement and ability variables, the odds ratios are less than 1, meaning that increases in the scores are associated with reduced likelihood of membership in the referral and diagnosis groups. Taken together, the four variables accounted for slightly more than 50% of the variance (R²) for both referral and diagnosis.

The key comparison here is how well ability and achievement--the typical benchmarks for an LD diagnosis--predicted referral and diagnostic status when contrasted with information processing and attention. As predicted, information processing and attention, taken together, accounted for more of the variance in referral status (44%) than did ability and achievement (36%), whereas the opposite was true for diagnostic status, which was better explained by ability and achievement (52%) than by information processing and inattention (24%).

Children who are referred for evaluation of learning problems differ from nonreferred children from the same community on a number of dimensions that certainly include but are by no means limited to their academic achievement. Not surprisingly, children who were referred for evaluation during the 2-year study period exhibited poorer scores on tests of cognitive ability and academic achievement than children who were not referred. Of particular interest, the mean ability and achievement scores of these referred children--80% of whom were deemed eligible for special education support-were at the expected mean for the general population, with no obvious indication of an ability-achievement discrepancy. Indeed, only 11.8% of children in the community-referred group exhibited discrepant achievement, and the same proportion of children in the nonreferred group demonstrated a discrepancy. Moreover, the achievement scores of the referred children were poorer than those of their peer group within that community, undoubtedly contributing to the concerns of parents and teachers about their academic success.

It is also worth noting that 14.3% of children in the community general education group met criteria for diagnosis but had not been referred for evaluation. This could reflect inadequate sensitivity on the part of parents and teachers. These children, however, actually had higher cognitive ability levels and fewer attention problems than referred children who did not meet criteria for diagnosis, but they did not differ in terms of academic achievement or information processing. The behavioral characteristics of these children, therefore, may function to increase or decrease risk in particular settings, influencing the likelihood that a child will be referred for evaluation.

Confirming the first prediction, community-referred children consistently demonstrated more attention problems and poorer nonverbal information processing than did their nonreferred peers. Even when the referred children were matched with nonreferred children by level of academic achievement, the referred group demonstrated poorer information processing, a lower level of general cognitive ability, and more symptoms of inattention, again implicating behavioral factors in the adaptational and the referral processes.

Our findings, which incorporate neurobehavioral data, are entirely consistent with the observation by Bocian et al. (1999) that the role of psychometric testing in the LD identification process is to determine acceptability relative to legally mandated criteria but that psychometric testing can be less sensitive to the child's actual functioning within the local school context. Our data further highlight that such testing can also be insensitive to other pertinent neurobehavioral factors--such as the efficiency of information processing and task orientation (McKinney et al., 1993)--that can significantly affect the child's ability to successfully adapt to academic demands. The matched-pair analysis, in particular, suggested that these neurobehavioral factors may increase the likelihood that one child will encounter greater difficulty in an academic setting than another even though both have the same level of academic skill on formal achievement testing.

We also generally confirmed the second prediction--that there would be no differences among the referred groups, whether community based or hospital based, and that all three groups would differ from the nonreferred group. Although there was variation among the referred groups in the severity of their academic skills problems, with the HR group showing lower achievement scores than the CR group, there was a striking consistency across groups in more generic information processing and attention behaviors. Children with more severe academic problems may be more likely to be referred for an outside evaluation, but the more global neurobehavioral substrate, in terms of information processing, cognitive ability, and attention problems, appears to be comparable nevertheless.

Our findings are also consistent with the third prediction, supporting the view that the social context plays an important role in the referral and identification process and, indeed, the broader view that the educational process is necessarily highly context dependent (Bruner, 1996). Scores on the ability and achievement tests of the children from the community who were referred for evaluation were most remarkable for their "normality." Group means for all the ability and achievement measures were at or within a few points of the expected mean of 100. From a strictly legal standpoint, therefore, these children should not qualify for special education. By contrast, however, the scores of their nonreferred peers on the same tests were approximately two thirds of a standard deviation above the population mean. Thus, relative to local norms, the referred children did indeed show a deficit. Further supporting the validity of the referral, their scores on the nonverbal information processing tasks were also consistently lower than those of their nonreferred peers, and they exhibited more attention symptoms, although only a few met screening criteria for a diagnosis of ADHD. The finding that 82% of these children were subsequently enrolled in special education suggests that the school teams who evaluated the children recognized a legitimate need for support, in line with the profitability criterion that according to Bocian et al. (1999) motivates the decision phase.

Children referred for evaluation exhibited a higher prevalence of attention problems and less efficient information processing than did nonreferred children, whether they had a formal diagnosis or not, confirming the fourth prediction. Although children who met criteria for a diagnosis generally performed more poorly on these measures than those who did not, there was also an effect of referral status; referred children performed more poorly than nonreferred children whether they qualified for a formal diagnosis or not. Moreover, confirming the fifth prediction, information processing and attention taken together were better predictors of referral status than were ability and achievement, suggesting that they play an important role in the decision to refer a child for evaluation. Finally, referred children who did not meet the criteria for a diagnosis generally performed more poorly than nonreferred children who did meet the criteria for a formal diagnosis. Consistent with our earlier work, the present findings, which are based on the prospective observation of a community sample, strengthen the hypothesis that neurobehavioral factors extending beyond academic achievement play an important role in the process by which a child comes to be referred for evaluation of learning problems and eventually enrolled in special education.

The consistently high prevalence of inattention symptoms among the children referred for evaluation of learning problems raises the possibility that many of these children actually had an undiagnosed attention disorder. However, when we repeated the group comparisons for attention problems excluding the children who met the criteria for an attention disorder, the results were unchanged. Thus, it is unlikely that our findings reflect a failure to identify children with attention disorders.

Many of the inattention symptoms set forth in the DSM-IV system, such as "often has difficulty organizing tasks or activities" or "often does not follow through on instructions and fails to finish schoolwork, chores or duties in the workplace" (pp. 83-84) may have a cognitive basis that does not necessarily reflect a primary failure to attend (Barkley, 1998; Weiler, Bernstein, Bellinger, & Waber, 2000; Weiler, Bernstein, Bellinger, & Waber, 2002). These behaviors are presumably domain general--that is, not specific to particular skill or content areas--and are more prevalent among children with learning problems. They most likely are subsumed within the broad domain of metacognitive skills or executive functions. The finding that these information processing problems are seen to a greater extent in referred than in nonreferred children who have the same level of academic skills (Singer-Harris et al., 2001) suggests that remediating specific skills may not ultimately be sufficient to normalize their adaptation to academic demands. It further suggests that a child who has adequate academic skill levels may nevertheless experience cognitively based difficulties that interfere with academic adaptation (Morgan et al., 2000). The cognitive and neurobiological bases of these domain-general processes, which appear to play an important role in academic success, merit more careful analysis.

The present study was carried out in a suburban, generally middle-income community. Had we implemented it in a different socioeconomic setting, the specifics would undoubtedly have differed. The basic principles, however, should remain consistent. First, children referred for evaluation of learning problems will demonstrate deficits relative to nonreferred peers within their community not just in terms of academic achievement but also of more basic information processing and metacognitive competence. Second, referral can be prompted by legitimate neurobehavioral issues that will not necessarily be documented by standardized tests of isolated academic skills. Moreover, even when neurobehavioral indicators such as attention and information processing are measured, there may be no valid, fixed criteria for classifying a child as typical or atypical; the significance of these findings can depend on the ecological context. It would be important to replicate a study like this across different social settings.

One unexpected finding was that the majority of referred children from the community were girls, which was at variance with the more typical finding of a preponderance of boys among children identified with learning problems. The youngest children at the start of our study were already in second grade. One possible explanation for our finding, therefore, is that the children with more significant language-based disorders, who could have been predominantly male, might have already been identified prior to the start of the study. Indeed, the children who were already enrolled in special education in that community (CSE group) were predominantly male.

There were, however, some important limitations to our study. As noted earlier, although the demographic characteristics of the referred and nonreferred groups were comparable, the range of backgrounds among the children in these samples was limited. Also, the measures of cognitive ability and achievement were only screening measures and would not be sufficient for a diagnostic decision in a clinical setting. Furthermore, we did not measure the full range of functions specified in the federal definition of LD. More extensive measurement, which was not feasible because of time limitations, might have yielded a somewhat different result, especially in terms of diagnostic decisions. Even so, it is not likely that the result would have differed dramatically. Finally, because we did not have access to school evaluations and education plans, we do not know the specific conclusions of the school team or the composition of the services that these children received.

Our work, together with that of educational researchers like MacMillan et al. (1998), raises fundamental issues about the enduring quest for a valid and objective system for diagnosing LD. When parents and teachers initiate a referral, they are responding primarily to the quality of the child's functioning relative to the particular social context and set of expectations within which the child functions, and this may or may not be reflected in the child's performance on tests of discrete academic skills. Given this contextual relativity, attempts to ascertain on some objective basis whether a child does or does not have a medical type of "disability" that is analogous to blindness or cerebral palsy based on a fixed set of psychometric standards, may ultimately be futile. The struggle to arrive even at first principles about the diagnosis problem has spanned 3 decades (Farnham-Diggory, 1986; Fletcher et al., 1994; Gaddes, 1976; Rudel, 1980) and has served more to highlight the fallacies inherent in various approaches than to provide workable solutions.

Although children who meet the criteria for a particular definition of LD will differ expectably from children without learning problems on a variety of neurobehavioral variables, so too will children who are referred for evaluation who do not meet conventional criteria for any disorder. Moreover, in practice, the identification of children for LD services is by no means a purely objective process; the empirical evidence from schools indicates on the contrary that it is context dependent and subjective for reasons that are very often justifiable (Gottlieb, Alter, Gottlieb, & Wishner, 1994; MacMillan, Gresham & Bocian, 1998).

This dilemma also has implications for LD research. MacMillan and Speece (1999) have commented that

The conflicting paradigms evident within the schools are mirrored in current emphases in federal agencies concerned with LD. On the one hand, the National Institutions of Health (NIH) recently established the Learning Disabilities, Cognitive and Social Development Branch. Based on the nature of past LD research funded by NIH (Lyon, 1995), we anticipate future projects will retain the 'within-child' definitional focus that we have characterized as more consistent with a cognitive paradigm than either a behavioral or sociocultural paradigm. On the other hand, the Office of Special Education Programs (OSEP) proposed dropping the requirement of placement in a specific disability category in favor of a generic category defined as 'a child who has a physical or mental impairment and who by reason thereof requires special education' (OSEP, 1995, as cited in Coutinho, 1995, p. 665). (p. 123)

Our NIH-funded research program does in fact integrate these two important perspectives. Although our research program was designed primarily to focus on neurocognitive mechanisms, generally regarded as within-child measures (Waber, Weiler, et al., 2000; Waber, Wolff, et al., 2001; Weiler, Singer-Harris, et al., 2000), our findings indicate that even these more biologically referenced indicators are probably better understood contextually. The inclusion or exclusion of children from research studies on the basis of psychometric test score criteria can generate study samples that are unrepresentative of the population of children who actually experience legitimate, neurodevelopmentally based learning problems in the school setting, nor can it be sensitive to critical contextual factors. A model such as the one proposed here, which emphasizes functional adaptation rather than disability, may be better suited to the inherently systemic and developmental nature of the phenomena to be described and ultimately managed than the more rigid definitional models that are currently in use. The educational and programmatic implications of such a model, in addition to its implications for the direction of LD research, merit exploration.

1. This work was supported by NICHD Learning Disabilities Research Center Grant P50-HD33803 and in part by Mental Retardation Research Center Grant P30-HD18655.

2. The authors are grateful to the Learning Disabilities, School Function, Neuropsychology, and Language and Auditory Processing Programs of the Children's Hospital, as well as to the children and families from those programs who participated and to the staff and students of the public school system of Melrose, Massachusetts, for their cooperation. They are also grateful to Jacki Marmor, Sethany Rancier, Laurie Kirshner, Allison Morgan, Naomi Singer-Harris, and Michael Kirkwood, who assisted in sample accrual and data collection.

DIAGRAM: FIGURE 1. Number of children in each group tested at baseline (Time 1) and follow-up (Time 2).

DIAGRAM: FIGURE 2. Sample stimulus array from the Serial Reaction Time task.

DIAGRAM: FIGURE 3. Sample stimuli from the Visual Filtering task.

GRAPHS: FIGURE 4. Mean scores and standard errors for Kaufman Brief Intelligence Test (K-BIT) Composite, inattention, achievement, and low-level information processing (LLIP) by referral and diagnostic status.