Results

The goal of the PCER initiative was to identify the impact of the 14 preschool curricula on five student-level outcomes (reading, phonological awareness, language, mathematics, and behavior) and six classroom-level outcomes (classroom quality, teacher-child interaction, and four types of instruction). Each outcome was based on one or more of the measures (see table D); thus, the process of determining a curriculum’s impact on the outcomes required two steps. First, the models were estimated to identify average differences in the 27 measures between the students receiving the treatment curriculum and those receiving the control and determine whether they were statistically significant. Second, criteria were applied to the set of measures that made up each outcome to determine whether the results for that group of measures showed a finding that the curriculum had an impact on that outcome. This process is described in the following order: (1) the model results for the 27 measures, (2) the criteria applied to the measures for each outcome, and (3) the findings derived from applying the criteria to the results for the measures.

The analysis tested the statistical significance of the difference between the means of the treatment versus the control group for each measure. Tables E-G display this difference as an effect size and note which differences are statistically significant (using a significance level of .05 and a two-tailed test). In the tables, the measures are grouped under their corresponding student-level and classroom-level outcomes. Table E identifies the impacts of each curriculum on the student-level measures in pre-kindergarten (note that Creative Curriculum is listed twice as it was implemented by the Vanderbilt University (Tennessee) research team and by the University of North Carolina at Charlotte (North Carolina) research team). Ten curricula show no statistically significant impacts on any of the student-level measures while five show significant impacts on some measures. Table F identifies nine curricula showing no statistically significant impacts on any of the student-level measures in kindergarten and six that do. Table G shows that with seven curricula there are no statistically significant impacts on any of the classroom-level measures and eight curricula show such impacts.

The statistical significance of these results depend, in part, upon the evaluations having adequate power to detect significant impacts. The original IES Request for Applications to which the 12 research teams successfully responded required that each team include a minimum of 10 classrooms or preschool programs (half treatment and half control) with a minimum of 150 total students. Minimal Detectable Effects were calculated after data collection using the smaller achieved (not expected) samples for each team on a set of four composite measures (combining the measures for reading, language, mathematics and behavior respectively). The Minimal Detectable Effects ranged from .34 to .69 across the composites and teams.

Four of the five student-level outcomes had two to three outcome measures associated with them (phonological awareness only had one per grade), as did three of the six classroom-level outcomes. The measures within an outcome are conceptually related to one another and sufficiently inter-correlated that an effect on one would not be expected to appear, except by chance, without indications of some effect on the others. To minimize the potential for false positive findings that may arise from multiple comparisons made among related measures, a criterion was applied to the set of measures within each outcome (rather than a post-hoc statistical adjustment). These criteria were used to determine whether a curriculum had a treatment effect on each student-level outcome for pre-kindergarten and for kindergarten. They include:

• The reading, mathematics, and behavior outcomes each contained three measures. The finding that a curriculum has an effect on any of these three outcomes required at least two of the three measures to have had a statistically significant effect with the same sign and no significant effect with the opposite sign.
• The language outcome contained two measures. A finding of an outcome effect required at least one of the two measures to have had a statistically significant effect and no significant effect with the opposite sign.
• The phonological awareness outcome contained one measure. A finding of an outcome effect required this measure (Pre-CTOPPP in preschool and CTOPP in kindergarten) to have had a statistically significant effect.

A similar set of rules was used to determine whether a curriculum had a treatment effect on each pre-kindergarten classroom-level outcome:

• The classroom-quality outcome contained one measure. A finding of an outcome effect required this measure to have had a statistically significant effect.
• The teacher-child relationship outcome contained four measures. A finding of an outcome effect required at least two of the four measures to have had a statistically significant effect in the same direction and no statistically significant effects with the opposite direction. For these measures, direction concerns desirability of the effect; a desirable effect would be a positive sign for the Positive Interaction scale and a negative effect for the other three scales.
• The early literacy instruction outcome and the early language instruction outcome each contained two measures. A finding of an outcome effect required at least one of the two measures to have had a statistically significant effect and no significant effect with the opposite sign.
• The phonological instruction outcome and the mathematics instruction outcome each contained one measure. A finding of an outcome effect required the measure to have had a statistically significant effect.

These criteria were applied to the results for each set of measures within the five student-level outcomes (for preschool and for kindergarten) and the six classroom-level outcomes for kindergarten presented in tables E-G. In this way, each curriculum’s impact on each of the 16 outcomes was determined. Below, these findings are presented in two sections: the first organized by outcome and the second by curriculum. Under the Findings by Outcome, those curricula affecting each of the five student-level (for pre-kindergarten and kindergarten) and six classroom-level outcomes (for pre-kindergarten) are identified. Under the Findings by Curriculum, each curriculum is discussed with regard to its effects on the outcomes.

The findings described in both sections are presented in tables H and I. Table H shows the impacts of each curriculum on the student-level outcomes for both pre-kindergarten (pre-K) and kindergarten (K). A blank cell stands for no effect, a plus sign (+) means a positive effect, a minus sign (-) means a negative effect, and a zero (0) signifies no effect in one grade when there is an effect in the other. Table I shows the impact of each curriculum on the classroom-level outcomes using the same symbols.

Findings by Outcome

Two of the 14 intervention curricula had impacts on the student-level outcomes for the pre-kindergarten year (table H). DLM Early Childhood Express supplemented with Open Court Reading Pre-K positively affected reading, phonological awareness, and language. Pre-K Mathematics supplemented with DLM Early Childhood Express Math software curriculapositively affected mathematics.

In the kindergarten year, four of the curricula had impacts on the student-level outcomes though three of these did not have impacts during the pre-kindergarten year (table H). DLM Early Childhood Express supplemented with Open Court Reading Pre-K continued to have positive effects on reading, phonological awareness, and language in kindergarten as it did in pre-kindergarten. Curiosity Corner, which had no effects in pre-kindergarten, was found to positively affect reading in kindergarten. Early Literacy and Learning Model (ELLM), which had no effects in pre-kindergarten, was found to positively affect language in kindergarten. Project Approach, which had no effects in pre-kindergarten, was found to negatively affect behavior in kindergarten.

Eight of the 14 treatment curricula had a positive effect on the pre-kindergarten classroom-level outcomes (table I). Bright Beginnings affected early literacy instruction and phonological awareness instruction. Creative Curriculum (as implemented by the North Carolina team but not by the Tennessee research team) affected classroom quality, teacher-child interaction, early literacy instruction and early language instruction. Creative Curriculum with Ladders to Literacy affected early literacy instruction. Curiosity Corner affected early language instruction. DLM Early Childhood Express supplemented with Open Court Reading Pre-K affected phonological awareness instruction. Doors to Discovery affected early literacy instruction and early language instruction. Let’s Begin with the Letter People affected classroom quality and early literacy instruction. Literacy Express affected classroom quality and phonological awareness instruction.

Findings by Curriculum

Each curriculum is discussed separately and cross-curriculum comparisons are not made. The type of pre-kindergarten program involved in the evaluation and the control curricula are described (though the results should not be used to evaluate any control curricula). Impacts on the outcomes are then presented in the following order: (1) student-level outcomes in pre-kindergarten, (2) student-level outcomes in kindergarten, and (3) classroom-level outcomes in pre-kindergarten.

Bright Beginnings
Bright Beginnings and its control were implemented in state pre-kindergarten classrooms in Tennessee. In the control classrooms, teachers used teacher-developed curricula with a focus on basic school readiness. No impacts on the pre-kindergarten or kindergarten student-level outcomes were found. A positive impact was found at the classroom level on early literacy instruction and phonological awareness instruction.

Creative Curriculum─Vanderbilt University
Creative Curriculum and its control were implemented in state pre-kindergarten classrooms in Tennessee. In the control classrooms, teachers used teacher-developed curricula with a focus on basic school readiness. No impacts regarding pre-kindergarten or kindergarten student-level outcomes were found. No impacts were found on the classroom-level outcomes.

Creative Curriculum─University of North Carolina at Charlotte
Creative Curriculum and its control were implemented in full-day Head Start programs in North Carolina and Georgia. In the control condition, teachers used teacher-developed, nonspecific curricula. No impacts on the pre-kindergarten or kindergarten student-level outcomes were found. A positive impact was found at the classroom level on overall classroom quality, teacher-child relationships, early literacy instruction, and early language instruction.

Creative Curriculum with Ladders to Literacy
Ladders to Literacy was implemented in full-day and half-day Head Start classrooms in New Hampshire as a supplementary curriculum in conjunction with Creative Curriculum. In the control condition, teachers used only Creative Curriculum. No impacts on the pre-kindergarten or kindergarten student-level outcomes were found. A positive impact was found at the classroom level on early literacy instruction.

Curiosity Corner
Curiosity Corner and its control were implemented in full-day preschool programs in three different states (Florida, Kansas, and New Jersey). In the control condition, teachers used a variety of preschool curricula including the Creative Curriculum and Animated Literacy curriculum models, and teacher-developed curricula. No impacts regarding pre-kindergarten student-level outcomes were found. A positive impact on reading was found at the end of kindergarten. A positive impact was found at the classroom level on early language instruction.

DLM Early Childhood Express supplemented with Open Court Reading Pre-K
The evaluation of DLM Early Childhood Express supplemented with Open Court Reading Pre-K took place in public pre-kindergarten classrooms in Florida. In the control condition, teachers were provided with the High/Scope curriculum. A positive impact was found on reading, phonological awareness, and language development in both pre-kindergarten and kindergarten. A positive impact was found at the classroom level on phonological awareness instruction.

Doors to Discovery
Doors to Discovery and its control were implemented in full-day Head Start and public pre-kindergarten (Title I and non-Title I) programs in Texas. In the control condition, teachers used teacher-developed, nonspecific curricula. No impacts on the pre-kindergarten or kindergarten student-level outcomes were found. A positive impact was found at the classroom level on early literacy instruction and early language instruction.

Early Literacy and Learning Model (ELLM)
The Early Literacy and Learning Model (ELLM) curriculum was implemented in combination with the existing comprehensive curricula that were in use in the control group classrooms in Florida. Several curricula were used in the control classrooms including Creative Curriculum, Beyond Centers and Circletime, High Reach, and High/Scope. No impacts regarding pre-kindergarten student-level outcomes were found. A positive impact on language development was found at the end of kindergarten. No impacts were found on the classroom-level outcomes.

Language-Focused Curriculum
The Language-Focused curriculum was implemented in full-day Head Start and public pre-kindergarten classrooms in Virginia. The control teachers reported using High/Scope curriculum materials. No impacts on the pre-kindergarten or kindergarten student-level outcomes were found. No impacts were found on the classroom instruction outcomes. Impacts on classroom quality and teacher-child interaction outcomes could not be determined because of unreliable (inflated) data from 8 of the 14 participating classrooms on the relevant measures.

Let’s Begin with the Letter People
Let’s Begin with the Letter People and its control were implemented in full-day Head Start and public pre-kindergarten (Title I and non-Title I) programs in Texas. In the control condition, teachers used teacher-developed, nonspecific curricula. No impacts on the pre-kindergarten or kindergarten student-level outcomes were found. A positive impact was found at the classroom level on classroom quality and early literacy instruction.

Literacy Express
Literacy Express and its control were implemented in public pre-kindergarten classrooms in Florida. In the control condition, teachers were provided with the High/Scope curriculum. No impacts on the pre-kindergarten or kindergarten student-level outcomes were found. A positive impact was found at the classroom level on classroom quality and phonological awareness instruction.

Pre-K Mathematics supplemented with DLM Early Childhood Express Math Software
The evaluation of Pre-K Mathematics supplemented with DLM Early Childhood Express Math software took place in Head Start and public pre-kindergarten classrooms in California and New York. Several curricula were used in the control condition including Creative Curriculum, High/Scope, Montessori, specialized literacy curricula, and local school district and teacher-developed curricula. A positive impact was found on students’ mathematical knowledge at the end of pre-kindergarten. No impacts on the kindergarten student-level outcomes were found. No impacts were found on the classroom-level outcomes.

Project Approach
The Project Approach curriculum was implemented in public pre-kindergarten classrooms in Wisconsin. In the control classrooms, teachers reported implementing their own teacher-developed, nonspecific curricula. No impacts on the pre-kindergarten student-level outcomes were found. A negative impact on behavior was found at the end of kindergarten. No impacts were found on the classroom-level outcomes.

Project Construct
Project Construct was implemented in full-day child care centers in Missouri. In the control schools, teacher-developed generic curricula were implemented. No impacts on the pre-kindergarten or kindergarten student-level outcomes were found. No impacts were found on the classroom-level outcomes.

Ready, Set, Leap!
Ready, Set, Leap! was implemented in pre-kindergarten programs in New Jersey. In the control condition, teachers used the High/Scope approach. No impacts on the pre-kindergarten and kindergarten student-level outcomes were found. No impacts were found on the classroom-level outcomes.

Top