Impact Evaluation of Title I Supplemental Educational Services

Contractor: Mathematica Policy Research, Inc (MPR)

Background/Research Questions:

The Title I program which began in 1965 as part of the Elementary and Secondary Education Act targets resources primarily to high-poverty districts and schools to help ensure that all children have the opportunity to obtain a high-quality education and reach proficiency on challenging state standards and assessments. Under Title I, districts with schools that miss AYP for a third year are required to offer Supplemental Educational Services (SES) to students attending those schools. SES are additional academic instruction offered outside the regular school day by state-approved providers. Districts are required to make available up to 20 percent of their total Title I funds to support SES and transportation for students using the school-choice option. Priority for these academic support services is required to be given to the lowest-achieving students in identified Title I schools, particularly when the number of students participating in SES is large enough that costs reach the SES expenditure cap. Findings from prior non-experimental studies have shown differences in individual student achievement before and after enrollment in SES suggesting a positive achievement trajectory. This evaluation will assess the potential benefits of offering SES to applicants denied services due to limited district funds. The following research questions will be addressed:

• What is the average impact of offering Title I Supplemental Educational Services to eligible applicants who are on the cusp of having access to services, in school districts where services are oversubscribed?
• What are the characteristics of SES provided to students in oversubscribed districts? Are the characteristics of services, providers, or practices in the host school districts correlated with the estimated impacts?

Design:

The impact study will use a regression discontinuity design with student achievement measures used as a quantitative eligibility variable for student applicants within each participating district. Participating districts had more applicants for SES than could be served by the funding required by the Title I program, and services were prioritized to the lower achieving students eligible for SES. The study includes a sample of 6 oversubscribed districts and 24,113 students in grades 3–8 eligible for services across districts. Data collection included state student assessments in reading and math from 2006 to 2009, student participation in SES program information for the 2008–2009 school year, and SES service and staff characteristics information from provider surveys administered in spring 2009.

Cost/Duration: $2,039,560 over 56 months (September 2007–May 2012)

Current Status: A report was released on May 3, 2012 (see http://ies.ed.gov/pubsearch/pubsinfo.asp?pubid=NCEE20124053).

• The study found no evidence of impacts on achievement of offering SES to students near the cusp of having access to services in six oversubscribed school districts. For grades 3–8, there were no statistically significant impacts of offering SES on student achievement in reading or in mathematics. Furthermore, the study found no statistically significant impact of participating in SES on student achievement in reading or mathematics.
• An average of 21 hours of SES per student for the school year were offered in either one-on-one or group sessions by providers that relied extensively on local school teachers to serve as SES instructors. Among students in the six districts who had received SES, 36 percent received tutoring in both reading and math, 55 percent received tutoring in only reading, and 9 percent received only math tutoring. Reading services in the study districts averaged 17.2 hours of tutoring for the school year, and math services averaged 12.5 hours for the school year. Teachers employed in the school district comprised, on average, 60 percent of the providers' instructional staff in the six study districts. On average, 44 percent of provider services in the study districts were offered in groups of 2–5 students, 34 percent in one-on-one sessions, and 21 percent in groups of 6–10.
• The study found no evidence that observed provider characteristics and practices, including intensity of services, were significantly associated with stronger impacts. Providers varied in the average number of hours of math and reading services their students received, ranging from 0 to 27 hours of math services and from 0 to 43 hours of reading services across providers. However, intensity of services was not significantly related to the estimated size of impacts on reading or math achievement.