Evaluation of the Effectiveness of Educational Technology Interventions

Contractor: Mathematica Policy Research, Inc. with SRI International

Background/Research Questions:

The primary purpose of the Enhancing Education Through Technology program under Title II, Part D (funded at $267.5 million in FY 2008) is to improve student achievement through the use of technology in elementary and secondary schools. Grants are awarded to states based on their proportionate share of Title I, Part A funds. The reading and math technology programs evaluated were competitively selected and are an appropriate use of funding under this program.

This mandated evaluation is assessing the impact of competitively selected educational technologies that are intended to improve student academic achievement in reading and/or in mathematics. Sixteen educational technology products, in four groups (reading in 1st and 4th grades, 6th grade math, and 9th grade algebra) were implemented in schools during the 2004–05 school year. The impact of educational technology on reading or mathematics achievement was measured through an experimental methodology in which volunteer teachers were randomly assigned to use a given educational technology or the control group. The study examined:

• Does the use of educational technology improve student achievement in reading and/or mathematics?
• Under what conditions and practices does the use of educational technology improve student achievement in reading and/or mathematics?

Duration: 5 years (September 25, 2003–September 24, 2008)

Current Status: The mandated report was released in April 2007 (see http://ies.ed.gov/ncee/pubs/20074005/). The second report was released on February 17, 2009 (see http://ies.ed.gov/ncee/pubs/20094041/).

Key Findings:

Results were reported for each of four groups of products (reading in 1st and 4th grades, math in 6th grade, and algebra 1), and in no group was the effect of using the software product on student achievement statistically different from the control group.

During the 2005–06 school year, the same teachers used ten products with a second cohort of students. For the reading products, the second year of use by teachers was not more effective than the first year. The evaluation found no significant difference in student achievement between the classrooms that used the technology products and the classrooms that did not use the technology products, in any of the four groups, in second year of use by teachers. Only for LeapTrack in the fourth grade is the effect on student achievement positive and statistically significant.