Timeframe

The mail survey directed states to respond to all questions according to their monitoring procedures during the 2004–05 school year or monitoring cycle, even if those procedures have since changed. This retrospective timeframe¹¹ poses several challenges for the interpretation of the data. In addition to the expected effects of time on respondent memory, there is also a preference—observed during site visits—among state personnel to talk about what they are doing now. It is possible that some states responded to the mail survey based on their current monitoring procedures. Staff turnover, and the resulting loss of institutional memory, also poses a threat to the validity of the mail survey data. It is possible that in some states the respondent, or even all of the state-level monitoring staff, may not have been involved in monitoring during 2004–05. Finally, there is the potential impact of the reauthorization of IDEA on responses to the mail survey. It is conceivable that some states responded to the survey based on the new requirements of the law rather than reporting about their monitoring and improvement practices prior to reauthorization.