REL The Early Warning Systems Learning Series

Print Page

Early Warning Systems

An important part of keeping students on track for graduation and postsecondary success is to identify and address problems before they grow into major obstacles. For that reason, many schools and school systems are implementing Early Warning Systems. These systems use a wide range of data to identify students that are at risk, allowing educators to provide timely interventions and supports.

Publications

Using Data from Schools and Child Welfare Agencies to Predict Near-Term Academic Risks (REL Mid-Atlantic, July 2020). This study provides information to administrators, research offices, and student support offices in local education agencies (LEAs) interested in identifying students who are likely to have near-term academic problems such as absenteeism, suspensions, poor grades, and low performance on state tests. It describes an approach for developing a predictive model and assesses how well the model identifies at-risk students using data from two LEAs in Allegheny County, Pennsylvania. It also examines which types of predictors—including those from school, social services, and justice system data systems—are individually related to each type of near-term academic problem to better understand the causes of why students might be flagged as at risk by the model and how best to support them. The study finds that predictive models which apply machine-learning algorithms to the data are able to identify at-risk students with a moderate to high level of accuracy.
Understanding the role of noncognitive skills and school environments in students' transitions to high school (REL Southwest, October 2017). The purpose of this study was to: examine differences in students' perceptions of their noncognitive skills and school environments by race/ethnicity, and explore whether students' perceptions of their noncognitive skills and school environments were related to three outcomes that have been identified in the research as mattering most for a success transition to high school—grade 9 GPA, grade 9 absences, and grade 9 course failures.
Getting students on track for graduation: Impacts of the Early Warning Intervention and Monitoring System after one year (REL Midwest, April 2017). At the request of its Dropout Prevention Research Alliance, REL Midwest will work in Indiana, Michigan, and Ohio to evaluate the impact of the early warning and intervention monitoring system (EWIMS) model in a cluster randomized trial. This study will fill an important gap in the knowledge base, and districts and schools in the Midwest and other regions will be able to use the results to inform their own decisions about implementing an EWS. The study builds on initial work to validate indicators of student risk for failure to graduate in Ohio districts
Early Warning Indicators of High School Dropout for English Language Learners in Road Map Districts (REL Northwest, March 2017). Improving retention and graduation rates for English language learners (ELLs) is a concern throughout the Nation. This study focuses on seven South Seattle and South King County (Washington) school districts that participate in the Road Map for Educational Results initiative. This study looks at whether early warning indicators used to predict high school dropout in Road Map school districts are accurate predictors for ELLs in those districts.
Are two commonly used early warning indicators accurate predictors of dropout for English learner students? Evidence from six districts in Washington state (REL Northwest, March 2017). Improving retention and graduation rates for English language learners (ELLs) is a concern throughout the Nation. This study focuses on seven South Seattle and South King County (Washington) school districts that participate in the Road Map for Educational Results initiative. This study looks at whether early warning indicators used to predict high school dropout in Road Map school districts are accurate predictors for ELLs in those districts.
District guide for creating indicators for Early Warning Systems (REL West, December 2016). Across the country, many K-12 school districts have implemented early warning systems that use readily available data to identify students who are at risk of failing to achieve a desired outcome, such as graduating from high school on time. These systems allow educators to intervene early to keep students on track. The guide lays out a seven-step process for creating local indicators, starting with reviewing possible measures, moving through exploratory analyses of local data, and examining the reliability of different indicators or composites to inform local decisions. At each step, templates are provided to illustrate how data can be analyzed and displayed. The templates are also available as Word documents so they can be tailored to local contexts. Developed by researchers at the American Institutes for Research, the guide draws on several years of work partnering with districts to help them look at their local data in new ways. Now others can follow a similar process.
Early indicators and academic mindset in the Clark County School District (REL West, December 2016). Previous research strongly suggests that beliefs regarding the nature of ability and the payoff to effort (academic mindsets) and the related actions (academic behaviors) play an important role in supporting student success. Not much is known about the distribution of these beliefs among teachers and students in different academic contexts. This study examined the distribution of reported academic mindsets and behaviors in Nevada's Clark County School District. The analysis revealed that most students reported beliefs that are largely consistent with a growth mindset. However, reported beliefs and behaviors differed significantly depending on students' English learner status, race/ethnicity, grade level and prior achievement. For example, Black and Hispanic students reported lower levels of growth mindset than White students. English learner students reported significantly lower levels of growth mindset and higher levels of performance avoidance than their non-English learner counter parts. Lower achieving students reported significantly lower levels of growth mindset and significantly higher levels of performance avoidance than their higher achieving peers. Teachers reported greater beliefs in growth mindset than students, and their beliefs regarding growth mindset did not, for the most part, vary significantly depending on the characteristics of the students attending their schools.
Identifying Early Warning Indicators in Three Ohio School Districts (REL Midwest, July 2016). The purpose of this study was to identify a set of data elements for students in grades 8 and 9 in three Ohio school districts that could serve as accurate early warning indicators of their failure to graduate high school on time. The three districts varied in size, demographic composition, and locale. The research team collected student-level data on two cohorts of grade 8 and 9 students and used logistic regression and Receiver Operating Characteristic (ROC) curve analyses to identify the indicators that were the consistent predictors of failure to graduate on time and the cut points for these indicators that most accurately distinguish students who were at risk from those who did graduate on time. Results show the optimal cut point for classifying students as at risk varied significantly across districts for five of the eight potential indicators included in the study. End-of-year attendance rate was the only indicator that was a consistent predictor for both grades in all three districts. The most accurate indicators in both grade 8 and grade 9 were based on coursework (GPAs and course credits). Consistent with prior literature, failing more than one class and earning one or more suspensions also were strong predictors of failure to graduate on time. On average, indicators were more accurate in grade 9 than in grade 8.
Online Credit Recovery: Enrollment and Passing Patterns in Montana Digital Academy Courses (REL Northwest, June 2016). Online credit recovery options are popular, especially in rural states, because they allow schools to serve students in remote areas throughout the year, across a range of subjects, and with few additional resources. Despite the growing popularity of online credit recovery courses, there is still little research about which students take these courses or how well they perform in them. This REL Northwest report addresses that gap by examining 2013/14 data from the Montana Digital Academy (MTDA), the only statewide-funded program offering online credit recovery courses in Montana. The report offers educators an early look at the potential of online credit recovery courses to help struggling students get back on track to graduation.
Early Identification of High School Graduation Outcomes in Oregon Leadership Network Schools (REL Northwest, April 2015). This study examined student characteristics related to completing high school in four years, with emphasis on graduation outcomes for male and English language learner students. The authors looked at a cohort of students who began grade 9 in the 2007/08 school year in four Oregon districts. Findings confirmed previous studies showing that attendance and grade point average (GPA) in grades 8 and 9 are significantly associated with graduating on time.
Comparing Methodologies for Developing an Early Warning System (REL Southeast, February 2015). This report explains the use of logistic regression and classification and regression tree (CART) analysis in the development of EWSs. It was motivated by state education leaders' interest in maintaining high classification accuracy while simultaneously improving practitioner understanding of the rules for identifying students as at-risk or not at-risk readers. Results indicate that CART is comparable to logistic regression. Details of each method are provided to assist analysts interested in developing EWSs using one of the methods.
A Practitioner's Guide to Implementing Early Warning Systems (REL Northwest, January 2015). To reduce the number of students dropping out, many schools and districts are turning to Early Warning Systems that signal whether a student is at risk of not graduating from high school. Little information exists about the actual implementation strategies being used across the country. This report summarizes the experiences and recommendations of EWS users throughout the United States.
Does Raising the State Compulsory School Attendance Age Achieve the Intended Outcomes? (REL Mid-Atlantic, December 2013). Maryland recently raised its compulsory school attendance age from 16 to 18 in two stages: from 16 to 17 at the beginning of the 2014–15 school year and from 17 to 18 at the beginning of the 2016–17 school year (Maryland Senate Bill 362, 2012). The Maryland State Department of Education, a member of Regional Educational Laboratory Mid-Atlantic's School Completion and Engagement Research Alliance, requested technical assistance from Regional Educational Laboratory Mid-Atlantic in fulfilling reporting requirements to the legislature and in implementing the policy changes. Against this background, this review addresses the following research questions: (1) What changes have occurred in dropouts, truancy, and disciplinary actions in states that raised their compulsory school attendance age during 2002–11? (2) What broader social outcomes have been identified in studies using national datasets? (3) How have these states measured changes in these expected outcomes?
High School Dropout and Graduation Rates in the Central Region (REL Central, July 2008). This report presents comprehensive and detailed information on grades 7-12 dropout rates and on high school graduation rates in the Central Region. The rates provide a comprehensive reference for state and local educators and policymakers on the student subgroups most at risk of not completing high school.

Works in Progress

First-Year Effects of Oregon's Early Indicator and Intervention System (REL Northwest). This study will examine whether the Oregon Early Indicator and Intervention System is supporting improvements in student attendance, behavior, and course performance outcomes and which school districts are showing early signs of success with the program. The Oregon Department of Education will use the results of the study to improve the system and the coaching that is provided to districts.
Student Performance, Academic Mindsets, and the Transition to Middle School in a Large Urban School District (REL West). This study focuses on identifying the factors that predict school disengagement and course failure as students make the transition to middle school. The results will inform efforts to identify and support students who may be at risk of falling behind or disengaging from school.

Videos

After the ABCs: Integrating Qualitative Data to Develop Effective Interventions and Community Support (REL Mid-Atlantic, March 2015). This presentation by Daniel Leland highlights the need to pair EWS data with qualitative information to gain accurate and comprehensive student profiles. He presented examples of how school-based intervention teams can best use both quantitative and qualitative data, as well as how combining data sources can help schools gain stakeholder support and community resources. The presentation was made at the 2015 Pennsylvania Department of Education Data Summit.
Data Illustration and Visualization for EWS (REL Appalachia, October 2016). This whiteboard animation captures the important steps in building functional EWS data visualizations.
Communicating with Parents about EWS (REL Midwest, August 2016). The REL developed a video for states to use to communicate with parents about EWSs and how school districts and schools use these systems to provide supports and interventions.
Districts discuss the benefits of implementing an early warning system (REL Northwest, February 2017). This video from REL Northwest spotlights three districts in Montana that have boosted their graduation rates by using the Montana Office of Public Instruction's statewide EWS. Learn why school leaders continue to invest in this strategy and how it's changing the way staff members and students interact with one another.
Early indicators of High School Graduation and Academic Mindsets in Clark County School District — Community Partners Meeting (REL West, November 2018). REL West conducted a workshop for high-school principals and data teams in Clark County as the early warning indicator (EWI) system was introduced district-wide. This event included an orientation to the EWI data dashboard and time for schools to work in data teams to answer specific questions about their students. A final segment focused on research evidence about effective interventions and introduced some new options that schools might try.
How Early Warning Systems Can Help Your Students and School Succeed (REL Mid-Atlantic, March 2015) In this video, Dr. Robert Balfanz presented recent findings on how schools can use EWSs to keep all their students on track for graduation and to improve the overall school climate and academic outcomes. The presentation was made at the 2015 Pennsylvania Department of Education Data Summit.
Implementing an Early Warning System in a District: Experiences from Pennsylvania and Beyond (REL Mid-Atlantic, May 2016). A panel of stakeholders from districts within and outside of Pennsylvania share their experiences and answer questions about implementing Early Warning Systems. The panel also briefly reviews the latest EWS research. This event took place at the annual Pennsylvania Department of Education Data Summit, attended by roughly 1,000 district-level staff from across Pennsylvania, many of whom have the authority to drive district policy.
Montana's Early Warning System (EWS) videos (REL Northwest, April 2018). The Montana EWS, which all public schools in the state can use for free, captures live data to determine the dropout risk for all students. To help the Montana Office of Public Instruction spread the word about the system, REL Northwest produced two videos that share successful strategies of districts in the state that are using the EWS.
Reducing Chronic Absenteeism: Every Day Counts! (REL West, July 2016). This 19-minute video from REL West first describes the prevalence, causes, and academic consequences of chronic absenteeism in grades K–12 in Utah and across the nation. It then addresses promising strategies for reducing chronic absence, highlighting three schools in Utah that have built schoolwide cultures of good attendance, used data to identify students missing school to trigger outreach, and personalized outreach to support students. The video is designed to be viewed in its entirety or can be stopped after each segment for discussion and reflection.
Supporting Baltimore City Schools in Implementing a Continuous Improvement Process around Data Use (REL Mid-Atlantic, September 2016) The Baltimore City School District has a robust data collection and management system and an Instructional Leadership Tool that includes data for an EWS. However, the use of these data varies across schools and depends on the knowledge and functioning of school-level instructional leadership teams. This video highlights how three high schools implemented a continuous improvement process focused on the schools' use of data to support the career and college readiness of their students.
Tackling Chronic Absenteeism in High School Grades (REL Southeast, March 2016). In the recorded webinar, Dr. Robert Balfanz from John Hopkins University presents work on the often overlooked prevalence of chronic absenteeism in low-performing schools and the role of chronic absenteeism in undermining school reform efforts. Actionable implications of these data, including policy and practice steps at the state and local level are discussed.
Understanding Risk Factors for English Learners in Completing High School. (REL Southwest, August – September 2016). REL Southwest developed three short videos including interviews with research experts and practitioners on the topic of EWS for English learners. The first of these videos begins with a definition of an Early Warning System (EWS) and overview of how an EWS is used in an educational setting. Three main aspects of an EWS are described. The speakers then address the importance of using an Early Warning System to look not only at individual students, but also groups of students, giving special consideration to the unique needs of English learners in EWS design and implementation.
The second and third videos provide a short overview of The Early Warning and Risk Prevention for English Learners technical assistance project from the perspectives of the researchers at REL Southwest, and the participants in three Texas school districts.

Using Data to Identify At-Risk Students (REL Mid-Atlantic, March 2015). During the 2015 Pennsylvania Department of Education Data Summit, panelist Joetta L. Britvich, Principal of South Middle School in the Albert Gallatin Area School District, described how her school uses data to identify at-risk students.
Whiteboard Animation of a District Guide for Selecting Early Warning Indicators (REL West, October 2016) REL West developed a guide for districts to select locally-appropriate indicators and cut-points, based on collaborative work with the Davis School District (see webinar from May 24th). To communicate the purpose and use of this guide, REL West will develop a whiteboard animation as an illustration. The REL will conduct user testing to determine the best style and method of communicating the information.

Archived Webinars

Supporting Emotion Regulation in Early Childhood Early Warning Systems (REL Central, September 2018). This recorded webinar presents strategies for creating supportive environments and using specific teaching strategies to foster the development of emotion regulation skills in early childhood settings.
Using Attendance Data for Decisionmaking: Strategies for State and Local Education Agencies (REL West, April 26, 2018). This REL West webinar addresses the importance of accurately tracking student attendance data and how it can be used to make decisions in policy and practice that will support students who are chronically absent get back on track with their attendance.
What Research Tells Us About Reducing Chronic Absenteeism: A Webinar for the Pennsylvania Department of Education (REL Mid-Atlantic, February 5, 2018). This webinar offers an overview of the What Works Clearinghouse practice guides, on dropout prevention. The presenters provide an overview of the four recommendations included in the guide that have been shown to prevent dropout. Ms. Julie Bruch discusses each recommendation in more detail, including specific steps for implementation and examples, and Dr. Balfanz and Dan Kaplan, of The International High School at LaGuardia Community College, speak about their experiences implementing these types of strategies in schools.
When Dropouts Return to School: How an EWS can Help Identify and Support Re-enrollees (REL West, September 27, 2016). Dropouts who return to school, or re-enrollees, are a subgroup of students especially at risk for dropping out again. However, few schools, districts, or states identify or track the progress of re-enrollees as a subgroup in their early warning systems (EWS) or other data systems, even though these students may have been flagged in an EWS for attendance, course credit, or other problems prior to dropping out. Knowing who these returning dropouts are, how many are in local schools, what specific supports they need and receive, and their academic progress upon re-enrolling is critical to dealing comprehensively with the dropout crisis. This webinar makes the case for identifying these students in a district EWS or other similar data system when they return as re-enrollees in order to provide the support they need to succeed.
Four Signs Your District is Ready for an Early Warning System: An Example from Guam (REL Pacific, September 20, 2016). Early Warning Systems (EWSs) can be a critical component of school and district strategies to mitigate academic achievement and dropout issues for at-risk students. But if you don't have an EWS in place, where should you start? Webinar attendees will learn about the following four "readiness signs" that your district is ready to develop and implement an Early Warning System.
EWS 401: Using Early Warning Systems To Identify and Support English Learners (REL Southwest, REL Northwest, REL Pacific and REL West, September 13, 2016). Schools nationwide are using Early Warning Systems (EWS) to identify students who are beginning to fall off track for graduation and to plan interventions that will get them back on track. This webinar focuses on the unique risk factors faced by English learner students and how districts can remove obstacles and/or use early warning indicators to help them graduate successfully.
Connecting with Parents about Early Warning Systems (REL Midwest, September 13, 2016). This webinar is intended for a state education agency audience and will include a companion video that states can use to communicate with parents about what EWSs are and how school districts and schools use these systems to provide appropriate supports and interventions. The webinar will also outline specific strategies for communicating with parents about EWS and how states protect student privacy in their data systems.

Early Warning and Risk Prevention for English Learners (REL Southwest, September 7, 2016). Early warning systems (EWS) enable districts to identify and support students at risk of falling behind academically or dropping out. Learn from the research and the field as we examine how early warning systems can be refined to target at-risk English learners (ELs). Researchers share the experiences of eight districts with large EL populations and established early warning systems. In addition, staff from one of the districts, Fresno Unified School District, described their EWS, which includes EL-specific risk indicators and interventions.
Connecting with Parents about Early Warning Systems (REL Midwest, August 25, 2016). This webinar is geared toward districts and schools and will present a resource booklet that administrators can use to communicate with parents about dropout prevention efforts and Early Warning Systems.

Data Illustration and Visualization for EWS (REL Appalachia, August 25, 2016). Webinar attendees will learn strategies for visualization of an EWS and available software, as well as best practices in preparing and conditioning data for use in an EWS. Presenters also discuss the links between underlying models and data display.
Developing a Culture of Data Use to Maximize the Use of an Early Warning System (REL Northeast & Islands, August 9, 2016). In this webinar, REL Northeast & Islands provide strategies and materials on ways to deepen data-use practices and support robust implementation of Early Warning Systems. This session uses and customizes the "Toolkit for a Workshop on Building a Culture of Data Use".

Going Districtwide: Implementation Lessons for Using Early Warning Systems and Local Risk Indicators (REL West, May 24, 2016). This webinar provides a case study of EWS implementation in Davis School District in Utah. It will highlight the process of initiating an early warning system and the importance of communication and support between schools and districts as they iterate and improve student data collection and reporting for dropout prevention. The webinar also features a discussion of the research partnership between Davis School District and REL West to conduct a local validation study to ensure rigorous data use and build trust among EWS users.

Massachusetts, Minnesota, and the State Role in Supporting Early Warning System Implementation (REL Northeast & Islands, May 18, 2016). What is the role of state education agencies in supporting the implementation of early warning systems (EWSs) to identify students at risk of not graduating from high school? Join Susan Therriault, REL Northeast & Islands, in this webinar for an overview of the state role in EWS implementation, along with Kate Sandel, Massachusetts Department of Education, and John Gimpl, Minnesota Department of Education. They discuss their processes for organizing, implementing, monitoring, and adjusting their EWSs and the supports (team members, training, guidance, resources) available in their states to support EWS work and confront challenges to implementation. Strategies to connect EWS work to other state agency programs and priorities are also discussed.

Considering the Best EWS Model to Fit Your Needs (REL Appalachia, May 12, 2016). Attendees discuss several different types of Early Warning Systems. Participants also explore how other types of state, district, or national accountability systems can be marshalled to serve as a type of EWS or an alternative model for a system.
Success and Lessons Learned: A District's Perspective on Early Warning Systems (REL Central, REL Northwest, and REL West, April 19, 2016). The webinar provides a case summary of one district's use of an EWS. Participants will learn how Washoe County (Nevada) School District developed and implemented its EWS. Presenters offer practical advice on how a district can create an EWS, the steps taken to analyze student risk factors and develop early warning indicators, and how to overcome challenges.
Planning High School Academic Interventions for Students Identified as at Risk by an EWS (REL Southeast, Part 1: March 25, 2016, Part 2: April 8, 2016). The goals for this webinar include helping district and school-based practitioners conduct self-studies for planning and implementing high school academic interventions. The webinar describes and shows participants how to use a working document self-study guide tool developed by REL Southeast Improving Literacy research alliance members.
Early Warning Systems 201: How to Build Effective EWS Teams and Provide Interventions within a Tiered Framework System (REL Central, REL Northwest, and REL West, March 31, 2016) Participants will learn how to build an effective team that is able to incorporate EWS into everyday practices and frameworks such as Multi-Tiered Systems of Support (MTSS), Positive Behavioral Interventions and Supports (PBIS), and Response to Intervention (RTI). This interactive session includes an opportunity for participants to practice matching sample interventions to indicators and evaluating whether there are enough interventions to support the number of identified struggling students. The webinar features REL representatives who have helped schools and districts nationwide implement EWS, as well as practitioners from Montana and Utah who will share their journey of using data to increase the success of their students.
Early Warning Systems 101: An Introduction to the Five Core Components of an EWS (REL Northwest, REL Southwest, and REL West, February 25, 2016). The one-hour session provides an overview of the five core components of EWS: building a team, identifying appropriate indicators, creating data reports, mapping interventions to indicators, and evaluating interventions. Practitioners from Oregon and Ohio will also share their experiences in team building and mapping interventions.
The State Role in Early Warning Systems (REL Midwest, March 24, 2015). Following an overview of EWS use across the country, representatives from Wisconsin and Minnesota shared details about their states' EWSs, including the role of the state and lessons learned from working with the districts. An assistant principal from a high school in Minnesota talked about how his school has used the EWS to identify students at risk of dropping out and the interventions provided to those students.
Early Warning Systems to Support Students at Risk of Dropping Out of High School (REL Northeast & Islands, December 14, 2012). The Puerto Rico Research Alliance for Dropout Prevention at REL Northeast & Islands hosted this webinar to share with educators and policymakers research on key indicators for predicting high school graduation, as well as guidance on developing and implementing a robust EWS. The presentation was part of the Puerto Rico Alliance's ongoing efforts to support the Puerto Rico Department of Education and Puerto Rico Institute of Statistics in their mandate to establish an EWS for all public schools in the territory.

Infographics

Comparing Methodologies for Developing an Early Warning System (REL Southeast)
Data Illustration and Visualization for EWS (REL Appalachia)
Data visualization can help educators address chronic absence (REL West)
Defining Tiered-Models and Early Warning Systems: Similarities and Differences (REL Central and REL Northwest)
Does Raising the State Compulsory School Attendance Age Achieve the Intended Outcomes? (REL Mid-Atlantic).
Early Warning Intervention and Monitoring System (EWIMS) (REL Midwest)
Early Warning Systems Indicator: Course Performance (REL Southeast)
Graphic summaries of "A Practitioners Guide to Implementing Early Warning Systems" (REL Mid-Atlantic)
Identifying Early Warning Indicators in Three Ohio School Districts (REL Midwest)
Preventing High School Dropout Across Vulnerable Student Groups in Oregon (REL Northwest)
Using Data to Monitor and Improve Handle with Care in Schools (REL Appalachia)
Using School and Child Welfare Data to Predict Near-Term Academic Risks (REL Mid-Atlantic)

Tools

Four Signs your District is Ready for an Early Warning System: A Discussion Guide (REL Pacific, September 2016). For districts without an EWS in place, this guide is intended to focus conversations and decisions about whether the district is ready for an EWS and when and how to proceed with EWS implementation if the district is ready. The discussion guide will help readers assess the degree to which each of four readiness signs exist within their districts and data systems. Those signs are: a clear vision for addressing the dropout problem, a high quality data system, access to expertise in data analytics, and the capacity the deliver a tiered intervention system.
Keeping your student in school: A parent guide (REL Midwest, August 2016). This brochure can be used by district and school administrators to communicate with parents about the common signs of being off track to graduate from high school, and how they can partner with their schools to help their student reach graduation day. The brochure is available in English, Spanish, and includes customizable versions in each language.

High School Intervention Self-Study Guide (REL Southeast, August 2016) . This Self-study Guide for Implementing High School Academic Interventions was developed to help district- and school-based practitioners plan and implement high school academic interventions, including those based on EWS data. It is intended to promote reflection about current district and school strengths and challenges in planning for implementation of high school academic interventions, spark conversations among staff, and identify areas for improvement. The guide provides a template for data collection and guiding questions for discussion that may improve the implementation of high school academic interventions and decrease the number of students failing to graduate from high school on time.

For more resources in ERIC on the topic of Early Warning Systems, click here.