Dr. Loeb wondered how much research is generalizable. That is, when local leaders decide what to implement, how can their findings reliably translate from one area to another? Dr. Kane said generalization is not happening now; interventions do not "trickle-down" from expert consensus to practice. He advocated for providing tools to decisionmakers to learn the implications of their decisions. If the effects of an intervention are heterogeneous, as Dr. Kemple suggested, then the only way to understand the effects at a local level is to implement it and evaluate it. Thus, it is important for LEAs and SEAs to build the capacity for decisionmakers to learn about the consequences of their decisions.

Dr. Kemple said that IES funding often aims to improve equity in education at the local level. Localities need to see that when they receive supplemental resources in addition to local investment, the results are worthwhile and, therefore, local investment should continue. Ideally, localities are also contemplating sustainability. Dr. Kemple added that generalizability is "more art than science," but evaluation to ascertain generalizability should continue.

Dr. Loeb agreed that there is a role for testing the decisions made at the federal level, but she hoped there is also a role for gathering knowledge about education that can apply more broadly. Even if a decision is made locally, it makes sense to evaluate at a higher level and share the findings—which is the theory that drives IES research. Dr. Loeb said both presenters seemed to be saying that for knowledge to be generalizable, a better understanding of heterogeneity and the reasons for it are needed, and she agreed.

Dr. Kane said it is important to find ways to get state and local governments into the business of evaluating performance, which would fundamentally change the rate of progress. He hopes to build a model that state and local policymakers would invest in to provide tools for school leaders to pool their resources around research.

Dr. Hedges pointed out that the causes of heterogeneity can be scrutable or inscrutable, and little is known about which aspects of heterogeneity are scrutable and which are inscrutable, posing a challenge for research design. He said more strategies may be needed to get auxiliary information that improves understanding of heterogeneity.

Dr. Gamoran noted that NCER is encouraging researchers to collect contextual data to facilitate comparisons across sites. He envisioned several schools assessing common interventions, then coming together for a meta-analysis that could explore heterogeneity. Dr. Hedges supported the idea of prospectively planning studies with an eye toward evaluating heterogeneity.

Dr. Singer agreed that the sources of heterogeneity are sometimes clear, but the "local exceptionalism" that pervades education is unhealthy. She said that personalizing education fosters the perception in every local school system that its students and context are unique, so results demonstrated elsewhere do not apply. Dr. Singer questioned why school systems refuse to buy in to an intervention unless they see it work in their own context. Other organizations and businesses readily adapt and implement ideas from around the world.

Dr. Kane said that whether local exceptionalism is real or perceived, providing local data is psychologically useful to overcome resistance and facilitate engagement in research. Dr. Singer advocated for pushing back against local exceptionalism with concrete examples that make the case for an intervention so that every method need not be tested in every population.

Dr. Loeb pointed out that most resource decisions are made at the local level. She said it may be beneficial to learn what kinds of tools, incentives, and communication strategies are needed to encourage local decisionmakers to implement effective interventions. Dr. Loeb asked what ED can do to encourage knowledge accumulation when local decisionmakers decide how to use resources.

Dr. Kemple said that local participants must recognize themselves in both the problem and potential solutions, especially when the researchers are of a different race/ethnicity then the community. It is critical to build relationships so that the local participants see their role in balancing relevance and rigorous research. Empirically, researchers can demonstrate that different schools share similar problems. Federally funded researchers should ensure that the local participants benefit from the research in some way.

Dr. Kane said that keeping stakeholders up to date on interim findings from research projects can inform decisionmaking at state and federal levels. Many researchers keep their results secret; it would be better to let the field know what is in the pipeline and when projects are likely to have results that may be useful.

Dr. Phillips pointed out that in early childhood education, researchers dig into questions more deeply through secondary research, often to learn whether an intervention deemed unsuccessful will work in some subpopulations or settings. She asked whether IES funding for large RCTs includes support for collecting data for later analysis. Dr. Phillips also asked how NCER plans to evaluate the success of its proposed research networks. That new research mechanism could foster the kind of collaboration and evaluation that is at the heart of this discussion, she noted.

Dr. Ford said the conversation has focused on how to improve the quality and dissemination of research. However, all classrooms have teachers, and the way teachers are trained has not changed. Dr. Ford proposed that IES focus on teacher improvement. He asked how all the research affects teachers and the schools of education that train them. Dr. Kane added that he believed school districts will begin creating their own professional development around Common Core State Standards, but there is no infrastructure in place to learn from those efforts.

Dr. Chard said schools in his states are "chasing exceptionalism" by constantly seeking out what is new instead of what works. The infrastructure Dr. Kane describes could be a network of state and local agencies that could force school boards to commit to an approach, which would provide longitudinal data on outcomes. Dr. Chard said there is a need to reeducate school and district leaders about how to look at what works. Dr. Kane said he hopes to build some consensus around effective learning so that schools can change the outcomes for their students.

Dr. Kemple supported Dr. Ford's notion of focusing on a single area and building a knowledge base around ideas that are likely to make a difference. He called for a unified IES culture that is visible, aggressive, and transparent to the field about what education science is and what standards IES promotes for the field. Strong partnerships can convince the field that IES standards really do matter. A clear vision of IES should come through in its funding priorities and requirements.

Lunch

The Board adjourned for lunch at 12:04 p.m. During the break, NBES members participated in ethics training, delivered by Marcia Sprague of the Ethics Division of ED's Office of the General Counsel. The public meeting resumed at 1:01 p.m.

Following lunch, Dr. Chard called for volunteers to serve on a task force to draft a letter to Congress about legislation under consideration related to the Family Educational Rights and Privacy Act (FERPA). Once drafted, the Board will hold a public meeting to discuss and approve the letter before sending it to ED. Dr. Loeb, Dr. Gamoran, and Dr. Hedges volunteered to serve on the task force.

Follow-Up Item
Dr. McKenzie-Thompson will facilitate the task force's effort to draft a letter to Congress expressing the Board's opinion about FERPA-related legislation. She will coordinate a public meeting of the Board to review and approve the draft.

Discussion of the RELs
Opening Remarks: Ruth Neild, Commissioner, NCEE
Panelists: Nikola Filby, Ph.D., Director, REL West, WestEd
Neal Finkelstein, Ph.D., Associate Director, REL West, WestEd
Barbara Foorman, Ph.D., Francis Eppes Professor of Education and Director, REL Southeast, Florida State University

Dr. Neild began the discussion by saying that the REL program is an example of a distributed approach for encouraging the production and use of research evidence in education. There are no easy answers to how to encourage states and districts to use research, and there are pros and cons to every approach. Moreover, resources are always limited, and priorities must be set. The current contracts for RELs will expire in January 2017; IES anticipates that the statement of work for the next REL contracts will be released by the end of FY 2015.

Dr. Neild provided some basic background about the RELs. She pointed out that each REL must have a governing board that includes state chiefs or their designees as well as other regional stakeholders. Because the RELs operate through contracts, not grants, IES staff are substantially involved in management of the RELs. NCEE seeks to ensure coordination across the RELs to eliminate duplication, to enforce high standards of quality, and to give a common appearance to reports.

The 2006 contracts focused on positioning the RELs as emissaries of science who primarily conducted RCTs. The 2012 contracts emphasized both research rigor and partnership. Rather than ask whether the pendulum is swinging away from rigor and toward relevance, Dr. Neild said she believes the RELs raised the bar with the 2012 contracts by attending to rigor while also building partnerships and addressing the usefulness and clarity of research.

Dr. Neild posed some questions for consideration:

What other improvements are needed in how we generate and use research in education that the REL program could help to advance?

How can IES better prepare and incent individuals who are good researchers and good collaborators to work with RELs (or on other similar projects)?

For the upcoming RELs, how can we encourage more robust competition in each region?

Facilitating Research in REL Southeast Research Alliances
Barbara Foorman, Ph.D., Francis Eppes Professor of Education and Director, REL Southeast, Florida State University

Dr. Foorman said her organization sees rigor and relevance on the same continuum. Practitioners often want answers to questions that will help them solve an immediate problem, but with limited time and resources, the challenge is prioritizing research. The REL works with practitioners to ensure that their questions are grounded in research and can be answered through studies. It also aims to build the capacity of practitioners to conduct their own research.

To address practice problems, the REL:

• uses its governing board and research alliances to identify significant, meaningful research questions;
• applies a logic model to clarify inputs and outputs;
• conducts bridge events to build research knowledge and workshops on gathering and interpreting data; and
• encourages states to share lessons learned with other states.

Dr. Foorman stressed that getting the right group of people to form a governing board means including legislative liaisons and finding SEA officials who are closely connected with programs. She said recently retired superintendents who are well respected by their peers have a lot to offer (while current superintendents rarely have time to participate). Foundation and business representatives are also important.

Dr. Foorman described some of the successful aspects of REL Southeast research alliances. All include LEAs and SEAs at the center of activities, which helps promote research across the state. Alliance members help pick topics for workshops and events and coordinate the logistics. A bridge event on language and literacy research in Florida led a school district to request a focused study on language among non-English-speaking students, which sparked a bridge event on the new practice guide on ELLs. That event prompted a teacher to document her implementation of the practice guide, which became the basis for a professional development offering. Dr. Foorman said the "circularity" of the work is healthy and exciting.

Dr. Foorman said the REL held events to promote results of the Beat the Odds study, which demonstrated better-than-expected results, but it did not get much attention in Florida until the SEA's communication director saw the infographic and distributed it to other leaders. The mode of dissemination matters, Dr. Foorman noted.

Dr. Foorman described one early literacy RCT recently completed. The participants were happy to take part because there was no "business-as-usual" arm and both of the conditions being compared were attractive. The study built capacity, because schools hired experienced staff as interventionists who stayed on after the researchers left. Dr. Foorman also outlined two other successful REL efforts to help states collect and share data on targeted issues of concern.

Research alliance members help disseminate research, facilitate bridge events, and serve as presenters and discussion leaders at REL events, Dr. Foorman pointed out. They also provide feedback on products and help distribute REL products. Dr. Foorman said the best products employ good graphic design and accessible, accurate language. They are also easy to identify and share. Dr. Foorman concluded that the REL Southeast offers several examples of how rigor becomes "contagious" when stakeholders are engaged in solving relevant and scientifically-verifiable problems in education through research.

REL West
Nikola Filby, Ph.D., Director, REL West, WestEd
Neal Finkelstein, Ph.D., Associate Director, REL West, WestEd

REL West has eight research alliances, all of which differ in their makeup. The goal of REL West is to increase the use of data and evidence in decisionmaking, representing the intersection of relevance and rigor. To achieve this goal, the REL actively partners with research alliance members; uses a range of rigorous analytic methods; provides tools and models processes (e.g., for collecting data); informs local decisionmaking processes, and disseminates research broadly.

Through its partnerships, the REL seeks to build a culture of continuous improvement. For some partners, using evidence in decisionmaking represents a different mindset, so the REL encourages them to understand their current systems, think about alternatives, act with intention, and use data to inform the next steps.

The Educator Effectiveness Alliance involves three states. All three were required by legislation to develop education evaluation systems, so they were motivated to partner with the REL. REL West assisted states in planning pilot studies and conducting psychometric analyses of measures. It enabled states to build the capacity to run their own analytics and to use the data to make changes. For each initiative, the REL created a simple, graphic logic model to help those involved think about the process as a whole and provided detail in a separate handout for the alliance partners. Research findings are disseminated through partners and other RELs.

Most importantly, said Dr. Filby, the states used the information they gathered to improve. They provided guidance and training based on the initial results and modified their evaluation models. They enacted legislative changes to fix unrealistic timelines and weightings. The states developed tools for collecting data, and they conducted more pilot programs to understand how teachers are using the data.

Dr. Finkelstein described the Silicon Valley Research Alliance (SVRA), which focuses on improving math skills, beginning in middle school, as a step toward college eligibility. The SVRA determined that algebra plays the role of "gatekeeper" for progress in math. Analysis of high school and middle school data indicated that students in area schools had a problem with the transition from basic to more advanced math. There was no standard, linear sequence of math courses to support advancement. A number of students took algebra twice, and some never took senior-level math courses.

In 2014, the REL West and SVRA published the results of their collaboration, which focused on using assessment data to guide math course placement. The study found that an inexpensive diagnostic test can predict students who will successfully complete algebra I in eighth grade and was a better predictor than a standards-based assessment.

Later this year, a follow-up publication will describe the results of the Elevate Summer Math Program on math achievement and algebra readiness. Elevate is a short-term, intensive intervention on algebra for rising eighth-graders. It incorporates extensive professional development for teachers and embeds support. Results of the program feed back to the SVRA within 1 year to inform program improvement.

Another SVRA effort provided technical assistance and professional development to teachers who were grappling with transformational geography, a new component of the Common Core State Standards. The effort uses student data to assess teachers' use of embedded assessments.

Discussion

Dr. Gamoran praised the RELs for creating networks of practitioners and policymakers to develop the topics of research that the RELs then address. He urged the RELs to go further to enhance engagement with their partners, so that the RELs are not just disseminating the information but building close relationships with practitioners and decisionmakers to understand the research.

Dr. Neild clarified that the RELs do not just disseminate information but truly partner with practitioners and policymakers to engage in research. For the RELs, the primary audience is research alliance members. Beyond that, the broader audience is the region that a REL serves and, even more broadly, the whole country.

Picking up on an earlier point about defining the audience, Dr. Foorman said that when the REL Southeast targeted principals directly and engaged them in meetings around research, the results were more effectively disseminated. Dr. Filby said more models are needed to help RELs meet the challenge of engaging partners at multiple different levels.

Regarding Dr. Neild's question about training researchers to better partner with others, Dr. Phillips said the question should be how to train researchers, practitioners, and decisionmakers to work within research partnerships. She asked what capacities school districts have that facilitate partnerships and how such capacities could help create more joint partnership models.

Dr. Finkelstein responded that the capacity to conduct research varies widely within and across school districts. Ultimately, successful efforts require a champion who has a keen interest in research and strong desire to create mechanisms for research within the district. Districts must also have the technical capacity for research and an understanding of the tradeoffs (e.g., inconvenience and burden). Dr. Filby said it is challenging to find school districts that want to partner in research.

Dr. Foorman pointed out that researchers need training on how to work in partnerships, and apprenticeships are among the few ways to gain that training. She suggested that IES' pre- and postdoctoral training involve working with the RELs. By apprenticing to a senior investigator in a research partnership, a trainee not only gains skills but also builds relationships that facilitate the continuation of the partnership over time. Dr. Chard suggested that such an approach could be incorporated into the REL contracts.

Dr. Yoshikawa said that investigators are expected to demonstrate the impact of their research but not the impact of their partnerships or training models. Incorporating partnership in pre- and postdoctoral training and integrating trainees into the RELs would be great, he stated. Dr. Yoshikawa asked whether the accountability structure for the RELs has changed over the years. He also asked whether the RELs have evaluated research alliances as a whole to understand what works.

She said the REL West seeks to understand the differences between partnering with an existing group to form an alliance or starting anew. In some cases, an existing structure helps facilitate research, while in others, it is not always clear that research fits into the mission and goals of the group.

Dr. Foorman said funding is a barrier to sustaining research alliances. In some cases, however, an alliance fails because the members are so different that they are unable to learn from one another. Some alliances work better at a small, local scale. Sometimes, a small alliance will mushroom. To improve the potential for sustainability, Dr. Foorman said the RELs should identify the pressing policy issues that stakeholders face and the extent to which they are shared within a region. When common interests are identified, members are motivated to join and continue an alliance.

Dr. Yoshikawa said alliances can be assessed from the perspective of costs, duration, or sustainability. The current contract fosters one model, but efforts should be made to identify other models for forming research alliances and to determine whether such alliances could achieve more and better results.

Dr. Loeb asked what the RELs have learned that make the process of creating and sustaining research alliances easier in the future. Dr. Filby said the tools developed through the alliances are helpful, as is sharing practical advice and finding areas of convergence around research. Dr. Neild clarified that the IES website houses all the tools and products created by the RELs, and she is working to raise awareness about them.

Dr. Finkelstein said that research alliances need to be "fed" regularly or the members lose interest. One challenge is determining the best "food." Ultimately, the research must be packaged in ways that meet the members' needs. On the basis of the REL West model, Dr. Finkelstein said, one cannot overstate the importance of maintaining a presence in the field to ensure that the partners stay engaged and contribute to the generation of important topics for research. Sustaining research alliances is a resource-intensive endeavor, he emphasized.

Dr. Foorman said another important mechanism for sustainability is pairing researchers with practitioners who are immersed in research and can act as a liaison. For example, she partners with a former practitioner and policymaker who is highly respected in the field and who is completing his doctoral work at the REL Southeast. Dr. Foorman added that the REL is figuring out ways to automate dissemination and information-sharing so that feeding the alliances is not a drain on the REL.

Dr. Singer pointed out two areas of concern. First, the research alliances described by the presenters center on topics that the presenters have pursued in their own research efforts. Dr. Singer questioned whether the REL contracts are acting as an umbrella for researchers to continue their research careers. Also, the examples given again highlight local exceptionalism by suggesting that the research findings are unique to the geographic areas in which the research was conducted. Dr. Singer wondered whether the regional RELs could partner with each other. She also said the RELs are well positioned to survey their alliance members to identify common areas of interest across states and regions.

Dr. Singer hoped that IES would think about mechanisms to expand the reach of RELs. At the end of the next round of REL contracts, she hoped that IES would be able to articulate what the RELs produced that could not have been achieved by others under another mechanism.

Dr. Chard asked for clarification of the peer review process for REL products. Dr. Filby said the RELs are still required to go through the peer review process, and the most recent contracts made the requirements clearer. Dr. Neild replied that all products are reviewed for technical quality, relevance, and readability. In general, the RELs like and want the IES "stamp of approval" that comes from the review process. The primary audience for the RELs is practitioners, but contractors are encouraged to publish peer-reviewed articles about their work for other audiences, such as their peers in education research.

Dr. Yoshikawa asked Dr. Neild to describe the range of products that RELs produce. Dr. Neild said the RELs still conduct plenty of RCTs and develop research reports. The category of "tools" encompasses a range of things that the RELs can use to expand on what works. Examples include REL Southeast's summer camp self-study guide, facilitation guides, or slide sets. REL Pacific created software for developing a customizable logic model. Dr. Neild said she would like to see more creative efforts to make tangible, practical products. Dr. Yoshikawa suggested that IES encourage innovation and development of new tools in the next REL contracts.

Dr. Finkelstein noted that over the next 16–18 months, the RELs will be working to ensure that the results of research completed are reviewed and disseminated, especially for topics that state boards of education are wrestling with right now. It is important to demonstrate that the research is relevant to current situations, he concluded.

Dr. Singer observed that, through the RELs, the federal government is playing a direct role in providing professional development regarding, for example, how ELLs are taught in Florida, and she questioned whether such a role is appropriate. As IES revises its REL contracts, Dr. Singer suggested thinking about the balance between technical assistance that verges on federal engagement in local school programming and the goal of research and dissemination.

Nexus Between Student Drug Use and Student Achievement: The Interagency Adolescent Brain Cognitive Development (ABCD) Study
Opening Remarks: Tom Brock, Ph.D., Commissioner, NCER
Panelists: George F. Koob, Ph.D., Director, NIAAA, NIH
David K. Mineta, Deputy Director, Office of Demand Reduction, ONDP
Susan Weiss, Ph.D., Director, Division of Extramural Research, NIDA, NIH

Dr. Brock said that, in recent meetings, the Board has discussed the idea of IES collaborating with other federal agencies. He explained that NIH is launching a study of the effects of drug and alcohol use on adolescent brain development. As part of the study, NIH is interested in data on students' school outcomes and school context. The study is in the early stages, and the investigators are open to input on its design.

Youth Substance Use and Academic Achievement
David K. Mineta, Deputy Director, Office of Demand Reduction, ONDP

Mr. Mineta said ONDP is alarmed and concerned about student substance abuse and its influence on academic performance, truancy, dropout rates, violence, unemployment, sexually transmitted infections, and other health indicators. As the central coordinator of drug control activities across the federal government, ONDP focuses on preventing and treating substance abuse and reducing relapses.

Mr. Mineta described several research findings that underscore the need to address substance use in the adolescent and teen years. Drug use escalates during the teen years. One study showed that as 12th-graders' perception of the risk of occasional marijuana use went down, their use of marijuana went up. Persistent marijuana use that begins early in life and extends into adulthood leads to significant drops in IQ. Students with low grades are more likely to use substances than those with better grades. As school climate improves, academic achievement improves.

Mr. Mineta referred Board members to the ONDP website and its National Drug Control Strategy for details about prevention efforts and ONDP's work with ED and education stakeholders.

The Science Behind Neuroprevention
George F. Koob, Ph.D., Director, NIAAA, NIH

Dr. Koob described recent evidence showing dramatic increases in the intensity of binge-drinking, particularly among 18–24-year-olds but extending into later adulthood. Binge drinking leads to risky behaviors, such as unprotected sex and fighting. Early substance use is tied to substance abuse and dependence later in life.

Dr. Koob explained that the frontal cortex of the brain does not fully develop until about age 25. Alcohol and drug use in adolescence disrupts the frontal cortex function and slows development. Specifically, marijuana impairs memory, and alcohol abuse impairs executive function, or decisionmaking. The combination of impaired decisionmaking and the underdeveloped frontal cortex lead to the impulsivity and compulsivity that drive addiction.

The National Consortium on Alcohol and Neurodevelopment in Adolescence is studying the effects of alcohol use on brain development in 800 subjects, ranging from 12–18 years, sampling them over time with brain scans. Early data demonstrate that brain size correlates with executive function and improvement on attentional tasks. Researchers can harmonize data across different types of brain scans, which provide information about the structure, pathways, and function of the brain. This study was the impetus for the ABCD study, a large, joint research effort around adolescent brain development.

Adolescent Brain Cognitive Development: A Longitudinal Study
Susan Weiss, Ph.D., Director, Division of Extramural Research, NIDA, NIH

The ABCD study is a national, longitudinal study of 10,000 people ages 9–20 to assess the effects of drugs on individual brain development. It involves multiple NIH Institutes, including the National Institute of Mental Health and the National Institute of Neurological Disorders and Stroke (NINDS). The latter is particularly interested in head trauma, often associated with sports injuries, and its interaction with substance abuse.

One impetus for pursuing this research now is the changing policies around marijuana use. Data show that alcohol use by 12th-grades in general is going down, but marijuana use is going up. As Dr. Koob explained, the adolescent brain is still developing and vulnerable to insults. Repeated, frequent use of marijuana in adolescence is associated with long-term changes in brain structure.

A large study in New Zealand showed a dose-response relationship between marijuana use before age 17 and adverse outcomes, including marijuana addiction, use of other drugs, and suicide attempts. Early marijuana use was also associated with lower high school graduation rates.

It is possible that substance users have different brains than nonusers or other genetic or physiological factors that affect current or later use. The ABCD study would enable investigators to gather baseline data as early as age 9. The large, nationally-representative, long-term study would allow researchers to see changes over time.

Dr. Weiss explained that NIH released a funding opportunity announcement (FOA) in which it outlined its vision for the ABCD study and asked applicants to propose how to accomplish the goals of the research. Response to the funding opportunity was robust, and awards should be announced by September. NIH hosted an expert panel workshop and put out a request for information for more input on the study's research design. It will use the input to revise the study design, then host a symposium for further discussion. Once awards are finalized, NIH will bring the researchers together for discussion about the final protocol. Dr. Weiss said NBES still has time to give its input on the research design. An external advisory board will also be convened to advise on the ABCD study.

The ABCD study will address the impact of diverse patterns of substance use on brain function as detected by imaging. Technology has advanced to the point that researchers can get good, reliable results with brain imaging from different laboratories. The study will look at the consequences of substance abuse, including academic achievement. It will assess how drug use affects the expression of psychopathology, including substance use disorders, and how the emergence of psychopathology influences drug use. Finally, the ABCD study will look at factors influencing drug use and how use of drugs contributes to other drug use.

Dr. Weiss anticipated that subjects would be tested every 2 years. NIH plans to release data for use as soon as they are available. The number of research sites is not yet known, and some sites may use a hub-and-spoke design. Dr. Weiss hoped NBES would provide insights on how the study could use the information that IES researchers are collecting.

Discussion

In response to Dr. McLaughlin, Dr. Weiss said she did not know how samples would be chosen, but the study is seeking a representative sample. Grantees will determine the sampling. Dr. Weiss anticipated that grantees would work with school systems, but the ease with which schools can collaborate with researchers varies across states.

Dr. Hedges asked whether "representative sampling" meant probability sampling, which ensures that individuals have an equal probability of being included in the sample and is the basis of claims about generalizability. Dr. Weiss said experts in sampling would address the specifics, but the goal is to have generally normative sample that includes, for example, people from rural areas and people of various race/ethnicity.

Dr. Loeb asked whether researchers already have some theories about the causes of substance use. Dr. Weiss said there is a lot of evidence about risk factors and investigators have ideas about what to look for. The study will collect biological specimens for genetic evaluation, but there is not enough funding to analyze all the samples.

Dr. Koob said NIH will have to limit the number of hypotheses in the study unless more money becomes available. Dr. Weiss said the study is not as hypothesis-driven as others. She expected that hypotheses would emerge and that other studies would be tacked on to the ABCD cohort as it develops over time. For example, NINDS is very interested in studying traumatic brain injury through the ABCD study.

Peggy Carr, Ph.D., asked whether the external advisory board or some other entity would address "the hard questions." Dr. Weiss said the advisory board, NIH staff, and grantees will handle technical issues. She recognized that the research may face ethical challenges, such as when to alert adults that drug use was detected, especially in very young children. The FOA identified the areas of expertise needed; applicants will address the requirements, and the external advisory board will tackle any areas not well enough represented in the applications.

Robert A. Underwood, Ed.D., asked what agencies are eligible for awards under the FOA. He also asked whether applicants in states that have legalized recreational marijuana use would have a better chance of being funded. Dr. Weiss replied that grantees can be individual sites or preexisting collaborations, and Washington and Colorado are well positioned. The study will have statistical controls in place for sites where marijuana is legal and will have to account for changes in policy throughout the study. Dr. Koob added that the award uses a consortium mechanism, and NIH staff have significant influence on consortia. With so many Institutes and Centers participating and making decisions about the study, it will not be "a free-for-all," said Dr. Koob.

Dr. Yoshikawa applauded the study, especially in light of the sudden demise of the National Children's Study. He noted that gathering contextual information about schools and peer groups is expensive. However, NCES has extensive data sets, and NIH could add biomarkers. Dr. Yoshikawa said the study could be a step toward developing national, longitudinal data sets that include biological and physiological markers.

Dr. Brock said IES collects a lot of data about student performance and school climate, for example. He asked whether early conversations about the ABCD study considered what kinds of questions could be answered with education research data. Mr. Chapman said NCES met with NIH a few times to discuss the administrative records data that NCES collects. He said NCES has a potentially strong set of assessment outcomes data and a good relationship with the U.S. Census Bureau, which agreed to provide district and school boundary mapping.

Dr. Brock asked how NIH is thinking about partnering with education researchers. For example, will the data become public? Can education researchers propose add-on studies? Dr. Weiss said the study is still seeking more funding, but data sets will be available, and secondary data analyses are encouraged. She thought education researchers would be an important part of the study, because education outcomes are a key issue. Dr. Koob added that the project is evolving, but it is his job to make sure it is not overloaded beyond capacity. He noted that the subjects would have imaging or neuropsychological testing every year, so any additional testing must be short.

Dr. Hedges suggested linking NCES items to NIH assessments and linking to existing data sets by state. In the future, it may be possible to get individual academic performance and contextual information. Dr. Hedges said NIH could leverage a huge amount of existing data without new testing. Dr. Weiss said concerns have been raised about the difficulty of getting individual academic data. Dr. Carr said NAEP was able to get information at the individual level through states, despite complicated privacy laws. She said NCES can provide suggestions on how to gather school data at minimal cost.

Dr. Hedges suggested NIH consider linking to international data sets, and Dr. Weiss responded that NIH is talking with several international groups. Dr. Hedges recommended that the research team include experts in causality. Dr. Weiss agreed to bring the suggestions to the advisory board for inclusion in the study design.

Dr. Phillips advised NIH to consider the lessons learned from the National Children's Study, which grappled with many of the same issues that the ABCD study is facing. Dr. Carr suggested talking with the Office of Management and Budget, because it keeps track of funding and resources across the federal government.

Closing Remarks & Adjournment
David Chard, Ph.D., NBES Chair

Dr. Chard said the next NBES meeting is scheduled for October 2. He will present a draft of the suggested letter to Congress regarding funding for education research for Board members for review. He will also take steps to convene those who volunteered to address issues related to two bills around FERPA. Both of these actions require swift movement.

Dr. Gamoran said he liked the organization of the meeting agenda, and Dr. Chard reiterated his call for feedback and suggestions from Board members for future agenda development. Dr. Chard adjourned the meeting at 4:05 p.m.

Report prepared for NBES by Dana Trevas, Shea & Trevas, Inc.

The National Board for Education Sciences is a Federal advisory committee chartered by Congress, operating under the Federal Advisory Committee Act (FACA); 5 U.S.C., App. 2). The Board provides advice to the Director on the policies of the Institute of Education Sciences. The findings and recommendations of the Board do not represent the views of the Agency, and this document does not represent information approved or disseminated by the Department of Education.