Location
Suite 206
Heritage Reporting Corp.
1220 L Street, N.W.
Washington, D.C. 20005-4018
(202) 628-4888
contracts@hrccourtreporters.com

Day one proceedings
Day two proceedings

View, download, and print the transcript of the December 4 meeting as a PDF (379 KB)
View, download, and print the transcript of the December 5 meeting as a PDF (313 KB)

Day one proceedings

The parties met remotely, pursuant to notice, at 10:00 a.m.

Participants:

Carol Lee, Chair, Northwestern University
Maria de la Concepcion Hernandez Legorreta, Maryland State Department of Education
Linda Darling-Hammond, Learning Point Institute
Douglas Fuchs, Vanderbilt University
Denisa Gandara, University of Texas at Austin
Elmer Guy, Navajo Technical University
Dana Hilliard, Mayor, Hilltop City
Stephen Klasko, General Catalyst
Ruth Lopez Turley, Rice University
Caroline Sullivan, North Carolina Business Commission for Education
Hirokazu Yoshikawa, New York University

Ex-Officio Members:

Mark Schneider, IES
Peggy Carr, IES-NCES
Elizabeth Albro, IES-NCER
Matthew Soldner, IES-NCEE
Nathan Jones, IES-NCSER
Diana Bianchi (Designee: Dr. Brett Miller), NICHD
William Wiatrokski (Designee), Bureau of Labor Statistics
Sethuraman Panchanathan (Designee: Dr. James Moore), National Science Foundation
Ellie Pelaez, Designated Federal Official
Anne Ricciuti, Deputy Director for Science

Proceedings
(10:00 a.m.)

DR. LEE: Good morning everyone. A special good morning to our colleagues on the East Coast. Not the East Coast, the West Coast. Sorry about that. I think we weren't thinking West Coast friendly when we set that time.

So this will be really sort of our first full meeting where I'm hoping we'll be able to sort of scope out a sense of our vision as a Board, and how we want to organize ourselves as a Board. And I want to thank everyone, the various subcommittees that we established just in terms of prep for this meeting for the work you've done.

I'm going to suggest, and if we can finalize this on our meeting tomorrow, that we establish formal subcommittees that will be able to meet in the interim outside of the Board meeting, so we knew we needed to do that for the committee working on the process for our hiring an Executive Director to support the Board, but the other committees need to be able to also meet outside of Board meetings. So I'm going to recommend that we do that.

So with that, Ellie, can you call the roll?

MS. PELAEZ: Yes. Will do. Dr. Anaya?
DR. LEE: He will not be able to attend today.
MS. PELAEZ: That's right. That's right. Thank you. Ms. Hernandez-Legorreta?
MS. LEGORRETA: Present.
MS. PELAEZ: Dr. Darling-Hammond?
DR. DARLING-HAMMOND: Present.
MS. PELAEZ: Dr. Fuchs?
DR. FUCHS: Present.
MS. PELAEZ: Dr. Gandara?
DR. GANDARA: Present.
MS. PELAEZ: Mr. Guy?
(No response.)
MS. PELAEZ: Dr. Harper?
DR. HARPER: Present.
MS. PELAEZ: Mr. Hilliard?
(No response.)
MS. PELAEZ: Dr. Klasko?
(No response.)
MS. PELAEZ: Dr. Lee?
DR. LEE: Present.
MS. PELAEZ: Dr. Scott?
DR. SCOTT: Present.
MS. PELAEZ: Ms. Sullivan?
MS. SULLIVAN: Present.
MS. PELAEZ: Dr. Yoshikawa?
DR. YOSHIKAWA: Present.
DR. KLASKO: This is Dr. Klasko. My microphone was off, so I was —
MS. PELAEZ: Okay. Thank you. Got you.
DR. KLASKO: I'm here.
MS. PELAEZ: Dr. Schneider?
DR. SCHNEIDER: Here.
MS. PELAEZ: Dr. Carr?
DR. CARR: Here.
MS. PELAEZ: Dr. Albro?
DR. ALBRO: Present.
MS. PELAEZ: Dr. Soldner?
DR. SOLDNER: Present.
MS. PELAEZ: Dr. Jones?
DR. JONES: Present.
MS. PELAEZ: Mr. Robert Santos from the Census Bureau, or a designee?
(No response.)
MS. PELAEZ: Dr. Brett Miller?
DR. MILLER: Present.
MS. PELAEZ: Mr. Wiatrowski?
MR. WIATROWSKI: Here.
MS. PELAEZ: Thank you. Dr. Moore?
DR. MOORE: Here.

MS. PELAEZ: Thank you. Okay. We have a quorum.
DR. TURLEY: Ellie, you did not call out my name.
MS. PELAEZ: Oh, I apologize.
DR. TURLEY: This is Ruth Lopez Turley.
MS. PELAEZ: Oh, I'm sorry. I'm sorry, Ruth. I had you on my list. Sorry. Dr. Lopez Turley?
DR. TURLEY: Thank you.
MS. PELAEZ: Thank you.
DR. LEE: All right. Great. And so now we are going to call for approval of the Board minutes. I assume that you all received, and had an opportunity to review, so unless there's any discussion, I'd just call for the vote.
DR. HARPER: I move to approve.
DR. LEE: And a second?
MS. PELAEZ: Second.
DR. LEE: All approve aye.
(Chorus of ayes.)
DR. LEE: All opposed nay?
(No response.)
DR. LEE: All right. That's approved.

So our first report is from the committee who will discuss a process for us responding to the National Academy, and other reports on IES. And that is Hiro, Ruth and Denisa. So I'm not sure who wants to take the lead; who is chairing.

DR. TURLEY: I will start. Thank you, Carol.

So yes, this committee is made up of Hiro, Denisa, and myself, and I'm going to share — I don't know if you all already have this, but I'm going to share in the chat a link to — there we go — a link to a document, just a Google document, and you all should have access to it, but, of course, if you don't, please let me know. Did you all receive the link just now? Okay. Good.

So what you'll see in this document obviously it's a draft document. We're very open to your input on what this process should look like, but what you'll see here is that we — as Carol just mentioned this committee met to try to develop a process that would ensure that our next Director of IES will develop a list of priorities that will be shared with the Board, and the Board will have an opportunity to review, and eventually approve that list of priorities.

And there was a discussion among us about the use of that language, so we are open to hearing your input about that as well, based on the IES website that describes the duties of this Board. It does describe one of our duties as approving — it actually uses that language of approving.

So that's why you see that language here as well. But we're open to discussion regarding the extent of the authority that we have to review and approve priorities, but right now this document is written with that language.

So what you'll see here is we divided it into a couple of sections, or three sections. The first one is focusing on the process for evaluating the IES Director's priorities.

The second and third are focused on the process for evaluating recommendations on any NASEM reports. The third one is for processing IES responses to those recommendations. So second and third are related in that way.

So going to the first section, the process for evaluating the IES Director's priorities, you'll see that there is a required — the IES Director is required to post those priorities on the federal register for a mandatory 60-day comment period, but we would like for the IES Director to submit to us, the Board, prior to that period to submit to us those proposed priorities so that we have an opportunity to review them, and discuss them.

And you'll notice that I just put a blank — blank weeks for now, because we are open to getting your input on what would be a reasonable amount of time. The goal here, of course, is we don't want to be a burden to the IES Director, but we definitely want to be a part of this process, and we would like for it to be a reasonable amount of time, so that we have an opportunity to review and discuss those priorities.

And you'll see here in these steps that there is an opportunity for there to be a little bit of back and forth, that we can actually, you know, discuss them, but then if we recommend any changes to those priorities, or anything along those lines, that we can convey that to the IES Director, and then that person will have an opportunity to respond, and make revisions as-needed, and that we can do this a few times.

The main thing I want to point out here is that in terms of the criteria that's recommending that are used to assess these priorities, there are mainly two that we have a starting point. First, alignment with the Education Sciences Reform Act, or ESRA, as well as alignment with recent NASEM report recommendations, which then brings us to the second and third pieces.

But I'd be happy to pause here for any comments before I turn to the second and third sections, or would you like for me to continue?

Carol, what do you think?

DR. LEE: I'd like to pause for a moment because I'd like to get some feedback from Mark on this. So there is, and, I believe, Mark had mentioned at some point at one of our meetings about some specific priorities that he had that I think he wanted the Board to comment on, and then there's the issue of the timing for the new Director, which I think starts in April.

And so I'm a little bit unclear at this point in terms of any relationship at this point to existing priorities that Mark may have, as it relates to the timing of the appointment of the new Director.

DR. TURLEY: That's a great question, Carol, and I have to admit that this subcommittee assumed that we were referring to the next IES Director. So all of — yeah, these details, these timelines, I assumed that it was probably too late for the current Director, but I'm open to suggestions about that regarding the current Director.

DR. LEE: Mark, do you have any comments on that?
(Technical issues.)
DR. LEE: Mark, are you there?
DR. SCHNEIDER: Yeah. (Audio interference).
(Technical issues.)

DR. SCHNEIDER: So first of all, yes, I agree that most of these comments apply to the next Director, and that person should be in place hopefully the last week in April, or early May. The White House has asked several people to fill out screen aires, but as you know, that's all being driven by the White House Office of Personnel.

So where that stands, and who is actually being actively considered, is pretty closely held, but I know that the intention is to have someone in place so that there's not the kind of hold that existed in leadership in the — several times in the past. That's number one.

Number two, with regard to existing priorities, as you know, there were only three people on the Board during — into the last week of the Trump Administration. There were a bunch of people, and none of them were ever finalized.

So there exists on the website priorities that were established by John Easton (phonetic) in 2011, I think, and that went through the formal process of review and approval by the Board.

When I came to the Director, I proposed different proprieties, but the Board was pretty much — I kept waiting for the Board to be constituted, and to turn into more than just three people, so I never submitted it to review by the three-person Board, which, of course, I could have done because the legislation just says that quorum is — would have been two of the three people. We could have had that meeting, but I didn't like the optics of it.

So if you look at the — if you look at what is there on the Federal Register, I mean, I think it's the right level of priorities for a statement like that. And I think you have the choice of either approving those, right, to take the place of the ones from 2011 or not, and just waiting until April when you start working with the — with whoever the new Director is who will hopefully be seated, as I said, at the end of April, or the beginning of May.

I mean, I think it's clearer for you to approve these, but if you don't like them, then you have the opportunity to just wait until five months from now.

DR. LEE: So thoughts from the Board the question of whether we want to visit the priorities that have been — that are already online, or simply wait until the appointment of the new Director.

DR. KLASKO: This is Steve. I mean, I just think that given the timing and proximity, having a new Director see what these were would have — then be able to look at what they now view as what they want to spend the next few years doing, I think is important as opposed to unnecessarily saddling them with what was already decided.

DR. TURLEY: Yeah. I'm not sure that it's worth our time to go through the effort of approving the current priorities under Mark Schneider given that he's got very little time left in this role. So I would prefer to just — and this is assuming, when I was talking about this proposed process that this would apply to the next Director.

DR. LEE: Okay. I have a sense, unless someone wants to speak otherwise, that the sense of the Board would be to wait until the new appointment to the new Director is appointed, and begin that process at that time.

Okay. We'll go onto the reports. So let me just say that part of my thinking is that the work of each one of these committees are ones that at this stage we're going to be sort of discussing, and giving feedback on process, but not that we're making a decision because there's so much information embedded just in the mere list of recommendations from the future of education report, but to get a general sense of the direction of those recommendations of that, and the other reports, and then to talk about a process and timeline through which we would respond.

DR. TURLEY: Correct. So the idea for today is not to actually make any decisions about any specific recommendations, but just to go over the process.

I do want to mention that I only talked about the process for evaluating the Director's priorities, but I also want to mention briefly the process for evaluating NASEM report recommendations that you can see listed here as well, that the idea would be to — for the Board to hopefully be alerted of any releases of new and relevant NASEM reports as soon as they become available, and then to have the Board review and discuss those reports at one of our Board meetings.

One possibility that you see here is it could be the whole Board, or we could assign a reading committee depending on how we'd want to proceed here. But the idea would be that then the Board would have an opportunity to share reactions, advice, or anything — or timeline if a timeline isn't already mentioned in the actual report, to discuss these things with the IES director.

And then the last part that I'll mention is just that there's an opportunity for the Board to evaluate the responses to any recommendations listed on NASEM reports. So that's also listed here.

The main goal is, of course, just to have — to do this in a timely manner, to actually come up with a timeline so that — both a timeline for responding to any recommendations, but also a time line for acting on those recommendations. But of course, there's an opportunity for the IES director to say, well, some of those recommendations are not feasible, or not — or maybe not necessary. So there's an opportunity to say that.

But either way, if the Director's response is this is a good recommendation, then we'd like to see a plan of action for those recommendations, and if the response is, no, this is not feasible, then we'd like see an explanation for that.

So yeah, so that's just the — just to wrap up the process of — you know, for the Board to be involved in both the Director's priorities, and NASEM report recommendations.

So with that, I do — as you all know, there are currently several reports that, I'm sure, many of us have already read, and I just want to highlight all the recommendations.

I'm going to go over briefly the Future of Education Research at IES report that came out last year, and then Denisa and Hiro will go over briefly the other two reports that are listed here.

But you can see here just for our purposes all the recommendations of this first report are listed in this document. I'm not going to read through all of them, but I will say that at a high level a few things that are worth highlighting are that the general aim here is to expand research topics, to expand research design, to expand research methods, expand even units of analysis, not just focusing on the student level.

So those are the kinds of recommendations that you'll see listed here. The general idea here is they mention a need for expanding the type of research that IES funds.

Another overarching goal here that stood out with these recommendations is, of course, an emphasis on equity, and really focusing on more transparency as well. Transparency in several ways, and transparency in terms of, you know, how decisions are made in terms of training, who is going through training, but also, of course, who is getting funding. So there is currently some level of transparency there, but the recommendations here are to — are requesting for more transparency.

And two last things I'll mention with these recommendations is just more — a better process for both adding and removing topics for funding IES projects, and so that you just — the need for a process for adding and removing topics was highlighted.

And the last thing that was mentioned that stood out with these recommendations is to just — a better process for partnering with — you know, between researchers and practitioners, and folks that are actually putting that research to use.

Currently the process, the application process, requests letters of support from practitioners, but the recommendation here to move beyond that to actually have a partnership plan in place, perhaps a working group in place, things like that.

So those are the overarching things that stood out in terms of these recommendations, and as Carol said, we don't necessarily have to make decisions right now about the specific recommendations, but we should definitely be aware of them, and have them in mind for perhaps the next time we meet, and especially once a new IES director is in place, to talk about how the new director will respond to these specific recommendations.

And then if there are any comments about this first report, I'd be happy to hear them, but then I'll turn it over to Denisa to talk about the second one.

DR. LEE: Denisa, you go ahead.

DR. GANDARA: Okay. Thanks, Ruth, for that overview.

So the second report was a vision and roadmap for education statistics. That one was also published in 2022, and that one was specifically addressing NCES, the National Centers for Education Statistics.

And here the recommendations — there's actually some alignment with the recommendations from the previous report that was just described, so some common themes across where strengthening the focus on equity in different ways.

So for the NCES report focusing more on data collection, and expanding research methods and design, examining — engaging in strategic planning. So that was really central to this other report for NCES, and having regular evaluations of responsiveness of research to contemporary issues.

And also similar to that previous report, engaging with stakeholders, improving transparency, and leveraging educational technology.

And one of the other key recommendations was to strengthen state longitudinal data systems. So that's one that was specific to NCES in strengthening the capacity to link data across state systems.

So again, the specific recommendations are listed on the document, but those are sort of the key themes, and the crosscutting themes across the first two reports. And I think Hiro is going to talk about the third report.

DR. YOSHIKAWA: Yeah. And I can jump in. Of course, there was one other 2022 report, which is on NAEP, and because that's under the authority of NAGB, we memorize it here. It's not under our Board technically speaking.

So I think we did want to highlight one prior report because there has been some follow-up from NCES, and it seems like some of it is ongoing, which was the report in 2019 on monitoring educational equity, which basically had as its heart a proposal for indicators for educational equity that IES could collect at the national level, though it also had recommendations for both state and district level collection of these elements.

So we pasted the relevant table here, which is in the summary of the report, which is a list of educational equity indicators, and as you see, it is not just about equity of educational outcomes, but equity of educational inputs. And so that, I think, was the point of that report.

And NCES in its summer — this last summer, its update, indicated that they were in the process of creating a composite indicator based on this report. So they have already been in the process of implementing based on this 2019 report.

But I just wanted to let our committee members know about this report, and that there was active activities to kind of follow-up on its recommendations. So that's why we included it here.

DR. LEE: All right. So we're going to have next on the agenda an opportunity for the various Commissioners to respond to report on how they have responded to the recommendations in these various reports.

I think that one of the recommendations that I'm going to make that we can resolve probably tomorrow because I think it will apply to the overall committee structure of the Board.

And that is — so we were made aware that one of the subcommittees that we had identified in planning for this meeting was one around the process for the Board identifying, and hiring an executive director to work on behalf of the Board, and we were told because that involved personnel issues we needed to create a formal subcommittee that could meet outside of the public meeting of the Board.

And I think we need to do the same for — we would create almost like standing committees coming out of this meeting. And I think this group would be one of those because resolving how we're going to work through responding to this range of recommendations is not something we can do within the context of a Board meeting.

I also am going to recommend that this body be constituted into a formal subcommittee of the Board with the authority to meet outside of the Board meeting.

At the same time, I think that because of the scope of this work, and the technical specificity of it, I'm imagining there's a lot of work that's going to need to be done in reconciling these recommendations with the existing priorities for not only IES, but the various centers.

I also am imagining that there's some timing issues in terms of how any of these recommendations might relate to issues that are under consideration for the ESRA reauthorization.

And in that case we probably need to be making some evaluations and recommendations relatively soon in terms of the timing for the process as that's unfolding, and that the committee that we would formally constitute as a formal committee of the Board with the capacity to meet outside of Board meetings, would also be able to recruit experts in various fields who might advise that committee in working through the details of how the Board should respond to his recommendations.

So any feedback around that structure? And I'm assuming that the Board has the authority in constituting standing committees to also in those internal meetings invite stakeholders who are able to give us technical advice.

The other thing that I think is relevant here, and again, I'm not quite sure how to handle that, but, I think, it would come under the purview of this committee, is we've gotten two statements of recommendations for the Board to consider; one from the Institute for Higher Education Policy, which we sent out, and shared with everyone, and the other from the Alliance for Learning Innovation.

And all of their recommendations, I think, again, are relevant to this question of priorities for IES that they want the Board to consider. And again, I think the details are such that I don't think we can respond to them today, but that it would be most efficient to assign them to this committee to look at in relationship to the NASEM reports, et cetera.

So any thoughts on that as a process for moving forward? I can't hear you, Ruth.

DR. TURLEY: Yeah. Thanks. Thank you for that recommendation. I really like the idea of sitting on subcommittees to keep the process moving bit faster than just depending on these big meetings with the entire Board.

But just to be clear, when you said this committee were you talking about this submitted, or this —

DR. LEE: No. This subcommittee.

DR. TURLEY: This subcommittee. Okay.

DR. LEE: Right. My general sense is that to work through the details of the areas that we're going to need to address, we need working groups who are going to delve into them so that when we come to the Board you have recommendations that you've been able to vet internally through a process that would involve, again, other people who can advise you and the Board through the process.

DR. TURLEY: All right. That sounds good to me.

DR. LEE: Yeah. I think that's a wise way to proceed. Just to publicly acknowledge, I'm assuming they may be in the public presence that's here, but to assure the Institute of Higher Education policy, and the Alliance for Learning Innovation, that the Board has received, and will plan to formally take up their recommendations.

I think before we move to the response of the various Commissioners to the recommendations from the various reports, we probably need to think about a timeline for the work of this committee, and perhaps if anybody here has any sense of — and I know this is nothing definitive, but any sense of the timeline for the process of ESRA moving through the Congress, because this would inform, I think, how soon we need to have the committee able to do the work.

DR. SCHNEIDER: Carol, this is Mark. So could I just add a couple of things on that?

DR. LEE: Sure.

DR. SCHNEIDER: So we know that the FACA committee, so FACA as it governs your activities, and I would suggest — and it's very complicated, and, actually, when we talk to the General Counsel's Office they refer everything to a separate FACA group within the OGC. So I would suggest that everything you do you would need to make sure that it's kosher with the FACA. That's number one.

Number two, in terms of timeline for ESRA, well, I mean, it's both very tight, and very loose. So very tight in the sense that the Health Committee is actively working on its reauthorization proposal, and depending on the day they're talking about moving to the next step within a week or so.

So they are actually moving very quickly. So you need to understand that. But there will be multiple bites at that apple, right? And the ultimate bite is if the House acts on the ESRA reauthorization, then it has to go to committee, but you know this.

I mean, so it's both a short timeline for help, and a much longer timeline that's much more problematic because ultimately the House has to pass a version of ESRA reauthorization, and then it would have to be a committee, to a conference committee, to iron out the final stages.

There are a couple of opportunities still for input into the Health Committee's discussions that they're intending to get a bill out pretty soon.

DR. LEE: So a couple of responses. We certainly will follow through in terms of any legal constraints. This is one of the reasons that we moved to formulate the committee working on the process for this Board hiring its executive director to a standing committee that would be able to meet outside of the public meeting because it dealt with personnel issues.

I am in communication with a staffer from the White House about the scope of our authority as a Board. We will certainly follow through, but I am very confident that we have the ability to constitute our working committees as we so choose.

The other is this is related to the — which we'll get to toward the end of the meeting the process for our identifying an executive director; that the executive director, as I understand it, is a staff person that is hired directly by the Board to be our support person, and liaison for our activities both with IES and with the Congress.

And I think we are going to try to have some more conversation tomorrow about the whole reauthorization process with ESRA, but, I think, we need to be able to create some line of communication with the Health Committee about our interest as a Board in weighing in on this process, so that we have some feedback relationship around the timing of our response.

Any thoughts on what I've just shared?

DR. HARPER: That sounds good to me, Carol.

DR. LEE: In other words, there's no point in the White House, it seems to me, appointing a Board if we have absolutely no relationship to the process just in terms of getting some sort of feedback, but we will pursue that, and figure that process out.

So now we're going to move on, which will be very helpful in getting reports from Nathan, from Liz, and from Peggy, and Matt, to how internally you all have responded to the recommendations in these various reports, and this will — since I think everything we — there's a direct transcript for this meeting, so that will be a helpful resource for this policy committee, to be able to review how internally you all have responded to the recommendations in the reports.

So we'll start with Nathan.

DR. JONES: Okay. And I'm actually going to hand it off immediately to Liz, who is going to get us started, and, I believe, Liz, you are going to be in control of the PowerPoint slides; is that correct?

DR. ALBRO: I am. Here, give me just a second here. Let me pull up the slides, and share, and we are going to view this in a shared fashion because the reality is that as we have been reading through this report it's clear that it will take more than a village.

So actually, Nate, I just — so you guys will hear me calling on my colleagues as we switch without calling; we'll switch. If you all can't hear anything, please just let me know.

I think the way we would like to do this is we would like to get through our deck, and then we will be happy to engage in conversation. So Nate, I actually think you are the first person.

DR. JONES: Oh, that is right. So just to get us started, our team thought it would be useful to just take a step back, and just at the onset provide just an overview of the structure of IES. This may or may not be helpful for folks.

This may or may not be too much, or too little information, so to the extent that before we jump in, and start talking about our responses to these recommendations, if folks do feel like they need a bit more, we're happy to pause, and hear folks out.

But one of the things that we wanted to just make clear, and lay out, is that IES does have four centers, so Liz and I, I think, were formerly tasked with making a response to the NASEM report, but to the extent that we thought it would be useful, we're also going to be calling in Matt Soldner from NCEE, and Anne Ricciuti from the Office of Science.

And I think the reason for doing so is just to give a brief overview of the charge of the NASEM committee. So they were charged with four tasks. The first was to identify critical problems where new research is needed.
The second was how to organize the request for applications. This is, I think, why we've brought in the Office of Science in our response.

The third is to focus on new methods and approaches.

And the fourth is new and different kinds of research training investments.

So to the extent that it is useful, we will walk-through our responses kind of around each of those, but we did want to just say at the onset both structurally, and because of this set of recommendations, we thought it would be important to at times turn it over to Anne, or to Matt, for responses.

Okay. Liz, you can move on.

DR. ALBRO: Great. Thanks, Nate. Sorry. I've got too many things going on here.

All right. So we've decided that we will — we thought it was really important to actually start with one of the last recommendations in the report because as Ruth really made it clear when she was summarizing at its very highest level what the report is asking us to do, it involved expanding, right?

Expanding our investments, expanding what we're doing. And we are, of course, constrained by the fiscal and personnel resources that we actually have present.

And so just to highlight what we're actually talking about in terms of the amount of money that Congress has appropriated to us, we only have numbers available for fiscal '22 and fiscal '23, as I think most of you are aware. Appropriations for fiscal '24 have not yet been made.

So we are operating currently with the assumption that we have flat funding, but we, in fact, don't know what our actual dollar amounts will be available for fiscal '24.

As you will note, there are two budget lines that support research here at IES. The first is our research, development, and dissemination line, which you all are going to hear me talk about as the R&D line. And importantly, that line supports not only the research activity at NCER, it also supports activities at NCEE, as well as across IES functions specifically focused on dissemination.

I also just wanted to highlight that you'll notice that there was a nice bump-up in '23, but I do want to underscore that that bump-up was actually associated with directive language in our appropriations connected up to coming up with high reward, solutions for seemingly intractable problems, which we haven't been acting on over the course of both '23, and activities so far this year.

You will also notice that the research in special education line, the mixer, the rise line, it's substantially less, unfortunately, than the available funds that we have in R&D, but it also did have a bump-up in '23.

So I just wanted to set that there, and again, one of the very last recommendations of the NASEM report was to increase the funding that is available for us to support education research, and unsurprisingly, in the role that I currently inhabit I agree with that recommendation wholeheartedly.

All right. So as we were thinking about, and as we have been thinking deeply about the report, we have been thinking about more than just the education research grants program, which is — and the research training grants program, which is where the report focused in part because the breadth of recommendations really pointed out to us opportunities to leverage existing mechanisms that we have in order to really meet the spirit in the full, we believe, of the NASEM report.

So we are also over the course of this morning going to share with you investments that we have been focused on in terms of the research and development centers, and NCER is required to support.

The research network is a portfolio of networks that we have leveraged, we think, to really answer critical problems of policies and practice.

Our investments in collaborations, and just to be clear, sometimes these things cross, as well as our required small business innovation research program. This is required by all federal agencies that have more than $100 million in investment and research, and NCER holds the responsibility for that program for the entire department.

Back to you, Nate.

DR. JONES: And so, I think I will just echo that within NCSER we have a set of levers that we can use as well to respond to recommendations like those in the NASEM report.

And I think we do want to prioritize these two main ones in our research grants program, and also our research training. I'll note there on the research training side that it's going to be useful later.

There is a long history of how the training came to be within NCSER, but one of the things that has been kind of a demarcation historically is that the Office of Special Ed programs, OSA, has run pre-doctoral programs, and so what that has meant for us within NCSER is historically where we've placed our emphasis on the training side is with regard both to early career training, so mentoring for early career faculty, and researchers, as well as post-doctoral training.

But, I think, what we wanted to emphasize is as we're thinking about these recommendations we do have this variety of levers, some of which are better suited to responding to the recommendations than others.

Okay, Liz?

DR. ALBRO: Thanks, Nate. All right. So for the next few minutes what we are going to do for you is share with you these crosscutting co-PIs that we have identified for the recommendations.

As you all know, there's a list of recommendations which is very, very long, and so what we can provide is an itemized list, but we really thought it was more helpful to sort of bump up a couple levels to share with you how we have been thinking about the co-PIs in shaping the work that we are doing.

So the first is responsiveness in community engagement. The second is expanding competitions, which Ruth has already showcased for us. The third is the broadening participation lens, and then, of course, equity, which crosscuts everything that we do.

So let's start with responsiveness in community engagement. We're going to share with you all some of the activities that we have already undertaken so far.

So the first we've used so far is three different approaches in terms of identifying and responding to community priorities within the two research centers.

So the first is that we have been leveraging our ability to put out requests for information, or RFIs, through the Federal Register. So this goes out to all the entire nation, and anyone is available to provide feedback.

So the full science of the fiscal '24 RDD centers we put that out in February where we knew we were required to do this, and as you all know, in law we have very, very broad topics, and we asked for input, and we still didn't receive any two responses out of 84 responses, including, just so you all know, someone's entire dissertation.

So there was lots of content to review, which we enjoyed, and have really pulled on, and as I hope some of you noticed, we will be posting our new competition notice for our R&D centers later this week.

We have also put out an RFI, which recently closed. Our new investment responding to the directive in appropriations that we're pulling from seedlings to scale, we've not had a chance to look carefully yet at those responses, but we did receive 62 comments, and we're very excited to provide a synthesis, and pull that together as we think about how we provide responses, both to the directive that the Hill has provided us with, but also with the broad needs that our communities have identified, and their hopes for this new line of investment.

Nate, do you want to turn it back to the working group?

DR. JONES: Yeah, because I think the first of these is a nice example of us trying to be directly responsive to recommendations in the report, and then I think the second of these, I think, reflects kind of how we are thinking more broadly about responding.

So in spring 2023 NCSER hosted a technical working group focused on the special education teacher workforce. The educator workforce was something that was clearly articulated as a highlighted topic within the NASEM report, and we wanted to be responsive to that.

So as we'll talk about a little later, one of the outputs of that type of a working group was an RFA for a new center focused on the special educator workforce that we announced this fall.

I think the second of these two that at least we're doing within NCSER is both because I'm stepping in as a new Commissioner, but also because I think we want to make sure that we are being responsive to the needs of the various communities that we serve.

We will be hosting in some form or another a technical working group in spring 2024 that is going to be a part of an internal strategic planning process on our part.

So the goal here is to say let's bring together stakeholders through a range of communities, and have them nominate pressing issues and challenges facing the field of special education, and then we will work internally to try to figure out how we can best respond to those nominations.

DR. ALBRO: Thank you, Nate. And then the last point that I wanted to highlight here is that we do have current opportunities, or current investments, that are happening right now that engage state and local partners in identifying the co-PIs for post-research, and where state and local partners are, in fact, partners in the work that is supported.

The first is a program that uses, leverages, the state longitudinal data system called using longitudinal data to support state education policymaking.

If you all are not familiar with that, I do encourage you to look at the range of projects that we've supported, again, in partnership with our state folk who are in charge SLDS, and this is inclusive — just so everyone knows, it's inclusive not only at the K–12 space, but also thinking about ways to connect SLDS all the way up to post-secondary education, to labor market outcomes, as well as the early childhood sector. So it is actually broader than the primary focus of the SLDS.

So it's pretty exciting. We've got some really, really interesting projects that have come through that.

As was said, during '22 we received funding through AARC, and we established two networks focused on pandemic recovery efforts in education agencies, as well as a highlight for folks that both of those networks require partnerships for folks who are the research teams who are participating in both of those networks. So the work that we're supporting through that is in response to what this (audio interference) on the ground using our funds in response to pandemic recovery efforts, and we're beginning to see some funding from that, again, both in the pre-K–12 sector, as well as in the community college sector.

So just some thoughts about where we are, some ideas, or discussions of what we've done. And I'm going to now turn this over to Matt, who is going to provide a different lens on some of the work that we've been doing. I believe I can push the right button. There we go.

DR. SOLDNER: There we go. Okay. Thanks, Liz, and good morning everyone.

So the work you just heard Liz and Nate talk about is part of the larger enterprise, a really shared enterprise across the work of all the four centers here at IES.

So we'll take you through this diagram, and help you have a sense of how we think about this work, and the various parts of the Center's kind of interactions which give you the broad goals and mission that I know we all share.

So you just heard on the far left in this notion around showcasing community priorities and needs. You heard Liz and Nate talk about RFIs, and TWGs, the technical working groups.

Here at NCEE we use many of the same tools. We have RFIs, we host TWGs. Our latest is a TWG we call Sweet, which I will talk a bit later about this morning, but a small working group of education practitioners brought in to advise us on how to work around topics that focus for NCEE, usability of our products, and strategic dissemination.

NCES also contributes to this notion of servicing priorities and needs. As you can see, all of their work whether it's NAEP, whether it's the work on the conditions of education, all designed to service and information about what's going on on the ground that IES might respond to.

And then finally, the fourth element to that kind of surfacing community needs, we all do direct need sensing, right, beyond using surveys, beyond assessments, beyond RFIs, beyond panels. We go on the ground talking with folks about what it is they most wish IES was doing.

So our regional education labs are done on a routine basis. There's education agencies. Shout out to NAPE helping us work with CEC to do some direct needs sensing around needs of folks who work with students with special education needs, all designed, as you can see, as we move from the far left to the second dock, all designed to inform our evidence-building work.

And inside there you can see how those needs inform not just the innovation activities at NCER and NCSER are undertaking, not just our shared applied RDD agendas at NCER, NCSER, and the co-PIs take on. Of course, our program evaluation work.

But it also involves, and it might not be immediately apparent in kind of surfacing needs as the role of evidence synthesis activities, accumulating what's already known, and making sense of it, and making sense of it, and importantly, servicing needs and gaps from a kind of different constituency, if you will, the extant literature.

And so most of the evidence synthesis activity happened inside, and NCEE is part of our overall house.

You probably are familiar with our practice guides. One of our most important publications, which is focused on educators, but an important new initiative inside NCEE is designed to compliment what we're hearing from the field, is this notion of servicing in a more rapid manner what we are seeing in the following literature, and in the gray literature, to inform where there may be gaps in high quality evidence that could be brought to bear to improve policy and access.

And this activity for what we're thinking about as evidence will be designed not to just surface gap the literature that — or not distribute literature to the practice guide, or surface gaps in literature, which might compliment what we hear from the community to help NCER, and NCSER program officers, to think about where they might shape their portfolio. So we're moving in the direction to making sure we're always responding to gaps whether they're field-initiated, or literature-initiated.

And then finally, all of that is kind of an input into the final box on the slide are knowledge use activities. You can see how the evidence-building activities, and our evidence based activities, are all designed to inform knowledge use.

Again, the whole goal of this is to see the high quality research that IES and others do, inform policy and practice. We do that through technical assistance of our RELS. Our grantees certainly do that as part of their dissemination activities. IES shares things wildly, of course.

But importantly, a bit of nuance, notice that there then is the cycle. Once we have knowledge use we inform that back to into our community needs and practice as we share information entering the world, and really have a question as to whether it's changing conditions on the ground.

And so it's a continuous cycle. Needs sensing, evidence building, synthesis, sharing knowledge, and then repeating, right, always moving hopefully in an upward trajectory to improve practice.

Finally, I should just say — well, not say. I was excited to talk about expanding competition. I know. I know. You want to get to it. Just in knowing that this cycle at the bottom, this knowledge use and needs sensing, really is bidirectional.

I think as a whole you would hear from the IES today, and over the two days we're together, about how it is we really want the community to be helping us improve our knowledge use activities at every turn, so it is not a one-way push. It's truly a communication between the community needs, and the activities we are kind of putting out there to the world based upon our high quality research.

So hopefully keep this virtuous cycle in mind as you think about our work together over the next couple of days, and I'm glad as we move through it to take questions on your thoughts on any part of it.

And now onto Liz to talk about expanding competition.

DR. ALBRO: All right. I was just trying to make sure that my arrow was in the right place.

DR. SOLDNER: There we go.

DR. ALBRO: All right. So Nate, I think this is yours, although I'm not sure here. I want to see if this image shows up if I hit this button. Ah-ha. Go for it, Nate.

DR. JONES: It does, and I'll get to the analogy in a moment. But I'm really excited about that last figure that Matt shared in part because I think it really aligns nicely with the report's recommendation.

So Chapter 5 of the report focuses on topics, right, And I think one of the charges of a committee was to identify critical problems where research was needed.

And I think one of the things the report does nicely is rather than simply say this is a laundry list of things that are important to that set of 15 people who are in the room writing that report, it instead says there is a real need to develop this kind of process of learning information from communities, generate evidence, synthesize that evidence, and kind of build this cycle.

So in doing that stuff the hope is that new topics will emerge over time, and that we will be able to pivot as an organization accordingly.

The analogy that we wanted to put out here because I do think it's a useful one is to think about the core of what we have historically funded, and at least on the NCSER side. If I look back in the last 20 years, NCSER has tremendously expanded our knowledge around promising effective interventions for students are both academic and behavioral, and we've really made tremendous progress.

And I think what is reflected in the report, and, I think, what is reflected in our perspectives as an organization, is really let's take that core of what we've been developing, and strengthen it by adding new strands, adding new strands to kind of strengthen the contribution we're making to the field, and the knowledge we're able to produce on behalf of the communities we serve.

So that analogy aside, which, I don't know, we may like, or may not like, but, I think, what we did want to do is much like the report nominates a handful of topics that would be useful for the field to consider moving forward on, we wanted to just take time to walk through a couple of topics that were highlighted in the report, and describe how we've been responding to them.

DR. ALBRO: Thanks, Nate. All right. And this is explained with many words, and I apologize for that; there are fewer words than there were to begin, but Nate, as he just said, we're going to — it should be sort of two examples in the research centers, and then Matt is going to have an opportunity to share some of the work he's been thinking about looking at some of these topics and scenes that have been going through.

And given that, this report came out in the midst of the global pandemic, and technology was pervasive, and all of the work that was happening with our students and learners, we wanted to showcase how we have been thinking about this request, this call to invest more in education technology and learning analytics at the two research centers.

And the reason I showed that very first slide with the six boxes is really to sort of showcase how we thought about that not only thinking about our verified A competitions for technology is a crosscutting theme, and technology is just in all of the topics that exist there, but how we've tried to embed education technology and learning analytics as deeply in our investments across all of the work that we're doing.

So I think I'm going to just quickly highlight a couple of things here. The first is that we know that there is a deep need in the field to build capacity, and broaden participation in the youth of data science and learning analytic techniques in the context of education.

So recognizing that, and in our limited funding year last year, we actually ran a competition, and are now thrilled to say that we have three training institutes across the nation focused on data science and learning analytics, and you also, I hope, are signed up to our news flash, and as we spread out, please share this as a community. There are opportunities for individuals to come and participate in this training without cost to themselves.

The second is that we've launched a variety of network type activities. AI Institute's broad investment looking at how can we think and learn together about the best ways to use ed technology, and lean into the learning analytics that can help people use the data that they collect every day in the context of a learning management system, in the context of how they're teaching their kids and learners in terms of that, all of the electronic stuff that exists right now, right?

The SEERNet is our digital learning platform network where we've been excited to invite our first research team to join that network, and we're excited to have the opportunity to join more folk to help us learn what can we do making use of an electronic data that already exists in school systems whether they're in the K–12 sector, or in the Pre-K–12 sector.

We've launched, I'm sure you guys have heard about this, two new AI Institutes working in partnership with the National Science Foundation, which we are thrilled to be able to be a part of.

The Learn Network, right, there's a call for adaptation and scaling in the context of the work that we're supporting. The Learn Network is intentionally designed to support teams of high quality evidence, products that have high quality evidence, to think about what do we need to do to be adaptive to the needs of communities who want to implement this work, and to prepare it so that these interventions can be used more broadly.

We have been under our 305-T are transformative application. If you didn't notice that that whole partnership brand, although it includes a different set of partners perhaps than one that we're already thinking about, trying to make sure that industry partners are part of the conversation because the reality is is that they are the ones who are pushing ed tech; they are the ones pushing learning analytics, and if we're not part of that conversation, we are worried that we will not as a research community be able to shape the products that come out of that.

We, as I hope you all notice, one of the topics included in our new R&D center competition that will be coming out later this week is focusing on the use of generative AI in classroom instruction, again, something that we know is happening, and yet the research community knows remarkably little about. And so we are really going to hopefully provide national leadership in that focus.

The ceilings to scale competition I've already mentioned quickly, and throughout all of this the Small Business Innovation Research Program, which we have been supporting, and have been charged supporting since 2002, sits at this interesting locus where we are really working with the tech developers who are small businesses, and trying to ensure that the products that they are developing and sharing using technology builds high quality evidence to the degree that we're able to, and support innovation so that we can take all the good work that the research community is doing, and make that it is invested, immersed, infused, whatever the right word is, in terms of the products that come out.

So that's just one example of the way that we have been trying to think about these broad themes across the body of work that we are (technical interference.)

And Nate, I think this is for you.

DR. JONES: It is for me, and it will always be for me. I am deeply committed to focusing on teacher education, and educator workforce.

This is something that clearly was identified in the report as a priority. I think what it is is an acknowledgment that if I go back to that rope analogy, especially I use the case of NCSER, that the interventions that we have been successful in developing do not get carried out in a vacuum, right? Those are implemented by educators who themselves are sense makers, who are kind of making sense of these new interventions vis-á-vis their existing set of priorities, and those things are existing within systems. And so the report really pushes us to make a more concerted investment in teacher education, and the education workforce.

Before we talk about some of our responses, I think this is an interesting case that let's you all into, I think, some of the challenges, or the tensions within a space like this, or in the complexities, because to an extent we can prioritize investigators to apply for certain projects.

But what we have seen historically at times is that both because of the conditions that are in place within IES, but also because of the conditions of fields, a field's ability to be responsive to new and emerging areas is not always in place.

Why is that? Part of that is measurement challenges, and part of that is simply the priorities within a field.

So I think when we think about teacher education, for example, I know that there is a group, Heather Hill (phonetic), Suzanne Alobe (phonetic), Pam Grossman (phonetic), David Chard (phonetic), who for the last several years have been trying to increase the amount of causal research being conducted in teacher education.

Liz and I both have met with that group at various times. And I think the fact of the matter is we see very see very few of those folks kind of doing this kind of work in the teacher ed space.

So we can do certain things to incentivize work, but I also think it is a deeper set of challenges, and one that I think we are all thinking about how do we respond.

Okay. That is kind of the high level diatribe, but, I think, in a more specific sense.

So I want to talk a little bit about our center. This one we announced this September. This is an RFA for research and development center on the Special Education Workforce. I think it is probably apparent to folks in this room that we have had longstanding shortages in special education that only appear to be getting worse over time.

So we are establishing the center with the hope of providing national leadership on ongoing staffing challenges.

The way that this RFA was written it requires that investigators are working in partnership with state Departments of Education. The goal here is so that researchers can be responsive to local state needs, and provide recommendations that hopefully will influence both policy and practice.

So that is us on the NCSER, and then, Liz, I will hand it over to you to talk about some of your efforts at NCER.

DR. ALBRO: Yes. And my two are very quick here. The first is that while we have provided investments, and invited applications focused on teachers for our entire history, it became clear to us in reading the report that sometimes there are challenges in folks picking that up, right?

We had changed the name of our teacher-focused topic in the RFA, and we've renamed it again, sort of really making it clear that, yes, we want to know about teachers teaching in the education workforce, and those are the applications that we're hoping to receive.

As you all know, A applications have already come in for '24. I don't have any data yet to share with you on that, but we're hopeful that that shift, right? You guys know that a new name can sometimes really change things, and we're really hopeful that that has changed the field.

And in parallel to NCSER's RFA, one of the other topics for R&D centers is we got lots of feedback from the community that there was a real need to understand teacher shortages in the K–12 sector, and really understand what kinds of policies and practices are states and districts putting in place to really try to address recruitment and retention in the K–12 genre education workforce.

Stay tuned for more details on that. Again, that will be coming out soon. And now, Dr. Soldner, this is you.

DR. SOLDNER: Thanks, Liz. Lastly, but not least in this section we want to talk a bit about our investments in research on the use of research evidence, or URE as we call it. Also sometimes called knowledge mobilization by folks.

You'll see on this slide that IES' founders must have been unbridled optimists when it came to the idea of the evidence us, right? That if we wrote the responses to this wonderful research, and we published academic journals, or posted this to our website, things would magically get from their hands into those of policymakers and practitioners to let all of us know, you know, Ruth, especially you know, not so.

Kevin Costner was not entirely correct. I mean, if you do build it, they do not always come, right?

And so there are many reasons why that is, and it is no party's fault at all, right? But it is something to manage if you hope to see your fellowship views and condition changed on the ground, and we very much do.

And so I want to — briefly about our investments. To be clear, IES has been investing in improving use of research evidence for more than a decade, so in one way with our emphasis on the use of RPPs is an investment in seeing research evidence used, and that dates back to the 2010s, right, both through work in the RELS, and through NCER.

NCER has also made grants to RDD centers specifically focused on this activity both in 2014 and 2015 at our National Center for Research and Policy and Practice, and our National Center for Research Used in Education.

So today we continue that thread, just in some different manners. So we're further extending this work in a new investment by being a bit more thoughtful about how we can use the REL program, something in our own backyard, not only as a place where we're doing the work of use of research evidence, but as a place where we can study it, right?

We have ten literal labs in which we can do this work. I know RELS are the nation's single largest investment in education RTTs, more than 70 of them, and so we are currently setting aside funds, investing $2 million in the first year to think about how we are, and maybe how we're not yet, improving the use for research evidence.

And this afternoon, or whenever we come to it, when I talk about the work that's at NCEE, I can talk a bit more about what that work looks like besides they were investing in this phase yet again after some real foundational work done by our colleagues at NCER.

DR. ALBRO: All right. I know this is a lot. Thanks, guys, for giving us the opportunity to share. There is a lot of stuff that we've been doing, and thinking about, and we're happy to have a chance to share it with you all.

Broadening participation, and transparency about broadening participation, we're definitely seeing a theme that came through the report. And so I wanted to share with you where we are, the information that we have so far, and where we hope to go.

So let's just share with you what we have done today in terms of trying to understand where we sit.

So we started with trying to understand what institute — which institutions apply for IES funding, right? I know this is a question, and a concern of the report, and so we will share with you some information about that in a second.

Describing the specific key personnel named on the project, we will share with you the information that we have, and I'm happy to talk about this.

We've listened to communities, as I think many of you are aware; that we have a whole variety of listening sessions. This was, I'm trying to think, about a year and a half ago, where (audio interference) about who completed (audio interference).

Someone is unmuted. Can someone please mute?

MALE SPEAKER: 245746.

DR. ALBRO: 245, yeah. The — okay. If we mute us, then no one can hear me, which perhaps you guys are ready for. I don't know.

All right. It sounds like that has been accomplished. Thank you.

We shared who completed our IES-funded pre-imposed sectoral training programs on our project abstract, if folks are interested. That information is now available.

And we have been systematically gathering information about best practices for broadening participation in federal grant programs, looking at what they have done at the National Institute of Health, and the National Science Foundation have done, as well as others, and including what folks have been doing around peer review.

So the next couple of slides are going to be some graphs, and some charts rather, and I will just share these with you. And I think what you will see is that the story is pretty much the same across each of these, irregardless of which characteristic we're talking about.

So let's first start with the named gender of PIs in applications and awards. These are for applications that were received in fiscal '21 and fiscal '22, and is inclusive of some of the funding now that's in place that we did in fiscal '23.

The data that you're seeing here is percentage of the total number of applications received, reviewed, and then the awards that we made. Just for context, we received 1,900 applications. Of those, 1,763 were forwarded for peer review, and we made 261 awards.

As you will see, this should not surprise anyone on this call, we have many females who are applying to receive funding for us, but I want to note, and again, this tan bar that is here, the tan bar is individuals who chose not to provide us that information, right?

As a federal office we ask, or we required, folks to complete the OMB form that includes information about gender and demographic features, but there is always an option to not wish to provide, and as you will see, both here, and throughout the other slides, anywhere from about 20 to 32 percent of folks who are filling in this form are not providing us with information.

So that given that data, I just want to put out there is a certain amount of caution that we try to try to think about in terms of drawing inferences from the data that we have received that has been shared with us.

DR. LEE: Liz, can I ask you a quick question?

DR. ALBRO: Yes, ma'am.

DR. LEE: Have you all considered — because I know we've been kind of wrestling with this at the National Academy, in terms of more than this dichotomous gender identification?

DR. ALBRO: Yeah, Carol. That's a really good question. Unfortunately, the OMB form that we are using, right, that that is a form that has been approved that we must use in order to collect data, currently only has these three options.

I do know at the federal level there is a conversation about expanding that description, and once we receive an OMB-approved form to use, that will replace this form, and they will have whatever the agreed upon classifications are.

So thanks, Carol. That's definitely an important question for just to take into consideration.

All right. Similarly for the race of PIs, and co-CIs, again, one of the things I think that — one of the things that's interesting to me is noticing how the patterns that we are seeing across proceed, to reviewed, and awards made, it's actually not that different across these different categories.

Because of the concern about protecting privacy, we had to collapse individuals who identified as Native American, Hawaiian American, and I realize I didn't write this down, but you will see that the vast majority of applications that we receive have PIs and co-principle investigators who identify as white.

We still have this large number of individuals who are choosing not to report their race to us. I do think that there seems to be some indication, again, I don't like to draw conclusions from this data, particularly with so much of it missing, but that the co-principle investigators are actually somewhat more diverse than the PIs that we're currently seeing, or that we saw in our applications that came in in '21 and '22. But again, that's perhaps the statement I shouldn't have made out loud, but that's what the data looked like.

In terms of ethnicity, I think we can — there's no argument with this. We receive very few applications from principle investigators who identify as Hispanic. Again, I do not wish to provide, and one could think about reasons why we don't have that information in terms of ethnicity.

But again, we have no reasons, we have no explanation, for why. These are simply people checking boxes on a form that they submit to us at the time of application.

Institution type. So for those who ever applied for federal funding you probably know well the Standard Form 424, and part of the Standard Form 424 there's an opportunity to that your sponsored project office has to select what is their institution type.

I went back and looked at the list. I think there's something like 16 different institution types, probably more, but that was what I was able to count. And so all I have here are the four greatest types of institutions who have applied and received for funding.

As you all can see here, public institutes of higher education are our predominant constituent, our predominant customer in terms of folks who are coming in and applying for money, followed by our nonprofit customers, followed by private institutes of higher education, and then we have state government, and MSI, and those categories are small.

Again, trying to draw conclusions from this data is complicated. I just ask that you're only allowed to select one of these categories, and we know that all of these MSIs, or many MSIs, could also be classified as either a public institution of higher education, or a private AAG. So in some ways it's — we just don't know to what degree there is additional overlap amongst those categories.

So the lessons that we've taken from this is that broadening our applicant pool has got to be something that we focus on, and as you all know, this is something that we've been concerned about even before we had availability of this data, which is in some ways was why we started to try to collect this data.

And so we have been trying to think about how we'd let folks with — who come with different backgrounds know that you can actually be an education researcher when you grow up if you wanted to. And so just under the leadership of Dr. Katina Stapleton (phonetic), one of the things we did was establish the pathways to the Education Appliances Program, which requires a partnership between minority-serving institutions, and a Research 1 institution, the idea being try to — to try to pull in individuals with bachelor's and master's degrees into sort of understanding and learning more about research.

We've so far run two cohorts in 2016 and — 2016 and 2017 is like one cohort, and 2021. As you might imagine, the 2021 cohort was deeply impacted by the pandemic, but the data that we have so far indicates that we have 264 pathways fellows who have completed their training program at these six institutions that were awarded in '16 and '17.

Early career is a newer opportunity for NCER, and we were building on the success of the NCSER program. As you all know, we established a program focused on early career faculty support for faculty who were at — who were on faculty at minority-serving institutions. That we have been working hard to build, again, the number of applications that we received, and we have made our first award, and we're excited about that.

So we also heard in our listening sessions that there are many individuals who bring different demographics characteristics who are working at predominantly white institutions, or other institutions, and so we thought it was really, really important to broaden the opportunities for early career scholars to put in applications to IES, and to have the — and to NCER, and to have the opportunity to work with my dedicated team to really help them build and improve their applications.

And, finally for institutions, again, this is really redundant, right? We've been trying to continue to find ways to invest in supporting work at minority-serving institutions. Part of this is through embedding these kinds of partnership requirements.

I will say that using longitudinal data programs we have been trying to right size the partnership on that, so initially the requirement was that the state come in as the lead, but it turns out that state SLDS systems are not always in the best position to quicken and support a research project in the context of a federal program, so now we've moved this to a partnership diagnostic about who comes in as the lead institution.

All right. Now, I'm going to turn this over to Anne because the other question that we got lots of input on was understanding and broadening who reviewed.

MS. RICCIUTI: Great. Thank you, Liz. Good morning, everybody. I'm Anne Ricciuti, Deputy Director for Science, and I have the responsibility for scientific peer review research grants in my office.

So I'm going to take just a couple of minutes because I know this is a lot that we're going through, but I wanted to address a couple of the issues around broadening who reviews, and then I am also on the agenda later today, and, actually, will try to provide a little bit more background around the review to conceptualize some of this information.

So just very briefly, there are two parts that we have been thinking about around broadening who reviews our research grants. One is around understanding who our reviewers are, and reporting out that information publicly while also preserving privacy, individual privacy, and the confidentiality of the peer review process.

Currently you can see on our website, if you are able to dig down deep enough on the website to find it, you can find lists by fiscal year of our reviewers by panel.

We do combine some panels, and sometimes we combine across years, again, to try to preserve some of the confidentiality of the review process, but we have, and we have had for a number of years, online the names of the reviewers, and their institutional affiliation, and we also indicate who has served as chairs of panels.

We will soon be posting the list for the most recently completed fiscal year, fiscal '23, so soon that will be up on the website.

We are working on figuring out best ways to collect demographic information about our peer reviewers in a way that preserves privacy, and that we can report out publicly in aggregate.

We have been looking at how and what NIH and NSF collect, and working on exactly what we should be collecting, how, and personing appropriate OMB approval.

So we are looking at wanting to collect information not just about race and ethnicity, but also about some of the kinds of factors that Liz was just reporting about around applicants, including gender, disability status, and others.

We are also working on expanding diversity, and being able to report about diversity of our reviewers, and a number of other aspects as well, including seniority, so you know, early career versus mid versus later career, disability status, et cetera.

And as Liz mentioned, all of this, of course, would be collected voluntarily from reviewers. We can work on ways to try to make sure that folks are answering in some way, but we can't require that anybody actually provides us the information.

We are also working on the systems and tools for collecting and maintaining that information, and I'll get to that a little bit later. At the same time where also we're working on expanding the diversity of our reviewer pool in terms of all of these areas.

I have a dedicated member of my team currently, my very small team, which you'll hear about later, and this past year that individual traveled to a number of different conferences to try to get information out, and recruit reviewers from a number of different areas that are new to us.

We also have participated in the listening sessions that NCER and NCSER did with a number of groups and communities, and were part of the contract that NCER and NCSER led looking at what other federal agencies are doing around broadening participation.

For a number of years we've been working to get reviewers with experience in different kinds of settings, so from state and districts. Also other nonacademic settings both industry and other kinds of settings.

It is challenging for a number of reasons; however, we have been continuing to work along those lines anecdotally because I don't have the status report, but anecdotally I think we've been able to improve in a number of those areas.

One example is the recent panel we held on the transformational research competition that Liz mentioned earlier. We did have on that panel reviewers representing industry, education, and educators and administrators, education administrators, as well as researchers. And I think it was a really, really good mix of folks and perspectives.

Our goal is to be able to provide lots of information publicly about our review process, these are the procedures, our people, our results, quickly and efficiently.

We've been working on — my sort of dream would be to be able to have the kind of information that NIH, Center for Scientific Review, has been able to put out. I believe very strongly in transparency while also preserving privacy.

And I will stop there. I am on the docket for later, so you'll hear more from me later, and I'm happy to talk more, and answer questions. And now I'm going to turn it back to Matt, I believe.

DR. SOLDNER: All right. Thank you, Anne. So all the issues we've discussed this morning whether it's engage in the communities with whom we work and serve, whether it is considering how our competition can help us innovate, and build evidence and serve supporting all learners on a wide range of topics, or whether it's participation, so that we are bringing more voices to the table in all aspects of our processes. All of those are examples of how we are responding to issues of equity that are raised by the Academy support.

There is a couple others that we'd like to highlight for you this morning before we wrap up. I promise the end of our conversation is in sight thanks to the next slide.

Great. So first, in June of 2022 with the help of the ISY DNI committee, we established the SEER equity standard. You'll see it on your slide.

If you aren't familiar with SEER, it began in 2018 as a series of initially recommendations, now principles and standards, that we believe that researchers could adopt beyond having studies which have strong internal validity that if they were adopted could result in their research being more transformational.

Equity was not one of the original standards, but in June of 2022 we added it. You can see the standard itself on the slides. It's supported by four additional recommendations.

There's things that we think are nice to have, or things that should be on researchers' minds, including recommendations that researchers be more transparent about how their conceptualizing of equity in their work, that they were to better understand the heterogeneity of impacts in their studies, so we can achieve the mission was often talk about, which is knowing what works for who, and under what conditions.

We advise researchers to design interventions that are responsive to the context in communities in which they're meant to be implemented.

And then finally, engaging those communities in all aspects to the RDD process.

So not now, but later, if we go to the SEER website, and check those out, since publishing that standard in 2022 we have convened one technical working group to help us initially identify tools and resources that are already in the community that might help researchers make forward progress on that goal, and we will continue to work to find ways we can support researchers achieving a trajectory standard in the years ahead.

And, I believe, I'm finally turning it over — back to —

DR. ALBRO: Nate.

DR. SOLDNER: — back to Nate.

DR. ALBRO: And then onto questions.

DR. JONES: And then, yes, we will turn it over to questions momentarily. I just wanted to highlight a couple of things related to NCSER on equity.

I think the one that's not here is we're giving a lot of thought to the ways in which individuals with disabilities are often multiply marginalized. So I've been excited that over the last couple of years we have funded a handful of studies focused on English learners with disabilities. I'd like to see more of that work moving forward.

But, I think, also one of the things that I think that we are seeing within the disability community, and especially the research community, is this kind of continued push around nothing about us without us, right, which is a slogan used by the disability community to ensure that folks with disabilities are included in policies, or practices, or programs that impact them.

So in response to that, in our most recent RFA we've tried out asking investigators to document how they're collecting the perspectives of students with disabilities within the research process.

We had specific grantees who have worked really hard to include researchers with disabilities in our projects, and we are trying to figure out how we can elevate those examples, and make them more commonplace in our field.

And I think kind of going back to Matt's original point where we're still exploring options to have routine ongoing feedback from the disability community about how our research maps onto their needs and perspectives.

So I will close there, and Liz, I don't know if you had any final thoughts, or if we were — if you just had —

DR. ALBRO: Well, first of all, thank you. I know that the Board wanted to have a chance to ask questions, and respond, and react. As you guys can see, our desire to try to sort of slim down how we responded given the breadth of responses in the report, I don't know how well that worked, but it was our attempt to really give you all a sense of how we've been engaging broadly with the report across all of the work that we're supporting across the three centers, as well as through our peer review.

And with that, I'm going to open this up. Carol, do you want to be traffic cop, and direct, ask questions? That would be great.

DR. LEE: Sure. Just one quick question, and I gradually see hands coming up here.

Nathan, and I know that at this point the way the field is established nothing on here is going to change, but I do know that in the special education community, if you will, there is discussions and debates around even the whole construct of disability versus diversity.

And to the extent to which you — the extent to which the focus of the work that you're doing in the center being able to try to shift the categorization. I'm just curious.

DR. JONES: Yeah. I mean, I think it's an important question. I think those — some of those thoughts, I think, reflect two things, right. One is the perspective that some within specialized research have historically approached things from a deficit-oriented perspective.

I'm not sure that that is always true, but I do think that we are working hard to think about how we can prioritize externally-facing research that is acknowledging the assets that individuals with disabilities bring.

So that's thing one. I mean, I think thing two is because of concerns around over-representation historically there have been kind of concerns about special education as an enterprise, and the extent to which we name something as a disability.

I will also say that within the disability community there are many who acknowledge that we have disabilities. We don't want to shy away from that, and we just want schools to be anti-ablest, and to ensure that schools are supportive of those individuals.

I don't know if there are other perspectives of members of the Board on this issue. I saw Doug's hand pop up, but —

DR. LEE: So we're going to —

DR. FUCHS: Carol, just — I know you want to move on. I just want to just briefly add to what Nate said.

I think the disability community is very aware of a potential tension between diversity and disability. And I think that the disability community in part strongly supported by IES has pursued policies and practices like response to intervention that try hard to distinguish in practice between kids with disability versus kids who may seem like they have a disability, but really are not disabled, and are performing poorly because they've not had access to appropriate education. So there is this sensitivity to the issue that you're raising.

DR. LEE: Thanks. Hiro?

DR. YOSHIKAWA: Yeah. First of all, thanks for this really comprehensive overview of all the activities in these four big bucket areas. So yeah, very helpful, and exciting to see all the activities that are related to the Board recommendations.

Just a couple areas I wanted to ask a little bit more about, which have to do with the definition of outcomes in education research, and then also the report's recommendations around methods and research designs.

Recommendations in the report around use of outcomes beyond the student level to get to indicators that are more structural and contextual, that are about measures related to educational equity, and how those — there's a research agenda around developing and validating those kinds of measures. So under that bucket, which is kind of more — yeah, whereas the traditional outcomes have been really at the kind of student, and maybe teacher levels, or that kind of thing.

Then around the choice of methods and research designs, I think the report made some recommendations for some methods that were certainly mentioned in what you presented like AI, but there is also one on supporting qualitative and mixed methods approaches to research design, and methods. There were a variety of other kind of design things in Recommendation 6.12.

So those two areas I'd love to hear what the activity has been related to those.

DR. ALBRO: (Audio interference).

DR. JONES: You unmuted yourself, Liz, so just mute yourself.

DR. ALBRO: Thank you, Nate.

I'm glad you saw that clearly. I was doing really well until you asked me a question, Hiro.

Two things. Yeah. Thanks. We were trying to slim down the slides, so definitely these are all the things that we are continuing to think about.

In terms of the definition of outcomes this is actually a really, really interesting and important question because for us one of the things that IES, and the way that we read the legislation, and as derived, that we are obligated to fund work that directly links to the student outcomes, right; that we need to have measures of student outcomes in addition to all of the other measures that we care about.

And so from my perspective nothing precludes anyone coming in and proposing to have additional structural — you know, measures of structural outcomes, or measures in educational equity, but I do know that it is challenging to include those types of measures, and also include measures connected up to student outcomes.

We would be happy to continue conversation around this, and help us think creatively about how we need our legislative mandate, but also broaden the kinds of measuring and outcomes that folks are including.

I will say that if you look at many of the research projects that we fund very few of them only include one type of outcome, right, and very few only include only education and (audio interference)

So we have all kinds of measures of attainment, of progression through the education system. We have measures observational measures, say, for things like school climate, of conditions of schools in the school systems that exist.

But, I think, we as a field need to work together to think about ways to improve and increase the measurement capacity and skills that we have to measures systems-level change, for example.

And Nate, I don't know if you want to jump in on that one before we move to the next one.

DR. JONES: No. The only way I'll jump in is I think clearly there is a need to invest on measurement kind of across the research enterprise.

I think IES has invested a lot in measurement of student outcomes. I think we have seen less proposals focused on educator outcomes, or systems-level outcomes, but, I think, our ability to make progress in those areas that kind of get further away from a focus on students is always going to be dependent on our ability to have high quality measurement systems surrounding that work.

DR. ALBRO: Yeah. And I do — and just to jump in. We have certainly been talking about the measurement project type, and about measures, and we're super proud of that work, and we're really glad that the committee is proud of that work as well.

But, I think, as I — I mean, my default is not always to go back to training, but I do wonder, and would love your all's thoughts, on how do we continue to build the community of scholars who have high quality psychometric expertise, and reliability, and validity who can help us tackle these really critical problems because we feel like — I mean, in terms of what we see there are fewer individuals who are doing that work than we think are needed.

So the second is around methods and research design. The first thing that I want to say is that we invite mixed methods work, and that has been part of our RFA for a very long time, and the reality is we actually see a bunch of mixed message work, right, as this is coming in.

And I want to say any time anyone puts an application in, we encourage them to do both kinds of work, right, to provide us with quantitative data that (audio interference). But we also are extremely interested in (technical issues.)

DR. LEE: Liz, we can't hear you now. We can't hear you, Liz.

DR. JONES: I think the phone is now muted, Liz, the 202 number.
(Pause.)

DR. ALBRO: The phone decided that we should know why you guys can't hear my voice anymore. They said nope, no more. Down with Liz.

Anyway, so when I look at the work that we support, we actually have lots of work that is in both qual and quan. It's really trying to understand the context within which this work is happening.

Anytime we support development in innovation work, which is actually in terms of numbers, the largest pop of applications that we support there's always qualitative work that's helping us understand how communities are responding to that work. So we appreciate its knowledge that is important for that kind of mixed methods work in the work that we support.

So with that, did I answer all of your questions, or is there more in there? Nate, did you want to jump in?

DR. JONES: Just when your phone cut out you said we are also extremely interested in?

DR. ALBRO: We're extremely interested in, good question, and you think I'm going to remember what I was going to say?

I mean, I think one of the things that I'm extremely interested in is trying to figure out how we, again, as a community can report the information that's collected on sort of both sides of this, if you will, qual and quan work together, so that the field is aware of both of these projects, right?

I sat on a mixed message committee that Spencer pulled together probably about a decade ago, and one of the real challenges that we talked about is how do we share this information together, right? The way our current publishing system is set up we actually don't have opportunities where people can sort of both share, if you will, an impact study outcome, the quantitative outcome and all of the contextual information that becomes incredibly important for interpreting and understanding those findings.

So that's, I think — may I say, again, I can talk for hours on this, but I know I'm not the only person on the agenda.

Nate, did you want to say anything?

DR. JONES: No. They have lots of questions. I'll hold off.

DR. LEE: Denisa?

DR. GANDARA: Thank you. I'll just start by echoing Hiro's appreciation for the presentation. I think you all did a really impressive job of distilling your responses to a 285-page report to accurate and concise, but very thorough, presentation. So thank you for that.

I do have a couple of follow-up questions related to transparency. Would you please comment on any efforts to make the data on applicants publicly available, and I'm especially interested in seeing these data in a way that allows us to capture this over time.

And my other question is about whether you have any data on study samples to share? That is also one of the recommendations of the NASEM report that we just covered, and I just think it's really important to understand who is being questioned, and I'm wondering were there any efforts around collecting data on that.

DR. ALBRO: Yeah. Thanks, Denisa. Super great questions. So yes, we are working hard to try to figure out ways to be transparent in terms of sharing this information. We will be sharing out tables, so just tables of that information, so as long as it conforms with sort of aggregate — the requirements of aggregate data.

And we've had to pool stuff together in part because the fiscal '22 funding years, and the fiscal '23 funding years were odd, right, in the sense that we didn't have most of our primary competition. So our numbers are really small.

Like Anne talked about, like NIH has their wonderful data that they shared on this DSR, we would love to be able to have something like the NIH data book, and we're actually doing some back-end work to try to make sure that we can have that.

I will say, however, that our numbers are small enough that the ability to track over time is going to be complicated by sort of the general number of applications, and information that we get. But to the degree that we're able to share that data, we will.

And we've just been delayed because we did this thing where we funded down this way where we funded grants from '23 — funded grants in '23 from the 22 applications. We had to come up with a shared agreement of how we were going to report that data.

So that is entirely the goal, and we will be sure to let the Board know when that information is available publicly so you all can see it.

In terms of some examples. So yeah, so we started work on — we did an initial survey, a small survey, I don't know. I don't want to mess up, and get in trouble here.

We reached out to our grantees, and asked them what kinds of information do you have available in the data that you're collecting about your study samples because what we are trying to figure out is what information is already available that folks could then report to us, and how much information would we really like adding additional burden to our grantees if we're asking them to do that because there are requirements in terms of OMB, and the Paperwork Reduction Act that we have to be responsive to.

So we have that data. Unfortunately, we've not had a chance to really dig into it, and come up with solutions, but it's absolutely something that we care a lot about, and we know, again, that reporting standards.

And I am focused on the publishing side of this, but reporting standards are all over the place in our journal, so it's very, very difficult to try to understand the demographic characteristics and features of individuals who participated in research funded not only by us, but by the National Science Foundation, and the National Institute of Health.

So we really appreciate that recommendation, and are trying to come up with a good solution that doesn't over-burden grantees, but also make sure that we have information that we think is really important for the community to know about.

Nate, anything to add?

DR. JONES: Nope. Now would be a good time.

DR. LEE: Doug?

DR. FUCHS: Yeah. Just briefly. First, thanks very much to Liz, Nate, and Matt. Wonderful presentation.

I'd like to — and I don't have a question. I'd like to just add a brief historical note to perhaps better understand the 2022/2023 appropriations for NCSER, 60.3 million, and then 64 million.

It should be understood, I think, in an historical context in roughly 2002 the Office of Special Education Programs was required to give up its research portfolio. It switched from the Office of Special Ed Programs to IES. And in that year 2002 the investment — the research portfolio which began NCSER's, was over $70 million, and that was in 2002 monies.

So we can look at 60 million going up to 64, and that's nice, but the historical fact is that way back when it was 70. That's all.

DR. ALBRO: Thanks, Doug.

MALE SPEAKER: Could I add to that? So we've been trying, and Doug knows this, and we've been trying really hard for years now trying to get to restoration of a more reasonable amount of money for NCSER.

So I think my understanding is, the history is, in 2011 about a third of NCSER's budget disappeared for payback for other work that the department was doing, and they had to find — a pay for, not a pay back, and they had to find money to do some other programs.

And I think it's arguable about how good the research community was for special ed in 2010 and '11. And I think when people look towards it because it's off the field wasn't, shall we say, as strong as it should have been in ten years later, 12 years later.

I don't think there's any question that the work on special education that I have supported since in the field, has just gotten, I know, an inordinate magnitude better, and the problem, of course though, is that we are in a hole from 2011 and 2012, and no matter —

I mean, I've tried in — and Nate, I mean, all — many other people have tried to get even a modest restoration of where we should be, which would probably be the vicinity of post-90 or $100 million.

And I think Carol said right at the beginning about the question of what the Board's role should be, so helping us make a case to the Congress, the appropriators for more money is one of the things that we really would like from the Board, and for me getting more money for the Special Education Research Program is incredibly important because I think what happened in, I think, it was 2011 where sort of the budget disappeared was not an appropriate action, and certainly the field has gotten so much better, and so much stronger that it deserves much more support than the money that it currently has.

DR. LEE: So Stephen, and other staff, and members of the Board, do you have any ideas about processes for the Board to engage in to support both this particular recommendation that (audio interference) increasing the SPED allocation even more broadly as we try to have some way of communicating with regard to ESRA reauthorization?

MS. LEGORRETA: This is Conchita. Sorry. I raised my hand. I'm not sure how to unraise it.

So Stephen and I's report actually go into that. We do have ideas, so whenever that would be a good time. I don't know if now is a good time, or if we should wait, but we do have some recommendations with the Policy Subcommittee.

DR. LEE: Okay. Great. We'll hold it until that time.

DR. KLASKO: And we're really sensitive to, you know, what we're willing to do versus what we're allowed to do from the Board's point of view. Because, I mean, I keep getting excited, and then getting regulations of what we can't do.

So I think that would be — once we do our report it would be very interesting to hear from that perspective.

DR. JONES: And I will just add — oh, sorry, Carol.

DR. LEE: Go ahead.

DR. JONES: Just one thing really quickly, and to the extent that we can provide data, or information to support your efforts, we'd be happy to do so.

You know, we have shared publicly that this year in particular there were more than a couple of projects that were deemed meritorious. We simply did not have funding for it.

I think it's evidence of where we're at as a field that our reviewers are saying we should fund this work, but we are in a position where our funding is just too limited for us to be able to be faithful on that issue.

DR. LEE: Hiro, your hand is up again.

Conchita, did you finish what you wanted to say? We're going to come back when we come to your report.
Hiro, is your hand up or down?

DR. YOSHIKAWA: It's down. Sorry.

DR. LEE: Okay. So James?

DR. MOORE: For what I would like to say is that I think it's very important that the Board consider some kind of action that it can take. IES, and EDU, and NSO, we're the largest entities in the federal government that funds STEM education research.

And it's clear that IES has its challenges even within my directory, we have our challenges primarily because the STEM education aspect of my portfolio is not congressionally mandated. So if I have to take cuts, I'm going to have to hit the core, and our core is education or research.

So we're the only game in town at the federal government. When you think about STEM research broadly you can go to almost every federal agency, I won't say every, but many other federal agencies, but in this educational research realm, particularly in STEM, you don't have a lot of entities you can tap into.

So typically when IES is hit, you know, we're hit as well. So in turn, the whole educational research enterprise is hit in a major way.

I don't know if Mark has anything to add to that or not, but I just thought it would be important to provide that context.

DR. SCHNEIDER: So there's a major opportunity with regard to STEM, and we — you know, James and I have been — we've had a road show going. We have another one on Tuesday, I think, is our next road show. But STEM is an area that is very, very contemporary; it's very much needed to increase the STEM workforce, the diversity in the STEM workforce.

And I've written about this many times. James has also. I mean, 75 percent of students with disabilities are below basic in science and math according to NAEP. Of the subgroups that NAEP measures, that's the largest percentage of students that are below-based, and we need to address that.

STEM is an area that mostly is in NSF's domain. We support a lot of work. We collaborate with NSF on this, but it's a window that I think the Board needs to think about about how to make the argument for more funding for IES because the nation is falling behind on its STEM workforce. In terms of students with disabilities we have to solve that problem.

DR. LEE: So just a general comment on time. I think if we can go one hour more — I mean if you need to just take a break, do it, and then take like a half-an-hour break for lunch, and then come back to try to complete.

And I can only see the hands of the faces, but I know there are more people, so that if you have something you want to say, and I don't see your hand go up, just speak.

But I wanted to make a couple of comments, and I think this discussion, again, has been very helpful, and again, to the staff I appreciate the breadth and synthesis of your sharing with us.

So on the question of like outcomes at the student level, and this is something — it's — there is this emerging — I won't call it a feel yet because it's not that, but sort of an emerging conception of something that people are calling the science of human learning and development that focuses on, and integrate a way of understanding human learning and development with some very interesting, I think, new work that's coming from work in the neurosciences.

And so, for example, in terms of thinking about even student level outcomes, I think that we tend to focus on what we think of as cognitive indicators in terms of achievement in some fashion or another, but less attention to the role of also — what I think about is psycho social indicators that are contributors to cognitive outcome and achievement issues having to do with identity, you know, self-perceptions, and the like. And so how that might be another kind of area of thinking about a sort of impact.

And equally around the methods question that in addition to thinking, as we sometimes want to think about dichotomies between quantitative, qualitative, and what we're thinking about is mixed methods.

Again, I think there's some really emerging work around data mix systems, and methods attached to understanding dynamic systems which would encompass both the level of the individual bias of the level of context and participation within all spaces, and the extent to which this is a set of issues that are emerging, the possibilities of IES sort of taking this up to support this.

And I was going to call on you, James, because I also had thought that these could be another interesting possible opportunity of relationships, some sort of joint kind of activity between the National Science Foundation and IES, and thinking about the National Science Foundation not only in terms of what we think about as sort of traditional STEM activities, but the science of learning, which seemed to me they are much situated in the work that NSF does, but the possibilities of really moving a whole new field forward.

And related to that is the question of — so the efforts that you all have done in terms of studying impacts, and some of the documentation that I've seen around that. And I think this is addressed in that report, the Future of Education, is what kinds of initiatives might be developed that the Board could potentially give some feedback on it, thinking about it, of how do you measure the impact of the work that's done.

So we have the What Works Clearinghouse, and Eric and the practitioner reports, but the whole question of who takes them up, and what happens with them. And I think Peggy will be next. I see she's back.

In terms of our sort of reporting, the field of education we still have these persistent — and if we could just take NAEP, NAEP — you know, I'm more on the reading side of things. NAEP you have a little jump here and there on the reading. It's still relatively flat.

And the vast majority of children even beyond the students with disabilities that you talked about, James, or I think Mark, at your measures of science achievement, on the whole we have such significant challenges that despite all the efforts and money we put in, we still haven't been able to fundamentally shift fields.

And so what it might mean to try to figure out, and what I'm thinking of is like a theory of change, of what are the variety of levers from the policy community at state levels, at federal levels, of all of the varieties of institutions we have, what is the theory of change that can sort of push the things that we research, and find afterward to be taken up in practice.

And then the last comment on the diversity, so it was great seeing the data that you all have collected, and the data shows — I mean, there are no surprises in the data.

So the question underneath that, I think, becomes what sort of practices, efforts, relationships, need to be built into — if you think about a theory of change, of expanding, the diversity of the pool of both applicants and persons through institutions have received awards, they're beyond the collection of the data. What would be the theory of change to be able to expand that?

So I don't have answers to any of these questions. I'm just saying I think it's part of thinking about IES priorities that these would be important issues to sort of take into consideration.

Any thoughts relative to that?

DR. ALBRO: Thanks, Carol.

(Simultaneous discussion.)

DR. ALBRO: I just wanted to say thank you, Carol. I am mindful of time, and so we have lots of things that we could think about.

I do think that the measurement challenge is interesting. I did want to call out for folks if you all had not been to edinstruments.org, this is the download. So initiated into this at Brown University, and it is really an attempt to crowd source, and pull together extant measures that exist in the field.

And so as we're thinking about like communities, and trying to understand what's available, Ed Instruments is a place to start, and there's lots of measurement on that — in that particular database, focus on like the social, the social side of all of this work.

So if you're not familiar with it, I just wanted to make sure that you had a chance to see what is there, and if you have better, different, new, other measures, to please submit it to that group so that we can continue to collect.

We're always looking for opportunities to continue our collaboration with the National Science Foundation. We appreciate the suggestion around thinking about dynamic systems, and different ways of approaching human learning and development. I mean, as a cognitive scientist I appreciate, of course, thinking about human stuff, human learning and development as a group together.

And theory of change is hard. We've been working on it. We've got some theory of changes, but you know, coming up with a single theory of change is really challenging, and so we appreciate thoughts and insights that you all have as we're trying to think about how we as a federal bureaucracy can make changes, and shift and change the lives and outcomes for individual learners, right, that we're all deeply committed to.

DR. LEE: So one of my takeaways from this discussion, which I have found to be very rich is, one, part of what I had hoped to come out of this meeting, was for us as a Board to sort of develop our own theory of action in terms of the scope of what our commitments are, and how we want to operate.

And so, one, I think, is thinking about the role that the Board can potentially play in supporting some of these big ticket issues that we've raised. Certainly the reauthorization of the funding around us, SPED, but equally important some way of engaging in some fundamental self-reflections about how we achieve impact, and what are sort of new frontiers that perhaps are not currently captured in existing priorities. That would be helpful.

So maybe now if we can move onto Peggy. I see you're back now.

DR. CARR: Yes. Thank you. And thank you for waiting. I had to drop off for the PETA release that are scheduled a year ahead of time, so the date I could not control. So thank you for waiting.

If someone could put up my presentation. Is it part of the — no one has it? Okay.

MALE SPEAKER: Peggy, I could share it if you would like.

DR. CARR: Yes. That would be great, so I have that.

MALE SPEAKER: One second.

DR. CARR: Let me say while he's bringing up the presentation the NASEM report for NCES was bold, it was daring, it was forward-thinking, and we really were excited to receive the report.

I likened this report, this NASEM report for NCES addition for the future, NCES to be very similar to what Emerson Elliott received in the '90s from the Academy, the National Academy of Science, and they were very blunt. They said that if NCES did not improve what it was doing to take seriously its role as a statistical agency, they might as well close the doors, shut the doors.

Emerson took what that report said to heart. And I think what we see today, as an NCES well-respected organization, had a lot to do with what Emerson did, and how he responded to that report.

So I'm glad to see that we have good, bold, and forward-thinking as part of our frontier, and so I'm going to take time today to tell you what we've done to respond to it.

I should point out that in our conversation — we can go to the next slide, Stephen. In our conversation with the panel they made it clear that they did not want to be confined by the lack of resources, the fiscal resources, the Human Resources.

They wanted to explore what should be possible, not be confined by any limitations of resources. So with that as the context we have done as much as we can to forward the good thinking of this panel.

Next slide. What I thought I would do is just basically go through the themes. There are 15 recommendations here. We have addressed, I think at some level, almost all of them, but to go through the themes, and to identify how these things relate to the recommendations, and then take the bulk of the conversation to tell you what we have done in response to these things.

The first thing was to develop a bold, and there's that word again, bold, strategic plan to make tough decisions, and we've done that, and that will be the basis for my discussion with you today.

In April and May we actually released a plan, a strategic plan. It's on our website. And then following not long after that we actually released an action plan, an implementation plan relevant to this strategic plan, what we have been able to accomplish, what we're hoping to accomplish in the future. So I invite you to go online and look at the details with regard to this plan.

Another thing that was in bold, NCES to set its own priorities, and there are a lot of thoughts there about independence, and the Evidence Act. Actually, I think these next two things are related, maximize NCES' ability to fulfill the Evidence Act.

As you know, ESRA is being reauthorized, and there are a lot of components about the independence and autonomy of NCES embedded in ESRA, particularly in Title 1.

And the version that we have seen, I think, struggles, in my opinion, with what independence should look like. It is not commensurate, and my opinion with what the trust regs, and the Evidence Act reports that independence and autonomy should look like for an agency such as ours. So I think that that ship is still out.

Next slide, please. Diversity, and awareness of equity issues, is a major part of this report. Chapter 2 would ask you to focus on that. If you really want to understand what they're asking us to do is, I think, a major theme throughout this report, and most notably in this chapter. And we have taken it very seriously as we implement it, and, well, developed, and now implementing our strategic plan.

Another theme was to expand data acquisition strategies for a new insight. And I think what they're asking us to do there is to look at data science, and newfound sources of data, administrative data, and we've taken that challenge very seriously as well.

Next slide, please. Prioritize data to increase relevance is something that we're going to have to work on operationalizing because we want to be cost-efficient, we want to examine what it is that we've been doing for decades, and perhaps figure out ways to do it differently, and that's going to take — that's going to take considerable effort.

And I think we're going to need our experts, our stakeholders, to help us get through that, which really brings me to that next thing there, create engagement feedback loops.

The report asked us to put together a nimble consulting body, and I don't think I get to say much more about that later in this presentation, but I want you to know that we have put together some experts to develop maybe a charter for our consideration of what that nimble consulting body might look like.

And they will help us in our vision to help us prioritize our data collections, maybe even content of what we're collecting, but this is an important part of how we want to engage consistent with what the report is saying with our stakeholders moving forward. They are also asking us to enable data access in a way that we have not done before.

So they're asking us to think more creatively about our state longitudinal data systems, and how we can not just provide technical assistance for their development of longitudinal data sets, but how we can ask them to, or hopefully get them to, allow us as a federal agency to be part of that community, not just providing technical assistance.

That is what they want. It's going to be difficult to achieve, but we are exploring possibilities there.

Next slide, please. Improved dissemination, and focus on accessibility and usefulness. I will say more about this later in the talk, but we are looking at ways to track our products, and monitor them to see what stakeholders are finding useful about them, or not finding useful about them, so we can use that information to improve our products, and to improve accessability to them. What kinds of products are we producing, not just reams and reams of tables, for example, that people will not always find very useful, or accessible.

We also are going to examine consistent with the recommendations of internal structure, and operationalization of NCES' responsibilities.

There's a lot of silos, and you know we have four centers, but within NCES we have divisions, and these divisions, these three divisions that we have, and we have two units as well, often work more in silos than one might imagine. So we're looking internally how we can recentralize some of our operations, for greater efficiency consistent with the recommendations of the panel.

So on page 94 of the report is this — don't worry if you can't see it, but it is a roadmap, and the report actually gave us a roadmap. They didn't just talk about it conceptually, and in terms of a framework, but they actually gave us a roadmap as to when we should be getting these major activities done, such as getting the strategic plan together, getting the action plan associated with implementing the strategic plan.

Well, I can just tell you we are not on the timeline that they've laid out here, but we are on track.

Next slide, please. So this is our strategic plan that I think is the major part of what the panel wanted us to do. We developed this plan not just from the top down, but from the bottom up, so all of NCES employees, and managers, had an input in what is in this plan. And we also shared it with our colleagues across IES.

There are four goals, and here are multiple objectives beneath these goals. I won't go into them, but the first goal is to align products to data on needs. And I'm going to take some time today to go through each one of these.

The second goal is to improve and innovate NCES operations, and I think we have some really good examples with what we've done, and what we're planning on doing with regard to this particular goal.

Foster and leverage mutually beneficial partnerships. I mentioned the nimble consulting group that we are excited about, but we've also started to develop stronger relationships with stakeholders that we really haven't had that kind of ongoing, mutually beneficial relationship before, such as HCBUs.

And to then go forward to embed and integrate these principles, diversity, inclusion, accessibility, into all of what NCES does. And I don't mean just sampling. I don't mean just reporting. I mean everything, and so we're going to go through that, and tell you what we're doing there.

Next slide, please. I thought it would be interesting to lay out why I think — going through the strategic plan today is the best way to sort of respond to your inquiry about how we are addressing the NASEM report. There is a huge overlap between what's in our plan, and what NASEM asked us to do.

Next clip, please. These are the things that I just went through, and you can see the recommendations behind them, and we're going to go through the strategic plan, but what I want to demonstrate is that all of these things are embedded in all of these themes from the NASEM report, are embedded in the NCES strategic plan.

But there are some parts that are not that we still need some collaboration, some work with stakeholders both in terms of IES, and the Department.

So their recommendation in the plan — I'm sorry, recommendation in the report that encourage the Department and IES to collaborate for ensuring the independence of NCES, and to support the Evidence Act, and the NCES role in the Evidence Act, and not just NCES, but also the statistical official also identified in the Evidence Act.

So I think this is still a work in progress. The trust regs, if you have not seen them, delineate

— the draft trust regs because they're not final yet, delineate how the IES, the parent organization, is to operationalize, or support, and enable, the independence and autonomy of NCES as a statistical agency.

So I think we need to have some finalization of that trust reg. And also what Congress wants to do with regard the independence of the NCES commissioner whether it remains a presidential appointee, for example. And some of the other very clear independence signals in the reauthorization of ESRA.

The new contracting arrangements that's still something that we need to think more about, but you will not see much of reference to those activities in our strategic plan.

Then there are those things within our strategic plan that are not in the overlap between the report, the NASEM report, and our plan. I want to point those out to you.

We are revisiting as I speak, our statistical standards. One of the things that Emerson Elliott was asked to do back in the '90s when NASEM gave him his report for the future was to develop some standards, some real strong defensible standards. And we have revised these standards a couple of times since I've been here at NCES, but it's time to do it again.

And we are in the process of doing that from the top down, and from the bottom up. Everyone is involved in the revision of these standards, and they're going to include DEIA, not just as a chapter, or a portion of the standards, but everywhere throughout the standards is our goal.

We also have a goal to have psychometric standards, which were not sufficiently dealt with in the statistical standards, so we now have a chief of psychometrics, and that chief will work with the chief of — the chief statisticians to develop standards. And we have some outside experts that are helping us to articulate what the scope and framework might be for psychometric standards.

We want to continuously address issues of data quality. There are data quality issues as all statistical agencies are struggling with, for example, around response rates. So that is part of our strategic plan, and not really dealt with that much in the NASEM report.

And we also think that we need to work on internal and externals, and internal, but we need to work on internal communications both within NCES, and within the Department about what our role is, particularly now as it has been defined in the Evidence Act, and will be defined more, and in more detail, in the trust regs about what it is that we should be doing as a statistical agency.

I have found that as these documents have developed, have been circulating in the federal statistical system, and is parent organizations and partners, that we were not all on the same page. So I think there has to be some improved communication in that regard.

Next slide, please. So these are the goals. And go to the next slide.

So what I'm going to do now is sort of go through each of the things that we have done, and give you some exemplars with regard to each one of these goals.

Just as a reminder, this particular goal addresses the NASEM themes of data acquisition, prioritizing data for relevance and feedback loop that I've described several times now as I've gone through my talk.

We are most excited, I think, when it comes to a new and innovative product about our School Pulse Panel, our SPP. As you know, IES was asked — was given funding, and asked to collect information about the impact of COVID, and to monitor it. Some of those funds were allocated to NCES to collect data using this new instrument, the School Pulse Panel survey, that goes out every 30 days, with a set of modules some of which are constant, but many of which rotate in and out every 30 days.

Here we're getting realtime information from the K–12 schools about the impact of the pandemic, what the recovery looks like, not just for students, but for teachers, for the schools, the entire context.

What you see here on the right are the latest results. We just released some results to stakeholders from the August Pulse Panel, and you can see a very interesting finding; 45 percent felt they would understand; 45 percent of the schools, K–12 schools, felt they were understaffed entering into the '23-'24 school year.

I happen to know the results. We just released them. It's better than it was when we did this last year, but there are still challenges with schools being understaffed.

On the right here, the furthest to the right here, you'll see what we're planning for December. That will be school facilities, something that many have expressed interest in, and also learning recovery. This is one of the modules that we continue to cycle in and out of.

So we are very, very proud of this new product, and aligning these products with the needs of our stakeholders. I'm reminded by teachers, for example, when they see these data they say they for the first time see themselves in data that we are releasing. So we're very happy to hear about that.

We have also undertaken a listening tour of districts. I mentioned earlier that we have challenges with response rates. There are certain states, and certain large urban areas within states that always say no to our requests, or rarely say yes to our requests, to participate in our studies.

So what we have done is to develop a listening tour asking for nothing, but just to talk to the chief superintendents of these districts, sometimes within the state at a higher level, and just tell them what we have to offer, what kind of data that we produce, and what we could do to make their participation more meaningful for them.

A

nd we've held a series of these listening tours over the course of about three months, and we've gotten districts who normally would just say no. Baltimore, for example, is good for saying no. We really can't do it. Montgomery County where I live almost always say no, but we've gotten some good response from this listening tour.

The other thing we've done to align our products to data needs is to launch a different way of developing our portfolio of products that we release in terms of our publications.

What we used to do years ago is to sort of sit around the table very much like you would do in a dissertation defense, and people would defend — each unit would defend why they think a particular publication is worthy of being part of our portfolio.

When we have put that process in this writ system, and it has an opportunity for everyone to comment on why a particular area should be the focus, and we can also see what everybody is proposing to plan for their next publication, so we can decide if there needs to be an alignment, there needs to be some editing, or trimming of what our portfolio, or releases of publications, might look like for the coming up year. It's a wonderful system.

Next slide, please. If we had more resources, I know the panels that don't worry about resources, but we have to worry about resources.

We would increase granularity of our data of products by doing things like more small area estimations. We have done small area estimations for adult literacy, data collection, a PEAC (phonetic), for example. Just smashing success. But we think we could probably use this methodology more with more funding, and more FTE to do it.

Another example would be concatonating samples across low response rates for private schools. We are having a heck of a time getting private schools to understand why it's important for us to have their responses to our national surveys as well. They're part of our education system.

And so often we're not able to report on them, so we're looking into some methodology that will allow us to concatenate across years, so that we can increase our samples, and decrease the bias around those samples, and report on Lutheran schools, or more than Catholic schools, I should say.

We want to build state and local capacity by leveraging geospatial data. This is the pot of gold that no one seems to, at least internally, realize that we have, that this geospatial data is such a valuable asset. And we get millions of hits on geospatial websites a month, not a year, a month.

People are touching these data, and we could do a lot more if we could figure out how to link across states to use state of the art methodologies to link to our data, our state data, and to data from other agencies using this kind of leveraging with something like geospatial data. It's very exciting work.

We would also, if we had additional funds, we are now working in a pilot, for example, with three other statistical agencies on ways to use AI to crawl around in peer review journals to gather information about how our data sets are being cited in these journals.

Researchers are using them, what they're saying about these data sets, how they're using them, which variables they're using, and then to put all of this information in displays, so that we can share it internally, and share it with our stakeholders.

GOIs are something that we're working with our colleagues across all of IES to mark every publication that we have so we can track it, and see how it is being used.

And, of course, AI is everywhere, and we want to use AI with additional funding not just in NAEP, but other places, in other surveys, in NCES to do things like item generation, or doing item scoring, but item generation as well.

Next slide. So Goal 2 is to improve innovation — is to improve and innovate operations. I mentioned this earlier. We want to address the silos that we have, and these are the things that this particular goal, and these objectives, address.

Next slide, please. So I mentioned earlier that we hired a chief psychometrician, and this is important because we often think about NAEP when we think about assessments, but we have assessments elsewhere in NCES.

We have longitudinal ECLSK, the high school longitudinal assessments; there are assessments there. But the advances, and the innovations, and assessment activities that we see in NAEP are not necessarily being applied to these other large scale data collections that we have, for the national data collections that we have.

And this psychometrician is also going to look to other areas where we can develop indices, such as the equity indicator that was mentioned earlier.

The edge data, that's just this geospatial data that I mentioned earlier, has so much potential, and stakeholders know that it has a lot of potential, but we haven't done, we believe, as much as we could do moving forward.

The other thing I wanted to mention I've already talked about the private schools in the automated scoring, but I wanted to mentioned that our longitudinal data collections are exploring using remote administration, remote testing, not just going to the schools, and this is going to be very efficient if we can actually fully operationalize this because some of these longitudinal study participant students by the time they get to, say, middle school, or in the higher grades, they have dispersed so much so you might only have a handful of students in a school. So it's very expensive to go there to just test a few students.

So testing them in settings outside of the brick and mortar is going to be very efficient, and very cost-efficient if we can do it.

Next slide, please. We would love to have an R&D unit, and the NASEM report says a little bit about this, but with additional funding, and resources, and for staff, we think that this would go a long way to modernizing our methodology, ways of collecting data, ways of using data not just survey data, but other data that we perhaps don't collect that might be useful. We want to do more of that.

And I've mentioned the geospatial work, but I want to point out that this Geospatial work that we want to do with additional funding could address these issues of better poverty measures than the ones that we've had. Free and reduce indicators, free and reduce price lunch has been used since the proxy for SES, and, of course, that indicator was not developed for that purpose.

But these Geospatial studies that we are exploring we believe will help us to develop not just school level poverty measures, but student level poverty measures that can be generated in such a way that we can have more accurate poverty measures for a school.

We also want to look into centralizing our sampling methodology. A lot of the contractors who do our sampling across NCES are the same contractors working for different data collection activities. We think we might be able to centralize some of the contracts, or some of those sampling methodologies, so that we can have more efficient sampling, and more cost-efficient activities across the center.

Next slide, please. Goal 3, fostering and leveraging mutually beneficial partnerships. Let's just go to the next slide.

To address this particular goal we hired a director of partnerships. We have a lot of outreach through a form, which is actually mentioned in our legislation, and the National Post-Secondary Education Cooperative, so we have a lot of outreach, but it's not as well-coordinated as we think it should be.

Our director of partnership is going to work to improve our coordinated outreach. She is also going to work at the level of operation to make sure we know why our partners are having challenges with our data collections, who is saying no, why they are saying no, how often they say no, what did they say yes to, those sort of nitty-gritty pieces of information that we kind of know in silos, but we don't know as a collective.

And similarly, we need to take a stronger look at the state longitudinal data system to see if there are ways that the federal government can work with these grant recipients to the benefit of us all.

Next slide, please. We would love to have a state coordinator group very much like NAEP has a state coordinator for every state. And I think this is actually mentioned in the NASEM report.

If we had funding to do that, we would perhaps not have at the state level, we might have at the district level, but we would like to have a coordination group of that type to further our mutually beneficial partnerships.

We have some of the most renowned psychometricians and statisticians in our field who came through internships with NAEP — I'm sorry, NCES, many, many years ago; the Larry Hatchers of the world, or the David Kaplans of the world.

They came through NCES years ago as young scholars, and through an internship, or fellowships as it will, and now they're some of the best in the world, and we would like to, if we have funding, to rekindle that activity.

I think I'm going to move on. I think I have actually already said enough about this latter two points.

So let's move onto the next slide. And more on the embedding DEIA principles across NCES.

I think maybe since the last time we met perhaps aligned with that time period. We released an equity in education dashboard. We actually have a dashboard now that pulls all of the publications, or products that we have, tools as well, into one dashboard where it's like a one-stop shop for equity with regard to our product, or products that we partner with our other statistical agencies in collecting.

And this particular dashboard is based on that equity report that someone mentioned earlier published by the National Academy back in 2019, has 16 or 17 indicators, and a really solid framework for not just outcomes, but also variables that mediate equity, and that's how we pull together that dashboard.

That NASEM report is also the framework for our equity indicator that hopefully will be not just national, but state level as well, using as a guideline the indicators identified in that report.

Our IPEDS data collection is also exploring some equity indicators through piloting of some particular areas of concern by our stakeholders, like how we operationalize sex or gender in our report, so there is a question that's being piloted with IPEDS now.

Legacy admission is another one that has been fairly — has been at the top of mine, by many in the post-secondary world. So we are piloting a Legacy question, and also race ethnicity admissions. We have information about who applies, but we don't have information about who is actually admitted.

And you know the Supreme Court in their decision, this is kind of related to this, but not because of that. We have included a pilot question here. We were actually thinking about it long before the pilot — sorry, long before the Supreme Court made its decision.

And last, but not least, I wanted to point out that we've taken leadership roles in various interagency equity working groups, as many have across IES, to provide a presence in the statistical practitioners' community.

NCES is leading, for example, the

ED SOGI plan, which is required by House, the OMB.

We also are co-chairs on a couple of the interagency subcommittees on defining, redefining race ethnicity SPD, Statistical Policy Directive 15 that the chief statistician has been working on, and will finalize in the summer of 2024.

And I am one of the four cochairs of the White House Equity — the EEO, the Equity Initiative for 13-985 that the President signed on his first day of office back in 2021. And we have a lot of different projects there.

So we have been a presence in the community, and taking leadership roles in the community, and this, we believe, is relevant to the NASEM recommendations, and to our strategic plan.

And, finally, I think there's one more slide, we want to do more. We think the resources that are needed here are probably more in terms of number of FTEs as opposed to funding. We just are short on people as it is. Our FTE is smaller than it has been in ten years by 20 head counts. And we want to do more in this area in reaching out with MSIs, and HBCUs.

For example, the first time I've seen in a decade attended the HBCU conference that they have here annually, and made a big presentation. It was standing room only. We need to do more of those sorts of things, but we need to do it with concentrated FTE that can be assigned, and can focus on this kind of work.

And we also need to be able to publish more in this area. Actually analyzing our own data. Resiliency analysis, for example. We can't just continue to report on gaps, how big the gaps are, where the gaps are. We need to do more diagnostic work. That is what we would do if we had more people, more FTE.

I believe that is the conclusion of my presentation. I said a lot, and I'd be happy, Chair, if there's time for questions. I'd be happy to come back if your break — if you are targeting your break for right now, but —

DR. LEE: Well, if people are willing to stay on for a few minutes more, I think it would probably be most efficient to address questions right now, and then once we do we'll go onto our break.

So do you want to close that screen down, so I can see who's —

DR. CARR: Stephen, if you could close that. Thank you.

DR. LEE: So Conchita?

MS. LEGORRETA: Yes. So thank you so much for that information. Stephen and I, when we give our report, the information you gave that's going to be kind of the model that we're going to look for for others. So thank you.

I have two questions. One is how many schools, or systems, about are taking part in the SPP? Just to know like is there a large part? How big kind of the sample is.

And then the second question I have is around the DEIA focus, and I love that it includes the DEIA, as the executive order mentioned.

Is there a requirement, and, I guess, it would — my question is kind of broader than just the specific center. Is there a requirement that all applicants be required to include their results, products, research, in an accessible manner, and is this included as part of the RFPs?

DR. CARR: The Pulse, I believe, has something in the range of 1,200 respondents, and it is — I think the most important thing about the Pulse is that it is nationally representative. It is a strong sample.

We drill down to the local level, locale level, I should say, as well as the regional level, to make sure that parts of — all parts of the United States are represented. So even though it's not a huge sample, it is a very strong and reliable sample.

Your question about equity issues, inclusion, and diversity in our RFPs is a really good one.

We have — there's room for improvement there. I think NAEP has a really strong model with regard to requiring as vendors to talk about how they're going to ensure equity in their administrators, for example, or how they're going to encourage professional opportunities for up and coming psychometricians. There are like some internship programs that are in some of the NAEP contracts, and they were collaboratively across the alliance contracts.

But there needs to be more about who is sitting around the table, who is involved in item development, who is involved in the analysis of the data because that is the perspective they're bringing to those particular activities.

If that's your question, there's lots more to do in that area.

MS. LEGORRETA: And my question is more specific on like accessibility. So do we require inner RFP —

DR. CARR: Oh.

MS. LEGORRETA: — for all of the products, and everything be accessible to both the participants who made the — in the research, but then also their final products.

DR. CARR: Oh, yeah. They have to be 501 — 508 compliant. Yes. Everything has to be 508 compliant.

But in addition, for accessibility regarding data collection, we also require accommodations. We require universal designs in our data collections, the devices, the instruments. Yes. Those are part of the RFPs.

MS. LEGORRETA: Thank you.

DR. CARR: But 508 compliance is required across the Board.

DR. LEE: Okay. Shaun?

DR. HARPER: Thank you. Peggy, thanks for such a spectacular, very, very, very thorough presentation. It was very impressive.

Two questions. One, could you give us a sense of the integrated R&D unit? What level of resources do you imagine would be required to pull something like that off?

DR. CARR: Well, we need expertise that we probably don't have now. We want to be able to crawl around, you know, do web scraping. We have researchers that do web scraping, or vendors that do web scraping for us, but we need someone internal to NCES to manage that kind of work.

Data scientists. I know that, you know, everyone is talking about data scientists, they're unicorns.

We need the kind of unicorns that really understand education data, and that can take their knowledge of data science, and apply it to our work.

Figure out creative ways to use found data, whether it's transaction data, whether it's administrative data that's at another agency.

HUD, for example, has reached out to us, and they want us to figure out how we can work together to help them understand equity issues with their data.

So we need dedicated staff more than anything else, to be quite honest, to implement this. It's not — funding, of course, would be helpful to have contractors to help you operationalize those activities, but I think it needs to start with highly specialized, focused FTE.

DR. HARPER: That's very helpful. My second question is a quick one. The equity and education dashboard, how do we get to it?

DR. CARR: We'll send you the link. Stephen, you can put the link —

Carol, Dr. Lee, if it's okay, we'll just put the link right in the email. I mean, in the chat.

DR. HARPER: Okay. Fantastic. Thank you.

DR. CARR: I just want to say I love this dashboard. Everyone loves this dashboard, but it is so much more we could do. You talk about what else needs — what else we could do. We need to push it down to the district, and state level. We need to add more tools. We have a lot of tools across the center that we think could be tweaked, so that they are focused on equity and inclusion.

There's item mapping. There are all sorts of things that we have that just with a little bit of a tweak they could have an equity focus. So that dashboard is just the beginning. As happy as we are about it, it could do a lot more.

DR. HARPER: Fantastic. Thank you so much, Peggy.

DR. LEE: In addition to putting in the chat, Ellie, could you maybe after the meeting just send it to us via email since everybody isn't necessarily downloading from chat right now? Denisa?

MS. PELAEZ: Yes. Thank you.

DR. GANDARA: Thanks, Peggy. It's really exciting to see all the excellent work that you're doing at NCES. Thank you for sharing with us.

I'm thinking about the public comments we received from the Institute for Higher Education Policy regarding post-secondary sample studies, and in full disclosure, I'm on the Board for IHEP, and I'm also a higher ed researcher who uses a lot of these data products.

So I'm curious to learn about the process for evaluating the relevance and usefulness of data products, and to what extent you engaged stakeholders in the decision-making process.

And just for further context in case others haven't seen the letter from IHEP, IES has decided to — well, my understanding is IES has decided to discontinue baccalaureate and beyond, and potentially scale back NPSAS, which is the post-secondary student aid survey, and PBS.

And again, as someone who uses these data, and as a member of the community who uses these data, I was just really surprised to learn about these decisions. I'm curious to know more about the decision-making process. So if you could please share it, that would be great. Thank you.

DR. CARR: Thank you for your question. I, fortunately, did read the letter, and thank you for your interest.

NPSAS is one of the few mandated data collections that we have. IPEDS is mandated. NAEP is mandated. NPSAS is mandated. And so we have to do it. I mean, it is the only source of information about how low income families, families of different — at the student level, I should say. Students from low income families of different racial ethnic backgrounds, where we can get that information, and understand how they are paying for college whether it's Pell Grants, or loans, or whatever.

So this is valuable, but it is expensive. And I should point out that the decisions about B&B were made before I took on this role. So I am not totally privy to exactly how that decision was made, but baccalaureate and beyond is one of those — it's a spinoff, as you know, of others on our call who may not know, but it's a longitudinal spinoff of NPSAS.

But as I understand it, prior to my coming we have to figure out how to stay within our budget. Our budget is not improving, it's not increasing, and so we have to cut back, I think is the bigger point.

With regard to what we're doing with NPSAS itself, what we're planning to do is to have an administrative data collection year to be alternated with a sampling year in which we also collected administrative data.

So we would go in that cycle every three years, and that's going to save us a lot of money because collecting information from students is very expensive, but we believe that we are still going to have meaty information about how students pay for college.

Our biggest problem with NPSAS, I don't want to belabor this point because it wasn't in your letter, is that it looks like we will not have access to federal tax information, which we used to get from our partners over in federal student financial aid when the data got here, but because of the way FAFSA now collects information from families, it just gets put right into the system, it's protected, and we can't get it.

So the biggest concern I have is, quite honestly, and Matt, my colleague over in evaluation, has expressed a similar concern for his activities, is that we don't have access to these data for NPSAS.

B&B, yes, we need B&B. I would love to have funds to implement it, but the biggest problem I see coming in the next couple of years is that at some point someone is going to realize we don't have that data.

We are in contact with our stakeholders, so stakeholders, let us know when we make hard decisions. Remember that first recommendation I showed you that NPSAS said we had to do? It said you have to make hard decisions because we can't do everything.

So I don't think we're in a vacuum this nimble supporting, advising group that I mentioned earlier. That's one of the things we're going to ask them to help us do.

How do we make these decisions? What will be the criteria for making these kinds of decisions? Who should we talk to? What are the tradeoffs? So it's not an easy answer unfortunately.

DR. LEE: Denisa, your response to Peggy's response? Our response to this request from — how do you pronounce it? IHEP?

DR. GANDARA: IHEP. Yes.

DR. LEE: Our response to IHEP is going to be a purview for this subcommittee to work on, but I'm just curious as to whether you have any thoughts or response?

DR. GANDARA: Thank you, Peggy, for your response, and for sharing the additional context about the fact that this decision about B&B was made long ago it sounds like, and in the additional context regarding FAFSA, and tax information, it's not something that I was familiar with, and it does seem like another cause for concern.

I will be really curious to know more about the advising group, and to learn more about the process that will be put in place for making these kinds of decisions about which products to discontinue, and what areas to expand in. So I'd like to get an update on that once that advising group is in place.

DR. CARR: Well, we can certainly provide — Madam Chair, we can certainly provide more information at your request regarding the FAFSA group because we are working on a charter, and perhaps this group might have some input, or some ideas about what should be in the charter before it's finalized, because this is one of the goals that we're hoping this group help us to address prioritizing of our data.

DR. LEE: So two issues that have emerged, not to discuss at this point, but, I think, should come up perhaps even tomorrow when we get to sort of bigger issues, is, one, is that I'm hearing that, one, there are ESRA questions, and one of which I understand has to do with some potential recommendations about the appointment of the director of the center that I think is emerging in the ESRA discussions on the Hill.

And the other is — again, I think this would be useful as a topic tomorrow after we've gone through all of these reports, are the funding questions relative to certainly the ESRA reauthorization.

There seemed to be a lot of complicated kind of issues about what rationales for increased funding for whom, you know, under what circumstances. So I've heard this generally on the IES side, and now specifically relative to NCES that would be helpful.

I have one quick question, Peggy, and that is how do you all — and, I guess this is a Peggy, Mark, other directors kind of question, about relationships across the centers?

So there seems to be some common issues like around equity, or the issues of the kinds of data that NCES collects, and its relationship potentially to priorities that other IES centers develop.

So I'm just curious as to what kind of structures do you all have in place for communication, and feeding on one another's data? Like how does some of the kinds of reporting coming out from NCES inform some of the kinds of targets that the other centers seek to address?

DR. CARR: Mark, you can start.

DR. SCHNEIDER: So a couple of things. So first of all NCES — I'm sorry, IES is a relatively small organization. We were all in close physical proximity, and that really matters.

And so you walk down the hall and talk to people, right? And even after the pandemic people are here an awful lot of the time, and that makes coordination a lot easier. I mean, we all know this, right?

I mean, these meetings — we have to have meetings like this, but the fact of the matter face-to-face contact and discussion is actually quite important.

And just an aside, which is a very human side, we're a five-minute walk from the Wharf, which I don't know if any of you have been to the Wharf, which is a relatively new area in Washington, D.C. with lots of bars, and lots of restaurants, and like a 5:30 trip to a bar is actually, again, incredibly important for human interaction.

So, I mean, this is real important. The second thing that I'll note is that I've been in various circumstances, education, universities, et cetera, and I would say that the senior leadership team is one of the most cohesive and friendly groups that I've ever encountered.

So again, I mean, the close proximity, the bars, all this matters, and it's actually an amazingly collaborative and friendly senior leadership team. So you have to keep that in mind also.

The third thing, I think the third is that when I came on Board IES had a terrible history with regard to communications, and communication really matters because we're producing reports, and agreeing on reports, and things like that actually matter. So I hired a director of communication about four years ago, and just recently promoted her to be the deputy director, which is a pretty serious step forward for Comms, and that actually becomes another mechanism for ensuring discussion.

So I think that the — I'm sorry, the last point is Matt laid out his, I'm not sure, theory of action. Is that what you want to call it, Matt? Where there's like this virtuous cycle with data that comes out of NCES, and there's studies from NCER and NCSER get fed into each other, and that really matters.

DR. LEE: Well, let me just say I think it's wonderful that you have the personnel infrastructure. That's really very, very important. I was more concerned a question about a strategic set of strategies or policies that you all have in place for formal ways of thinking about the data that's collected, for example, and its relationship to priorities for funding, or the joint interest across all the centers around issues of equity where your efforts to address them.

Like Peggy, you were talking about going to the HBCU conference for not necessarily only on behalf of NCES, but the broader across centers. So it's more I question about strategically how these relationships inform one another.

DR. CARR: Well, from my perspective we have lots of room for improvement in terms of having a systematic, strategic process for ensuring that that kind of shared purpose and activity is actually happening.

There are some good examples, however, notwithstanding of when it has worked. NCES, for example, collects process data on not just how students answer a question yes or no, correct or not correct, we also have information about the process they undertake to arrive at that arrive, and we put that data set together, and to Matt's group — Nate's group, sorry, Nate, you're still new, give it to Nate's group, and they put out an RFA using that data.

Or whether students with disabilities are using accommodations, or universal designs that we have built into it, and is it really helping them. That's the kind of coordination that we should be doing more of that we're not doing more of, I think, and we would do if it were put together in a very strategic way.

I think another good example would be the IES summit that was just done, and we all participated in the summit. Over 2,000 participants, over 120 representatives, who made fantastic, very impressive presentations, including the Dr. Bernard. I hope I got his name right, the black astronaut who kicked it off. But many of those participants recommended by our colleagues down the hall that they need to be involved.

So I think that there are good examples, but we haven't codified it, and stratified it into goals, and objectives, and procedures in a way that it happens, and it just shouldn't happen because somebody came up with a good idea, and we go down the hall and bring everyone in.

DR. LEE: Well, again, that's helpful, and for me the reason that I'm kind of raising the question is that I think a fundamental issue for the Board, and IES together to wrestle with, is the question of uptake and impact of all the work that's being done.

And so it seems to me the relationship between the breadth of kinds of data that NCES collects should be an important sort of indicator, if you will, on level of the impact of the work that IES is supporting. Denisa?

DR. GANDARA: Yes. I'm sorry for keeping us from our lunch break here. Just a quick clarifying question.

Peggy, I think I heard you say that there might be an opportunity for us as a Board to provide input on the advisory group's procedures for prioritizing data products, and I just wanted to ask, it sounds like you're nodding yes.

Carol, is that something we could also discuss either later today, or tomorrow, the possibility of having NBES offer input on NCES' advisory group's procedures for prioritizing data products?

DR. LEE: So one of the big issues that we're going to discuss, and probably that will be tomorrow, is the relation — as we're talking about forming sort of permanent committees is to whether or not we want to establish some formal committees that have relationships with specific centers. So we can — we'll kind of, I think, come back to that tomorrow.

So why don't we —

DR. SCHNEIDER: Could I just add a point of information —

DR. LEE: Sure.

DR. SCHNEIDER: — or whatever order, something, some point to that?

So remember under the existing legislation you're an advisory board, and almost all of your formal responsibilities as specified in ESRA are — is to advise the director.

So just keep that in mind as you start thinking about these committees, and who is talking to whom. But under the existing legislation the — I think there are probably nine or ten activities that are specified in ESRA, and almost all of them are designed to say the Board shall provide advice to the Director.

DR. LEE: So these again are some issues. I think one of the challenges, I'm just going to speak personally, is that I think that the charter, and I'd mentioned this to Jacob I think is the contact that we've had with the White House, that I think that the charter for the Board is very vague and conflicting, and it moves back and forth between sort of advising, and more direct kinds of roles.

But, I think, that — and part of this may come back to the conceptualization, Mark, that you have about the pathways through which you as director get advice, and I'm thinking that the ability of the Board to interact with centers can still be viewed as ultimately, you know, going up the chain of command, if you will, to you as opposed to everything that we do is only at the level of interacting with you, which I think —

I mean, the reason we had all these reports is because getting data from the various centers, and understanding the challenges that they're working with informs, any kind of recommendations that we make.

But we can, you know, pursue that further because I think this is going to be a learning journey for all of us in terms of the work, and the focus of this Board to be effective as an advisory board.

So why don't we take a break. I have 12:24 Central Time. So why don't we come back at 1 o'clock, 1 o'clock Central, 2 o'clock Eastern, and you all can go down the line. So basically a little over 30 minutes. Does that work?

MALE SPEAKER: That works.

DR. LEE: Ellie, we'll all just sort of stay here online, and just leave, so that you're not having to keep readmitting people. Will that work?

MS. PELAEZ: Yeah. That's fine.

DR. LEE: Okay.

MS. PELAEZ: All right.

DR. LEE: Thanks everyone. Thanks, staff, for all the wonderful reports, and we'll see you in a half-an-hour.

DR. SCHNEIDER: Thanks.

(Whereupon, at 1:25 p.m., the meeting in the above-entitled matter was recessed for lunch, to reconvene at 2:00 p.m., later that same day.)

Afternoon Session
(2:00 p.m.)

DR. LEE: How are we looking, Ellie, in terms of we're back?

MS. PELAEZ: I mean, it appears that we have a quorum, folks are back.

DR. LEE: Okay. Great.

MS. PELAEZ: Mm-hmm.

DR. LEE: So I'd like to move on to Conchita and Stephen's report around the Board and upcoming policy legislation.

MS. LEGORRETA: I don't see Stephen. I'm not sure if he's back.

Okay. I will go ahead and get started. If Stephen joins, we can get that.

So we had different conversations looking at the lens of legislation in different areas. So one of the things we talked about was internal review of systems at our centers, and kind of looking at it in that way, and then also given the Administration's concerns about learning loss, and educational equity, and the pandemic, what does that look like for us to be able to improve our kind of internal policies and procedures.

And then we also wanted to touch on the NIH designation. They recently designated people with disabilities as a population with health and educational disparities. So we wanted to kind of focus on how can we make sure that our information is kind of aligned with that.

And then also what does it look like to require demographic indicators in research for the projects that we support, specifically around disability. So that was kind of the broad implications.

Mark and Steve had a call, and as we all know, there's a lot of things that we can and cannot do in terms of Board members.

So one idea that we had specifically around the funding, and how as different people have mentioned before, over time, the funding has gone down, and what does that look like.

So we're proposing, and we definitely want to get the feedback of Mark, and others who know kind of what we can and cannot do, but one of the ideas was a day on the Hill, and this would be specific with the Help Committee, and the Appropriations Committee, in keeping it very bipartisan, very — not bipartisan. Sorry. And as a way to not advocate, but rather, educate the people on the Appropriations and Help Committee as to this is our need.

And the example that Peggy gave was really perfect in her presentation. That's the kind of information we would need if this is something we would want to do. The best time to do Hill visits are in March and April, so we could start small this year where it's just the Board who goes, and then the following years we open it up to our stakeholders to support.

But what this would look like is they — I've been a bunch of times, but they can provide us with space on the Hill to have like a pre-meeting where we could have people who are kind of supporters of our work come give a couple words, and the main focus would be creating a fact sheet.

So this fact sheet would include like, for example, the information Peggy shared, but coming from kind of all of the different centers, and why it's important that these initiatives be funded, and it would be very simple, easy to understand. They would be pretty much training everybody during that visit on these are the things to talk about, and then breaking up and visiting the different people on the Help and Appropriations Committee.

And it does take time for this tour, but, I think, beginning with the visibility of this is who we are, this is why it's so, so important, and getting people in the legislation on our side in order to support us with this.

Again, we're not sure if this is something that we can do, but what a lot of organizations do to kind of get around this is as a way to educate legislators, not as an advocate, and so the fact sheet would reflect that in making sure we have kind of that information.

So that's our brief and mighty report. So if we were to decide if this is something we, a) can do based on the regulations, and we would want to do, what we would need from the different centers is kind of similar information to what Peggy shared, and then we would work on putting together a fact sheet.

And I have contacts with the Appropriations Committee, so we can get the meeting and rooms reserved, and get that set up with the idea that this year is kind of smaller in the Board, and the following year we would have a bigger, inviting educators and constituents, and then not only hitting the Appropriations and Help Committee, but also representatives so that people can go to their offices.

So Stephen, are you on by any chance?

Okay. So that's kind of our report. We'd love feedback from a) is this possible with our parameters of the Board, and if people are interested that this is a way to do that to kind of uplift funding that we really need.

DR. LEE: So I'm assuming this might fit in with broader discussions that I think we ought to hold tomorrow about any relationship that we may have relative to ESRA reauthorization. So you might include what is the scope of specific recommendations we would make, some of which we will vet in a broader discussion tomorrow.

But I am curious, Mark, or Ellie, or any staff who can advise us on this question that Conchita has raised about our role — I would assume that I would be interested in hearing back from them that as an NDS Board advising IES that we could, in fact, advocate. It's not like we are personally advocating, taking the political positions, but in some sense I would assume that's part of our role as a Board.

Mark? Is Mark back? If not, maybe Liz, you may know.

DR. JONES: They may be having trouble with their microphone.

DR. ALBRO: We got it. I got it, Stephen. Thank you.

Yeah. Hello, Carol. We were just thinking that this is probably a question for the FACA attorney, and we were all just kind of trying to look to see if the FAFSA attorney is on in terms of making sure that we follow the guidance.

So I don't know who that person would be, but Ellie can connect us, connect you all, up with the appropriate attorney, who can help you provide guidance, and sort of a framework by which you should be interacting with our colleagues on the Hill.

DR. LEE: So part of the discussion that I had with David, I think is his name, the last Board chair, around the functions and the process that they had from the last Board of hiring an executive director, and part of what he said was that that person can also serve as a kind of liaison among other responsibilities between this Board, and the Congress, and the Secretary of Education's office.

So again, we will pursue that, but I would assume that this should be within our purview. Doug?

DR. FUCHS: Yeah. Conchita, thank you for that report.

As someone who has participated in education efforts on the Hill in the past both for IES generally, and NCSER particularly, I have a sense that it can be helpful, but as I'm sure you know, it's not the necessary and sufficient condition for moving people in the direction of greater funding.

So, I mean, it's got to be part of a larger strategy, and I'm certainly not asking for the specifics of a larger strategy right now but would be very interested to hear about that at some point.

DR. LEE: Do you all as staff have any — I mean, what are the activities in which you engage certainly as you're, you know, arguing for increased budget funding, for example, or other kinds of priorities that are going to be impacted by legislation at the congressional level? How do you all engage in communications or seek to inform them?

MALE VOICE: Yeah, Carol, that's a great question. And so we do often have occasions to provide feedback to both our appropriators and authorizers. We tell them what we — you know, what we are doing, and what we might be able to do is increase resources, but it is in the spirit of giving information only to inform it's their deliberation; it's not for the sake of lobbying.

So it really is kind of a descriptive for lack of a better word kind of account, you know, what we're doing, where we'd like to head, and what the kind of delta between those two things are, just so we're sure we don't run afoul of any actual kind of legal requirements.

DR. LEE: So I would assume once we get a sense of the sort of scope of authority we have relative to the relationship of this Board to the activities of the Congress coordination of information would certainly be very important between internally what you generate, and what we would attempt to try to support, you know, externally.

DR. FUCHS: Carol, if could just quickly add. My limited experience in work being — you know, working the Hill to try to get more money for IES, a really great group of potential supporters, collaborators, are the representatives — what do they call them? The Washington, D.C. representatives of respective institutions of higher education.

These people — the people that I've worked with are extremely sophisticated. They know exactly who to speak to, and it's rarely — you know, it's rarely a Congress person. It's staff, it's key staff, who oftentimes seem to run the show. And they know these people, and were able to get us to speak with them directly, and for long periods of time, and it was — I think in the long run it was very helpful.

DR. LEE: Does anyone know about this group that's reached out to me that's — they're connected with A.E.R.A., but the friends of IES? And presumably they would be a kind of entity I think — some of these stakeholders you're talking about I think are part of that — part of that group.

Caroline?

MS. SULLIVAN: Yeah. Some of this probably, and, I guess if we're having a discussion tomorrow about sort of larger communications, because I think a piece of it is, you know — in this hearing you have your external communications to the public, and to parents, and whatnot, and then your internal communications. This goes into that.

Some of this is sort of what is the story of what we're doing. I think especially it's quite difficult when you're a very data centric organization because that is your job, but we have to tell the story because that's how human beings, and really politicians, they think in story, right?

And so I think maybe the Board can be helpful with some of that comms strategy, but then also who are your people because whenever you're going to advocate for something it is — I think you need to have a strategy in that too, right? Who are all of the people who are your families, and think about family I'm meaning like who can help you with this, and it's not just — it's the associations, and it's also the association of say school superintendents and the associations of all of the higher ed people and the ed prep schools and all of these groups, you know, and we can even drill down in the special ed and to the Autism Society and the ARC and all of these groups that can help amplify what we're doing as well.

MS. LEGORRETA: Yeah. This is Conchita. What you're saying is exactly what we discussed. So the idea was to bring these groups together, and do it at — because March and April is so close for this year, if we could do this, it would be something smaller.

But in the following years bringing together all of those supporters and constituents who can tell those stories because what staffers like to hear is they like to hear both, so they want to hear the data, and they want to hear the data, and they want to hear the personal stories on how is this impacting a student in special education, or whatever.

And so the more we can provide the stories, and the data with all of these people who are allies for lack of a better word, who are providing this information would be super, super helpful.

So thank you. Yeah, that's absolutely one of the things we discussed.

DR. SCHNEIDER: So could I just add to that? So one of the problems that IES has had, And I think why it's not as visible or successful as it could be, is that we were never in the storytelling business. And we have many really, really truly compelling and successful stories.

Liz has been working on them. We had Ogilvie as a contractor because contracting law, and processes, and administration in the Department is not so good shall we say.

We have no contract comms in place right now. We are struggling to get it back in place, but one of the reasons we need a good strong comms shop is exactly what you're talking about.

There's nothing as compelling as stories, especially when we talk about successful stories of students, and there are so many — there are so many stories that we have, so many contributions that are both at a very high level, and ultimately it gets down to the lowest level of a student, you know, who all of a sudden can, in fact, have support services that they would never have before.

So Liz has been working on this. Nate actually have this too. I mean, so the question is whether the stories — and this has to do with the research side, right, what are the stories that our research has — to show that our research has actually done something.

So the other part of our problem is that — and I think problem is the right word, is that we — most of our money in research goes academic researchers, and their incentive structure is not the same as what we're talking about, right?

And we are — I mean, it's been a struggle, but we are reorienting IES to be more — I mean, we do basic research obviously, but we need to do the transnational work. We need to do the communications work. We need to do all that kind of work that's essential for turning all the work we've done, all the good work we've done, all the accomplishments, into stories that people could grasp.

And that is — I mean, it's not — it's a cultural issue as much as anything else, but we're changing the culture so people understand the importance of getting those stories into the world because that is how we build support.

So what — just one last point. In earlier discussions there seems to be some confusion between authorization and appropriations, right? So we've got to be careful about that, right?

I mean, we call them the four corners, right, the two appropriations committees, and the two substantive committees, authorizing committees. So the ESRA reauthorization is in only one of those corners.

The Senate Help Committee, the House Education Committee, may never take up this bill, right, and this bill is going to happen — I mean, it's happening fast, right, at least in the Senate side, whether or not they could get a floor vote, and whether or not it passes the floor vote in the Senate is a different thing.

If it goes to the Senate floor and gets a lot of bipartisan support, the chances of it going through the House are greater, but right now the House has not touched ESRA reauthorization and don't seem much interested in doing it. So that's one thing.

But you've heard several times that part of our problem is our budget, so, I mean, we have our budget around $800 million, which in the real word sounds like a lot of money, but in the world of science, federal science agencies — I mean, someone said, oh, you must be a rounding error in NIH's budget, you know, one of our fellow science agencies. We're not even a rounding error in their budget, right?

And I don't know if James is back on, but, I mean, you know, they've had so much more success in the last year or two in terms of getting money, new monies. And, I mean, the Chips Act gave tons of money to Commerce and NSF. We got nothing. We weren't even mentioned in the Chips Act, which is crazy.

So part of our problem — and part of that is the sales. I mean, it's literally sales, right? We haven't sold ourselves correctly. So then it's easy to ignore us when the money machine starts cranking out money for Commerce and NSF, and like — I mean, we lost money actually. In the recision that happened, we lost like $44 million.

So this is — I mean, so this is part of a problem on our side, right, that we haven't done our job in terms of telling stories. So we need your help on that.

So, Conchita, I mean, you're absolutely — or I'm sorry, Caroline, both of you are totally correct. We need a — it's a P.R. campaign as much as anything else.

MS. SULLIVAN: And, I mean, I think while it would be great to have a whole bunch of money and hire an LV, I mean, it is — you don't need to — I mean, a lot of times this does need to happen organically anyway, so I think just saying, oh, because we don't have a giant budget for a big old comms shop doesn't mean we can't do some of this work, right?

And also let's face it, a lot of your research partners, all those universities have comms shops too, and it behooves them to tell these stories as well, especially if it's coming out of their institutions.

So there are ways to do it, but it is, I think — it's just problematic, especially in this time where a lot of people aren't paying attention to data to be able to knit that through, look, we found out this that's a best practice for helping students with special needs be able to do better in school, and this is where we do it right, and this is how it can help you.

DR. FUCHS: You're absolutely right. All you guys are absolutely right. We need to tell stories, but we also need to be able to speak the language of the people we're talking to.

And Vanderbilt's got a very clever congressional liaison person, and she said to us before we went to the Hill she said, look, one of the people you're going to be talking to is a congressman from Tennessee. He's a small businessman. Speak to him as a small businessman.

And so one of us in conversation with him talked about research as an engine for innovation. Research is an innovation engine, and he — that metaphor really connected to him.

He didn't — we were talking about research, you know, research for like 10-15 minutes, and he didn't know what the hell we were talking about. When we talked about innovation, you know, an engine for innovation, that connected with him.

So that's also — I mean, part of this is really art, you know. I mean, you really — you really need to know how to be a good salesperson.

MS. LEGORRETA: How visits work is knowing — so let's say we're targeting the Help Committee. Knowing who all of those people are, and knowing their values, and their backgrounds, and being able to train.

That's why the training would also happen before the sessions, is being able to train everybody on this is who you're going to go see; this is how you frame it differently.

It's the same issue, but you need to frame it differently based on even if they're Republican or Democrat. The values are going to be very, very different.

So that's part of the research that goes into it is figuring out who is — who you're going to be speaking to, and then as well — because ideally the way this is done is you have a couple of people who are kind of people that support this work who are on the Hill who would come and do a little intro. So maybe like, you know, someone from Senator Casey's office. They do a lot with students with disabilities.

But then you also need a Republican, so that it shows that we — when we go and talk to these individuals that like it was we had people from both sides of the aisle, and this is people that —

So yeah, there's a lot of work, and a lot of kind of research that goes into making it in a way that makes sense, and then also trying to pair if there is somebody who is going that's from — that's a constituent of one of these people that's the ideal because that's who they're going to listen to more than somebody who is not a constituent.

So all of these things are kind of part of those visits. The question is how much can we do, or not do.

DR. SCHNEIDER: I'm sorry. Just one last thing. So I think in terms of authorization the timing is not going to work for you, quite frankly. I mean, there's going to be some discussion, but almost everything is baked already.

But it's the appropriations side that will have some more runway, right?

And I think — I mean, Carol is obviously very forward-thinking, and whatever you said — I mean, look, it's an annual process, right. I mean, the authorization is going to happen or not, right, and most people don't think it's going to get through, but what we're trying to do is get as strong a bill as possible to be the floor, the guidepost for next year, right.

So I don't want to — I don't want to discourage you from working on the ESRA Reauthorization, but the timing for this cycle is probably a little compressed.

I mean, the House hasn't taken it up yet, so there's entry points in the House. And then — so Stephen wanted me later to talk about the NEED Act, but there's another act in the House which may or may not get through, and the NEED Act — I don't remember what — National Education Evaluation whatever, it actually does two things.

The first one is — it sets up NCADE, the ARPA Ed, and some day we can talk about that, or not some day, maybe later. And the second one is SLDS.

So the NEED Act has two titles to it. Both of them are really good, and really important, right? So we could talk about ARPA Ed, NCADE, and, you know, whenever you want.

But the SLDS is really worth discussing because that is — Peggy, you alluded to this before. So that is an incredibly important data source. The federal government has spent $1 billion building it.

Almost all that money was done by 2011, or 2012, for tech process where the biggest investments were 12 years ago. I mean, I don't need to finish the dot, dot, dot in that. And we need SLDSV 2, and that would require whole bunches of money. Probably not another billion, but a lot of money.

But SLDS, if we properly conceive of it, can solve so many of our problems, right, because the vision of SLDSV 2 is actually to integrate data into the state longitudinal data system.

So rather than — so we spent 20 years — 17 years actually, figuring out how to build SLDS from a K–12 system pushing it down to early childhood, pushing it up into post-secondary. That's what we've done for the last 10-12 years, right? And we haven't talked about the importance of labor market outcome data, but that's in ESRA reauthorization.

So the importance of labor market outcome data, but that's — in ESRA reauthorization the labor market outcome data is all over the place, right?

We internally, and using our own authorities have expanded, but to have the Congress say we want these data, we want labor market outcome data, is a huge expansion of our ability to find out actually what happens with education.

So that's number one. So we're expanding this, right. It's gone down, it's gone up, but for many issues that we care about in education, chronic absenteeism, you need data coming in from everyplace, right?

So Peggy mentioned the Evidence Act, which is on the federal level where agencies are supposed to share their data. My reaction is good luck getting the Census, or IRS data, to — you know, giving us these data. Peggy mentioned that also.

But at the state level, I mean, what we want to do is we want authorization in SLDS so that the education department that runs these systems can, in fact, bring in data from other — I mean, it can already, some states are doing it, but we want to encourage states to bring in data about housing, for example, about food security, or insecurity. Criminal justice, whatever it is, so that we get a 360 view about what the kids are experiencing.

So, to me, SLDS is like something that we all should go to bat for that is in the NEED Act, and there's a longer runway for that. And then again, I think most of you know about our plans, or attempts to get NCADE and ARPA Ed into the legislation, but it didn't get into the Help Committee. We're still trying to get them to revisit that. But the NEED Act has it in it.

DR. KLASKO: This is Steve Klasko, and when you and I talked, you know, And I think this gets down to what I'd really like to hear from Carol, and Conchita, and I, have talked about is, you know, we talked about, I think four things that could really matter.

You mentioned the absenteeism and getting the funds to help the states get the resources to obtain actual data, you know, the whole issue of the seedlings to scale piece about research is an area for students with neurodiversity, you know, almost like an A.I. institute for neurodiversity like you talked about sort of looking at how we can scale academic progress through partnerships with — between new technologies and academic research. I'm in the VC world now that would love to do that. The government ends up, you know, very much creating a barrier to some of the companies in the — I guess I think you called it a $10 million opportunity to create the actual efforts from school Pulse data to be analyzed in months as opposed to years. You'll have a more rapid turnaround from the point of modern technology.

So, I guess my naive question is let's just — I mean, forgetting the whole health authorization, and centers committees, and stuff like that, you know, how can we help you even if we just take to those things that would seem obvious as being needed, you know, all over the lay press about absenteeism, nor diversity.

And other countries, frankly, I was just in Brazil, on the nor diversity side, and they're doing in some respects a better job than we are.

So how do we get that word out, and not have it just be another thing, and what's the role of this advisory Board to you in our relationships with representatives. Like I don't even know can I call my representative as a member of the National Board, and say, hey, you know, you should really — you should really look at this.

So to me, I think, that's where Conchita and I were really trying to get some direction.

DR. SCHNEIDER: So two things. So the first thing is about what your authority — of what your freedom is as both a member of the Board, and as a U.S. citizen. They may not align perfectly, but you need to have advice about that, right?

But you need to have advice about that, right? And again, I think the FACA attorneys, and OGC, could give you advice about that.

The other thing — I'm sorry. Just to follow that up. So my daughter is a pretty good attorney, and her job is to get to yes. Government attorneys, often their job is to get to no.

So I don't know if any of my friends from OGC are on there, but — okay. So the fact of the matter is like you need to be pretty assertive about this is what we need to do. Tell me if this is illegal, right?

And look, I don't want to visit you in jail, quite frankly, right? So let's get good advice about what you can do as a citizen and a member of NBES. That's number one.

Number two, I think — so I don't want to — I don't want you, I don't want any of us to be in the position of going to the Hill to the appropriators and say, oh, give IES more money, right, because they're good guys, they need more money, right?

So what I've been trying to do, and only with modest success, is to pitch, you know, bright, shiny objects, right?

So NCADE, ARPA Ed, we got $40 million for ARPA Ed, okay? And that was like — we were never going to get $40 million for statistics or NCER, right? We made a case, which is unfortunately a true case that we needed a modern education R&D infrastructure. We needed to have more timely releases of our data. We needed to focus on more applied issues. We needed to scale, right? So we haven't even talked about scaling. The scaling is so incredibly important, right, and part of the reason that people like DARPA, and all the ARPAs that are beginning to show up, is because they admitted to taking science and turning it into products, and things that change facts on the ground.

So in part because ARPA is a thing across the government we were able to pitch ARPA Ed, which by the way when I was in IES, and when I was at Peggy's job as Commissioner, we started in 2003 or '04 we started talking to DARPA about ARPA Ed, and now 20 years later we are getting close almost — we're almost there, right?

It's not in the — it's not the Help Committee write-up, but we haven't given up on that yet. And then the NEED Act is yet another possibility to get ARPA Ed done.

But you need — so for me SLDS if we can get like NGA, or CCSSO, to endorse the importance of these data sets for the Governor, or for the chief of the school system, I could talk forever, I do talk forever about SLDS. It doesn't matter, right?

We need NGA to stand up and say the Governors need this. We need CCSSO to stand up and say, you know, the school superintendents need this. So that to me is a marketing strategy, but we have to have something to sell.

If we just say, oh, we're good guys, look at all the wonderful things we've done, which we should be doing anyway, right? This is why telling stories is so important. But people are going to be much, much more interested in something innovative, right?

We haven't been able to crack — we haven't been able to crack this nut, but give us some money in the ARPA situation, and we'll push. We'll push, we'll push, we'll push in a totally different way than our standard approach. That to me is the — is what we — we're doing that, and that's what we need to do more of.

DR. KLASKO: That makes a lot of sense. Thanks.

DR. LEE: So what I'm hearing is that this can be a useful project of support from the Board to IES, and that one of the things — and also that we need a targeted group in terms of some targeted persons from this Board, certainly frm this committee, staff from IES, and possibly may even want to pull in some other advisors.

This is why I'm suggesting that at the end of the meeting I want to propose a vote to create standing committees that are able to meet outside of the Board meeting to work through details that will be recommended to the Board, and that such committees can include pulling in stakeholders who can give advice to how they reason through whatever the project may be.

So to that, Mark, what I'm hearing in particular would be the need of multiple stories. One is a general branding set of stories, but the other around particular initiatives that may be related to particular policy initiatives, or that may be related to part of the argument for increasing overall funding to the institute.

I think it would be helpful tomorrow when I wanted to have a broader discussion around the ESRA issue, and thanks, Mark, for the distinction between reauthorization and funding to be able to perhaps create a representation of a number of initiatives that are on the horizon, and timelines associated with them, so that as we begin to try to create a plan and infrastructure we have a sense of the scope of what we'd want to do, and what might be different timelines for that. James?

DR. MOORE: I'll try to be succinct. You know, I can easily give you the names of our Board members for NSB and NSF. I know our Board have ongoing meetings on the Hill. We got the biggest budget increase that we've had since we were founded.

But I can tell you some of the talking points because I sat on a lot of committees beyond just the educational research space, the workforce development space. It's one of those spaces that Congress is really interested in.

And so part of the vernacular that people use when — regardless of what side of the aisle is around the Chips and Science legislation, and if you haven't reviewed that, I encourage you to review that because that is the space in which people tend to operate is — some would say is the spending moment of our time, the Chips and Science.

So that might be a good frame to kind of frame some of your discussions, but like I said, our budget — our Board have continuous meetings with people on the Hill to advocate for increasing budgets.

But I will say I agree with Mark, it does seem that things are going downstream; however, you know, we had the omnibus bill that we got to supplement, and that's in my directorate is the biggest beneficiary of that omnibus.

But when you talk about, yeah, NSF did get funds, but it's like taking a five-gallon bucket out of the ocean and say you stole the ocean. It was $200 million over five years. We're talking about $25 million per year the first two years, and $50 million for the last three years.

Now, I won't ever say $200 million is chump change. It's more money than I ever, ever dealt with in my lifetime, and probably ever will, but nevertheless, when you think about the grand challenges around these issues, it's really not that much money.

So I would — I'm willing to share the names of our chairs, and I know they have ongoing meetings with individuals on the Hill, and because it's clear that the way my budget comes in is very different from how everybody else's budget comes in in NSF I always know what my budget is.

It's a blessing, and this occurs at the same time, and you'll probably say what's the blessing. The blessing is I always know what my budget is because it says EDU; however, the curse is that some people still don't think educational researchers do research. We're marginalized sometimes in an Agency like mine.

And so I just think there is a path forward, and I do know the sense of the emergency, but how I kind of navigate is when I talk to people on the Hill, outside of the Hill, they really know the significance of the educational space, particularly as it relates to STEM, although their world view is STEM. But like you said, Carol, the science of learning is a big part of what we do within our portfolio.

And so, yeah, my budget is 1.3 billion, but it's driven, or it's a lot of congressional mandates, right, that the money is already carved out in many ways.

And so I really think in these times, particularly around there's this emphasis on rural STEM education, which is underscored throughout Chips and Science. And I think that is one of the many reasons why we have bipartisan support in Chips and Science.

So framing is going to be really important, but rural in Ohio means something different, and Shaun could probably bear witness, than being in Georgia and South Carolina. It's just totally different.

And when you think about some of the demographic shifts, and when you think about the diversity, when you talk about what we call an ed scores states, the correlations with poverty, and the states that get, again, the least amount of funding in the federal government.

So these are priorities of the Biden/Harris Administration, and so I won't be long-winded, but that's just something to think about as a Board.

DR. LEE: So one of the things, and then Caroline, I'll call on you, that I think — and we obviously would have to pull together a group, and relatively soon, to sort of try and map out the plan for this. But part of the storytelling seems to me, obviously, is around the impact of work that has been done.

But also it seems to me as one of the other story lines around the possibilities of additional kinds of work that meet needs that have been persistently not addressed, some of which I think do include — I'm going to keep coming back to this science of learning and development. That's the new horse I've been riding for the last few years, and it goes back for me to this notion of theory of change that the extent to which human activity takes place within and across ecological systems, and not in any single side of activity.

And the fact that all human actors involved in trying to design and support learning are bringing to bear their perceptions of themselves of others, the relevance of whatever the activities that are engaged in, and wrestling with implicit biases, and conceptual change.

That there's basic scientific work to be done in that area that may fundamentally even shift how we think about this notion in moving to scale because I think even the understandings we have around reaching scale, one, don't take into account the complexity of the education system in this country from the city or town to the district to the state to the feds, the multiple parties, the different assumptions about what are the outcomes of interests. A very complicated, you know, system to manage.

And I think that on the one hand there's this powerful story of the work that IES, and all of its centers, have accomplished over the years, but there's also standing this sort of bear in the back of saying whatever kinds of data we're looking at, we're nowhere near where we should be, and what does it mean to wrestle with that question, and convince other stakeholders that they need to be investing in.

And also the issue of developing partnerships, so it's not only NGA, the what is it, Council for Inner City Schools, practitioner organizations, the whole variety of stakeholders we may need to just outline, you know, a variety of kinds of stakeholders in terms of relationships that we want to form, some of which obviously — much of which you would be doing internally, but also ways in which the Board could potentially help facilitate some of those relationships. Caroline?

MS. SULLIVAN: Yes. So like, James, the workforce thing, I've never seen such a focus on workforce, and I've been doing workforce for a while. So I think however as we talk — talking about how this impacts workforce, it is compelling to elected officials, as well as to pretty much everybody else. So I think that's critical.

And on these large acts like the Chips Act, it is critical that we're growing that workforce. I mean, this is something that I think is just sort of dawning on people is that if you're going to bring everything back to the states, and need a technically skilled workforce, you got to start that way deeper in K–12 than just trying to do solutions at community colleges, or get people to do short-term credentials, because we need to grow more.

And also we've got demographics that really aren't working in our favor just because of the birth rate, and things like that. So I think some of this caging around workforce, and things like that, I think, and knowing the data, and understanding what works, and understanding how you help students reach their full potential so they can participate in these emerging sectors is critical.

And Matt, I think maybe it might be helpful to not think about this as just some data people in education. It's sort of laying a foundation for all of these other things that are in different departments, and different acts.

And the other thing, the NGA I'm happy to help with that. We do a lot of stuff with the NGA. But also what Governors, and elected officials at all levels desperately want is some of this longitudinal data; what are the outcomes, and it's very difficult.

And I think another thing that I think the Board might be able to help is talking about that, and how with additional funding we can support states to help their common follow-ups in longitudinal data because it's rough out there, and people do need to know what happens to people.

DR. LEE: So I think too that the earlier comments that you were just making, Caroline, to me exemplified this point I'm trying to make about the sort of theory of change having to address the multiple levers in ecological systems that impact whatever outcomes are of interest, and the extent to which we can articulate that kind of vision in a way that's accessible, and human-focused, but also is the case for the funding that's going to be required to take that shift, or expansion of the efforts of all of the centers to address these issues in more complex ways.

So my kind of takeaway from this is that it sounds like this is a target in terms of as we're trying in this meeting to come up with kind of a vision of priorities for this Board that one of them would be how we can establish collaboration, support, partnership with IES, and all its entities around developing a strategic plan for this in a timely manner.

So as Conchita was saying about even whatever limited things we might be able to do as of March or April, that, one, for me this would become a rational for constituting this group as a standing committee of a Board that would allow the committee to meet outside of the public before a public Board meeting to work through the details of such a plan, including in working through that close work with whomever IES determines, I would think, some representative from all of the centers would need to be involved, and our ability to invite some other stakeholders not as members in any way of such a committee, but to advise the committee on how to do the drafting.

And then when we determine, which I think will probably come up at the end of the meeting tomorrow in terms of setting a calendar for our meetings, we can set a standard calendar, but we also may want to set a special date for another meeting that would allow this committee to report back on the specifics of a plan to begin to actually start this work in March.

Does any of that seem reasonable as an approach?

MS. LEGORRETA: Yes. Sounds great. Thank you.

DR. LEE: So I'm going to — I think I'd like to hold off on the vote for establishing committees of the Board until we've gotten the next two reports done because I think that those discussions will inform the breadth of what standing committees we think we should have as a Board.

So anything else on this upcoming policy issues in the world of the Board? And in the meantime before the Board — the committee meets to start as planning Mark will give whatever feedback we need from the lawyers about the scope and restrictions of whatever it is we can do as a Board.

I do suspect, however, that when we're speaking as individuals we should speak as individuals, and that the only time we should speak as Board members is when we are speaking on behalf of some decisions that the Board has made. I think that's going to be a very fine line that we're going to have to navigate.

Okay. So the next group, the notion of organization of NBES subcommittees. Caroline and Elmer.

MS. SULLIVAN: Well, we weren't able to spend — weren't able to like have official meetings, but, I think, just from the discussion that we've been having today it sort of — this is coming pretty clear on some of the buckets that we might need to think about committee work.

One of them, I think, is communications both external and internal, And I think that — at least it seems to be it doesn't really involve any of the advocacy, or the lobbying, and so I think that is something that is certainly doable for the group.

The policy work that was shared earlier, I think that is another area where we can have a subcommittee that also seems like it would not run counter to whatever we end up understanding is our place going into the Hill.

Yet when we were going through the minutes from the last meeting there was an awful lot of interest in supporting research for exceptional children, and So I think that crosses different — I think different buckets, but, I think, that is one that a lot of us have expressed to, and interest in. And I think it might make sense to have it be a separate committee.

Another one is the equity focus, how are we supporting staff to have that equity and accessibility focus both in the studies that are coming out, and how we can — how can we support some of the institutions that are receiving funding with those goals that we have.

Let me pull up my document. I want to see something.

Because I was also thinking that part of it is I think some of how we want to work, I think, is going to be gleaned then from our discussions today and tomorrow, but I really also think some of this we keep talking about alignment, and crosscutting, and the different systems, and how people talk to each other.

Carol, your whole system of change thought, I think, is an interesting one as well on how we can do this work. I spent a lot of time silo busting in my job, and it is sort of amazing how nobody knows what we're doing, and I think that is helpful as well.

DR. LEE: Other thoughts? So part of what I was thinking about this question, And I think maybe we could come back to this again after we have some discussion about the executive director search in part because it's my understanding that this is a position of someone who works directly for the Board, is paid out of the Board's budget, an the scope of what kinds of support we want that person to fill I think should be related to the scope of the kind of broader commitments and vision that we have as a Board if we want, you know, support for.

But I think the discussion — so I think this idea of a communications committee is really important. I think the policy committee, again, is very important because we've had a lot of discussion, and I can imagine that both communication and policy would want to have a lot of crosscutting conversations because they're going to be interrelated in the scope of work.

I think the issue, and I'm just speaking for myself at this point, so the focus of discussions that have come up relative to exceptional children, the issues of equity, a number of these, I think, are embedded in some ways in the recommendations from the various reports, evaluation studies of the different centers' work. And also just lack of kind of confirmation.

My sense from Peggy is that NCS pretty much agreed with the recommendations of that report, and statistics on education.

The other centers when you all did the sort of summarization, which makes sense because there are so many recommendations, is how do you kind of get a handle on it, but I didn't necessarily get a sense, but it would be really helpful to know if there were any of the specific recommendations that were made that you all said no, we just really don't think these are valuable.

If that's not the case, then I think a committee, and this is complicated, and kind of gets back to one of the questions Mark had raised earlier, but I had been thinking that if we were able to form maybe a committee, and it might have — this is sounding complicated, but maybe sort of two subgroups within it in terms of what — the issue of what kinds of supports, and understandings of the unfolding of the work of the various centers to address the recommendations, and the visions that they have of which themes around equity, around special need populations, all the variety of kind of buckets if you will in the report could be a valuable group, but the understanding that's going to take time because there's a lot of stuff, right, to do. Could be buckets of standing committees.

And, again, my understanding of the logic of standing committees is that they have the authority to meet outside of this public meeting where we'll never be able to work through the details of complex stuff in the context of a Board meeting like this, but we can have specialized groups that prepare all the necessary research, collaboration, whatever it is, to come and be able to make recommendations, including how the two — the feedback we've gotten from IHEP, and I'll ask for a learning innovation, a cite for their recommendations to be taken up so we're not getting feedback from the public that somehow get lost in the mix could be a way of moving forward.

And again, with the understanding that these committees would have the authorization to invite not as members of these committees, but as advisors to come in who bring specialized knowledge that could help inform whatever the work that we're doing.

So I'm just — I'm going to turn this into a formal recommendation after we discuss this search for the executive director, but I'm just curious for your feedback about this sense of a structure for the Board moving forward.

DR. HARPER: I'm in favor of what you proposed, Carol. It makes perfect sense to me.

DR. LEE: You know, we're going to be the most active NBES Board in years, so we're going to maybe pat ourselves on the back or not. I don't know. Ruth?

DR. TURLEY: I was just giving the thumbs up to say, yes, this is a good plan.

DR. LEE: Okay.

DR. FUCHS: I do too, Carol. I think it's — having subcommittees, and subcommittees able to draw upon others with expertise as consultants is really the only way to go about this to get as much done as I think we would like to get done.

DR. LEE: Great. All right. So why don't we just transition, and we'll do a formal vote on this after the discussion of the search for the new executive director.

So I'm going to just express what I think I understand the position to be, and then Mark, or staff, anybody else, can correct me if I'm wrong.

So it's my understanding that we are authorized — that we have a budget of something like $345,000, something in that range, to support the work of this Board, and that part of those funds can be used to hire an executive director. It's a very weird-sounding position to say that you got an advisory Board, and you're hiring somebody to help you, and you call them an executive director.

But anyway, that's what they seem to have in the charter, or someplace, that they're hiring someone to basically fulfill the role that Ellie and Andrea had done, and take this time to publicly thank Andrea. And she's moved on professionally, but how supportive she had been from the very beginning of this process with all of us. And Ellie has taken her place, and been very wonderful in the transition.

But it's my understanding that the executive director is a position that the Board conducts the search, and hires this person, and identifies the duties that we want this person to fulfill, which is basically to support carrying out the work that the Board has identified.

When I talked to David, the prior chair, and that's why I'm recommending that that committee once we form it meet with David. And he also indicated that he could put us in contact with the former executive director that he hired, or that last Board hired in terms of the scope of the work.

But it seemed to have included things like the work we're talking about in terms of connecting with the stakeholders on the Hill for establishing relationships with the other potential organizations that could be helpful. Our contacts internally with IES that this person would handle all of those kinds of work for carrying out the wishes of the Board.

I think there's a question also in terms of our giving this committee some direction about the scope of the work we want, the qualities, and professional experiences we think that person should have, and also whether this should be a full-time or part-time position.

I'm sort of leaning to thinking that it could be a part-time position except I can't imagine who would take it as a part-time position for the scope of experience that we want.

DR. FUCHS: Carol, if could just — if I could just add a —

DR. LEE: Go ahead.

DR. FUCHS: If I could just — if I could just add to that. Jim, and Dana Hilliard, and I, as part of this subcommittee thinking about the executive director communicated by email, and we're in agreement that we will go forward.

I contacted — I have an email into David Chard, whom I know, and we're going to set up a contact with him, and then after him the — and David Chard was the former chair of the Board, and then after speaking with him the executive director under him.

And just speaking for myself, I think the commonsense way to go here is to use our conversations with David Chard and the former executive directors as a kind of data gathering to get a sense of, as you just said, what seem to be the desirable professional and personal characteristics of an executive director.

And I think to me it kind of makes sense for us to come back to all of you with a very preliminary set of — criteria is too strong a word, but a set of characteristics that we think would make sense as we move forward in trying to find somebody.

And I'm not suggesting that we first conceptualize what it is, or who it is that we want, and then to start the process of trying to find people. I mean, they don't have to be in lock step, but it seems we should all be in agreement, at least in principle, about the kind of person we want as executive director.

So I think it would be helpful to get some feedback from Board members at this stage about what — how you're thinking about the sort of skill set that you think we should be recruiting for. I assume we would have to write a job description.

And then a question — Mark, this may be a question for you, or one of the other staff, and that is technically how does this process work, the process of sending out the call, promoting the position?

I assume the interview process should take place within the committee. I don't see that we would be doing interviewing potential employees at a public meeting, and then that the committee would then make a recommendation for hiring that would be voted on in the public meeting of the Board.

DR. SCHNEIDER: Okay. So Carol, the one thing that you have to understand is that you're hiring a government employee, which means that the hoops that you have to jump through to make this happen are rather consequential, and rather difficult.

You can see Nate is nodding his head because he's two months into this, and he's run his query into all the H.R. problems.

So you have one major advantage, and that is Ellie is the designated official, and she is unbelievably good, incredibly well-organized, and does a lot of the H.R. work us, you know, for the institute.

Right now, I believe, and she could add what's needed, right now we're working with historical records, and practices, to get a job description, which you need, and has to be classified. I mean, it then has to go through H.R., and has to be publicly advertised. I mean, there's just a lot, a lot, a lot of stuff. I mean, we all in IES bear the scars of these processes.

And I keep looking at Nate. He's gotten himself under control. I think that's earlier when we started talking about hiring you could just see like, oh, my God.

So you just have to be — you just have to know that — two things. One is that the hiring process is rather difficult, and you want to be careful about this is designated — we're expecting this to be a half-time job, not a full-time job, and I'm not sure — I mean, you have a lot of ambitions, but a full-time job is expensive, and you have a fixed budget like all of us. So you need to think this out.

But hiring is not simple, and Ellie, fortunately, has a lot of experience, and a lot of scars negotiating this.

DR. LEE: So let me say that one of the things that I have found absolutely fascinating about this whole process of joining this Board, and understanding the expectations of it, is that it's my understanding that we are required to hire this person, right?

I had assumed coming in before I even learned that we were required that we would just have a staff liaison from IES who would support the work of the Board only to find out we're suppsoed to hire somebody to help us.

And the time — and so all of these hoops I can quite imagine on the one hand — and, you know, Mark, I've raised this with you, and I did with Jacob, and with Andrea, I don't think any of us even know the term of our appointments, or when our appointments end, and when and if we're going to be reappointed considering that in theory we were appointed two years ago, and we're just now, you know, getting to meet, that I can easily imagine the hoops that we have to go through to hire someone we're required to hire. Our terms could be over by the time you go through the hoops.

So I just find the whole process to be quite fascinating.

DR. SCHNEIDER: Well, fascinating is a nice word. I would use words like horrifying, you know, words like that.

DR. LEE: And one of the issues relative to the finances of this too, Mark, that you had raised is my understanding that we have a fixed budget of something like 345, or whatever, and that is to pay both the staff position along with the meetings, face-to-face meetings, that we would have.

So again, this will be part of the equation, and we need in budgeting out — like this budget of this 345, whatever it is amount, is for what period starting when through what period of time, so as we are anticipating face-to-face meetings, and what the anticipated costs would be.

I mean, all of these sort of budgetary issues have to be worked out as well, and it's clearly part of the equation as to whether this is full or part-time. But David said he had a full-time. I don't know.

DR. FUCHS: Carol, I got two questions for Mark.

Mark, I was listening carefully to what you were saying before. It sounds like when you said the position was part-time that was more of a recommendation than anything else, and the reason it was a recommendation was because to hire a full-time person is very costly, and the budget is a given.

So my question is am I correct in that assumption? That's my first question.

DR. SCHNEIDER: Yes, right. Absolutely.

DR. FUCHS: And my second question is Carol has mentioned several times that the budget is several hundred thousand. I seem to recall, and I may be wrong, that it was closer to 600,000. Can you tell me whether you know exactly how much our budget is? And also, can we actually see the budget?

DR. SCHNEIDER: I'm not sure what you mean see the budget.

Ellie, are you here?

MS. PELAEZ: Yes. I'm here.

DR. SCHNEIDER: So the number, where is — that number in the budget document, right?

MS. PELAEZ: Yes. It's in the FY-24 President's budget. Uh-hmm.

DR. SCHNEIDER: And what is —

MS. PELAEZ: It's 339,000.

DR. SCHNEIDER: 339,000. That's public — that's a public document?

MS. PELAEZ: Uh-hmm.

DR. SCHNEIDER: So we could obviously — obviously we could send it to you?

MS. PELAEZ: Yep.

DR. FUCHS: Thank you.

MS. SULLIVAN: Is that the President's budget, or is that the omnibus that was passed?

DR. SCHNEIDER: Right, right. No, Caroline, that's the right question. So we'd have to go back and look at what was in the controlling budget, and that's the fiscal — and to answer Carol's, that's a fiscal year.

DR. FUCHS: That's '24. That's not been approp'ed yet.

MS. PELAEZ: Right. Just to be clear — this is Ellie again — it hasn't been appropriated yet. That's what has been recommended I guess you could say, right?

MS. SULLIVAN: And that's the levels of funding, correct, right? So the deal that was reached with '23 funding that's where this is, correct? No?

DR. SCHNEIDER: No. So the '23 level was zero, right? Because there was no executive director.

MS. SULLIVAN: Right. Right.

DR. SCHNEIDER: And right now you have zero dollars. So until you have a budget you have zero dollars for R&D, right?

MS. SULLIVAN: Yeah. That's what I was wondering.

DR. SCHNEIDER: Yeah. I mean, that's something to float you through until they pass the budget in February, but that's — I don't know about that though.

DR. FUCHS: And Mark, not to put you on the spot, but, I mean, Carol did raise in her characteristically diplomatic way the fact that none of us knows how long we're going to serve on the Board, whether we're going to be reappointed. I mean, it seems to me this is a fundamental issue that needs to be addressed almost before anything else.

DR. SCHNEIDER: So to be fair, right, I mean, this is — you're appointed by the White House, and not by the Secretary, and not by me. So all these issues that you're talking about I sympathize, but the fact of the matter is I have no control over them, and no input into that.

So you all — but my understanding at one time didn't you get a memo that said that — So I think Linda is not on the phone anymore, on the call, but she was appointed, and then reappointed, even like months ago because the way the legislation worked was that there are staggered terms, and people get appointed into those existing terms.

So when she got appointed it was — I'm going to — she got appointed in the fall, and her — the seat that she was taking expired, the term of office expired at the end of October. I'm making these dates up, but, I mean, you get the point. So the White House reappointed her already.

So I'm not sure in your letter of appointment it must say when — didn't get a letter that said you're done on 2029, 2028, 2027? You got nothing that looked like that?

DR. FUCHS: I believe I did get the letter sometime ago saying when my term would be — would finish, and then I got an email from somebody saying disregard that, don't take that seriously.

DR. SCHNEIDER: Okay.

DR. FUCHS: So I think that's where I'm at.

DR. SCHNEIDER: Okay. So I would take the letter with dates on it as a good starting point, but the fact of the matter is that — and quite frankly, this is all driven by the White House, your appointments, as you know, your Presidential appointees.

At one time we were also Senate confirmed, which meant that we wouldn't even be having this meeting probably for four more years if we were waiting for you all to be confirmed by the Senate.

But the fact of the matter is — and Carol obviously has good ties in the White House, So I think we need — you need to get that straightened out with the White House.

DR. FUCHS: Thank you.

DR. LEE: I'm sorry. Just a quick response, and then Shaun. Jacob more or less said the same thing to me, this kind of warning. It could be helpful if each of you could maybe just send me the dates that are on the letters that you have received, and then as we start this process then maybe we can talk together with this liaison from the White House. Shaun? Sorry.

DR. HARPER: Yeah. So I'm going to be a bit less diplomatic than Carol, unfortunately. Despite Carol's extraordinary leadership really, and a heroic effort, I'm just going to name it, this doesn't feel like a real thing to me. It feels like we are a low authority, no authority group that's going nowhere fast. It doesn't feel like a real thing, at least to me. That is enormously frustrating I must say.

DR. SCHNEIDER: So I understand your frustration. Welcome to federal government. Got it. So that's number one.

Number two, I mean you are governed, like I am, like Liz, like Matt, by existing legislation. So your activities, and your authorities, are clearly specified. There is some confusion in the charter, but the charter doesn't supercede the law. The law is the governing document. So that is part of the deal. And I understand your frustration, and I totally understand it.

Look, we have spent now five, six, seven hours. I mean, you've met the staff. They're incredibly responsive. They're incredibly professional. They will do what they can to support you, but, I mean, quite frankly, again, they too are limited by the law, right?

They also have full-time day jobs that they have to do. So there's always going to be this tension. I mean, look, that's the reality.

And Shaun, I mean, I understand why you'd be frustrated, but that's the reality.

And I'm sorry, you know, when I start talking like this it drives me crazy because like I know in all of our personal lives we're like get this done, get this done, like I need this done, I need this done, but the fact of the matter is that we have governing authorities and legislation, and we have to abide by them.

DR. LEE: So let me just say, Shaun, that one of the things that's been very helpful for me is the fact that there is a staffer from the White House who has reached out to me, and we've had several conversations, and I have found when issues come up that are kind of conundrums from me he's been very helpful.

So I think as we move through this process then I suspect one will be the sort of legal questions around the whatever boundaries around advocacy work that we do on behalf of IES, you know, relative to policy, so we can get clarity on whatever the bounds of that may be. And also clarity around this position, this executive director position.

Again, I think talking to David is going to be very helpful. They did, as I understand it, hire this person that's a full-time employee, and he said that he would be willing to connect us with that person so we could talk with him directly about what they're experiences in the scope would be, And I think that will be meaningful to do as we come up with a possible job description for this work.

I will also say that it may be because Andrea knew that she was planning to make a transition out of IES that we had some discussions about trying to move forward on the EDD search, so that she would not have to continue to carry that — you know, that work out.

This may be a different situation with Ellie. I don't know. But as Mark said, welcome to the federal government. This is the playing field that we're on right now.

So what I'm going to do —

DR. HARPER: You know, Carol —

DR. LEE: Go ahead.

DR. HARPER: Carol, I want to ease up on this. I will say that there is something about this that is particularly and unusually off. I've done several things with the federal government over the past 20 years, several, several things in collaboration with the White House, and I don't know, like forward movement was not as stalled in those previous engagements.

So I'm just not sure that we can attribute it entirely to, well, this is just a bureaucracy of the federal government because it's unlike other federal government experiences that I've had.

And again, Carol, my problem for sure is not with you. You are doing the very best that you can to lead us, and you're leading us great, but, I think, we all said yes to this because we saw it as an opportunity to contribute, and we're people who are accustomed to accomplishment, and it just feels like it's a low authority, low accomplishment group at this point.

DR. LEE: Well, again, I totally understand, and sympathize with what you're saying. I think we just have to be forward-looking, and create our own vision, and our own action plans, and then if they want to step in and tell us we can't do it, then we respond at that point, but I don't think we should restrict the vision of the kind of support that we want to create.

And at the end of the day, and Shaun, this is precisely, you know, for all of us, and this is why I'm saying I want us to try to figure out a way of how to continue to support the really productive, positive work that IES has done.

You know, I've been in several grants, and I'm very thankful, Liz, for both of them, and to be able to continue to support those efforts on the one hand, and on the other hand for all of us to continue to wrestle, which is not on any person, on any division, et cetera, about the inability that we have as a nation to figure out how to restructure opportunities to support children and adults.

And Peggy was saying she had this meeting around PISA. It absolutely fascinates me that the PISA data comes out, the PEARLS data comes out. We're supposed to be the city on the hill at the top of everything, and no matter what those education statistics outcomes are, we are never in the top.

And yet all the things that they do in other countries, we're talking about the teacher workforce, you know, conditions of the healthcare system, we pay absolutely no attention to the fact empirically that other people who are able to get strong educational outcomes don't do anything like what we do, and it doesn't seem to matter.

And so, Caroline, one of the questions, you know, of the stories we tell somehow has to wrestle with there's some kind of underlying kind of fixed deep-seated beliefs.

I don't know if it's the pull yourselves by your bootstraps, or whatever the case, or education doesn't matter, you know, educators are like factory workers, that is so deep-seated that it's hard to shift people's notion that why in Finland do teachers make the same thing that doctors and engineers make. Doctors and engineers don't make in Finland what they do in the United States. But the fact that there's parity says something, right?

Linda's work in that world in education, these places — teachers are not in classrooms with teachers all day. They have time for professional development, the kind of PESA data that's captured around supports in a system that's — we're not doing that stuff.

And that's why I want to keep planting this little seed around what I think — and James, I'm glad you're here because I still think that NSF and IES need to be collaborating in some way around this.

And I keep coming back again to the work in the neurosciences in supporting this that participation and culture for practices matter. Genes don't determine. Genes follow experience. Genes are inherently malleable, malleability in their expression.

Diversity is what allows humans as a species to survive. So why is it that we have people who are in special education — because you've got to have somebody who is in and out, when these are people you're talking about neuro diversity that have different sets of resources that they use to navigate the world, and because all of this is evolving in dynamic systems with dynamic relations, and we're still doing, you know, causal studies, and quasi experimental designs.

So whatever the error is we don't pay any attention to it because we have to find that there's some fundamental kind of shifts in reorganizations that as we continue to support the work that all of the centers do, you're also able simultaneously to interrogate some questions about some fundamental reconceptualizations as some streamline, you know what I mean, of the work. So that the NCES data shows — you know, gave them NAEP, and they're jumping off the chart. You know what I'm saying.

My speech, sorry for it, but this is part of what drove me to even being interested in trying. It's been an interesting journey.

DR. JONES: Carol, you know, you mentioned PESA, and you're right. When you look at the PESA data, the international comparisons, the United States at best is middling. It's not — it's hardly in the top 50 percent, or top half.

DR. LEE: Right.

DR. JONES: But if you look at the PESA data by states, Massachusetts the last time I looked was sixth in comparison to all countries in the world, and why was it sixth? Well, probably for lots of reasons, but one of which was it tends to be more research, or evidence-based.

And so if someone is trying to make a case to a skeptical staffer on the Hill that research — you know, when we say that research matters, there's a story right there.

DR. LEE: And so again, I think that idea of having a standing committee that may have some subgroups in terms of relationships with the various — the work of the various centers can just become — these kinds of cross conversations, I think, are really important for new ideas to emerge, and this become an important part of the story that that — that in our complex, you know, decentralized system we do have good stories to tell, and can identify the research, including research that I'm sure has come out of the work of IES over the years that inform how Massachusetts thinks about organizing their system not only of education, but healthcare if my memory serves me correct.

So I want to ask for a — I'm probably — I think probably older than anybody here. I just turned 78. I have all this gray hair, and I can be starting to say something, and then completely forget what I was going to say. Hopefully the gray hairs is an excuse for it.

But a resolution, whatever you call it, to form four standing committees of the Board. One that focuses on supporting a communication policy of a collaboration, again, between the Board and IES.

Two, a policy committee that will focus on identifying policy targets to which IES and this Board should respond, and make the recommendations for how we engage in that work.

Three, a standing committee focusing on the hiring of an executive director.

And fourth, a standing committee that will focus on working closely with the various IES centers around responding to both the recommendations of the National Academy of Reports, and transformation targets that the centers themselves have established. And as standing committees they then would have the authority to meet outside of the regular public Board meeting.

It's my understanding, Mark, you can correct me, that either Ellie, or the executive director we hire, would always need to be present at those meetings with the expectations that those committees do not make decisions; they do the research to make recommendations to the Board.

DR. SCHNEIDER: That is my understanding too.

DR. LEE: Okay. So with that, can I get — I don't know if I'm making the whatever you call it, I need a second.

DR. ALBRO: A motion? Do you need a motion?

DR. LEE: Yes. That's what I need is a motion.

DR. JONES: I move — go ahead. Go ahead.

MS. SULLIVAN: I was going to move that we establish four committees, or standing committees, to advise — and we might want to put — if this is going to be a formal motion, what Mark was sharing before, like our primary job is to advise the Director, so maybe do we want to put that in the language as well?

I don't know. Or do you all want to make this — do you all want to make this more broad, or do you want — do you feel that it needs to be a little more specific?

DR. LEE: I think that's a question for Mark. At the end of the day we advise the Director. From that point in terms of these committees, and particularly the one to support the various centers, is a mechanism through which to at the end of the day advise whomever the representations from the various centers.

That's a Mark decision, whatever feedback that the directors, or whomever he appoints to be the sort of liaison to those committees, that was all his decisions.

DR. SCHNEIDER: So I'm not sure if I'm supposed to comment on that. I mean, that's factually correct.

DR. LEE: All right.

MS. SULLIVAN: So Mark, would that be an appropriate motion, or do you need more specifics, or do you think that would work?

DR. SCHNEIDER: Well, I'm just going to — I'm going to harp back to the comparison of earlier when I said like all our good decisions come from hanging out in a bar together as compared to Peggy's much more formal view of how things get done.

Look, I mean, I think — so first of all, I just want to tell you that the Board is a welcome addition because of all the voices, and hopefully all the political energy that you could help mobilize to get us the things that we need, right?

I think that — I some of you, not all of you, I think that there's going to be a lot of give and take, and a lot of, shall we say, evolution as we go forward.

I myself think that — and again, you're going to need advice from people that live in the legal world what is possible within the terms of FACA, et cetera.

My own preference would be because we really — I mean, look, we're embarked on a journey, and I have no idea where it's going to end up, right? I would prefer — And I think if I were you, I would prefer something that's not as specific, and much more narrowing since we really don't know — we really don't know what we're — where we're going.

MS. SULLIVAN: Okay. Well, then why don't we say I move that the Board establish four subcommittees to guide the Board's work, and that's pretty open.

And if you want Carol — if we wanted the motion, I can put the names of the four committees, or we can just move to create the committees, which might give us a little more flexibility if we needed that.

DR. LEE: I think we have to identify the committees in order for them to be constituted, not just that we'll — we already have the capacity to create committees under ESRA.

DR. SCHNEIDER: Okay. So we know where the subcommittees — we know what the target of the subcommittees are, so that's fine. And I'm not sure if we want more sentences then to advise, or to — you know, to gather data, whatever the right words are, in order to help the Board come up with recommendations and advice to the Director. That to me is — anything more than that is, I think, we're looking for trouble.

DR. LEE: Well, that is the essence — I think the essence of this is that committees do not make decisions. Committees are the forms through which data is examined, points of view are brought to bear, and that then come to the Board to make recommendations in the areas that we have identified.

And again, just the complexity of the discussions today just around the future of education report, and all of those recommendations. There's no way we can wrestle through this in a Board meeting.

DR. FUCHS: Yeah. Caroline, I agree. I agree. I would — and I agree with everyone, the general sentiment that it should be more general than specific. And I'm just wondering whether we should limit ourselves by saying four. I mean, are we sure that it's going to be four, and it's not going to be five or six?

I mean, I just don't see the imperative right now to specify a number. I mean, we could say — we could say, you know, that the motion could be about forming subcommittees, four of which are, if you want to name them, and that leaves open the possibility that we add to them later.

DR. LEE: We can certainly do that. I just don't want to lose the four areas of focus, but to say that we are creating a structure of standing committees of the Board of which the original selection will be focused on these four topics.

To me, the importance of creating the standing committees is it allows that group of people to meet and to vet through ideas before we come to Board meetings. So can I get a second?

MS. SULLIVAN: Whoever is taking the minutes, do you want to read back the formal motion? Do we have a Secretary?

DR. LEE: Ellie, are you taking notes that you could —

MS. PELAEZ: No. We have a court reporter on who is taking the minutes.

MS. SULLIVAN: Okay. So it's a transcript, not necessarily minutes.

So all right. I will move that the Board establishes subcommittees to support and advise the work — it's very hard to dance between specific and vague. To advise the work of the Board. Why don't we just leave it wide, wide open? Including standing committees to address communications, policy, research for the executive director, and support of IES centers.

Carol, do we need to go to recommendations, or just — that's pretty vague, but, I think, that would — that hits it.

DR. LEE: I think that's fine.

MS. SULLIVAN: All right.

DR. LEE: Can I get a second? Because this is a big question, Ellie, why don't you call the roll.

MS. PELAEZ: Okay. I can do that.

DR. HARPER: And we're doing yea and nay?

DR. LEE: What did you say?

DR. HARPER: No. I was just going to say the options are yea and nay, or abstain?

DR. LEE: Correct.

DR. HARPER: Okay.

MS. PELAEZ: Dr. Anaya? We know he's not on.

Ms. Hernandez-Legorretta?

MS. LEGORRETA: Yea.

MS. PELAEZ: Okay. Dr. Fuchs?

DR. SCHNEIDER: That would be yea. He gave a thumbs up.

MS. PELAEZ: Oh, he did. Okay. Dr. Gandara?

DR. GANDARA: Yea.

MS. PELAEZ: Mr. Guy?

MR. GUY: Yes. Yea.

MS. PELAEZ: Dr. Harper?

DR. HARPER: Yea.

MS. PELAEZ: Mr. Hilliard?

MR. HILLIARD: Yea.

MS. PELAEZ: Dr. Klasko?

DR. KLASKO: Yea.

MS. PELAEZ: Dr. Lee?

DR. LEE: Yes.

MS. PELAEZ: Dr. Lopez Turley?

DR. TURLEY: Yes.

MS. PELAEZ: Dr. Scott?

DR. SCOTT: Yea.

MS. PELAEZ: Ms. Sullivan?

MS. SULLIVAN: Yea.

MS. PELAEZ: And Dr. Yoshikawa?

(No response.)

DR. LEE: So that's it, right?

MS. PELAEZ: Yes. That's it.

DR. LEE: Okay. Great. Thank you.

MS. PELAEZ: Uh-hmm.

DR. LEE: So we're going to be able to finish early today because I'd rather wait and take tomorrow to talk about both the ESRA, and other policy initiatives that have a timely basis to them in terms of how we're thinking about them. And also at that meeting to discuss a calendar for Board meetings. And I'm thinking that it might be a particularly formal policy committee.

So one, I would say that we had sent out early a request for people to decide on what committee they wanted to work.

I would say that we can send perhaps Ellie, you know, a communication out to the Board members identifying the four subcommittees so that people can decide if we want to stay with the same committee, or another committee. You can be on more than one committee, you know, if you like.

But the two committees, I think maybe three really, that are going to need to meet very quickly will be one, the executive director search, and the other is the group — some combination between the policy group and the communication if we're going to take up Conchita's recommendation about possibly being able to prepare something, small though it may be, by March or April for issues that are on the table coming up.

So you can kind of think about that, you know, tonight, and we'll meet tomorrow morning. And we probably won't need to take the full time for that tomorrow.

I'm praying that Linda's husband is okay, so she'll be able to be on Board of the meeting tomorrow, but she's been gathering a lot of information relative to the ESRA authorization. That will be kind of helpful information, as well as some communications had with a number of stakeholders who are kind of weighing in on that upcoming legislation.

DR. FUCHS: Carol, what time do we meet tomorrow Eastern Standard Time?

DR. LEE: 10:00.

DR. FUCHS: Okay.

DR. LEE: Same time. 10:00 East Coast, 9 o'clock my time. 9:00, 8:00, 7:00 Eastern. Shaun, 7:00 again in the morning. Get a good night's sleep, you all, and Shaun.

DR. HARPER: Maybe that's why I was so cranky today. Maybe.

DR. LEE: We'll make sure we have later times so you'll be in a good mood when you come, right?
All right. Thanks, everyone. I really appreciate everybody's time and effort.

MS. PELAEZ: Thank you, guys. Take care.

(Chorus of goodbyes.)

DR. LEE: Take care, everyone.

(Whereupon, at 3:50 p.m., the meeting in the above-entitled matter was adjourned, to reconvene at 10:00 a.m. the following day, Tuesday, December 5, 2023.)

NATIONAL BOARD OF EDUCATION SCIENCES OPEN PUBLIC VIRTUAL MEETING
Day two proceedings

Suite 206
Heritage Reporting Corporation
1220 L Street, NW
Washington, D.C.

Tuesday,
December 5, 2023

The parties met remotely, pursuant to notice, at 10:00 a.m.

Participants:

Carol Lee, Chair, Northwestern University
Maria de la Concepcion Hernandez Legorreta, Maryland State Department of Education
Linda Darling-Hammond, Learning Point Institute
Douglas Fuchs, Vanderbilt University
Denisa Gandara, University of Texas at Austin
Elmer Guy, Navajo Technical University
Shaun Harper, University of Southern California
Dana Hilliard, Mayor, The Hilltop City
Stephen Klasko, General Catalyst
Ruth Lopez Turley, Rice University
Caroline Sullivan, North Carolina Business Commission for Education
Hirokazu Yoshikawa, New York University
Derrick Scott, Virginia State University

Ex Officio Members:

Mark Schneider, IES
Peggy Carr, IES-NCES
Elizabeth Albro, IES-NCER
Matthew Soldner, IES-NCEE
Nathan Jones, IES-NCSER
Brett Miller, NICHD
Ellie Pelaez, Designated Federal Official

Proceedings
(10:00 a.m.)

DR. ALBRO: It looks like we have a quorum, Carol. Do you want me to start with the attendance?

DR. LEE: Yes. Go ahead.

DR. ALBRO: Okay. Dr. Anaya?
(No response.)
DR. ALBRO: Ms. Hernandez Legorreta?
MS. HERNANDEZ LEGORRETA: Present.
DR. ALBRO: Dr. Darling-Hammond?
DR. DARLING-HAMMOND: Here.
DR. ALBRO: Dr. Fuchs?
(No response.)
DR. ALBRO: Dr. Gandara?
DR. GANDARA: Present.
DR. ALBRO: Mr. Guy?
(No response.)
DR. ALBRO: Dr. Harper?
DR. HARPER: Present.
DR. ALBRO: Mr. Hilliard?
(No response.)
DR. ALBRO: Dr. Klasko?
DR. KLASKO: I'm here.
DR. ALBRO: Dr. Lee?
DR. LEE: Here.
DR. ALBRO: Dr. Lopez Turley?
DR. LOPEZ TURLEY: Here.
DR. ALBRO: Dr. Scott?
DR. SCOTT: Here.
DR. ALBRO: Ms. Sullivan?
MS. SULLIVAN: Here.
DR. ALBRO: And Dr. Yoshikawa?
DR. YOSHIKAWA: Here.
DR. ALBRO: Okay. Dr. Schneider?
DR. SCHNEIDER: Here. Here.
DR. ALBRO: Dr. Carr?
DR. CARR: Here.
DR. ALBRO: Dr. Albro? Here.
Dr. Soldner?
DR. SOLDNER: Here.
DR. ALBRO: Dr. Jones?
DR. JONES: Here.
DR. ALBRO: Mr. Santos?
(No response.)
DR. ALBRO: Dr. Miller?
DR. MILLER: Here.
DR. ALBRO: Mr. Wiatrowski?
(No response.)
DR. ALBRO: And Dr. Moore?
(No response.)

DR. ALBRO: And we have a quorum, Carol.

DR. LEE: All right. So the primary task for today, we said we wanted to discuss ESRA, as well as upcoming policy legislation that we should be attending to and also setting a schedule for our meetings.

I did look last night. I think I sent out, I think I did, the charter relative to standing committees, which indicates in my reading of it that we can establish standing committees. The majority of the members of each standing committee shall be voting members of the Board whose experience is needed for the functioning of the committee. Other members of each standing committee may include experts and scientists in research, statistics, evaluation, or other development who are recognized in their disciplines as highly qualified to represent such disciplines and who are not members of the Board but who may have been recommended to serve by the Commissioner of the appropriate National Education Center and approved by the Board.

So bottom line — where are we, Teams — bottom line, we have the authority to establish subcommittees. We have the authority to include non-Board members as working members of the standing committee. So, based on what we had decided yesterday, we're able to move forward.

So I'm glad that Linda was able to join us today. Hope that your husband's doing better.

And we discussed yesterday — I think this was after you had left, Linda — that we wanted to — that we were establishing a standing policy committee, and we began some discussion of the need to develop some protocols for how as a Board we would communicate with stakeholders, including Congressional. And, again, my reading of the charter is that we do — we are authorized to speak to Congressional representation on behalf of the Board, but whatever that communication is obviously is approved by the Board.

And Mark had raised the question of not — had raised the issue of not only attention to the reauthorization of ESRA but also separate funding policy discussions that were underway. And I know that you and I had had some discussions that we thought would be helpful for the Board in terms of our responsive feedback around some items that are at least under consideration in the ESRA authorization that we may find problematic and would want to address as well.

So maybe — and also, there was some considerations around the timing of any activities that we might engage in relative to how the legislation is slowly or otherwise moving through the appropriate Congressional committees.

So, Linda, if you might want to say something maybe to start our discussion about both ESRA reauthorization, ESRA fiscal funding, other policy initiatives that you're aware of. There were several that Mark had mentioned yesterday that perhaps ought to be in our purview and if there are any particular issues relative to either the reauthorization or the funding that you think we ought to be discussing.

DR. DARLING-HAMMOND: Well, I'm glad to say a couple things, and, obviously, we'll want some guidance around how the NBES has historically engaged with Congress. There is a move, many people here probably know, to get a version of a reauthorization proposal for ESRA out of the Committee like this week, so it's moving very quickly, and then it'll go to the broader — this is the Senate side.

And it does include a number of changes. I've just begun to see the redlines. I'm sure other folks are beginning to see them. There is some proposals that actually deal with NBES itself and with the governance of IES and NCES, including whether the NBES would be appointed by the President or by the Secretary. There's a proposal for that and to reduce the size of NBES from I think we're at 15 down to nine. There's a proposal in the draft to make the Commissioner for NCES appointed by the Director of IES rather than by the President, so that's another kind of major governance proposed shift.

There will be some discussion, I know, and I've already began hearing from folks at the AERA, American Statistical Association, I'm sure others of you are too, about some of those kinds of concerns. And then there'll be a robust conversation, I'm sure, about the definition of evidence-based and what do we mean by adequate evidence. And so these are among the things that this group might want to be engaged around. And the draft that's out there just landed Friday, so people are still, of course, looking at it and figuring out what else might be a major shift from where we've been.

And then, of course, there will be appropriations conversations, you know, and those are always important with respect to the capacity to do the work, the periodicity of the data collections. I know that's got to be on Peggy's mind and everyone else's minds. So those are things that we would want to be able to engage with as well I would think.

And I do not know how the process of NBES engagement with the Congress has worked in the past, so I'd love to hear from people who have experience with that.

DR. LEE: I'm trying to glance back at the charter because I think there is a statement somewhere in here about this Board reporting to the Congress. But, while I'm doing that search, I'd like to get some discussion and feedback.

Also, before I open it up for discussion, Linda, do you have any sense in terms of the Board organizing to provide some feedback on the legislation or elements of the legislation that are being considered about timelines for when it would make sense for us to try to share any information or recommendations that we had? I say that because what had come up in the discussion yesterday was being prepared through the policy committee that we're forming to be able to have something to share by March, but I'm imagining at this point that's going to be too —

DR. DARLING-HAMMOND: That feels a little late.

DR. LEE: Yeah.

DR. DARLING-HAMMOND: If the ambitions of the committee — the help committee are, you know, realized, they're hoping to move much more rapidly. Of course, you know, things can move for a moment and then get stalled again, so one never knows about that, but I would say we should get the committee, the standing committee, comprised as soon as possible so that we can get feedback from the entire Board and be prepared to offer input.

The proposed legislation, yeah, I think it is now available to be looked at, and I can actually probably put something in the chat. I'll look and see what I've —

DR. LEE: I think also too we're going to want to as much as we can have some discussion about what Board members are interested in what subcommittees, but I'm thinking that the policy committee at present is just Conchita and Stephen. And I think, for the work and the timeliness of this, we're going to need some additional people.

I think, Linda, you had indicated you would be willing to serve on that committee as well.

DR. DARLING-HAMMOND: Mm-hmm. Yeah.

DR. LEE: So I'd like to get some feedback in terms of how people are thinking and particularly these three issues of the reduction of the size of the Board, the issue of the NCES director being appointed by the director versus being appointed by the President, as I think is currently the case, correct, Peggy? Right?

DR. SCHNEIDER: That is correct, yeah.

DR. LEE: And any ideas that are popping up as the legislation's evolving around conceptions of evidence-based research.

DR. LOPEZ TURLEY: I just have a question. So only the NCES director was mentioned. None of the others — of the directors of the other centers?

DR. SCHNEIDER: That is correct. So, at the current time, three of the commissioners are appointed by the director, the NCES Commissioner by the President. So, in the existing legislation, proposed legislation, as Carol just noted, the committee has chosen to change the status of the NCES commissioner to a director's appointment.

So just a couple things on that. So there is an internal debate within the Department of Education about that specific recommendation. That fight has been going on for some time. And the Department has recognized the internal division within IES and is itself debating what position to take with regard to that provision. So that is a pretty serious conversation going on at the current time, and you should just be aware that that is going on in the Department and outside the Department also.

So, at the current time — again, at the current time, the legislation calls for changing the appointment process for the NCES commissioner to make it like the other commissioners.

DR. LOPEZ TURLEY: What are the implications or what are the pros and cons? If anybody has reads on this, that would help me.

DR. SCHNEIDER: Well, I mean, so Peggy will give one side and I'll give the other, but we've been through this many times already.

DR. LEE: Peggy, go ahead.

DR. CARR: Well, what's new this time around is that we have the Evidence Act of 2018, which makes it clearer what Congress wants to do with a statistical official, a recognized statistical agency. And this debate that Mark is talking about that just surfaced again a few days ago, the Department made it clear that it wanted to be aligned with the Evidence Act. And so I think that is a clear signal that there's been a shift in the thinking to be more in line with what the statistical community sees and wants to do with this recognized statistical agency.

I would also add that there is a draft regulation, trust regulation, about how to implement the Evidence Act. It's been through several iterations here in the Department and, of course, across the larger federal government, and it is very clear that the independence and the autonomy and the responsibilities of the recognized statistical agency is of the utmost importance.

So I don't think it's as clear as it maybe was earlier in the debate a few years ago. I think legislation has clearly laid a stronger guideline.

DR. SCHNEIDER: I'll say just two points to that. So, clearly, the Congress is the ultimate arbiter of the decision about how to appoint the NCES commissioner. So just as the Evidence Act was passed by Congress, ESRA, the authorization could be passed by Congress and change that status. That is totally within the purview of the Congress. And to say that one act means that they can't act differently is not correct.

The second thing is regulations are at the bottom of the pile of hierarchy of what rules government agencies and government policy, so, obviously, there's the Constitution, treaties, at a level above us, but then there's legislation, and regulations cannot trump legislation.

So there are many things in the regulatory statement, the proposed trust regulations, that contradict ESRA and that are probably contradictory of other practices in government. And we are not the only agency that has many comments about the trust regulations. State, Treasury also had many comments about trying to rein in the trust specs.

DR. CARR: Well, I would add that to —

DR. SCHNEIDER: So — I'm sorry. So, obviously, there's a big debate and you should just know that when you wade into this.

DR. CARR: Well, I would add to the point about other statistical agencies NCES is the third largest statistical agency by a lot of dimensions. Census and the Bureau of Labor Statistics are appointed by the President and they may also — and my recollection are confirmed by the Senate. This is not in the best interests in my opinion, and I've been around the statistical agency and the whole community for decades now. To demote the independence and the autonomy of this agency is not in anyone's best interests to do that.

So this certainly is my opinion, but as Mark has indicated, it's not only my opinion. There are those on the outside of the Department, those who are advocates of the federal statistical agencies that really support this position as well.

And I just would also add that what needs to be considered — you know, Mark and I have been colleagues for, I don't know, 20 years or so, and so our personalities and how we interact is different than what — we don't know who's going to be in these positions in the future. So it should be mutually exclusive from the personalities or the relationships of the people in the position and what they bring to the position. So I think that's an important component. The objectivity of the position should be mutually exclusive from who's in it or who is the — who has the administration, whether, you know, it's one party or another.

DR. SCHNEIDER: So the only thing I would repeat is it's obvious that there's a lot of tension and a lot of debate going on both in IES, in the Department, and wider than that about this decision.

So the only thing I would just say is I would repeat the same warning, right, that this is an intense debate, ongoing. It divides. Even in OMB, there are fights over this. So I would just warn about, you know, going into this with your eyes wide open.

DR. DARLING-HAMMOND: I was going to say I think we've got the sense of the debate, and there is a debate going on in, you know, many places in the research enterprise around this.

I think a question for us as a Board is what is the process by which we want to in general be engaged around the policy work, including this obviously important piece of proposed legislation.

I did put all of the — for those who are speed readers, I did put all the links in the chat, but, obviously, it's not something we can address and resolve around the content of our feedback today. I think the process is what we should try to figure out.

DR. LEE: So I would like to hear from other Board members. I'm raising the question because, as we make decisions about weighing in with information or advice relative to legislation that's arising, we need to be aware as a Board of the full dimensions of those pieces of legislation as they're unfolding, as opposed to just a simple decision. There's ESRA reauthorization coming up and, obviously, we are supportive of that. But, if there are issues that are under consideration that we think are questionable, then we need to be able to weigh in on that.

So I would like to hear from other Board members. This will be one of the issues about how we want to respond to the ESRA legislation, both the reauthorization and the appropriation. Process-wise the Committee is going to come up with a plan to advise us about how to proceed, and, obviously, that's going to have to move forward quickly.

But, at the end of the day, it is this Board that will decide the content and focus of the recommendations we might have. So I would like to hear from other Board members.

DR. KLASKO: Carol, this is Steve Klasko. Does it make sense for us post-Board meeting to get a group together to come up with a position paper of, you know, the four or five major issues and where we stand as a Board and can that be shared or is that just something that we would use as an internal document?

DR. LEE: I think process-wise that would be very appropriate and I think not only for this ESRA legislation but going forward for other pieces of legislation that are coming up that are going to affect IES that we think as a Board we should be able to weigh in.

And, again, I don't have the time at this point to go back, but I'm almost certain that I did read in the charter a responsibility of us to communicate to Congress, that this is not something that's beyond, you know, the purview of the Board to do. Yeah. But I think — but I still would like to get some feedback, some sense of the Board as the policy committee would sort of move forward with the details of this. I think it would be helpful to have some sense. I'm not saying we take a vote or anything at this point, but just some sense of the sort of breadth of thinking that Board members have. I think they call this wait time in classrooms, right?

DR. LOPEZ TURLEY: Well, I had one reaction to the proposal to reduce the size of the Board from 15 to I think nine you said.

DR. LEE: Right.

DR. LOPEZ TURLEY: You know, at first, obviously, there are pros and cons to that. One advantage is that sometimes it's easier to have discussions and make decisions with a slightly smaller group, but then, in thinking about these subcommittees and all the work that needs to be done, you know, it's been helpful to have, you know, three or four people within each subcommittee to get the work done, especially between meetings. My initial reaction is to lean toward keeping it at 15 simply because of the work that needs to be done and because of the structure that we're trying to put into place. But that's just an initial reaction to that proposal.

DR. DARLING-HAMMOND: If I could just add to that, I think there's another question, which is how do you represent the various areas of knowledge and domains of expertise that might need to weigh in on the various decisions. So I think that's another reason to have room to appoint sufficient breadth of Board members.

MR. HILLIARD: Morning, Carol. I agree with both Ruth and Linda. I mean, I manage a nine-member city council here, and there is not a day that goes by when I am not making emergency appointments to subcommittees or committees so that they could meet. I just made two this morning so my appropriations committee and my public works committee could meet, which was important because there was snow coming down.

So nine is very hard because you are constantly moving the coconuts to try to fill in a vacancy or just someone doesn't show up and suddenly you can't legally meet, but you can meet but can't make any decisions, which is, you know, just having coffee and catching up on the gossip in town.

So I think the 15 is important, one, just to legally be able to obtain a quorum and, two, to get like Linda said, that broad spectrum is important so that suddenly you are not tunnel vision on one form of opinion.

DR. HARPER: Yeah, I also agree for all the reasons that have been stated, that 15 is definitely a much more appropriate number than is nine.

DR. YOSHIKAWA: I agree too, but there's also the issue I guess of the appointment structure, right? NBES was set up by some circumstance as a Presidential appointment and it did have confirmation of the Senate that added I would have to say, you know, six to nine months to appointment times. And I'm sure the Presidential nature of the appointment adds to all the White House clearance process, right. So just wondered — want to make sure that that is also part of our discussion here and whether there's been any kind of input that IES itself, Mark or others, have around that issue.

DR. LEE: Is that a question that's arisen in terms of moving back to Senate approval for NBES appointments?

DR. DARLING-HAMMOND: Currently, obviously, there's no Senate approval, but there is White House appointment. And the proposal is to move the appointment I think to the Secretary.

DR. SCHNEIDER: That's correct. And so the closest model that we have that NBES would be moving closer to would be NAGB, the National Assessment Governing Board, which is a Secretarial appointment and in contrast to what's gone on with this Board and appointments and dismissals and appointments and lag times, the NAGB — and Peggy could comment on this — NAGB as a Secretarial appointment is much higher priority. It very rarely has empty seats for very long.

So we actually proposed moving it from Presidential to Secretarial appointment. Any decision — any discussion about going the other way and putting Senatorial confirmation should be dead, right. I mean, as someone noted, it would take — I think, as Hiro noted, it would add six to nine months to the appointment process. So my recommendation and the Department's position is that this should be Secretarial appointments.

DR. LOPEZ TURLEY: And what was the reason behind the recommendation to reduce it to nine?

DR. SCHNEIDER: I have no idea about that. I was shocked by that also.

DR. LOPEZ TURLEY: There was no stated reason?

DR. SCHNEIDER: There is no stated reason. It came as a total shock to everybody in IES.

DR. LOPEZ TURLEY: Okay.

MS. HERNANDEZ LEGORRETA: This is Conchita. I have two points. I agree with everyone on kind of keeping it larger. And also, due to the fact that this Board hasn't met in so long, that there's so much work to get done, that at this point, if it was reduced, it would hinder the work.

The other point I want to mention is the policy subcommittee in my mind has two different purposes. So one is as things come up, so as we're getting a proposal, right? During the legislative cycles and how they work, something can happen in a week, in two days, things happen very, very quickly. So that would be the first part of dealing with these type of things and providing recommendations.

The other part, the kind of if we can do the meetings on the Hill that we talked about in March and April, that would be kind of a stand-alone thing that would really target appropriations regardless because that is something that we need to focus on, regardless of what is happening with ESRA and everything else.

So there are kind of two separate things. So I just wanted to kind of point that out. And we could do, like, a yearly day on the Hill if it is within our purview but also support the things as they come through legislation.

DR. LEE: Caroline?

MS. SULLIVAN: Yeah, I agree on the 15 and I think having all of the sort of diversity of thought and experience is good because especially, like, data and science mean nothing if people don't know how to use it and don't understand it and don't see it. And I think sometimes, you know, people just get focused on what they're doing day to day and aren't thinking of how this is useful. So I think having more advocates on the Board is helpful, along with everything else that people have said.

The one thing about the Presidential appointment, I do wonder because this Board is Presidential, is appointed by the President, does it elevate perhaps some of the work that we do and would it, you know, perhaps bring us a little more — would it sort of elevate our thoughts and our discussions and what we do? Just a thought —

DR. SCHNEIDER: So —

MS. SULLIVAN: — because I get the whole it would be easier the other way, but, number one, we've all been through the Presidential — the White House vetting process. But just wonder if sort of — there had to be a reason why they did it this way in the first place —

DR. SCHNEIDER: So I can speak to —

MS. SULLIVAN: — have it be Presidential appointments. Just curious if anybody knows why that was.

DR. SCHNEIDER: I can speak — right. I can speak to the history on this. So, in 2002, when ESRA was passed, the vision, quite frankly — and maybe this relates something to Shaun said yesterday about, you know, the Board level that we're at. So the thought was that education research and IES was going to end up the size of NIH or NSF. That was the thinking. And if you look at their high-level boards, right, they're Presidential appointees. Many of them were Senate confirmed, but Senate confirmations disappear from almost all of this.

But, if you look at the original legislation and the people still here that come from — you know, the director of NSF, the director of BLS, the director of the Census, and one other. I can't remember what the other office. But these were really, I mean, really high-level people that were designed to make this Board the equivalent of the National Board of Science, for example, which is NSF's, right?

So we never got anywhere near the size of NIH. We never got anywhere near the size of NSF. And so the image of why this Board was going to, you know, have all these ex officio members at the highest level and why this was Presidential appointment and why this was, you know, Senate confirmation was because the image was that NBES was going to be the same thing as the National Board of Science, NBS.

So, obviously, we spend more money now than we did in 2002, but taking inflation into account, you know, whatever the $800 million is in comparison to NSF and NIH, this compared to that, we never got to the size of the other science agencies. So this doesn't mean that your work is not important. It means that the Board was set up with a vision that nobody ever got to, so that's number one.

Number two, I look at because I'm an ex officio member of NAGB. When I was the commissioner of NCS, I also reported to, not — reported information to, not reported to NAGB, they — and they've been very effective, right. They've been extremely effective, and there was one empty seat that was just filled, right? I mean, it's not like it takes forever to get these positions filled.

Now, as to the allure, if you will, of a Presidential appointment to join the Board as compared to a Secretarial appointment, I mean, you'd have to look inside yourselves and answer that question.

DR. LEE: Shaun?

DR. HARPER: Yeah. Mark, that was incredibly helpful historical context.

I think my view is that, certainly in light of my observations yesterday about the pace of our work so far, it feels to me like a Secretarial appointment would remove at least some of the bureaucratic, you know, hurdles.

I also think that, yeah, we just need to move faster, which I think is important. I do agree with Mark, you know, given my other agency experiences, I don't think that our work would — the importance of our work would be diminished by, you know, a Secretarial as opposed to a Presidential appointment.

There was one more thing I wanted to say about it. It's still very early here where I live, so maybe it'll come back to me.

DR. LEE: Well, one question relative to that — and, again, Mark, you may be able to answer that for us — is whether or not the federal regulatory constraints, if you will, that we have would be any different whether we were appointed by the President or by the Secretary. I'm thinking that that would be better, but I don't know.

DR. SCHNEIDER: No, so you're a FACA committee and that's what drives. And you would be a FACA committee also, so — and internally — well, you saw the email we got this morning about this is what you can do. It came from a FACA attorney. So I think until we — I mean, until the — and I want to — this is I think Shaun's point also.

The slow pace was in part because of, quite frankly, some disorganizational issues, not organization, but disorganizational issues right at the beginning. Ellie, I mean, you have one of the best people on the administrative side of IES as the Designated Federal Official. She's amazingly good. I mean, again, you know, I must admit putting her on to supporting you, the Board, was a very selfish — I mean, I agreed to it as very selfish and almost said, no, find somebody else because she's — but you need the Board — I mean you need her right now until you get an exec director.

So I want to apologize because there were obviously many false steps setting it up. But, look, I mean, what you're planning going forward is certainly not slow, right? I mean, there will be inevitable, you know, hiccups along the road. You're all busy people, and Carol is obviously very ambitious about where she wants the Board to weigh in on. But, yeah, I think it's going to speed up.

But I do want to thank you for the comments. I don't think Secretarial versus Presidential appointment will have any effect on your standing and your ability to go to Congress or anything like that. I think that's immaterial, and, again, we have the counterfactual or the counter-example of NAGB, which has always been Secretarial appointment and a very effective advocate for me.

DR. HARPER: Carol?

DR. LEE: Uh-huh.

DR. HARPER: It came back to me. So one of the other observations I wanted to make is in terms of at least my appraisal of White House bandwidth and where we fall in the priority of all of the things that the White House has to do. I think that that is a contributing factor to the slow pace of our work whereas, if it were a Secretarial appointment, it is an Education board, so I would imagine that the Secretary of Education would have — we'd just be a much higher priority to the Department given the specificity of what we do.

DR. LEE: Thank you. Hiro?

DR. YOSHIKAWA: Yeah, just following that. I wanted to kind of — I think, Mark, you implied this, but I wanted to hear this kind of more clearly, which is that we did not start until at least two years into the current administration. And so, if this were a Secretarial appointment, would we have been appointed quicker? And as a comparison, did NAGB function earlier in this administration, which I imagine it did?

DR. SCHNEIDER: So the answer is yes to both parts. I think, yes, it would have — the lag between the election and the appointment of this Board would have been shorter, I believe, as a Secretarial appointment. And, again, I don't think the standing of the committee as a Secretarial versus Presidential appointee would differ at all.

And, actually, I wonder — I don't know because we don't have a counter to this, I wonder if it was a Secretarial appointment if the Secretary would pay more attention to NBES than it does at the current time. Definitely, there would not be the kind of lags that we have.

So remember, at the end of the Trump Administration, despite incredibly hard work by me, by other members and staff, we had three people on the Board. I mean, it just wasn't — it just was not a high enough priority. And I'm just going to mirror what Shaun said.

I mean, the White House has a million things to do or more, and the Board is not — I mean, look how long it took to get you appointed. Look how, you know, at the end of the Trump Administration we had three people appointed. It's just not high enough priority for the White House.

DR. LEE: Derrick?

DR. SCOTT: Good morning. I think Shaun hit it on the head in terms of the priority even for my reappointment. I think we started this probably in June or July, and I don't think there's any end in sight for the reappointment because I don't think it's a priority. And also, backing up a little bit, I definitely think 15 is the way to go for the number just based on the breadth of diversity of this room that we're sitting in right now. I don't want that to be diminished.

And then, yeah, I think if each person asks themself would they have taken this appointment if it was Secretarial versus Presidential, of course, I think I'm definitely honored to say that it was a Presidential appointment, but I would have said yes either way just for the work that we're doing. So I think that's also something to consider as well.

DR. LEE: Thank you. Stephen?

DR. KLASKO: I just have one procedural question. In the previous administration, I think there were three directors. Would that matter? I mean, would each time there was a new cabinet member, DHHS, would they have the opportunity to create a new board?

DR. SCHNEIDER: So, I mean, one of the unfortunate things that's happened is that the Board has, in fact, become more politicized between the end of the Obama Administration, through the Trump Administration, and through the beginning of this Administration. I mean, and so I cannot tell you what, you know, what's going to happen, right? I mean, so one time the norm was that the Board was appointed. It was advising an independent autonomist, non-partisan institute, IES. And it was supposedly, you know, much more expert-oriented, less politically oriented. And, I mean, I don't like what's happened since the end of the Obama Administration to the Board, but that's the fact and, I mean, that's just the way it is.

But, you know, NAGB, just again to keep going back to NAGB, NAGB has not had these kinds of purges that NBES has had, quite frankly.

DR. LEE: Dana —

DR. SCOTT: Can I say one more thing?

DR. LEE: Oh, sure. Go ahead, Derrick, and then Dana.

DR. SCOTT: I just wanted to say — I just want to give a shout out to everybody that made these appointments happen. Looking at the political nature of how hard it is to have a board and we were able to get 15 people in this short a period of time now that I'm on the other side seeing how much it takes, I just want to say you guys did a great job just getting us to where we are right now, and that's a mini-miracle in itself. So I just want to say thank you for that.

DR. LEE: Okay. Dana?

MR. HILLIARD: Thanks, Carol.

Actually, Derrick, that was a great leeway. My only caution and, you know, my service on this Board is I'm periodically going to pull the football flag and throw it on the field and just say, okay, just look at this penalty that might occur, is that right now I am proud to be part of this Board just because of the wide variety of spectrum and service that has been brought to this field from politicos to statisticians to, you know, people that, like I say, are in the mines serving in secondary positions or elementary positions and those that are in collegian positions. It's that all of us are coming to the table with a wide variety of spectrum of view, which means we'll be able to blend those opinions and find hopefully a clear pathway.

That's a lot of hard work, but in the end, when you are on that clear pathway, you're able to reach success. It's the conversation and it's getting there that is the hardest.

My caution with moving forward to perhaps a Secretarial appointment is that we will get narrow in the scope of who comes to the table, and that's what I don't want to happen. There are ways of correcting that, which is that you do defined appointments, which means you then specify who is occupying that specific seat from a specific area.

So, if we are going in that direction, then that's really what I would like to see, is to ensure that, again, this wide variety of who is sitting at the table when we all eventually meet together in person, that that continues because that offers up a difference of opinion as opposed to that narrow scope.

And, again, it's very hard work. As you know, when you cast that net and you get a lot of fish in there, there's a wide variety and it's going to be a lot of hard work to get to a point where we all agree, but the end product is always stronger. It is always stronger.

DR. LEE: Shaun, is your hand up again?

DR. HARPER: Yes, it is. I wanted to weigh in on what Dana just said and actually pose a question. Might we have some authority to offer some guidance on the kinds of people who would be, you know, appropriate prospects for this group if it were a Secretarial appointment? Obviously, not naming individuals but, you know, just sort of naming the kind of diversity that would be important and offer it as guidance?

DR. SCHNEIDER: But that's already in the legislation. There are slots. People, you know — so, I mean, you — so let me suggest one thing. So I think, Shaun, your point is well taken. We should look at that list, right, and make sure that the list — because NAGB has the same thing, right? The seats on the Board are designated for not specific people but specific roles, right, so you might want to consider, like, the roles that need to be presented.

And so just in terms of strategy, I think there's growing consensus — there's clear consensus. I'm sorry, I don't — I mean, there are other people that have their hands up. But, if I were the Board, I would think that the recommendations with regard to nine versus 15, Secretary versus Presidential, that's totally within your bailiwick. And rather than get involved in the short term with fights over this or that, because I have a whole other list of legislative activities that I am going to pose to you for you to consider, but this is a very particular activity, right? It's clearly affecting the Board. And I would recommend that you all come up with a policy statement as soon as possible and get that to the health committee as soon as possible.

No one's going to accuse you of getting involved in any — you know, getting outside your lane or over your skis or whatever. These are your skis. This is really you and I would say you should come to a consensus as soon as possible with regard to nine versus 15, Secretary versus Presidential, following up on Shaun's points about, you know, who's on there, and I would get that to the health committee as soon as possible.

And, again, I don't think there's any — you know, you wouldn't get — there's no act — it's impossible for you to be accused of wandering into places you don't belong. This is where — I mean, this is clearly within your purview, and I would recommend that you move as fast as possible to get your position in front of the health committee while they're still debating and working on the bill.

DR. LEE: Stephen?

DR. KLASKO: Yeah. So, I mean, to that point, Mark, I think there seems to be consensus about the 15 rule.

And just to follow up on Dana's piece, though, I know in my situation I was president of a university and health system and was not allowed to do a lot of things and had been offered, you know, certain cabinet appointments.

It's very hard for a Board to say no to a Presidential appointment and the only question we'd have to ask is whether or not it would affect the group that we get or that's allowed to do this, and you would know better than I.

But just to follow up on Dana's piece, there's such an important piece in getting this incredibly diverse group that we've got, I don't really care personally, from my own personal point of view, but I'm not sure that my board would have been such a terrible battle if it was, you know, Secretary Becerra, for example, that was, you know, asking me to do it.

DR. LEE: Derrick, is your hand up again?

DR. SCOTT: It's not. My bad.

DR. LEE: Okay. So what I'm hearing is, one, I totally agree, Mark, that the reason I wanted to have this discussion today that we need to create whatever we want to communicate relative to ask for reauthorization. I want a very quick turnaround because of how it's moving.

I wanted to have also the discussion because, and I think this will be an issue going forward in terms of the work of the Policy Committee, is making sure that we examine as closely as possible any piece of legislation that's under consideration that we think we want to respond to so there's clarity about the expectation isn't just, yes, we support it or we don't support it but that there may be particular features in any legislation that's emerging around which we want to communicate our position and thinking.

So I'm hearing a relative consensus around the argument for maintaining 15 versus nine, and the rationale for that will be in the transcript. I think that makes perfect sense.

I'm not sure that we have consensus on the appointment moving from a Presidential appointment to one by the Secretary of Education, but what we may want to do is to create a statement about the sort of pros and cons of either form of appointment with particular emphasis on the representation, the diversity of representation, and in so doing, the committee could look at the categories, as Mark has indicated, that are already, you know, represented to determine whether or not we think that's appropriate or there should be some, you know, additional considerations of categories of representation.

What we have not had any more discussion on is the question of the NCES — I'm still working with all your acronyms here — appointment being made by the director versus the White House, and I don't think it's inappropriate for us to have a position.

DR. DARLING-HAMMOND: Well, I will maybe speak for some in the both statistical and research communities who are strongly in favor of maintaining the Presidential appointment for NCES commissioner. I'm worried that the stability and status of the statistical enterprise would be harmed if it were changed to a director appointment, and I know there's a lot of activity going on to make that point of view clear to the Congress from folks in those associations.

DR. LEE: So is part of the logic about a unique sort of status and function, if you will, that NCES has that's different from the other centers within the IES purview?

DR. DARLING-HAMMOND: Well, yeah. One part of the argument is that, you know, when the — I'm sorry. I hear somebody else was about to chime in. Let me just get one sentence out and I'll pass the ball.

One of the first reasons for any federal role in education was statistical, was to maintain the information that is needed about the educational system. So it is a particular — we rely on it in a particular way more than the coming and going of other centers or labs or other components.

Somebody else was about to speak. I'm sorry we bumped into each other.

DR. LEE: Actually, it was me.

DR. DARLING-HAMMOND: Oh.

DR. LEE: And it was similar to what you just said, and that is the sense of its relationship to other statistical agencies of the government having a function that seems to me to be a little different than the nature of the other centers funding research —

DR. DARLING-HAMMOND: And in a way, it's sort of like the Census and other things we do that, you know, allow us to know what is going on in the world on a regular and reliable way on which we build a lot of other policy, and so the importance of that infrastructure is part of the argument, I think.

DR. HARPER: Linda, I think that's very helpful. I guess I wanted to first respond to the question of the Presidential appointment. Does it matter who the President is? I mean, I think we've seen in recent years perhaps the up and down with the valuing of science and knowledge. I wonder about that.

And then, secondly, I am persuaded by your point about I imagine my interpretation as being at the table with the other, you know, statistical agencies. I think that's important. So would that be totally lost if this group were a Secretarial versus a White House appointment?

DR. DARLING-HAMMOND: I was not speaking to this group. I was speaking to the appointment of the NCES commissioner.

DR. HARPER: Oh, the Commissioner. Got it.

DR. DARLING-HAMMOND: Yeah.

DR. HARPER: Sorry. Sorry.

DR. LEE: But I will say in terms of the way in which who's in the Presidential office impacts all of this work because the President's going to appoint the Secretary of Education. So whoever the Secretary is going to be following, if you will, the sort of lead and philosophical orientation of the President. So that aspect of it, I think, won't matter in terms of whether it's Presidential or Secretarial appointment for NBES board members.

Dana?

MR. HILLIARD: Carol, you made my statement. I think both of us, we can both agree that we've experienced both sides of the spectrum with both a President and a Secretary of Education. That could, you know, completely change the dynamics of this Board, so well said.

DR. LEE: Hiro?

DR. YOSHIKAWA: Can I just clarify? I think, for the NCES commissioner, the question has been whether it's a Presidential appointment or a director of IES appointment.

DR. LEE: Correct.

DR. YOSHIKAWA: Not Secretarial versus President, right? Just want to clarify that that's —

DR. LEE: Correct.

DR. DARLING-HAMMOND: That's correct.

DR. LEE: Correct.

DR. YOSHIKAWA: That's the conflict on which we're currently — that we're talking about, right, and that what's in the language proposed right now is that it would shift to the director? Is that what's in the language right now? Sorry. I didn't do my speed reading.

DR. LEE: That's correct.

DR. SCHNEIDER: Correct, do we change it from a Presidential to a director's appointment. Correct.

DR. DARLING-HAMMOND: And I —

DR. LEE: Peggy? I'm sorry.

DR. DARLING-HAMMOND: Go right ahead.

DR. LEE: Linda, were you going to respond to Hiro?

DR. DARLING-HAMMOND: No, I think we got that clarified, so —

DR. LEE: Yeah. All right. Peggy?

DR. CARR: I was just going to point out that the term that I'm in is a six-year appointment for a reason so that it transcends administrations. And that is the case for Mark's appointment as well. Mark can speak for himself.

And Mark and I are from different administrations. I mean, he's from the Trump Administration; I'm from the Biden Administration. But therein lies the reason why six years work. I will stay in this position even if the Democrats don't win. And Mark has been in this position even though the Republicans are no longer in office. And that's the way I think it remains independent and the autonomy is protected.

But, if the IES director is allowed, appointed by whoever, to also appoint the NCES commissioner, there in the value of having independence goes — or the autonomy then is at greater risk. Mark should jump in. He knows this as well as I. We've both been around this rodeo quite a bit here, but that is the reason why I believe it is set up the way that it is. So changing it would threaten that independence.

DR. SCHNEIDER: So just to note, every commissioner has a secure appointment for the exact same reason that Peggy just noted, so that it's trying to insulate the IES from the vagaries of Presidential appointments and changing political. So the six-year term is for every commissioner.

So the reason — so we have beaten around, like — okay, so the independence is critically important. But remember, IES is an independent science agency and we have all kinds of protections also, so it's not like we're throwing NCES into OPPD, the policy wing of the department, which would be terrible. But the relationship between a statistical agency and a science agency is actually rather — it can be rather challenging, shall we say.

The problem in the existing relationship is the lines of authority and the lines of reporting, which are pretty well deep into the ESRA, are difficult and that's fundamentally — I recommending the changing of the appointment from Secretary to — I'm sorry, from Presidential to director is really in terms of helping this organization move forward in a more united away. So that's my position on this. And remember, I was both NCES commissioner and IES director, the only person on this entire planet who served in both of these positions.

So my view — when I was in Peggy's position, I would have fought like crazy to keep the — I did — to keep the Presidential appointee. I think I was the last Senate-confirmed commissioner.

But looking from after six years of managing the science agency, of which NCES is a part, but so is NCEE and so is NCER and so is NCSER, and NCEE has all kinds of authorities and responsibilities based on the Evidence Act. And, I mean, by some level, we could just say — and there is a whole evidence community and evaluation community and that is the commissioner of the evaluation officer, which also has status in the Evidence Act. So, I mean, on one level, we could say, well, he should be a Presidential appointee and entitled to all the protections, the protections that supposedly go with NCES.

Anyway, I mean, look, we can talk about this forever. We have talked about this forever. There's just an internal disagreement and I'm just telling you that, you know, as you wade into this thing you should be careful about the degrees that this is being played out or the different fields this is being played out in.

DR. LEE: So what is the downside of our weighing in?

DR. SCHNEIDER: Oh, I mean, look, you have the freedom and the opportunity to weigh in and you should do what you want. I'm just telling you that it's not a simple world. It's a politically charged world. I mean, there are people weighing in all the time. So I'm not dissuading you. I'm just saying that this is a complicated issue.

DR. LEE: As is everything we're dealing with, to be sure.
Peggy and then Caroline.

DR. CARR: Well, I think there's a lot of pros of what Mark is saying about a scientific agency, and, in fact, I think OMB has taken to heart that argument that when a statistical agency is embedded in a science agency that it does give you reason for pause.

But there are pros and there are also cons. So we talked about NPSAS yesterday and whether NPSAS should have been stopped or not stopped. I'm not sure. I wasn't in this position at the time. I'm not sure that would have been the position I would have taken. Perhaps I would have considered other factors. Not that it wasn't important to balance priorities with budgets, but I'm not sure I would have landed in the same space, and that is, I think, what we are trying to protect.

The opinion of whoever's in this position should be the opinion that is paramount when decisions are made about statistics, about what should be collected, when it should be collected, what the priorities should be with input from stakeholders clearly. So there are pros and cons, and as you consider what your position is, I would strongly encourage you to consider both of these points.

DR. LEE: Caroline and then Hiro. You're on mute, Caroline.

MS. SULLIVAN: On this proposal, Mark, would the appointments that you're — would they be concurrent or would it be staggered, right, because I would think you would not want both positions to be on the same cycle, right? I mean, I would think the whole point of being independent and trying to stay away from politics, if you did it that way, you wouldn't get that. You wouldn't get that sort of diversity of thought maybe.

DR. SCHNEIDER: So the only person — when I showed up, the NCER Commissioner was NP and NCES was filled. NCE was empty, right? So I appointed those two commissioners, right, and NCSER was occupied and that was — I don't know if you — Joan McLaughlin was the Commissioner for NCSER, and she was doing a fine job and she stayed until she retired.

MS. SULLIVAN: Well, now my question is, if this is going to change from both Presidential appointments, I think there needs to be some guidance on when those appointments are made because — you know, versus just — I mean, I think, if this is going to be a change, which, you know, to what Peggy was saying is impactful, I think we have to think through if it is the will of the Board to make this change how those —

DR. SCHNEIDER: It's the will of the Board to recommend —

MS. SULLIVAN: — how that works.

DR. SCHNEIDER: Right. It's a recommendation. You're not making the change just to be clear. So —

MS. SULLIVAN: But, I mean, you know, if we're going to communicate with the committee that's doing the reauthorization, I think it would be good to give them our ideas about guidance on how this would work.

DR. SCHNEIDER: Yeah. Look, I mean, so all I can tell you again — and I'm just repeating — when the open seats were appointed by me, the commissioners that were in place stayed in place. I mean, so I don't think — look, I mean, it all depends on —

MS. SULLIVAN: No, I'm just talking about six-year terms, not about whether when there's a change in the director do you do that. I was just wondering, you know, how like in boards oftentimes, you know, people are serving different lengths at different times. That's what I'm trying to say.

DR. SCHNEIDER: Right, but —

MS. SULLIVAN: It has nothing to do with whether the Commission — whether the director can fire people or not. It has to do with are they going to be appointed together, right? So the President appoints a director and then would the direct — for six years and then would — you see what I'm saying? Because are we going to start with Peggy's term?

DR. ALBRO: NCES has a term-limited appointment that starts in January of — or, I'm sorry, in June of a year and then goes for six years.

MS. SULLIVAN: Right.

DR. ALBRO: So Peggy is one of the very first people who started her term in June of 2021, and then it goes through June of 2027 regardless of who the director is.

DR. SCHNEIDER: Correct. And that is actually the practice with all the other commissioners. They get their six-year appointments. In contrast to Peggy, they don't have a start date and an end date. So, when I became commissioner of NCES in 2004 I think it was, my term had three years remaining and that's what I got. I got a three-year term.

MS. SULLIVAN: Right.

DR. SCHNEIDER: Right? So that is — BLS has the same thing, but most other statistical agencies don't have that fixed term start on day one, end on day plus six, right.

MS. SULLIVAN: So it's more like Senate appointments when there's a vacancy in the Senate versus some of the other boards where it is you get six years when you start. Correct. Thank you. Sorry for the —

DR. SCHNEIDER: Right. And —

MS. SULLIVAN: But I think we needed clarification on that because I think that has to do with consistency and if you're trying to get politics out of it that that would have repercussions.

DR. LEE: Elmer. I think you're on mute, Elmer.

MR. GUY: Thank you. I was just saying the Presidential appointment I think is very crucial, that we should maintain that. To me, there's no guarantee that the Secretary will make appointments any sooner. I've seen Secretaries where they did not make appointments readily. And perhaps a recommendation could be that the appointments could be expedited. And I think, when a vacancy occurs, the Board, this Board, should make that recommendation to the President that those vacancies be filled. Maybe that's what our recommendation should be. That's what I was just kind of thinking. Thank you.

DR. LEE: So I think what I'm hearing is our recommendation to the Policy Committee, one would be to meet immediately. And I think it would be helpful to find out today — I'm assuming that Conchita and Stephen want to remain and that Linda has indicated her willingness to work with that committee and to find out if there are any other members who would want to work with them, because their work right now is going to be the most immediate in terms of the turnaround, and that you all would create a draft of a statement that will come back to the Board.

One challenge here is going to be — how much lead time do we need for public meetings in terms of the notice?

MS. PELAEZ: This is Ellie. Fifteen days.

DR. DARLING-HAMMOND: And do you have to post the substance of what you're going to discuss or just the topic? In other words, if we —

MS. PELAEZ: I think it's just the topic, the agenda. Yeah.

DR. DARLING-HAMMOND: If we knew that 15 days from today or tomorrow — I guess, from tomorrow, we wanted to meet, we could post that tomorrow and then have a conversation that was —

MS. PELAEZ: I don't think it could get out that fast, Linda. There's an internal review process and it has to go over to the Federal Register, so —

DR. DARLING-HAMMOND: So the timing of the subcommittee, I mean, it could get held up, but right now it's much faster than that. Nonetheless, you know, we probably ought to figure out how to post something and have the conversation and bring some recommendations back quickly.

DR. LEE: So, in terms of the agent — the process, because the craziness of that process is why we didn't meet back in November, two things. One, I think it will be helpful in the general scheme of things to create a standing rotation for our meetings and we also have standing topical areas in terms of Board reports, blah, blah, blah, that are sufficiently generic that we could just get this out without having to run through this every month, you know, jumping through hoops to announce a meeting.

As a way of generally moving forward, for the purpose of this, I'm assuming that the recommendation that would come from the Policy Committee would, in fact, need to be publicly approved in a Board meeting because of the consequence of it, right? Go ahead, Linda.

DR. DARLING-HAMMOND: Just ask a question on process. We're talking about the very consensual area about our views — our shared views of maintaining a 15-person board. Mark made the point that we should get that written down and over to the committee ASAP. So there must be some way that we can make a judgment about something like that today and then act on it, rather than having to schedule another meeting and go through another set of hoops. So maybe we could get some guidance on that.

DR. LEE: So I think that's certainly something that we can make the decision on today. I think we could also decide — part of what I'm hearing — and you correct me if I'm wrong — that around the two other questions, that is, the appointment of NBES members either by the Secretary of Education or the White House and the question of the appointment of the director of NCES by either the President, the White House or the IES director, that we could, in pulling from the transcript, create a document that identifies what we see as the pros and cons of each decision which we, in fact, have discussed today, that would be reflected in the — you know what I mean, in the transcript that the Policy Committee could create such that we could then create the document that we would approve in terms of the sort of focus and content of it to be able to share that with the health —

DR. DARLING-HAMMOND: The health committee, yeah.

DR. LEE: Senator Sanders' committee.

So I think there are two ways to move. That would be one way, which would allow us to move in a timely manner in the next week to get some statement out from the Board and a decision that we could make today relative to that.

And the other would be — I'm newer to all of this probably than anybody on this Board, so it's quite ironic I would end up being the Chair. But it has been helpful to me in being able to have conversations with the White House staff-person who's weighed in as we, you know, work through challenges of indicating that we need to be able to make a fast move if we needed to come back to meet to vote. And I'm sure that there's some mechanisms within the White House to facilitate whatever the hoops we have to jump through to get the announcement publicly made.

Conchita?

MS. HERNANDEZ LEGORRETA: I don't know if this is within our purview, but considering kind of the time we had at the end of the meeting yesterday, would it be within our purview to use this meeting when we've dealt with everything else or anything else that we need to deal with to finalize what this document is so that we don't have to meet again and don't have to put it out.

Stephen and I, we've been, like, taking notes and we have, like, a little template going. So would it be within the purview to do that? And I don't know how, you know, based on what was shared on the agenda, so if there's some guidance on, is that something we can do to get it out as soon as possible?

DR. LEE: We certainly could do it because the reporting of the Policy Committee is on the agenda, so absolutely. That actually makes sense. We actually have hours left to meet.
Hiro?

DR. YOSHIKAWA: Yeah, I was thinking further we could — just a friendly amendment to what you proposed, Carol, is that we could if — I think all three of these decisions are kind of binary, right, so rather than actually going through a document that raises the pros and cons of each side, if these are simple votes that would shorten the process, it feels like if there are simply three positions on the language which have to do with the number of members of the NBES, how the NBES is appointed and how the director of NCES is appointed, then it doesn't seem like we need to actually describe all the pros and cons, which would be a lot of work. Might take up the rest of three-and-a-half hours, but yeah. So, if it's a simpler — we simply want to come to a decision on these three, I would suggest we just maybe call each of those to a vote after we feel like everyone's had the chance to express their opinion or ask any last questions. We've had a pretty good discussion, and then we could just go to making those recommendations and that would be very simple language to pass on to the committee.

DR. DARLING-HAMMOND: I think that makes a lot of sense. I was wondering about that myself, whether we could just figure out how to have a vote and then we will have a position and then we can write it up.

DR. LEE: Sounds great. Great.
Dana?

MR. HILLIARD: I would agree with Hiro. I think we have consensus on at least one, which is the size of the membership, and I'm happy to make that motion if need be, and I think we can at least remove that item from the agenda.

The other two I think we're still navigating our way through a little bit. I don't know if we have quite reached a comfort level, but we know we're not going to have consensus in everything, although I'm a big consensus guy. I love battling through the weeds to reach consensus, but I like the game of getting there too. So just shoot me now if we're getting in the weeds too much because I'm a go-through-the-weeds guy.

DR. LEE: Well, I think that makes sense. I think, if we can have three different motions on the three different questions, I agree on the first one. I think we will have total consensus and to see what the vote — the split of the vote may look like on the other two, and that can inform whether we've either taken a position or not or whether we need to go through a process of just trying to construct an argument. We could, you know, put up a little Google thing to share and to actually just create the language ourselves right now if it turns out we need to do the pro and con on the others.
Linda?

DR. DARLING-HAMMOND: I wonder, was Conchita in front of me or — it looked like her hand was up sooner than mine. No? Okay. Did you intend for your hand to be up? I just figured out to work that, and it just —

MS. LEGORETTA: That's okay.

DR. DARLING-HAMMOND: So I think it would be good to start with the size of the committee. We have a lot of consensus on that. Have a discussion about each of the others.

If we don't have an emerging consensus, I'm not sure it's going to be helpful to have a vote that, you know, just communicates that we are — you know, certainly, we don't have a lot of time to really dig into these issues, so I think our best bet is to weigh in where we develop a consensus and communicate that consensus to the committee, acknowledging that — I mean, we might want to take a vote if it sounds like maybe there's an emerging consensus. But, if we're, like, feeling like we just don't have enough information, then I don't want to — I don't know that people need to be forced to take a vote if they're feeling like they just would have needed more time and input. But I like the process that Hiro laid out, and I think we could decide which are the places where we have enough consensus to vote and communicate —

DR. LEE: I think the value of voting on all three questions is to determine on the last two —

DR. DARLING-HAMMOND: Whether there is —

DR. LEE: — how much consensus there is or are we actually very evenly divided, in which case we can decide either we won't weigh in at all —

DR. DARLING-HAMMOND: Mm-hum.

DR. LEE: — or we may try to just capture the — just at this point it would be sufficient, even in bullet points, about, again, the pros and cons on either.

DR. DARLING-HAMMOND: Having worked with a lot of people in the Congress, they'll typically prefer a decision to a list of pros and cons.

DR. LEE: Well, you know, the issue I think is that these are not simple questions at all.

DR. DARLING-HAMMOND: They're not simple questions. That is right.

DR. LEE: And, therefore, even if they don't listen, it seems to me there's some value in articulating what the pros and cons and the opportunities and constraints of a position on either side of the question might be, even if they ignore us, which they probably will.

DR. DARLING-HAMMOND: I see that Dana's got his hand up and then Mark.

DR. LEE: Mark?

DR. DARLING-HAMMOND: I'll get mine down.

DR. SCHNEIDER: Okay. So we —

DR. LEE: Dana?

DR. SCHNEIDER: Dana, you want your hand down or did you want to say something?

MR. HILLIARD: No, I'm good. I'm sorry. I must have pushed the wrong button, so I'm good. Thank you.

DR. SCHNEIDER: All right. So somehow we've ended up talking about these three things. But Linda at the very beginning had another thing that was critically important, and that's the definition of "evidence." So there's a paragraph in the proposed legislation that says what evidence is, and there's a giant internal fight about trying to get that clarified.

So, for example, in one paragraph, it talks about statistical significance, which the researchers among us know that that is an ongoing and evolving debate and to solidify or — what's the right word — I guess solidify the words "statistical significance" in legislation from our perspective is dangerous because it locks us into P equals .05 and a hundred-year-old — more than a hundred-year-old now standard that most people are abandoning.

There is a strong endorsement of RCTs in it, and it talks about the levels of evidence in trying to tie this IES to Every Student Succeeds Act, and that whole paragraph is worth consideration. I don't know if you have the time or the energy to do this, but there is a big effort to either change that definition or strike the entire paragraph.

It's an extremely dangerous paragraph for IES and it would affect work that, for example, the research centers support. And so that paragraph is dangerous and there is a considerable effort — this goes right to the heart of what IES is about, and, therefore, it affects the way the Board should think.

The last one, which seems to have disappeared, and Linda didn't mention this, but there has been an enormous push over the last year and a half to create an — I mentioned this yesterday — ARPA Ed. And the health committee dropped it, but it's not dead yet. There's still some windows to come, and for me, ARPA Ed has always been NCADE, the National Center of Advanced Development Education, has always been one of the highest priorities and it's been one of the highest priorities for the Department and, when we get their attention, often the White House.

So this is an opportunity to modernize IES as a force in the — a major force in the education sciences R&D world, and it would be consistent with the ARPA — the movement towards developing ARPAs in other agencies. But the Department — I'm sorry — the Committee has chosen not to pursue it.

So the Department — IES, the Department, others are trying to get NCADE back onto the discussion in ESRA. Whether or not that succeeds, we don't know. There's some backup plans, for example, to create instead of ARPA Ed as a separate center to create a designated program of innovative work in IES. It would be a program that could get a separate appropriation as compared to a center, which could also have its own appropriation. But I don't want you to neglect the possibility or the importance of having an ARPA Ed. That's number one.

Number two, on your list of legislations, I'm expanding it beyond ESRA, so I mentioned yesterday in the House there's the NEED Act by Bonamici and Fitzpatrick. That would create NCADE. That's in the House, not in the Senate. It's not been introduced in the Senate yet, but, I mean, one possible thing that could happen is that the provisions of the NEED Act could just be picked up and put into ESRA in joint conference committee.

Well, but anyway, but I think you need to know more about the NEED Act because that's critically important to the function of IES and the future of IES. And I mentioned a couple times yesterday about the importance of SLDS, the State Longitudinal Data Systems. So the NEED Act has two titles in it. The first one would create NCADE and the second one would be to redefine and increase the role of SLDS, the State Longitudinal Data System, to make it — it too is an old system that needs to be revisited. So we need SLDS V2 and we need hundreds of millions of dollars ultimately that would go to the states to help them build a more modern data structure.

So, again, those are further out, but they're not infinitely further out. And I just wanted — I mean, NCADE, for example, could be in the same list of your three things, right? I mean, it's complicated. There's a lot of background information on this. I've written about it many times.

But I don't want you — I mean, like you — if you can get the three done, that's plenty of work, but I'm just putting on your agenda, on your horizon, NCADE, NEED Act, SLDS as critically important things. And I don't know if you have the bandwidth to look at the evidence definition and weigh in on that, but that is really important.

DR. DARLING-HAMMOND: Yeah, that —

DR. GANDARA: Can you provide us the House number for the NEED Act if you have it off the top of your head?

DR. SCHNEIDER: I'm sorry, excuse me?

DR. GANDARA: Oh, can you provide the House number for the NEED Act if you have it off the top of your head?

DR. SCHNEIDER: I don't have it on the top of my head.

DR. GANDARA: Okay. I'll look it up. No worries.

DR. LEE: Okay. Linda and then Denisa.

DR. DARLING-HAMMOND: Yeah. I was just going to say that I do agree on the definition of "evidence-based" being problematic and that if we — I don't know if it's an easy enough thing to talk about today because it sounds like our strategy for the first input on ESRA would be to see what we can determine today and then communicate right away.

The NEED Act, if we get ourselves on a regular meeting schedule and we have our policy committee, we could start to work on that question perhaps after today or begin to talk about it today but not try to make decisions about it today. I think that seems like a lot to do in the little bit of time we have, but it would be good.

And I think somebody was good enough. Otto put the definition of "evidence-based" in the chat so everybody can see it. You know, it shows how little understanding of the world of research the Congress can easily have when it says something like a single study with a statistically significant PO 5 effect. And then there's some language about capable of causal inference. That will cause people to have a debate about what that will mean, particularly RCTs, et cetera.

So I wonder what the impact of simply referring or recommending that that paragraph be dropped would be, but I do think that it would be good to weigh in on the problematics and the constraints that could be posed by a fairly simplistic definition.

DR. SCHNEIDER: So the best strategy is the removal of it, but whether or not we can get that totally removed, you know, I mean, that's a political issue.

DR. DARLING-HAMMOND: And then there would be the question if the judgment is no, we can't remove it, then there'll be a lot of back-and-forth about what it should be and so on. I'm not sure whether we can get to alternative language. But I do agree with you, Mark, that it would be good for us if we can settle those first issues to get to this one also because it is very important.

DR. SCHNEIDER: But, actually, I think the definition of "evidence-based" is actually critical. I mean, it may be the most critical thing for this Board to address actually.

DR. DARLING-HAMMOND: Yeah. It's important. I totally agree.

DR. LEE: Well, I agree. I think that we should because this — all the big issues related to this reauthorization that we think we need to address in some way, I think we should try to do it in one statement because the timing in responding to this question about what constitutes evidence-based research is within the same time frame as the other three questions that we were raising.

So I do agree that we should weigh in on those and that there may be some — I'm assuming that there are other groups that are responding to this question, particularly groups, organizations of statisticians that may have statements or documents that we could somehow draw from, including their support of such statements being, you know, warnings for why to pay attention, so —

DR. SCHNEIDER: Linda made reference — I'm sorry. Excuse me. Linda made reference to other groups that are also equally upset about this definition.

DR. LEE: Right. Right. So I've got Linda, Denisa, and Hiro.

DR. DARLING-HAMMOND: Oh, I just need to take my hand down for now.

DR. LEE: Okay. Denisa? You're on mute, dear. You're on mute.

DR. SCHNEIDER: She's really not.

DR. LEE: You're still on mute.

DR. DARLING-HAMMOND: Try saying something. I guess she still is on mute. Yeah. Why don't we go to Hiro and come back to Denisa.

DR. LEE: Hiro, and then, Denisa, we'll come back while you're working through.

DR. YOSHIKAWA: Okay. So I like the idea of perhaps first possibly moving to a vote on whether to strike the paragraph, and then it might be a longer process if there's any kind of process that will address other language.

But I did want to check in on, you know, how the phrase "evidence-based" is used elsewhere in the language. I imagine that might not be that it's probably coming up in multiple parts of the language. But it does seem to me that that paragraph is actually harmful, so it's not just about something that is kind of neutral and we might have an opinion on wording but that it may be important to try to first vote to strike that language entirely, that paragraph. And then, if there's a longer process on alternative wording, we could be involved, but maybe that's the work of the Policy Committee, but it feels pretty urgent.

If there is a consensus that that is a paragraph that actually causes more harm than good, then that's probably worth a vote in and of itself.

DR. SCHNEIDER: So, Matt, do you want to add — I'm sorry, Hiro.

So Matt is the evidence guru in IES. I don't know if you want to add anything to this.

DR. SOLDNER: Just by way of information because that's what our role is, to give you information on which to make the possible decision, right.

So that phrase, "evidence-based" appears throughout the bill, with the exception of the part of the bill that deals with NCES. It does not appear there but affects all other parts of IES. There are related definitions in the Act or the proposed legislation that might be relevant for you to consider. So, if you have access to the redline that Linda shared, there's some longstanding language there about scientifically valid research which may be of interest to you. That's what I think I'd say for now.

DR. LEE: And is that language consistent with this paragraph?

DR. SOLDNER: Scientifically valid research, the definition proposed is much broader than the language in the evidence-based definition. And a version of it appears in ESRA and has been guiding our work for 20-plus years.

DR. LEE: All right. Denisa, you back on?

DR. GANDARA: I don't know. Can you hear me now?

DR. LEE: Yes, we can.

DR. GANDARA: Oh, good. I just wanted to share I'm also deeply troubled by the definition of "evidence-based," and I wonder if we could maybe take 10 or 15 minutes just to review the language of the proposed legislation. I also worry that if we wait on the Policy Committee to be able to meet, then we'll be too late in making recommendations. But I also don't want us to just arbitrarily select the top topics to address in our letter just based on what's been brought up. So I personally would like to see what else is in the bill. And one of the issues that's come up that I think is critically important is this definition of "evidence-based."

And just one other note is that we did receive a letter from public comments from an association called, I think, the Alliance For Learning Innovation, and they did also have a list of legislative updates that they wanted us to address, and one of them was NCADE or ARPA Ed, so I just wanted to note that also in response to Mark's comment.

DR. LEE: So just for clarity on that purpose in terms of what we can address today versus the Policy Committee working and we're having another meeting in another month probably at this point is that those latter two issues, because they're not directly in the act of reauthorization issues that are coming up, that we have a little more time to address those. Is that correct, Denisa?

MS. GANDARA: Well, the two issues are NCADE/ARPA Ed. That's one. That's the same one. And then I guess that's the only one that's not in the legislation currently. But the "evidence-based" definition is something that is in there that we could address.

DR. LEE: So I'm thinking again in terms of efficiency that it seems like there are four questions that we can take a vote on to get a sense of the Board that could inform on a statement that we could vote on and agree to today, which would not preclude the Policy Committee from further digging down in more detail with what's emerging to allow us at a later date. But I worry that if we wait things are going to move and we won't have any opportunity to weigh in.

DR. GANDARA: I see. I see what you're saying, Carol. So the other issue, I guess, is SLDS, right, was the other issue Mark brought up, so those are the two that could be discussed in the policy committee and not in this letter that we're drafting today, is that right?

DR. LEE: Right.

So I'm going to recommend in terms of moving forward at this point that we solicit motions on the four topic areas individually just to get a sense of where we have consensus and where we don't. And for the areas where we may still not have consensus, that we could have some discussion and maybe if Ellie or someone can, you know, post up a document that we could actually be sort of filling in the details in at this meeting to inform what we see as the pros and cons, opportunities and constraints on those topical areas that we don't have full consensus on.

MS. HERNANDEZ LEGORRETA: This is Conchita. Real quick, I have started drafting a document, just what everyone has been saying to be edited. So, if we agree on a vote, then I can share that with everybody.

DR. LEE: Wonderful. Bless you.

All right. So can we first get a motion to approve a recommendation that the ESRA reauthorization include maintaining the 15-member NBES board? Shaun?

DR. HARPER: I would like to move that.

DR. FUCHS: Second.

DR. LEE: Great. I think on all of these we should call the roll, Ellie.

MS. PELAEZ: Okay. Ms. Hernandez Legorreta?
MS. HERNANDEZ LEGORRETA: Agree.
MS. PELAEZ: Dr. Darling-Hammond?
DR. DARLING-HAMMOND: Aye.
MS. PELAEZ: Dr. Gandara?
DR. GANDARA: Yea.
MS. PELAEZ: Mr. Guy?
(No response.)
DR. LEE: He was — oh. Elmer?
DR. HARPER: Looks like his room is empty.
MS. PELAEZ: Maybe he stepped out. I can come back.
Dr. Harper?
DR. HARPER: Yea.
MS. PELAEZ: Mr. Hilliard?
MR. HILLIARD: Yea
MS. PELAEZ: Dr. Klasko?
DR. KLASKO: Yea.
MS. PELAEZ: Dr. Lee?
DR. LEE: Yes.
MS. PELAEZ: Dr. Lopez Turley?
DR. LOPEZ TURLEY: Yes.
MS. PELAEZ: Dr. Scott?
DR. SCOTT: Aye.
MS. PELAEZ: Ms. Sullivan?
MS. SULLIVAN: Aye.
MS. PELAEZ: Dr. Yoshikawa?
DR. YOSHIKAWA: Yes.
MS. PELAEZ: Mr. Guy?
(No response.)
MS. PELAEZ: He doesn't — it still looks like he's —
DR. LEE: He's still doing what?
MS. PELAEZ: — waiting for his computer maybe. So I have 11 yeas.
DR. LEE: So passed.
MR. GUY: So the Board is on mute.
MS. PELAEZ: Hi, Mr. Guy. Did you —
DR. LEE: Is that Elmer?
MR. GUY: This is Elmer Guy.
MS. PELAEZ: Yeah, mm-hmm.
DR. LEE: So we'll re-vote?
MS. PELAEZ: Carol, do you want to read the votes?
DR. LEE: We're voting on the question of recommending that the NBES Board remain at 15 rather than being reduced to nine. So we're asking you to vote, yes/no or abstain.
(No response.)
DR. LEE: Elmer, Elmer, where are you?
MR. GUY: I don't have — I'm in the — I agree. Thank you.
DR. LEE: Great. All right.
MS. PELAEZ: Agree. Okay.
DR. LEE: Yes.
MS. PALEAZ: That was 12. That was unanimous.

DR. LEE: The second question is of the appointment of the NBES members by — I'm going to say by the President's office. I don't know if we want to do it this way. Can we just go — procedurally this may not work — but to indicate that you either approve of the appointment by the Secretary of Education or the President, and we'll see what that tally comes out to be rather —

DR. DARLING-HAMMOND: Do you just want to go around and ask people to state that? Is that what you're asking?

DR. LEE: Yes. So, Ellie, again, if you'd just call the roll and just ask, are you in favor of the appointment of NBES Board members by the Secretary of Education or by the White House, or you can abstain. You don't have a position either way.

MS. PALEAZ: Okay. Ms. Hernandez Legorreta?
MS. HERNANDEZ LEGORRETA: Secretary.
MS. PELAEZ: Dr. Darling-Hammond?
DR. DARLING-HAMMOND: I would withhold an opinion at this point. I'm still deliberating.
MS. PALEAZ: Okay. Dr. Gandara?
DR. GANDARA: I would abstain also.
MS. PALEAZ: Okay. Mr. Guy?
MR. GUY: I support maintain the Presidential appointment.
MS. PALEAZ: Okay. Dr. Harper?
DR. HARPER: I am also going to abstain at this time.
MS. PALEAZ: Mr. Hilliard?
MR. HILLIARD: Until I am convinced otherwise, I believe the current structure works.
MS. PALEAZ: Dr. Klasko?
DR. KLASKO: Presidential appointment.
MS. PALEAZ: Dr. Lee?
DR. LEE: Presidential appointment.
MS. PALEAZ: Dr. Lopez Turley?
DR. LOPEZ TURLEY: President.
MS. PALEAZ: Dr. Scott?
DR. SCOTT: Presidential.
MS. PALEAZ: Ms. Sullivan?
MS. SULLIVAN: Presidential.
MS. PALEAZ: Dr. Yoshikawa?
DR. YOSHIKAWA: I think I'll abstain.
DR. DARLING-HAMMOND: Now I suggest we go back around and see if anybody else wants to add a thought.
MS. PALEAZ: Okay.
DR. DARLING-HAMMOND: I say that because I'm now prepared having heard from others to —
MS. PALEAZ: Oh, okay.
DR. DARLING-HAMMOND: — add to the consensus.
MS. PALEAZ: Okay. Okay. Dr. —
DR. DARLING-HAMMOND: — for Presidential — to Presidential.
MS. PALEAZ: Dr. Darling-Hammond? Okay.
DR. DARLING-HAMMOND: I would —
MS. PALEAZ: Doctor —
DR. LEE: You're changing — Linda, you're changing your vote?
DR. DARLING-HAMMOND: Yeah, I'm saying I'm now prepared to weigh in for Presidential appointment.
DR. LEE: Okay.
MS. PALEAZ: Dr. Gandara, you also abstained, right?
DR. GANDARA: Yes, I will abstain.
MS. PALEAZ: Still abstain. Okay. And anyone else who abstained who would like to — Dr. Harper?
DR. HARPER: Yeah. Presidential.
MS. PALEAZ: Okay.
DR. LEE: All right. So we have one move to appoint by the Secretary, nine to maintain Presidential appointment, and two abstentions.
MS. PALEAZ: So we have a vote.

DR. LEE: So is that sufficient to make the recommendation that we maintain Presidential appointment? So is the question whether anyone —

DR. DARLING-HAMMOND: You want to do a formal motion and a second and then do an actual vote now that we've kind of done our polling?

DR. LEE: Yeah.

MR. HILLIARD: That makes sense.

DR. LEE: Right. And then the question would be — and this will come up in the vote again as to whether anyone who voted to abstain or voted to move to Secretarial appointment feels so strongly against the recommendation for the Presidential. That'll kind of come out, but just to be in the back of the mind, we want to, I think, be sensitive to that. So can we get a motion to recommend that the appointment of NBES members be made by the White House?

DR. MILLER: So moved.

DR. FUCHS: Second.

DR. LEE: Stephen, you had your hand up there?

DR. KLASKO: I was just going to make a motion. I'm fine with that.

DR. LEE: Oh, great. Okay.

So, Ellie, you can call the roll.

MS. PALEAZ: Okay. Ms. Hernandez Legorreta?
MS. HERNANDEZ LEGORRETA: You all have strongly changed my mind, so I will vote with everyone else on Presidential as well.
MS. PALEAZ: Okay. Dr. Darling-Hammond?
DR. DARLING-HAMMOND: Aye.
MS. PALEAZ: Dr. Gandara?
DR. GANDARA: Abstain.
MS. PALEAZ: Mr. Guy?
MR. GUY: I vote yes.
MS. PALEAZ: Dr. Harper?
DR. HARPER: Yea.
MS. PALEAZ: Mr. Hilliard?
MR. HILLIARD: Yes.
MS. PALEAZ: Dr. Klasko?
DR. KLASKO: Yea.
MS. PALEAZ: Dr. Lee?
DR. LEE: Yes.
MS. PALEAZ: Dr. Lopez Turley?
DR. LOPEZ TURLEY: Yes.
MS. PALEAZ: Dr. Scott?
DR. SCOTT: Yes.
MS. PALEAZ: Ms. Sullivan?
MS. SULLIVAN: Aye.
MS. PALEAZ: Dr. Yoshikawa?
DR. YOSHIKAWA: Abstain.
MS. PALEAZ: Okay. We have 10 yeas and two abstentions.

DR. LEE: Okay. I think that's sufficient to move forward, unless the two abstentions have strong objections.

Then the third question is on the appointment of the NCES director by the White House or by the IES director. So I'm thinking maybe we could do the same thing, the same process that we did before, to take a poll and then create a vote. So if I can get a motion — I'm sorry, it was not a motion. So we're just going through the roll and the question of whether the NCES director should be appointed by the office of the President or by the director of IES.

DR. SCHNEIDER: Carol, could you get the terminology right? It's the Commissioner of NCES, not the director of NCS.

DR. LEE: Sorry about that. Correct.

DR. DARLING-HAMMOND: How did you want to proceed?

DR. LEE: The same way, just to go through it. Just take an initial poll and then on the basis of that poll determine what the motion will be.

Hiro, is your hand up?

DR. YOSHIKAWA: Yeah, I just wanted to ask a question and maybe — yeah, which is it's about the lines of reporting. I'm not entirely clear who the Commissioner of NCES reports to right now. So it sounded like the directors of the other centers report to the director but that in this case Peggy does not. Is that right?

DR. SCHNEIDER: That — no, Peggy, don't nod yes.

I mean, the problem with the existing legislation is that the lines of communication and authority and control are filled with gray areas that make extremely difficult governance and authority.

DR. DARLING-HAMMOND: But the point of the current structure is that the NCES commissioner is an independent — it's an independent agency.

DR. SCHNEIDER: Well, I mean, if that is your position, then NCES should be moved out of IES and made an independent statistical agency reporting to the Secretary.

DR. LOPEZ TURLEY: I'm still unsure about Hiro's question. So who does Peggy report to currently?

DR. SCHNEIDER: So that is the issue, right? So we've been trying to clarify that, and a lot of the words in the rewrite of the ESRA reauthorization are designed to clarify lines of authority and reporting.

DR. LEE: Peggy, do you want to —

DR. CARR: So there is a yes or no. Can I — is it — there — there's a yes or no to that question. Technically, I report to the Secretary, and the Chief of Staff handles the correspondence and evaluation of this position.

Now whether Mark or IES or others want it differently is I think part of the discussion, but the factual answer is that my evaluation is done by the Secretary's office.

DR. LEE: Thank you.

DR. SCHNEIDER: I'm sorry, but that's a unique situation because you were SES. So that is — I mean, I was — yes. I was supposed to do your evaluation, and I refused to do it and passed it off to the Secretary's office.

DR. CARR: Well, that's information.

DR. SCHNEIDER: Anyway, so the point is, obviously, there's a lot of internal debate. I mean, I think you'll take the pulse of the Board. I'm pretty sure what the outcome will be, but there is — but the issue of lines of authority, lines of control, the extent to which we have an independent statistical agency inside an independent science agency has been for the last five years an extremely difficult situation.

And this had nothing to do with Peggy. I mean, the lines of communication with her predecessor was also difficult because the statute is really — existing statutory authority and reporting is extremely complicated, filled with gray areas, and the purpose — some of the changes in ESRA were designed to correct that.

DR. LEE: So, in the discussions that have been unfolding around this relative to this legislation, have they had explicit inputs from a variety of stakeholders in their process of negotiating at this point? I'm just trying to —

DR. SCHNEIDER: Oh, absolutely. Absolutely. I mean, this has been — I mean, they had an RFI out. They had weeks of open discussion. I'm not sure how many comments came in, you know, once they made public the fact that they were doing this. But the Committee's consulted with many, many people. Many, you know, organizations have put in input.

DR. DARLING-HAMMOND: On this point, I would say there are organizations that one would think might have been consulted that weren't and that are busy writing letters to the Committee, and those include the statistical association and the research associations, so, certainly, something I'm hearing a lot about. So it's —

DR. LEE: Well, I have —

DR. DARLING-HAMMOND: — saying that the —

DR. LEE: Go ahead.

DR. DARLING-HAMMOND: Go ahead. No, go ahead, Carol.

DR. LEE: No, you go ahead.

DR. DARLING-HAMMOND: I would just say that there is concern that sort of demoting NCES in the way that it is being read, there's concern in the statistical association ranks and others that it will reduce the independence of the agency and its capacity to do its work most effectively.

I understand that there are these bureaucratic confusions, and I appreciate Mark and Peggy giving us their inside sense on that, but there is also a set of perspectives that are circulating in the research community as well.

And I think, you know, the National Academy report that was on the vision and road map for education statistics a year ago really talked about the need to strengthen NCES, and the folks involved in that effort see this as a weakening of the agency rather than a strengthening.

DR. LEE: So I'm thinking — and I know this is complicated — that if this initial poll that we're taking, we're not taking a formal vote at this point, reveals that we have mixed opinions about this, that this would be an important area, even if we were able just to create with — thank you, Conchita, taking the notes right now — from what we do know about the issues that are involved in those deliberations and some reference at least to the extent that we have the knowledge of stakeholders and organizations that are weighing in, as opposed to just saying if we're divided that we just ignore it altogether.

So any other discussion before we take — again, this is not a vote. It's just a poll on the question of the NCES commissioner. Is that right, Mark, my terminology, of being appointed by either the White House or by the internal IES director?

(No response.)

DR. LEE: Okay, Ellie, you want to call the roll again? This is just a poll. So you're just saying IES director White House or abstain.

MS. PALEAZ: Ms. Hernandez Legorreta?
MS. HERNANDEZ LEGORRETA: Abstain.
MS. PALEAZ: Dr. Darling-Hammond?
DR. DARLING-HAMMOND: I would weigh in to maintain the appointment by the President and ensure that the ESRA language is strong enough around the capacity of the agency to do its work.
MS. PALEAZ: Dr. Gandara?
DR. GANDARA: I also support maintaining the appointment at the White House.
MS. PALEAZ: Mr. Guy?
MR. GUY: It's a complicated issue to me. I think I'll support the Presidential appointment on this.
MS. PALEAZ: Dr. Harper?
DR. HARPER: White House.
MS. PALEAZ: Mr. Hilliard?
MR. HILLIARD: I support maintaining the Presidential appointment.
MS. PALEAZ: Dr. Klasko?
DR. KLASKO: White House.
MS. PALEAZ: Dr. Lee?
DR. LEE: White House.
MS. PALEAZ: Dr. Lopez Turley?
DR. LOPEZ TURLEY: I don't feel like I understand the implications fully, so I will abstain.
MS. PALEAZ: Dr. Scott?
DR. SCOTT: Likewise.
DR. LEE: Likewise meaning abstain?
DR. SCOTT: Yes. I don't fully — I'm not informed enough to make a decision one way or another.
DR. LEE: Okay.
MS. PALEAZ: Ms. Sullivan?
MS. SULLIVAN: White House.
MS. PALEAZ: Dr. Yoshikawa?
DR. YOSHIKAWA: White House.
MS. PALEAZ: That's it. We had nine for maintaining and three abstentions.

DR. LEE: So, again, we're in a slightly similar situation. Those who abstained, is there anything in the discussion so far that changes your mind or you want to hold your position around abstaining, which is fine? Just a question.

DR. LOPEZ TURLEY: I don't have a change of mind but rather just a follow-up question, and that is, so if I understand correctly, the Commissioner currently reports to the Secretary, but, of course, NCES organizationally, it's structured within IES, and so I'm trying to understand then why it's structured that way and yet it's a different line of reporting.

Yeah, because, you know, I direct an institute here at Rice, you know, and I have several directors that report to me, and I'm trying to put myself in that position of, okay, how would that complicate things. Of course, that's how it's currently done. So would want to understand — yeah, I'm still trying to understand I guess from an organizational structure standpoint why it needs to be different.

DR. LEE: So, again, I'm going to suggest what I think the argument is in terms of maintaining the Presidential appointment. I believe the argument is the standing and function of NCES as a statistical agency that happens to focus on issues of education but has a broader function in terms of the variety of agencies in the government that collect broader statistical data on issues that are important for the nation to understand. And I think the argument is trying to understand NCES in relationship to other agencies of the government that fulfill a similar function.

Peggy, am I off in that as the logic?

DR. CARR: No, I think you're on board with it.

DR. SCHNEIDER: So my problem with that is someone — we have feedback somewhere — so I would make the — I could make the exact same argument with regard to NCEE, and, actually, to the extent to which NCER and NCSER collaborate with other agencies across the government, I would make the same argument.

So I'm not sure that language is correct. NCES is a recognized federal statistical agency and does have different standing, but the fact of the matter is that the evaluation community has the same structure now. There is meetings of evaluation agencies, organizations within agencies. So I just want to be careful about this kind of centrifugal force that seems to be you're endorsing in that statement.

DR. CARR: So, if I could just weigh in, I don't think it is the same. I think that we need to recognize that recognized statistical agencies can lose their status from OMB if they do not fulfill their primary responsibilities under the Evidence Act. So there is a lot to be lost, a lot to be risked if there is not autonomy and the ability to implement the primary responsibilities, and that is not the same for the other centers. I don't see that parallel whatsoever.

DR. LEE: Any other thoughts?

DR. LOPEZ TURLEY: Yeah. Peggy, can you say a little bit more about that risk, about how that could happen, just to help me understand that because that does sound like a significant risk to me, but if you can explain maybe, like, a hypothetical example would help.

DR. CARR: Well, the current thinking of the White House is that there will be an evaluation periodically of recognized fiscal agencies — there are only 13 in the government — as to whether or not they are implementing the directives of OMB and now codified in the Evidence Act appropriately.

So we will risk as a department and a statistical community losing that status. It's not about just NCES. It's about the values, the principles, the responsibilities of the federal statistical system that the recognized statistical agencies are current of and must uphold. There are units as well, but there are only 13 recognized agencies and this is what I think is paramount to the value of the independence and autonomy as part of this larger system that they're asking us to adhere to.

DR. LEE: Ruth, is that helpful?

DR. LOPEZ TURLEY: Yes. Excuse me. Yes, thank you.

DR. SCHNEIDER: Well, I'll just make one last statement. I mean, if we follow this argument, then what we should — what the Board should be advocating for is the establishment of NCES as a separate agency reporting to the Secretary, which, by the way, if you read ESRA, the Commissioner of NCES does not report to the Secretary, so I don't know where that comes from.

But the fact of the matter is the lines of authority and control at the current time are a mess, and, again, I mean, following what Peggy's saying, then NCES should be — you should recommend NCES as a separate entity reporting to the Secretary and not with staying inside IES.

DR. LEE: Peggy, you're —

DR. CARR: If I can just say —

DR. LEE: Yes, then Hiro.

DR. CARR: — I would just sort of follow up that the whole basis — one of the major goals of trust regulations, which is, again, the implementation of the Evidence Act, is to codify the role of a parent agency, such as IES, and a recognized statistical agency. I mean, the statistical community recognizes that these agencies may be embedded in an organization and that that organization should be responsible for enabling and supporting the recognized statistical agency.

So it's not being ignored by the community, and there are lots of resources that need to be shared. You know, statistical agencies can't always be in a position where they can have access to these resources. It's okay to share these resources and to be supported by the parent agency. That's half of what the Trust Act talks about — I'm sorry, the trust regulations talk about.

DR. DARLING-HAMMOND: Can we — are we in a position to go ahead and do the vote and then take a comfort break?

DR. LEE: Right. Let's let Hiro and Denisa make comments, and then we'll go take a vote.

DR. DARLING-HAMMOND: I'm worried about our court reporter.

DR. LEE: Okay.

DR. DARLING-HAMMOND: Make it quick, Hiro.

DR. YOSHIKAWA: I'll be quick. Sorry. Wow, that's a lot of pressure.

I just want to say my understanding — and this is, of course, just from kind of Peggy's own reports — but my understanding from the global kind of perspective is that a responsibility of NCES, of course, is to report to the UN agencies around national education data. And I believe NCEE wouldn't have that kind of role or doesn't, right, to reporting to UNESCO on the country's education statistics, for example?

DR. DARLING-HAMMOND: It's one of many examples of the way in which it's somewhat unique.

I'd just like to say for efficiency, I think the first order of business was to get a decision on this question. Mark brought up the additional point that there may be other ways in which one wants to think about the role and governance of the agency, which we could take up afterward, after that, and after our court reporter's comfort break.

DR. LEE: All right. Denisa, and then we'll take the vote.

DR. GANDARA: I have a very brief comment. I'm just thinking about what Peggy reported on yesterday regarding the leadership roles that she has taken in collaboration with other statistical agencies, including around DEIA, co-chairing some work around the President's Executive Order related to equity, and I just worry that demoting the Commissioner might not enable those kinds of leadership roles for the Commissioner of NCES.

DR. LEE: All right. So let's move to the vote. So I'm requesting a motion to recommend that the Commissioner of NCES continue to be appointed by the White House. So we'll get the motion and a second and then we'll take a vote.

DR. YOSHIKAWA: I move.

DR. DARLING-HAMMOND: I'll second.

DR. LEE: Great. Ellie, can you call the roll?

MS. PALEAZ: Yes.

DR. LEE: So you're voting either yes for the Presidential appointment, no, or abstain.

MS. PALEAZ: Okay. Ms. Hernandez Legorreta?
MS. HERNANDEZ LEGORRETA: Abstain.
MS. PALEAZ: Dr. Darling-Hammond?
DR. DARLING-HAMMOND: Aye.
MS. PALEAZ: Dr. Gandara?
DR. GANDARA: Aye.
MS. PALEAZ: Mr. Guy?
MR. GUY: I vote yes.
MS. PALEAZ: Dr. Harper?
DR. HARPER: Aye.
MS. PALEAZ: Mr. Hilliard?
MR. HILLIARD: Yes.
MS. PALEAZ: Dr. Klasko?
(No response.)
DR. LEE: Where did he go? He maybe took a break.
DR. DARLING-HAMMOND: He might have taken that break, yeah.
MS. PALEAZ: Dr. Lee?
DR. LEE: Yes.
MS. PALEAZ: Dr. Lopez Turley?
DR. LOPEZ TURLEY: Yes.
MS. PALEAZ: Dr. Scott?
DR. SCOTT: Yes.
MS. PALEAZ: Ms. Sullivan?
MS. SULLIVAN: Aye.
MS. PALEAZ: Dr. Yoshikawa?
DR. YOSHIKAWA: Yes.
MS. PALEAZ: Dr. Klasko?
DR. KLASKO: Yes.
MS. PALEAZ: Okay. Okay. The yeas have it, 11 to one abstention.

DR. LEE: Okay. So that passes.

All right. Let's take a 15-minute break. It's 11:15 my time, so 11:30 then we'll come back and then we'll take up the question of the evidence statement in ESRA and create a vote for the contents and focus of the statement.

Conchita, you said you're taking some notes, so maybe you can share the preliminary document that you developed and then we can move on to talk about scheduling our meetings for the year.

DR. HARPER: Carol?

DR. LEE: Yes?

DR. HARPER: Before we depart for this comfort break, is it our only break?

DR. LEE: It depends on how far we want to go. I'm thinking that we — I don't think that we'll have business that will take more than the next hour, so I don't think we need to worry about a lunch break or whatever. I don't see us going until 4:00. Does that help?

DR. HARPER: Somewhat.

DR. LEE: Bring your snacks with you so you can — we can hold on.

DR. HARPER: Okay.

DR. LEE: All right.

DR. SCHNEIDER: That means refill the espresso mug, Shaun.

DR. LEE: Right. All right. See you all.

(Whereupon, a brief recess was taken.)

DR. LEE: The last item relative to this current legislation reauthorization has to do with the statement on what constitutes research evidence, with particular attention to the paragraph that's in the meeting chat.

And when we finish this, Mark, I think it could be helpful if you were to say something about the timing on the reauthorization relative to the appropriations, some sense of the timeline, the urgency of the timeline of responding there as well.

So we'll try to resolve this issue. And then if, Conchita, you could screen share — Bill, if you could give her screen sharing so Conchita could share with us the draft that she has worked on questions —

MS. HERNANDEZ LEGORRETA: And it's in everyone's email, so whatever email was in the calendar invite, it went to everyone on the calendar invite.

DR. LEE: Yeah. No, I got it. I'm sure we all got it. I think it would be helpful to actually see it on the screen —

MS. HERNANDEZ LEGORRETA: Yes.

DR. LEE: — just to finalize.
So the question now is the statement regarding research evidence. The consensus seems to be toward recommending that it be removed. Is there any discussion about that? We could take the vote, I think. What would be helpful would be for us to create a statement as to why, assuming that's the — so is there any — before we ask for the motion, any discussion about the focus of the statement that's in the chat regarding what constitutes appropriate scientific evidence?

DR. DARLING-HAMMOND: I'm having some trouble getting my Teams to work, but am I still here? Yeah. Okay. I'm still here.

DR. LEE: You're still here.

DR. DARLING-HAMMOND: I would just reinforce Mark's recommendation that we just ask for it to be — for the current moment, you know, delete it and make some framing around the fact that there are a lot of decisions that go into the right methodologies and, you know, what we are concerned about, being sure that there's strong methodologies being used for research to draw appropriate inferences. It's more complex than simply looking for a single statistically significant study or a single methodology. That could be part of the rationale, but I think that for the moment, eliminating the paragraph makes the most sense because wordsmithing it now would be difficult.

DR. SCHNEIDER: So I totally support Linda's position on that because I'm looking at this and this is a good starting point, but it needs a lot of work and I'm not sure if you have that much work. I mean, I think C goes — that goes beyond statistically significant is not exactly what's at risk here or what's at stake.

DR. DARLING-HAMMOND: Mm-hmm. Right.

DR. SCHNEIDER: So that C needs to be changed. And then again, I mean, then that requires some thought and some careful definitions. I would just say that the easiest thing to do is to recommend that that paragraph be deleted and then something innocuous like, "We welcome the opportunity to work further with you on a different definition if that's needed, but in the meantime, we recommend that that paragraph be struck."

DR. DARLING-HAMMOND: Yeah, I think that's a good way to go. I mean, they are probably going to have to say something. And, of course, there's the evidence-based definitions that are in ESSA, which are also, you know, not fully adequate to —

DR. SCHNEIDER: Yeah, so the things —

DR. DARLING-HAMMOND: — the decisions that have to be made about what kind of research to support.

DR. SCHNEIDER: Right. And there's some — there's been some discussion about trying to take ESSA standards and apply them to IES via this Act, and that would be — I'm not sure how strong a negative word I could use, but it would have negative implications for us.

So I think four is much simpler than this and I think the recommendation is that this be struck for a variety of reasons and Linda could probably just whip off two, three, Matt could help probably, and that we welcome the opportunity to work on a different definition in the future.

DR. LEE: So I totally agree and I'm going to call for the motion. I would just also like to say while it's on my mind that it would be helpful, Mark, I think once this meeting is over and we start the process with this policy committee to sort of meet with them and begin to articulate some of these other next-step projects for areas that you think will be important to be taken up.

DR. SCHNEIDER: Of course. So the most important thing with regard to any future discussion about "evidence-based" and future definitions, so Matt Soldner has to be involved in that because he is the evidence officer for the entire Department.

DR. LEE: Well, I'm saying two things. One, this would be around the future of attention to criteria for evidence-based reliability, if you will, on the one hand, but I'm saying, in the broader scheme of things, there are other policy issues that are emerging that you know are on the — you have comments already to be addressed —

DR. SCHNEIDER: Yes. Yeah, yes.

DR. LEE: — right, to share that timeline would be helpful.

So I'm going to ask for a motion that the Board recommend that the statement articulating criteria for evidence-based research be deleted. Can I get the motion and second?

DR. DARLING-HAMMOND: And do we want to make in —

DR. SCHNEIDER: But now I'm sorry, just —

DR. DARLING-HAMMOND: — that motion the other half of it that it be current — deleted for the moment and that we're — you know, and ultimately replaced with something that we would, you know, be happy to contribute to?

MS. PALEAZ: Or something broader. Maybe you could just leave it that vague, but just something broader.

DR. LEE: Broader than —

DR. DARLING-HAMMOND: Mark, you were about to say something also.

DR. LEE: I'm just saying broader but that we would also be willing to contribute to the discussion of what those shifts ought to be.

DR. SCHNEIDER: And we have to make sure that the specific paragraph that we're talking about is referenced in the motion.

DR. LEE: Right. So can Matt or somebody give us, like, where the paragraph comes from so we'll be specific as to which one we're talking about?

DR. SOLDNER: I can try to pull something up right now. Hold on. In the — yeah, let's see. This is — well, it's Section 15 — or No. 15 in the definitions section. Anyone know what the definitions section is off the top, Section —

DR. DARLING-HAMMOND: We can get that in before we send the —

DR. SOLDNER: Yeah.

DR. LEE: Right. It actually copied the paragraph and inserted, so the issue is left —

DR. SOLDNER: Okay. Section 102, paragraph 15.

DR. LEE: So what's important is that in voting the Board's aware of the specific language we're talking about now, which you can because it's in the chat. So I get a motion and second.

DR. SCHNEIDER: I so move.

DR. LEE: Second?

DR. GANDARA: Second.

DR. LEE: Okay. Ellie, can you call the roll?

MS. PALEAZ: Yes. Ms. Hernandez Legorreta?
MS. HERNANDEZ LEGORRETA: Yes, agree that it be removed.
MS. PALEAZ: Dr. Darling-Hammond?
DR. DARLING-HAMMOND: Aye.
MS. PALEAZ: Dr. Gandara?
DR. GANDARA: Yes.
MS. PALEAZ: Mr. Guy?
MR. GUY: Yes.
MS. PALEAZ: Dr. Harper?
(No response.)
MS. PALEAZ: Mr. Hilliard?
MR. HILLIARD: Yes.
MS. PALEAZ: Dr. Klasko?
DR. KLASKO: Yes.
MS. PALEAZ: Dr. Lee?
DR. LEE: Yes.
MS. PALEAZ: Dr. Lopez Turley?
DR. LOPEZ TURLEY: Yes.
MS. PALEAZ: Dr. Scott?
DR. SCOTT: Yes.
MS. PALEAZ: Ms. Sullivan?
MS. SULLIVAN: Aye.
MS. PALEAZ: Dr. Yoshikawa?
DR. YOSHIKAWA: Yes.
MS. PALEAZ: Dr. Harper?
(No response.)
DR. LEE: I'm sure he'll be back in a minute, so we can — so it's unanimous, with the exception of the votes that haven't come in, correct?
MS. PALEAZ: Yes. Mm-hmm.
DR. LEE: All right. This has been extraordinarily productive. Really, it has. So I'm thinking now if we could read through the document.

Conchita, are you showing or is Ellie showing?

MS. HERNANDEZ LEGORETTA: Yeah, I'm showing.

DR. LEE: Okay.

MS. HERNANDEZ LEGORETTA: Can you see it?

DR. LEE: Can you make it a little bit bigger?

MS. HERNANDEZ LEGORETTA: Yep.

DR. DARLING-HAMMOND: So I'm wondering about our process at this point. This is a moment of editing, and Committee editing is usually challenging, how we want to get to a final version. Do we need to get to the final wording in a way that we vote on that in this meeting? One possibility might be to give us a break to put edits into the Google doc rather than trying to do it as a group, but I just raise the question before we dive in to think about what process could be most useful.

DR. LEE: So that makes sense if we can — it already is a Google doc product, so we can just go into our documents. So let's say if we take the next 10 minutes to go into the doc files, insert our comments, and that way we could hopefully have an efficient process for the final edits.

DR. DARLING-HAMMOND: Yeah, I'm not sure we can do it that quickly, but, you know, we could certainly make some progress.

DR. LEE: Let's try. We'll do a check-in — come back and do a check-in in 10 minutes, and if people need some more time, then we can do it that way.

DR. DARLING-HAMMOND: Oh, I just — this is maybe me because of the way I like to write. I like to think about it, and so I think we should get our comments in the document in the next 10 minutes and then figure out how to take a little bit of a longer break to finalize it so that we're careful with the language ultimately, you know.

DR. LEE: Okay.

DR. DARLING-HAMMOND: So, if we can get everybody's comments in in the next 10 minutes and then figure out how to give somebody authority or the group authority to go off and work it into a nice document.

MS. HERNANDEZ LEGORRETA: And what would be helpful too is if everyone put their edits as track changes and that way we can kind of look at them and just be like yes, yes, yes, because then we're going to be writing over each other.

DR. DARLING-HAMMOND: Okay.

DR. LEE: All right. Let's start. Thanks.
(Pause.)

DR. LEE: So are we about finished? So I think, if, Conchita, you could pull up the document again, we can look through.

MS. HERNANDEZ LEGORRETA: Sure. The first suggestion is just delete some formatting issues, which is good. The next one is adding the work of IES more broadly, if that makes sense. And this one, adding some grammar to inform the Board's deliberations and decisions underlies. That looks good.

So those of you all that are much better at wording, this sentence here, "The Board has not met since," so I put in the date of the prior board with the dates of the current Board. So the good wordsmiths, feel free to reword that so that makes sense.

Membership, that totally makes sense. I guess membership. Emphasizing the scope and departments of the Board, as well as providing support for a diverse group of leaders. Looks good. Yeah.

Go ahead. Oh, sorry, no. This is Conchita seeing I'm doing a very bad job of, like, badly reading the comments and not fully kind of going through them just to make sure it makes sense. Is somebody — oh, so Carol added appointed by the NES board, which makes sense in the wording of responsibilities.

DR. LEE: So this sentence on the NCES Commission, I think we need to change that. I think this — that if you could go back over this first part. So, no, down to the NCES —

MS. HERNANDEZ LEGORRETA: Okay.

DR. LEE: Okay. So "worry about the statistical enterprise would be harmed if done by the director, maintain the information that is needed about educational systems more than the coming and going of other centers," I think that should be taken out.

DR. SCHNEIDER: I totally agree. I find that sentence actually wholly wrong.

DR. LEE: I wouldn't like it either, Mark, so I think that should come out. I think the argument —

MS. HERNANDEZ LEGORRETA: Excuse me, Carol, which one?

DR. LEE: That whole first "worry about the statistical enterprise would be harmed if done by the director," all that should come out, the "maintain the information that's needed about educational systems more than the coming," I think that should come out.

DR. SCHNEIDER: That definitely has to come out.

MS. HERNANDEZ LEGORRETA: Okay. So —

DR. LEE: I think I —

DR. SCHNEIDER: And I'm worried —

MS. HERNANDEZ LEGORRETA: — let me — I'm seeing no — okay.

DR. LEE: So I'm just — I think that what comes below that about the relationship —

DR. SCHNEIDER: Wait, I'm sorry, there was a — I'm sorry, Carol, that sentence about comparing it to other centers has to go also.

DR. LEE: Right, right, right. All —

MS. HERNANDEZ LEGORRETA: The maintaining and then —

DR. SCHNEIDER: Maintain the information and need about education systems, that is, again, insulting.

DR. LEE: Right.

MS. HERNANDEZ LEGORRETA: So what if — like that?

DR. LEE: So I think the essence of the argument is the relationship of NCES to other statistical agencies in the government and that the appointment by the President places it — I don't know if "on par" is the right language. The logic is the issue is coming up with the language, if I'm understanding, is that there are a variety of statistical agencies in the government whose responsibility it is to report to the larger public on particular topical areas and that there are activities of NCES that include but also go beyond the purview of the centers. Something to that effect. To me, it has nothing to do with the director.

You know, Peggy, maybe you could offer some wording.

DR. SCHNEIDER: I don't know why we need more than just the first sentence, that NCES remains — the Commissioner remains a Presidential appointment.

DR. LEE: Well, I think we need some rational statement. I don't think it has to be as long as it is. For each one of these recommendations we have some reason.

MS. HERNANDEZ LEGORRETA: Do we want to keep the part about the Evidence Act or no?

DR. LEE: I think that's fine. I think that's the strongest of the arguments at one level.

MS. HERNANDEZ LEGORRETA: So do you want to keep A and B?

DR. LEE: So keep the B. I think the variety of agencies in the government NC, blah, blah, blah, standing, opportunity — I think those are fine. Just all that stuff about the director I didn't think was relevant.

MS. HERNANDEZ LEGORRETA: Okay.

DR. SCHNEIDER: Well, you also have broader rather — it should be broader, going back up, right? "But has a broader function than the issues." So, look, I mean, I believe A and B are relevant and that you could just put C as the only paragraph that you need and to fix that, fix that C. I don't believe A and B add anything.

MS. HERNANDEZ LEGORRETA: Okay.

DR. SCHNEIDER: That's my — I mean —

(Simultaneous comments.)

DR. LEE: Well, Peggy, can you weigh in on this one?

DR. CARR: I think A and B are useful. I think that what we had before the opening stuff, I think, was not — I think it's the rationale for why C — why we have any attention to C at all. It's like C is the warrant for the claims in A and B.

DR. LEE: Peggy, do you have any thoughts on this language here?

DR. SCHNEIDER: I don't understand what the second sentence in A is.

DR. CARR: Am I audible? Yeah, I'm just going to point out that NCES is part of the larger federal statistical system under the Chief U.S. Statistician, and that is part of a larger scope, that we have duties and responsibilities that are outlined for all federal statistical agencies, especially the recognized statistical ones. And so I do think there is something to reiterating the scope of NCES beyond just its role as a center in IES.

DR. LEE: So can we —

DR. SCHNEIDER: Yeah, but that has nothing to do with Presidential appointment or not because the heads of other statistical agencies are not Presidential appointments.

DR. CARR: Well, many of them are and especially —

DR. SCHNEIDER: And many of them are not.

DR. CARR: — some of the larger ones. Well, you know —

DR. LEE: Well, the fact that some are —

DR. CARR: — when people have a valid position.

DR. SCHNEIDER: Yeah. So I still don't understand what "goes beyond the scope" means. What does that phrase mean?

MS. HERNANDEZ LEGORRETA: We can get rid of that sentence and add what Peggy was saying right now about —

DR. LEE: Yeah, what Peggy's saying.

MS. HERNANDEZ LEGORRETA: Yeah.

DR. LEE: So take that out and then, Peggy, just give us the wording of the first sort of sentence that you had. NCES is a part of blah, blah, blah.

DR. CARR: Well, NCES is part of a federal statistical system that has the oversight of the Chief Statistician of the United States with delineated roles and responsibilities actually now codified in law under the Evidence Act.

DR. LEE: And then that seems to me then leads to the B and C. Can you go down to B and C?

MS. HERNANDEZ LEGORRETA: Yes. Should we delete B?

DR. SCHNEIDER: B doesn't make any sense.

DR. CARR: I think B could go. I think I agree with that.

DR. LEE: Okay. So take B out.

DR. SCHNEIDER: And then C just supports the initial, the A, so I think that's good.

DR. LEE: Right.

DR. CARR: Yeah.

DR. LEE: Okay. So now let's go down to 4.

DR. SCHNEIDER: Well, someone's going to fix that B paragraph, right?

MS. HERNANDEZ LEGORRETA: Yeah, B needs to be fixed. Yeah, we'll work on it to make it better. I like this "We urge" — who put that in? It really doesn't say, but I like that it says, "We urge that the current" instead of "current position." Agree.

DR. SCHNEIDER: Right. And we urge that the current definition — so, in some way up there or in the phrase, the definition of "evidence-based," we have to put a reference — we talked about this before — a reference to the specific paragraph we're talking about. So the 15 —

MS. HERNANDEZ LEGORRETA: Yes, and that's in bullet point A, so bullet point A has the paragraph and then we're going to have it —

DR. SCHNEIDER: Yeah, but it says 15. You need to place either a cite at the end.

MS. HERNANDEZ LEGORRETA: Yes.

DR. SCHNEIDER: Or 15 is not sufficient without placing it in the section.

MS. HERNANDEZ LEGORRETA: Yeah, yeah, yeah, that'll be here at the citation part. Does everyone like this with "The definition that allows for thoughtful decision-making of other appropriate methods and standards of evidence needed to address a range of research problems. IES and NBES stand ready to participate in the development," I think that's really good.

DR. LEE: Yeah.

DR. SCHNEIDER: All right. Matt, Matt, are you still on? Are you still around? Maybe not.

DR. SOLDNER: No. No, no. Yeah, no, I am still here.

DR. SCHNEIDER: Right. Could you comment on this, or you think this is okay?

DR. SOLDNER: Well, as always, I think the Board ought to do what the Board thinks is, you know, the right way to go. Can you scroll down a bit here? I don't have your copy of the document open. Yeah.

So what I would suggest to you is, you know, we know that Congress is working quickly, and eliminating the definition is one way to proceed. If you think — I heard someone mention the idea of proposing an alternate definition at a later time. I'm not sure how much later time there is for you all to act and so, if there are things you would want to telegraph to the Committee, even if you don't have a fully formed idea, now might be a propitious time to do that.

So, if there are other structures in the law that seem like promising ways to bound evidence-based in ways you think are appropriate for IES, you might want to point them in that direction. I just don't want you to get caught not being able to provide feedback on your timeline.

DR. LEE: Do you have any suggestions or thoughts about that?

DR. SOLDNER: Gosh, Carol, I'm trying to not give you specific suggestions because I'm not sure that's my job as a commissioner, but I would say — I don't want to sway anyone's opinion. I want you all to come to it as you think you ought.

I guess my suggestion is, if there are things in existing ESRA or the reauthorization language you've seen that feel closer to you to what high-quality research looks like and that you would want IES to prioritize, you should point that way. And so I'm not sure — I mean, I know Linda had access — had shared around the whole redline. I'm not sure, Linda, if you think there are things in either original ESRA or the redline that describe research in ways that are helpful here or not.

DR. LEE: Well, let me just say, Matt, that we've already — the Board has already taken a position in terms of claims of this issue, but we do depend upon the expertise of IES staff to, you know, help us think about issues, so I don't think you should feel constrained in any way in making recommendations about what might be language for other ways of considering reliability and research methods and evidence.

DR. SOLDNER: Thanks, Carol. I guess what I would suggest to you then is there is some statutory language that we have operated with for a very long time related to scientifically valid research. I think it's called principles of scientifically valid research. It's on the redline that Linda shared earlier. And it's principles-driven, right, as opposed to being terribly specific and has lots of guardrails that have supported IES in producing high-quality research of all types of 20 years. And so that might be a place where I would start to look if I was trying to find inspiration for what might be used here.

So I'm not sure if Linda has that open and wants to show that to — whoever has access to that document and has it open and would want to show it to the Board, but that's existing text that you wouldn't have to come up with de novo right now in a short time frame.

DR. LEE: Is Linda still here? I know she had to step away at one point. It looks like she had to step out.

I mean, I have some thoughts, but I think they're too long-winded to go into a document like this because I think the issue has to do with the conceptionalization of the questions that we're asking and having the freedom to sort of recruit from a variety of analytical methods and design that will allow us to wrestle with the question at hand. And quite often, the randomized control experiments and the causal inferences make assumptions about sort of singular pathways of influence on an outcome versus work that evolves in things like studies and that this was studying dynamic systems, which are quite different, and they're warranted in very different kind of principles. But I think, oh, that's too long-winded to put in a document like this.

I think, at this point, we're trying to make our position clear and that it will likely be relationships that we try to establish with key folks on the Hill and other stakeholders that would bring about opportunities to go into more depth. I think, at this point, this isn't going to be the document that influences. This is going to be the document that makes our position clear, if that makes sense.

To me, the problem in trying to — on this is, for example, this cause about "Statistical difference is debated among statisticians and researchers. Key criticism include an over-emphasis on arbitrary threshold for determining statistical significance." The problem in that is that conceptually, the problem is that while we look at the value in the indicator, the statistical indicator of significance, there's always an error in it and we don't pay any attention to the error, as though it is not relevant, you know what I mean, to the question that we have.

So, to me, all of this at the end of the day is less about a set of procedures as much as it is about the conceptual basis of what it is we're trying to understand and not limiting ourselves in terms of the array of kinds of analytical models that help us wrestle with the question, because it isn't you do one or you don't do it. But I think that's too big a hole to jump down in a document like this.

MS. HERNANDEZ LEGORRETA: So are you suggesting instead of us talking about the definition that our only suggestion be deleting it from the language and not suggesting language change?

DR. LEE: I'm suggesting — I'm trying to just deal specifically with what we have in here, so I'm fine with A. If you can kind of go up a little bit. B. I think for the C it would be helpful — we don't have to identify it now, we can easily find that out — might be a reference to some citations in terms of the debates over the question of statistical significance.

DR. SCHNEIDER: So, yeah, I mean, the debates about the term statistical significance are pretty wide-ranging. You know, the P value is one of the major focal points in the sense that it's arbitrary. But worse than that is the American Statistical Association has said many, many, many, many times the misuse of statistical significance and misunderstanding what statistical significance means is endemic, and it doesn't tell you what it's supposed to — what people think it tells you. That's number one. And it moves us from an arbitrary level — an arbitrary bright line that's mostly a function of the size of the sample.

So NAPE, which has a huge, a huge sample at the national level, report has reported something as statistically significant where the effect — the change rounds to zero. So, I mean, so the — what's the right word? So, by talking about statistical significance in this way, it gets us away from the real question, which is the substantive importance, the substantive significance of a finding, right? So, if we could report a zero effect that's statistically significant because we have a large sample size —

DR. LEE: Right.

DR. SCHNEIDER: — well, I mean, that's — what do we do, quite frankly?

DR. LEE: So can you give us a couple of sentences for Conchita to capture the essence of what you said? Because I totally agree with you.

DR. SCHNEIDER: So, yeah, so, I mean, again, you want to add stuff and I just think you've simplified it, right? So the last part under emphasis on the practical significance of findings from quantitative research is the point. So I'm not sure you need a D.

DR. LEE: Just a sentence to add to C. In other words, the nature of the debate over the question as to statistical significance.

DR. SCHNEIDER: Well, I mean —

DR. LEE: The complications of it that can be misunderstood or whatever, some sentence like that.

DR. SCHNEIDER: Yeah. So I would add a citation to the — I could dig it up. The citation to the American Statistical Association has been talking — has multiple times, multiple times called out the misinterpretation of what the P values mean because they don't mean what people often think they mean.

MS. HERNANDEZ LEGORRETA: I just added the —

DR. SCHNEIDER: And I liked the last sentence. I liked the last sentence about underemphasizing the practical significance of findings. That's really important.

DR. LEE: So I can see Peggy and Denisa.

DR. CARR: Well, I, of course, underscore the importance of meaningful differences, not just statistical differences, and that doesn't always mean that it has to be qualitative research, I should point out.

But I think what is missing that hasn't been dealt with in the summary of points is the value of external validity, generalizability, because this definition seems to focus on internal validity, making sure that one can identify the cause and effect through, you know strong internal controls of a study, but there needs to be a balance between internal and external validity. All of this is about is generalizing the findings to the population of interest.

So the value of studies that have generalizability to different populations, different settings, et cetera, is also important here, and I think it's missing in this definition, by the way. But it has not been made a principal point in the need to revise the language because it is omitting what the whole endeavor is about; be able to provide information that can be generalized, used, scaled up, whatever, for the betterment of our education system. That's what it's about.

DR. LEE: So, if I could just add, but not to be added to the language, but just a little seed that I want to keep planting on the sort of sides of human learning and development that if the model of the real world involves dynamic relationships, dynamic relationships that are situated within and across ecological settings and time, and you have a model for analyzing the data that presumes a kind of linear causality, you're not going to have external validity because the conception of the problem doesn't fit reality, and so your methods are inappropriate for trying to capture it.

And just my little seed-planting not about this document is that I do think that this is a conundrum that both IES and in National Science Foundation need to take up because I think it's the cutting-edge shift that's going on in the field that needs to be supported. None of this goes in the document. This is just my little seed-planting here.

Denisa and then Liz.

MS. GANDARA: Yeah, I'd just like to mention I added the reference to the ASA statement that Mark was referencing from the American Statistical Association.

DR. LEE: Is that the Wasserstein or —

MS. GANDARA: Yes.

DR. LEE: Okay. Great. Thanks.

Liz?

DR. ALBRO: Yeah, hi, folks. I just want to recommend that the Board look at the principles of scientifically valid research, which I believe is Section 25. It's written pretty broadly, and I actually feel like it does provide an open — a more open framework than perhaps what — you know, than what we've been discussing here. So I just wanted to make sure folks looked at that section as they're considering this definition. Thanks.

DR. LEE: And, again, for the purposes of getting this document out, we could reference that.

MS. HERNANDEZ LEGORRETA: What was the number again of the section? Sorry.

DR. LEE: Liz, I think that's a question she's asking you.

DR. ALBRO: It's Section 25 on page 10.

DR. LEE: All right. I think that we have enough done on this that we can approve the document, give it to the Policy Committee to put the final, you know, edits, kind of clean up, et cetera.

DR. ALBRO: Oh, I just wanted to add one last point, a small point, that a single study is insufficient.

DR. LEE: All right. So this has been —

DR. SCHNEIDER: Well, I'm sorry. What about Point 5 about ARPA Ed?

MS. HERNANDEZ LEGORRETA: Yeah, we never got to that.

DR. LEE: So can you say more? We hadn't discussed that yet. So is that related to the ESRA reauthorization?

DR. SCHNEIDER: Well, it was — I mean, so the Department of Education, IES, ALI, many have put forward the position that ARPA Ed is needed. It's been high priority for the Department and endorsed by the Administration and it disappeared.

So the question is, I mean, do you — I mean, I could go through what it is. We talked about this yesterday. We talked about, you know, some of the activities that we are supporting, ARPA-like activities that we're doing, the transformative research program, the seedlings to scale. I mean, we have a bunch of things. We got $40 million in appropriations language directing us to do these kinds of activities. Ten million went to the School Pulse, which we talked about yesterday; and 30 million went to the kinds of things, the transformative research program and will support the seedlings to scale. I mean, for me, it's an incredibly high priority and it disappeared.

And there's still ongoing discussions about whether or not if you get back into NCADE — I'm sorry, back into ESRA reauthorization, there's some fall-back positions. So rather than create a new center, which NCADE would be, one possibility is to create a program for innovative research and as a program headed by a deputy director that would report to me and that would be a program with a separate appropriations line.

But I think — so here's the question, right? So the question is where the Board stands on the need for IES to undertake more modern R&D activities compared to the basic research that NCER and NCSER is mostly concerned with, right? So the question is, as an implied science agency, how much more work needs to be focused on more modern R&D activities and more modern, faster research activities?

DR. LEE: So the question — and then I'd like to open it up for discussion — is, are you recommending, Mark, that we address this issue in this document because you are recommending that it be reconsidered in terms of the full development of the ESRA reauthorization bill, or is this something that you're offering separately?

DR. SCHNEIDER: Yes. So, I mean, so, look, I mean, as Matt said, our job is not to tell you what to do but to give you advice and show you the light. And then whether or not you follow how — you know, you have to follow your own parts.

DR. LEE: Yeah, that's what we're looking for.

DR. SCHNEIDER: Yeah, I don't have to tell you that, Carol. I'm just trying to, you know, position what I'm about to say.

DR. LEE: Right.

DR. SCHNEIDER: So NCADE, this has been like ARPA — as many of you know, there's a giant push across government for ARPA — more ARPAs. DARPA, of course, the defense. ARPA was the first. ARPA Energy, ARPA Labor, ARPA-I, Intelligence, ARPA Transportation. There's a lot of — there are many ARPAs that are being formed. They're all aimed at encouraging more rapid-cycle, higher-risk activities. That's the core of them. And one of the distinguishing characteristics, which is something that IES historically has not been that — has not been sufficiently concerned with and that's scaling of activities.

So one of the distinguishing characteristics of ARPAs is that it is concerned with finding interventions that work and then bringing them to scale and often by partnership with tech companies, and DARPA is with obviously Boeing and military contractors. But the whole focal point on rapid turnaround, high risk, high reward, transformative work is something that is in wide demand across the government.

We have worked with OSTP on multiple occasions to fashion what ARPA Ed should look like. We've dealt with ARPA Energy, we've dealt with other ARPAs. And the question is, like, the time is ripe. It's an opportunity to introduce new ways of thinking into our work and the purpose of it obviously could be built on all the basic research that NCER and NCSER has done and to move forward into a different mindset with regard to, you know, what kind of research is needed.

So the other — the final part is that this has been a very high priority, a very high priority for the Department and it's gotten a lot of support in the White House, so OSTP has been a major supporter of this. So the question is, how are you going to think about putting this in as a request that they reconsider NCADE and put it back in?

DR. LEE: Do you have any sense as to why it has been removed?

DR. SCHNEIDER: Why? Well, there are many reasons, but I'm not sure how many of them are — you know, what the accuracy of my — of what I've heard is. So a lot of it is hearsay, shall we say.

DR. LEE: Mm-hmm.
Linda, do you have any thoughts on this?

DR. DARLING-HAMMOND: I'd like to know more about why it's been left out and what people are concerned about. I mean, I've heard, you know, various points of view on it, but I don't feel like I've studied it.

DR. LEE: What other questions would — where might we go to try to get further information?

DR. SCHNEIDER: Well, I'm going to send you a link to the last blog I did on this.

DR. LEE: Can you just summarize for us what's in the blog?

DR. SCHNEIDER: Pretty much what I just told you.

DR. LEE: Liz?

DR. ALBRO: So, yeah, I just wanted to make sure folks saw I put the Request For Information that I mentioned in my talk yesterday about From Seedlings to Scale, which kind of lays out the vision of how we're instantiating what we — a line of work that's similar to the work that might exist under NCADE if folks want to get a sense of the conceptualization from the research side of what it is we're hoping to accomplish.

DR. SCHNEIDER: So we've just gotten the results from that RFI. We got 62 comments, is that right, and we're going through them. But there have been — there are other documents that have been produced. This blog, my blog, is one of the last ones that I know of. But they're all making the argument about why we need a different infrastructure, a more modern infrastructure for at least part of the IES portfolio.

DR. LEE: So the challenge that I'm having is I'm just not clear that we have enough information. I don't really have a position one way or the other personally just because I don't think I know enough at this point. And I understand, you know, the time constraints. I do think that we want to set an agenda for the Board. At this point, I think we ought to try to meet monthly because I'm assuming that we can cancel meetings, right? We don't get beat up if we cancel. We have to have some advance notice if we're canceling in terms of this register business?

MS. PELAEZ: I'm not sure, Carol. I'm going to look into this. This is Ellie.

DR. SCHNEIDER: So, quite frankly, a monthly cadence, depending on what work you expect from the commissioners, is not possible, quite frankly. It's just — I mean, that cadence is — I mean, getting ready for this meeting for the commissioners was like days upon days of work. So I don't — I mean, if you expect this kind of input from the commissioners on a monthly basis, you're going to be disappointed.

And I'm also concerned that — I mean, look at the quality of your Board. I mean, do they have the time for monthly meetings? Look, that's not my business. It's your business. But I'm telling you, from my side, the IES staff are going to be hard-pressed to have a monthly cadence, if you —

DR. LEE: So I think the question — I understand that and appreciate it. I actually had initially thought about the Board meeting every other month, giving the committees time, a month in between, to do their work in preparation for the work of the Board. I do think that the timing generally speaking and particularly of the next meeting should be attached to work that we have to accomplish.

So you, Mark, had indicated, if I understand correctly, in addition to this particular piece around ARPA, several other policy-related initiatives that you were anticipating coming up that you thought would be useful for the Board to weigh in. So depending, again, on the timing needs for the Board to potentially address that, we could decide on perhaps if those issues that you were raising can wait two months, then we can start with a schedule of every other month. If those are issues you think some more immediate time and attention to by the Board, we might have a meeting next month that's specific to those issues, those policy issues that you raised, and then create a schedule going forward potentially every other month.

DR. SCHNEIDER: Well, look, I mean, in contrast to my commissioners, who actually are really hard-working, my job is nowhere near as defined as theirs, quite frankly, so I'd be more than happy — I mean, I could do this more easily than my commissioners, but I don't — you just have to understand that you can't expect the kind of time, prep work, et cetera, that they put in for this meeting on a monthly basis.

DR. LEE: I understand that. All I'm saying is, for me, the timing — I think every other month is meaningful. That's what I originally came in here thinking about. I'm just saying that the issue that seems to me to be time-bound are issues about our responding to upcoming potential policy legislation.

DR. SCHNEIDER: Well, if —

DR. LEE: Several have been raised.

DR. SCHNEIDER: Okay. But if you want to have —

DR. LEE: And if that's the case — —

DR. SCHNEIDER: If you want to have a meeting about SLDS and NDEC, the appropriations process, as you know, is independent of the authorizations. And, I mean, probably Linda, others here have probably a better fix on what's going on with approps. But, you know, sometime in the new year there's going to be voting on the appropriations, including the one that includes Education.

DR. LEE: Right.

DR. SCHNEIDER: So I'm not sure exactly what date. There are two — there's two functions, two steps, and I'm not sure where — which one Ed is in, if it's in the earlier expected deadline or the later one, but that won't take place until January. I mean, the vote for the budget's going to take place in January. Hopefully, an independent bill is not — not another omnibus.

So, I mean, look, as I said, I'd be more than happy to meet with you about these things, and, I mean, I don't know if the — so, if you invite the commissioners and they don't show up, I just don't want a level of unhappiness on the Board's part.

DR. DARLING-HAMMOND: That makes sense.

DR. LEE: No, we won't be unhappy. We won't be unhappy. So, basically, what I'm trying to do, Mark, is be responsive to the issues, the policy issues that you've raised both about the appropriation and these other bills. And if we were to have a meeting let's just say theoretically next month where that was the primary focus and you were the primary point person for information for which we may not need to have the other commissioners present, you know, I mean, that would be fine. I'm just trying to not let the issues that you raised around the timing of policy issues that are on the table to go, you know, unaddressed. Beyond that —

DR. SCHNEIDER: Right. And —

DR. LEE: — meeting every other month would be fine.

DR. SCHNEIDER: Right. And then the amount of time needed for that meeting would be a lot shorter than the day and a half that we have for this one, and I think one of the things that I think we always need to revisit, which I find extremely — many people have a hard time doing, is how long do meetings really need to be, right?

DR. LEE: Mm-hmm.

DR. SCHNEIDER: I mean, that — we need — and, again, if we're going to do hybrids, then, I mean, that's a — we have to readjust actually to — like, when we flew people in and we had in-person meetings, if we just had it for a day, then, obviously, that was a major imposition, but if we do hybrid meetings or online meetings, then, obviously, we could do shorter and tighter ones.

DR. LEE: Well, we had this as a two-day because we hadn't had one. This is our first full-blown meeting to actually deal with content. So, no, we don't necessarily have to have two-day meetings going forward.

One of the other issues — and I think this will also be another reason to have a January meeting and then potentially this will be a vote we can take today to bi- — moving to bi-monthly meetings is we're going to need to get some quick feedback on the Ed executive director search process. That does need to be kicked two months down the line.

So we could have — if we had a January meeting, we could have those two items, you know, on the table and then move to bi-monthly meetings, including the question — making decisions about when and if we want to have any face-to-face meetings considering that the ED position and paying for our meetings all come within that $350,000 that apparently we still don't have, but —

DR. SCHNEIDER: You do not have. That is correct.

DR. LEE: Whose hand is that? I can't see. Is that you?

DR. SCHNEIDER: Linda.

DR. LEE: Oh, Linda, sorry.

DR. DARLING-HAMMOND: Yeah, I was just thinking that there is this pent-up work that is needed that might mean we meet in January virtually, you know, to try to keep that going. Maybe even again in February, because we've got a lot that we want to catch up on and there's a lot happening and then go to every other month.

I think, Carol, part of what you're trying to solve for is the fact that we need to have notice. So it may be easier to notice more meetings than we may even need and we can cancel them, right, rather than, you know, getting up to it and saying, oh, we really need to meet, but we haven't had enough days of notice in order to do it.

So I was just going to endorse your idea of getting a few things on the books that are more frequent in the short run and then going to a longer run, you know, cadence after that. And I think Mark made a really useful point about the fact that the way we used to do meetings back in the old days was everybody flew in and stayed for two days. And I saw your comment, I missed this yesterday, about taking us for drinks at The Wharf, but I want to endorse that.

But I do think that we can have more variability in the ways we meet and in the, you know, cadence and maybe have some of these shorter virtual meetings to catch up on some things and then much less frequently, particularly given the size of the budget, get together for, you know, a more meaty in-depth meeting. So just to encourage us to go with the possibilities that, you know, the pandemic brought, you know, to us for varying the way in which we meet.

DR. LEE: I love when people complain. I love it. I'd rather do Zoom than get on a plane any day, so yeah.

DR. DARLING-HAMMOND: It does feel like we need maybe one or two meetings that are pretty short and pretty soon so that we can get caught up on some of these things that are going to be on our plate.

DR. LEE: Caroline?

MS. SULLIVAN: Yeah, I echo what Linda's saying. I've been on governing boards before and with a lot of statutory obligations and, you know, depending on what the agenda is, you don't need — Mark, everybody doesn't have to come to every meeting depending on what the agenda is. So I think — and if things veer off into one of the spaces where the commissioner is not present, we can table it for the next meeting, so I think that would work.

And I do think we need to go ahead and put some meetings on the books. You know, we've talked about subcommittees. I mean, there's a lot of ways I think that the Board can be helpful to your work and to the busy commissioners' work. And I think we can be a great resource for you all as well, but we sort of have to have a little more understanding and plan on how we can be helpful not just from the legislation that's happening right now but from all the, yeah, dissemination of information and communication and how do you — the DEI work and AI and all of those things. I think we can be a support, but we have to have some sort of regular cadence to the meetings because we can't — you know, we need to be as informed as possible to be able to do a good job with advice, as well as support.

So I'm with Linda. Let's do one in January, one in February, figure out a way to do at least one in person because, Carol, I know. I can get a whole lot done when I'm sitting in front of this screen.

DR. LEE: Right.

MS. SULLIVAN: But the best things I've ever done professionally are because I had personal relationships with people. I mean, so I think that meeting together is super helpful as well.

DR. LEE: Yeah, I agree.

MS. SULLIVAN: We don't have to do it all the time, though.

DR. SCHNEIDER: So the only other thing I would add is, like, January is not the best month (a) to travel because of snowstorms and —

FEMALE VOICE: We don't want a trip.

DR. DARLING-HAMMOND: No, we're going to do a virtual in January.

DR. LEE: No, Mark, we're coming in the spring when we can go out on The Wharf and sit by the water and drink.

DR. SCHNEIDER: Well, don't come during — actually, just come at cherry blossom, which is about a 10-minute walk also from our office, from IES's office. And that's always very crowded but always an interesting time to be in D.C.

DR. DARLING-HAMMOND: Well, I think we can also put Hill visits and anything like that that we might want to do on the same calendar, so I think we can be strategic about our dates.

DR. SCHNEIDER: Sure, absolutely.

DR. LEE: So I'm hearing, one, a January meeting at this point to address both these additional policy issues that Mark has raised, as well as a report on the process of thinking about the ED search. And then I'm thinking that February could be a useful time.

The other big-ticket item that's kind of hanging out there is the NBES — the Board's response to the recommendations from the Academy reports in relationship to what the centers are currently doing. And I think we could put that off until February. If we have the — if the focus for January is this additional policy work and the ED search, there's no reason — I don't think any of the commissioners necessarily need to be there, unless, you know, Mark, you determine someone needs to, I think that could be primarily with you. And then February, for that Committee — the Committee working on those recommendations can be in communication as standing committees with the relevant commissioners without a meeting so that when we do meet there is some — there's a consensus sort of reached about how the Board is responding to the recommendations and IES's uptake of them. So we did that.

And then we waited. That would be February. We'd hold off on March and then set a meeting for April, right? Then we could schedule these all in advance and we have enough time if we start right now, Ellie, on the January meeting to get the three weeks or whatever the time frame prep thing is.

Then we can — could we then also, Ellie, send out a Doodle for January, February, and April dates so we can get the dates decided and people could get them on their agendas because one of the problems is we have a board of very busy people and trying to find dates that we can all agree upon. Then that would give us at least a three-month cycle for moving forward. Does that sound reasonable?

DR. DARLING-HAMMOND: It sounds good to me.

DR. LEE: And then I would assume —

DR. DARLING-HAMMOND: And then, after April, would you then propose that we go to an every other month cadence?

DR. LEE: Yes.

DR. DARLING-HAMMOND: Is that your thought?

DR. LEE: Yeah, right. And I think we could continue that discussion in January and February so then by February we would be in a position to create a schedule essentially for the year, including identifying at least one where we meet face-to-face.

And, Mark, this can just be some internal conversation in terms — or Ellie, of how we get confirmation of the point at which we can have money to spend, have money to hire, have money to plan face-to-face meeting.

DR. SCHNEIDER: Well, just watch the budget process. I mean, until the budget is passed, we have no money.

DR. LEE: So it wasn't allocated for this past — for the past year and —

DR. SCHNEIDER: Well, and last year, though —

DR. LEE: — these meetings already would have been planned?

DR. SCHNEIDER: — Ellie —

DR. LEE: They didn't assume we'd have money?

DR. SCHNEIDER: There's no — right, there was no appropriation for this activity in last year's budget, and the only way that there will be money for this is in the current budget — in, I'm sorry, whatever budget is passed in the next — hopefully within the next month.

MS. PELAEZ: That's right. That's right.

DR. LEE: So this is so humorous. I'm coming back taking into Shaun's shoes that we were appointed roughly two years ago, so they didn't appoint — allocate any money for us on the assumption that they would appoint us, and so we wouldn't meet because we wouldn't have any money.

DR. SCHNEIDER: I'm not sure it goes exactly like that, but you can tell that story. But it's just —

DR. LEE: I've got to wonder.

DR. DARLING-HAMMOND: So, if they do a CR — if they end up in February doing a CR under — and just continuing on under this budget, we'll have another year of no funding, correct?

DR. SCHNEIDER: I believe that's correct.

DR. LEE: I think what I'm going to do is to take this issue up with the contact we have at the White House because this is —

DR. CARR: That's a good idea.

DR. LEE: — so patently absurd. I'm still waiting to get from the framer the huge, you know, certificate that says I was appointed two years ago to a Board that had no money to do anything with. But anyway, so I'll reach out to Jacob and get some feedback from him relative to that.

DR. SCHNEIDER: Well, it's probably a good thing we didn't have this meeting in person because then that would have been a real problem.

DR. LEE: Yeah, we would have bought our own tickets because I think the November meeting, someone did buy their own ticket. Actually, I bought my own ticket and then thankfully had enough, you know status with United to cancel it, but anyway.

DR. SCHNEIDER: Yeah, it goes on and on.

DR. DARLING-HAMMOND: Yeah.

DR. LEE: Conchita?

MS. HERNANDEZ LEGORRETA: Yeah, I just wanted to clarify on the ARPA issue for the document, is there any decision on if we want to keep it in or take it out? Because, ultimately, what's going to happen is this document would have to be finalized.

DR. LEE: I think, at this point, we need to — I think we need to take it out because we haven't had, but I think we can revisit it in January, Mark, where you can, you know, send us some information or whatever ahead of time so we could have a full-blown discussion.

DR. SCHNEIDER: So I sent you a blog that I did two weeks ago that lays out pretty much the highlights of what we conceive of as NCADE, ARPA Ed.

DR. LEE: Okay. So maybe you could just kind of re-send it, but just a sense of the framing.

DR. SCHNEIDER: It's in the chat.

DR. DARLING-HAMMOND: It's in the chat now.

DR. LEE: Okay.

DR. DARLING-HAMMOND: And then we can do some other, you know, background research and know more about what we're talking about.

DR. LEE: Right. So we have the document that we will be sending.
Conchita, you and Steve and will take a last edit review of the document and work with Ellie and whomever, appropriate staff around a process to send to whom whatever. And any recommendations that you and Stephen may have and Linda, because Linda is joining your committee now, may have about any of the kind of formal follow-up or whatever beyond sending the statement to whomever should receive it.

MS. HERNANDEZ LEGORRETA: So we'll want to, I'm assuming — I just took a look at it. It still has some sort of bullet point kind of things in there that are not yet sentences, and so we'll turn it into some paragraphs and —

DR. LEE: Yeah. Right.

MS. HERNANDEZ LEGORRETA: Right. And just put a fine point on it and get it out.

DR. LEE: All right.

DR. SCHNEIDER: So the one thing that I would suggest you do as soon as possible is you need to check with probably OGC about what the process is, the legal process or the — I mean, I don't know if a statement from the Board needs to go through clearance. I mean, I don't know if it goes to the Office of Congressional Legislative Affairs, OCLA, and they forward it. I mean, there are many — there could be different wrinkles and different avenues forward. And I think the most important — I mean, I think you probably, Carol, or somebody needs to call up OGC and get the process by which your documents go to be — what's the process for getting things cleared, if they need to be cleared, on their way to the Congress.

DR. LEE: That's fine.

DR. SCHNEIDER: Yeah. But, again —

DR. LEE: It could be just referencing back to in terms of the committee that I have to keep going back reading that charter for the Board, but I am pretty certain that there is language in there about communications between the Board and Congress.

DR. SCHNEIDER: Well, please —

DR. DARLING-HAMMOND: Yeah, but Mark is adding to that —

DR. LEE: But the process —

DR. DARLING-HAMMOND: — the fact that there's a bunch of, you know —

DR. LEE: Gotcha.

DR. DARLING-HAMMOND: — kind of regulatory appurtenances —

DR. LEE: Right. No, I understand.

DR. DARLING-HAMMOND: — beyond that. Yeah.

DR. SCHNEIDER: Right, right. And, again, welcome aboard and good luck.
(Laughter.)

DR. LEE: The good ship NBES on the rocky waters of the federal government. All right. Well, I think —

DR. DARLING-HAMMOND: So I assume you have the — basically, the Board is delegating to the Committee getting that thing out the door after —

DR. LEE: Absolutely.

DR. DARLING-HAMMOND: — ESRA gets those.

DR. LEE: And just keep us informed of the progress, you know, being made.
One last in terms of the committee structures, we'll send those out again and again would encourage everyone to think about whether you want to change membership, whether you want to serve on multiple committees or not, and then internally to ask each committee to elect a chair so we can have one point of communication to the committees will be very helpful.

So we're going to send the — authorize the Policy Committee to clean up the document between the committee, Ellie, and myself, clarification on whatever the legal process for communicating it to the Congressional committee. The Executive Director Committee will meet.

And one other point around the standing committees, the reason that we made the decision to constitute standing committees means that as standing committees, you can meet outside of the Board, the public Board meeting. I believe Ellie needs to be present, but you can meet to conduct your work outside of the Board meeting. You just can't make decisions. But the function of your committee meetings is to gather data, evidence or whatever to come back and make recommendations, you know, to the Board. So we would want the Policy Committee — again, that would be your task and you just can report back to us.

I think, Mark, if you want to do any general communication or even if you wanted to have conversations with the Policy Committee in anticipation of the January meeting, that's something you all can work out. And we need the Ed Executive Director Committee to now begin meeting. My understanding, you're going to meet with David, the past chair of NBES.

DR. SCHNEIDER: David Chard.

DR. LEE: Chard. Yeah, you're going to meet with him, and if you want, he will put you in connection with the last executive director.

You'll also want to begin to gather some — you know, consulting whatever, any experts that you think would be helpful in terms of defining the position, determining whether it's full-time or half-time, although I do think the charter says half-time, although, when I talked to David, he said the person he hired, he hired full-time, so who knows.

But, to figure that out and while you're doing it, probably need to be thinking about budget questions and that the 350 that we hope they'll eventually give us, we'll need to pay both for the positions as well as meetings, particularly face-to-face meetings that involve traveling, to be part of your considerations. And because you'll have time until February, I would suggest strongly that the subcommittee, the group focusing on working with the various centers, begin to have some discussions about how you're thinking about structuring yourself and probably including some discussions with the commissioners as you work through that. But you can do all of that outside of the Board meeting.

Is there anything else outstanding that you can see for our immediate work going forward?

MS. HERNANDEZ LEGORRETA: Yeah, I was looking to see if — would it be possible to — oh, I'm sorry, Denisa, I just saw your hand up. I should use the hand raising too, but I — would it be okay to go ahead and select the dates for our next Board meeting?

DR. LEE: Yes.

MS. HERNANDEZ LEGORRETA: Just to get those on our calendars? Thank you.

DR. LEE: Absolutely. So maybe people could indicate dates that you're not available in January. Actually, we need — yeah.

MALE VOICE: I thought you were going to send out a Doodle poll.

DR. LEE: We can do a — I think the Doodle can go for February and April. I think it probably would be simple just to eliminate dates now so that the January stuff can start the Federal Register process since we're in December and we need three months' notice, right?

DR. DARLING-HAMMOND: Three weeks.

DR. LEE: Three weeks. Yeah. So the 17th I can't do other than — and the 26th. Other than that, I'm open. Everyone's check — you're checking your calendars now?

DR. DARLING-HAMMOND: Do you want us to just stick those in the chat then?

DR. LEE: I think if you just — each one of us just tell us the dates you can't come. Whatever those are, we will schedule on different dates.

Ellie, why don't you just call the roll and ask people to tell us what dates they can't meet in January.

MS. PELAEZ: Okay. Ms. Hernandez Legorreta?

MS. HERNANDEZ LEGORRETA: Okay. This is more complicated than it seems, doing it this way, so I feel the pressure. Okay. I can't do the 4th or the 5th or the weekend of the 8th to the 12th. And then I believe — let me scan real quick. I believe the other days work for me. Yeah.

DR. SCHNEIDER: Then someone's going to have to coordinate this. I don't understand why you just don't send out a Doodle pool, poll or whatever they're called.

DR. LEE: Because people don't respond immediately to Doodles and they end up taking days instead of minutes.

DR. SCHNEIDER: Well, okay.

DR. LEE: This shouldn't take but five minutes. Just keep the roll going.

MS. PELAEZ: Okay. Dr. Darling-Hammond?
DR. DARLING-HAMMOND: I can't do the 11th, 17th, 18th, 26th.
MS. PELAEZ: Dr. Gandara?
DR. GANDARA: I cannot meet on Wednesdays in January.
MS. PELAEZ: Okay.
DR. LEE: No Wednesdays, right?
DR. GANDARA: No Wednesdays.
DR. LEE: Go ahead.
MS. PELAEZ: Mr. Guy?
MR. GUY: I'm still trying to look at some dates.
MS. PELAEZ: Okay.
MR. GUY: I think during the week will be fine for me.
MS. PELAEZ: Okay. Dr. Hilliard — or Mr. Hilliard?
MR. HILLIARD: I can be flexible.
MS. PELAEZ: Okay. Dr. Lee?
DR. LEE: The — I think I gave my dates. The 17th and the 26th.
MS. PELAEZ: Okay. Dr. Lopez Turley?
DR. LOPEZ TURLEY: I cannot do January 1st through 5th or 12th.
MS. PELAEZ: Dr. Scott?
DR. SCOTT: I could be pretty flexible as long as — my birthday is on the 16th. My wife might have something planned, so let me check with her first.
(Laughter.)
MS. PELAEZ: Ms. Sullivan?
MS. SULLIVAN: We can all eat cake in your honor.
DR. LEE: Yes.
MS. SULLIVAN: I cannot do the 4th or the 17th. I can make other stuff work.
MS. PELAEZ: Dr. Yoshikawa?
DR. YOSHIKAWA: Until the 12th I can only do Eastern Time mornings because I'll be in Japan, and then I cannot do the 22nd or the 23rd.
MS. PELAEZ: Okay.
DR. LEE: Well, what about the 19th of January? It's a Friday. No one has given that date.
DR. GANDARA: Works for me.
DR. LEE: I probably should have put it as a question mark. I might have state board, but usually not.
FEMALE VOICE: I have a faculty meeting that morning that I'm co-chairing. Sorry. I should have also mentioned that. I missed it.
DR. LEE: What about the 10th of January — oh, you — somebody can't do Wednesdays.
MALE VOICE: What about the 24th?
MS. PELAEZ: This is Ellie. We think maybe the 25th looks to be like it'll work.
DR. SCHNEIDER: I'm gone that entire week.
MS. PELAEZ: Oh, okay.
DR. LEE: The whole week of the 22nd, Mark?
DR. SCHNEIDER: Yes, that week.
DR. LEE: So —
DR. DARLING-HAMMOND: Maybe we could do a Wednesday where we work around the faculty meeting because the faculty meeting, whatever it is, is probably only an hour or two, right?
DR. LEE: That would be the 10th or the 24th. What time is your faculty meeting?
FEMALE VOICE: Sorry. My faculty meeting was on a Friday that was proposed, the 19th. On Wednesdays, I teach from 9 a.m. to 4 p.m.
MS. PELAEZ: Friday would be better then.
FEMALE VOICE: Yeah, that Friday, the 19th, yeah, that would be better. My meeting's from 9 a.m. to 10:30 a.m. Central, but I could join you.
DR. DARLING-HAMMOND: I may be chairing state board on the 19th, unfortunately.
DR. LEE: That's all day, Linda?
DR. DARLING-HAMMOND: It could be half a day, but I never know until we get up to it how long they're going to — it depends on how many, you know, charter appeals and things like that we have to hear.

DR. SCHNEIDER: Well, can we get a promise from everybody to do a Doodle poll by tomorrow? Because this is not an effective way of doing this.

DR. LEE: So we can go ahead and do that. We're going to run up against the same problem, and I think what we need is just the criteria for going ahead and perhaps it should be the dates that the vast majority can attend because all the dates we've come up with —

DR. SCHNEIDER: Well, that was going to happen no matter what, right? You're never going to get a meeting, you know, in this short notice, and even with longer notice, there are going to be people, like, who teach from 9 to 4. I don't know how you can manage that.

DR. LEE: You should tell your dean you've been appointed by the President, so you get an exemption to get out of teaching, right?
All right. So let's do —

MALE VOICE: The 29th looks pretty good, right?

DR. LEE: Let's do it for both January and February and ask that everyone please commit to completing it by tomorrow so we can move ahead and get the Federal Register, whatever it is they need to know.

All right. Well, thanks, everyone. We finished an hour —

DR. HARPER: I had a question. I had a question for you.

DR. LEE: Sure.

DR. HARPER: It seems like we were really close on the 29th except for a faculty meeting. Is that still the case?

DR. LEE: The 29th?

DR. HARPER: Yeah, working around the faculty meeting from 9 through 10:30 Central?

FEMALE VOICE: That was the 19th. I don't have anything on the 29th. I'm open.

DR. LEE: No one's mentioned the 29th. Does that work?

MALE VOICE: The 29th is good.

FEMALE VOICE: It's good for me.

DR. LEE: Yeah. All right. The 29th. See, Mark, we could do it. We didn't need to Doodle.

DR. SCHNEIDER: Well, I hate to say it, but I might not be here on the 29th. I'm not sure yet.

DR. LEE: Well, the purpose of that meeting involves you, so it has to be it. So we can go back to Doodle because the whole function of that was to entertain these policy issues that you were raising.

DR. SCHNEIDER: No, I understand. I understand that, and I don't mean to be difficult.

DR. LEE: I'm just saying relative to the date, one of the criteria is it has to be a date when you're able to be there, so —

DR. SCHNEIDER: Right. So I'm sorry. What happened to the 19th?

DR. LEE: The 19th? Who was having a —

DR. DARLING-HAMMOND: That's when I have a state board meeting.

DR. LEE: All right.

DR. DARLING-HAMMOND: I'm sorry.

DR. LEE: So let's go back to the Doodle. We have a sense of the limitations on here. It'll be majority, but it also has to be one when Mark can be present because most of our conversation's going to be with him.
Denisa, did you have your hand up?

DR. GANDARA: It's not about scheduling, though. It was just a couple follow-ups.

DR. LEE: Okay. Go ahead and then we'll be finished.

DR. GANDARA: Okay. So I guess two related things. One, Carol, I think yesterday you mentioned we need to reply to those letters we received in the request for public comments, and I'm wondering what the next steps are for that. Is that right? Did I understand that correctly?

DR. LEE: I think we do — what I had stated publicly is that those recommendations would be taken up by the committee that's looking at the National Academy reports that's all about sets of recommendations, right? So that was my attempt at least publicly to let them know that although we did not directly address them in the public meeting, they would be addressed by the committee.

And we can put together — I can work with Ellie and put together a letter to send to them indicating that the follow-up will have to take place in February at the point when your committee begins to dig in to looking at recommendations across the board.

DR. GANDARA: Okay. And that was my other question was about whether we had any tasks on the committee for considering based on recommendations, if there's anything we should be working on in the interim before the next meeting, but it sounds like there isn't.

DR. LEE: I think working on the Academy recommendations from those three reports is a big job because there's so many and part of that — and so that's what we would like you to recommend on in the February meeting. And I think it would be part of one being very clear meeting with the various commissioners that they agree with the recommendations in the reports. And if they don't, then that's an issue certainly to raise and that hopefully that we should be in a position based on the analysis of recommendations that you all make to vote to approve the plans in place that the various centers have to take up the recommendations.

And in so doing, once we take that vote, I think to also report back to the authorship or committees of the three reports that we have, in fact, taken a position to support them and the efforts of the various centers to take them up.

DR. SCHNEIDER: Okay. So, again, you said three reports, but you're not going to do the NAEP report, so there are two reports, right?

DR. GANDARA: There's the 2019 report also that he wrote.

DR. SCHNEIDER: Oh, we want to do the '19 report from 2019. Right.

DR. GANDARA: Right.

DR. SCHNEIDER: Okay. So I just want to put — again, to put these reports from the Academies in perspective, that is, these did not come from — you know, they did not come from heaven. IES authorized them, commissioned, them and paid for them, right. That's number one. And I'm not sure who paid for the 2019 one, but the two Academies' reports that are still — that you're actively discussing were ones that we commissioned. That's number one.

And number two, you keep using the word "recommendations," which I need you to remember that's all this was. These were recommendations that are not binding in any way.

DR. LEE: But it's also from my —

DR. CARR: Let me just point out that NCES paid for the other report, but you may not have known that, Mark.

DR. SCHNEIDER: The one for '19?

DR. CARR: The Academy report.

DR. LEE: Well, let me just clarify from my perspective that it is my understanding that the NBES stands as an advisory board to IES and, thus, any recommenda — any issues that come out around evaluation of IES should be the purview of the Board to weigh in on, even though —

DR. SCHNEIDER: Right, but —

DR. LEE: — even though IES paid for them.

DR. SCHNEIDER: Right. But the operative word is "recommendations." That's what I want to emphasize.

DR. LEE: Well, one of the issues that is going to be interesting to work through will be instances that may inevitably come up over our tenure as a board where the Board will have recommendations to IES and to you as the director that you or that IES doesn't agree with. And so we've got to, you know, figure out, conceptualize and deal with how we address those issues. I think that involves the complex relationship between the very complicated charter about the relationship between the Board and IES.

If we're not able to enter into debates where the Board, you know, takes one position and the director or commissioners or others take different ones, then there's no point in having us as a board. We're not simply — you know, our job is to engage in that discussion, so I'm just saying —

DR. SCHNEIDER: No, no, look, and we're totally — we are looking forward to the engagement, but, again, this is an advisory board, not a governing board. I mean, you always have to keep that in mind, that the recommendations that you're making are recommendations and almost every one of the pathways goes through the director, right? Almost all your responsibilities are to advise the director on X, Y, and Z, so you do always have to keep — you always have to keep that in mind. And, I mean, look, I mean, we're all here engaging with you, but that doesn't mean that you have the final say.

And the other thing is the — and I'm just — so the other thing and the justification or the basis for my statement is that you have a charter, but that charter cannot supersede the law. And the extent to which we believe — and, again, this may just — I'm sorry that — I want a cordial relationship. I want a friendly relationship. I'm going to support you as much as I can and I hope you support me as much as you can, but that we operate within the boundaries of the law and the charter cannot supersede what the law says is our relationship and the process by which you generate advice to the director.

DR. LEE: I'm very clear on that and I think you and I have discussed and gone back on that, and I think we have reached a good consensus around that.

I'm also just saying that part of that role in terms of advisement is IES also understanding that we have been appointed in this case by the White House to provide guidance and support. Obviously, we can't dictate anything that you or IES does, but I certainly would imagine that there are entities in the government who are watching how we operate, and I don't in any way anticipate that we would end up in a position where the Board would be strongly making some set of recommendations that the director simply — you know what I mean — ignored or pushed back on or whatever, but that won't speak well for any of us.

So it is at the end of the day I understand our role is advising, but, again, I'm sort of stepping into, you know, Shaun's role that if there is not, you know, a sense of how we truly work through coming to agreement in some general sense, you know, on issues, that it also becomes — I totally understand your position. I wouldn't want to be in your role. I wouldn't want to have to deal with such a board. I get that.

But the fact of the matter that exists is on the equal side of this is as Board members we also don't want to feel like we're doing this for nothing, that we're talking to, you know, an audience that doesn't listen in any way to anything we say. I don't expect that, but I do want to make clear that this is part of the balance that we're working through in part because we've been put into a position with a mandate that is very complicated and unclear in its hopeful intentions.

FEMALE VOICE: Can I just add also, Carol, that we were discussing the NAESM recommendations within the context of our task force's charge to develop a process to approve the priorities of the director of IES? And my understanding is that we do have statutory authority to approve the priorities of the director. Is that consistent with your understanding, Mark?

DR. SCHNEIDER: Yes. I mean, that is clearly spelled out as a function of the Board.

FEMALE VOICE: And I really like the idea, Carol, to engage with the director and with the commissioners to continue discussing responses and reactions to the recommendations of NAESM. We've spent a limited amount of time during this full Board meeting, but I would personally welcome the opportunity to continue those conversations.

DR. LEE: And so that's what we anticipate a good deal of the February meeting will be focused on, on that.

So conversation is always good. I think it is through, you know, wrestling with the virgin ideas that we actually come up with the best and most creative ways of thinking about issues.

So, with that, I'm going to call for a close to the meeting. Again, appreciate everybody's time and effort. I think this meeting has been really very, very, very productive, and so we're part of the reawakening of the National Board of Education Sciences. All right. Thanks, everybody.

MS. SULLIVAN: Do you need a motion to adjourn?

DR. DARLING-HAMMOND: Yes. Uh-huh.

MS. SULLIVAN: Motion to adjourn.

MR. GUY: Second.

DR. LEE: All in favor, say aye.

(Chorus of ayes.)

(Whereupon, at 2:00 p.m., the meeting in the above-entitled matter was concluded.)

REPORTER'S CERTIFICATE

CASE TITLE: National Board of Education Sciences
Open Public Virtual Meeting (Day Two)
HEARING DATE: December 5, 2023
LOCATION: Washington, D.C.

I hereby certify that the proceedings and evidence are contained fully and accurately on the tapes and notes reported by me at the hearing in the above matter before the Institute for Education Sciences.

Date: December 5, 2023
David Jones
Official Reporter
Heritage Reporting Corporation
Suite 206
1220 L Street, N.W.
Washington, D.C. 20005-4018

The National Board for Education Sciences is a Federal advisory committee chartered by Congress, operating under the Federal Advisory Committee Act (FACA); 5 U.S.C., App. 2). The Board provides advice to the Director on the policies of the Institute of Education Sciences. The findings and recommendations of the Board do not represent the views of the Agency, and this document does not represent information approved or disseminated by the Department of Education.