“Why,” she asks, “are so many of America’s brightest students apparently interested in teaching but not availing themselves of the training their school has to offer?”

A decade later, Harvard Graduate School of Education lecturer Zid Mancenido continues to study these same questions. As a former high school social sciences teacher in Canberra, Australia, the educator and researcher says in some ways his work is autobiographical. When he was an undergraduate student deciding to become a teacher, he said he heard his peers say, “I’d love to be a teacher but…” He now wants to better understand this apprehension.

“Often when we talk about teacher recruitment we only talk about things like how much teachers are paid or this amorphous thing like teachers aren’t prestigious enough … All of those extrinsic motivators [do matter],”, Mancenido said, “but there’s a lot of subtle, more social drivers around those things that really need to be paid attention to.”

Mancenido first began exploring these questions as part of his doctoral research at Harvard, when he collected the stories of over 100 college seniors or recent graduates to gain a greater understanding of what differentiates people who are categorically uninterested in teaching, those who are interested in teaching but ultimately pursue another career and those who are committed to teaching. 

Recently, he spoke with ����Ӱ��’s Amanda Geduld about his work and why, despite permeating defeatism about how to improve the education system, he ultimately remains hopeful. 

This interview has been edited for length and clarity. 

����Ӱ��: Can you share an overview of your “How High Achievers Learn That They Should Not Become Teachers,” and some key findings?

Zid Mancenido: This paper tries to get a little deeper into the questions of “What makes someone want to become a teacher?” or “What makes someone not want to become a teacher?” Often in the news or in public media, people talk about, “Well, it’s just about pay” or “It’s about pathways,” or “It’s about, you know, schools being really messy.” There are just lots of ideas about what it could be for why people want to or don’t want to become teachers …

What I found is that high achievers aren’t born knowing that they want to become teachers or they’re not born knowing that teaching isn’t an appropriate career path for them. They learn it and they relearn it through a number of signals that are built into their interactions with family and friends. They learn it through others’ expectations of them and their careers. They learn it through observing the career trajectories of their role models and their peers. And it really emphasizes that these sort of negative conceptions of teaching that we all sort of generally know but can’t really land entirely. They’re really fostered and reinforced by all the social signals that are around us.

So if we really want more people — more academic high achievers — to become teachers, we need to work on that social discourse. We can’t just be thinking about the individual’s choice in becoming a teacher. We have to think about how everyone perceives and thinks about teaching and how that influences people’s decision making.

What are some of those signals that high achievers are receiving? 

A lot of people might think that these signals are explicit. They’re things like parents and friends responding to you when you say, “Oh I think I’m becoming a teacher,” and those parents and friends say, “No, don’t become a teacher. You shouldn’t do that. You’re too smart for that,” or something like that. And that’s something I did hear amongst my participants. Many of my participants did mention explicit signals like that.

But rather what most participants reported was that it’s much more subtle. It’s things like going to career fairs and seeing that everyone is milling around corporate, high-flying jobs — legal pathways, medical pathways — but very few people standing around the teaching pathways, the sort of social, public service career pathways. It’s in lots of people coming into college and saying they’re really interested in social-impact careers, but then all of them taking pre-med, pre-law, economics, business, all of those sorts of majors.

It’s in the really, really subtle way that as one of my participants, Amanda, mentions that when she says that she’s interested in becoming a lawyer or a diplomat or something like that, people light up and go, “That’s a really exciting career. I’m so excited for you.” But then when she says something like, “Oh, I’m interested in becoming a teacher,” they go, “Oh, that’s really nice.” And then change the subject.

Part 1: Students are exposed to messaging — both explicit and subtle — that teaching is not a desirable career. Harvard’s Zid Mancenido wants to fix that

I know this is a question that you’ve been asked before, but what made you focus specifically on high-achieving students and how do you define what a high-achieving student is?

This is a really messy and challenging thing because truthfully there’s so many ways to be a high achiever in the world … And often because of the sort of standard, conventional ideas of what makes someone a high achiever, we’re really limited and we box kids into saying, “Oh, you’re an achiever or not.”

So I recognize that it’s a really complex social environment with these sorts of identifying figures. I wanted to step back and go — who do we want in the profession? We want smart, talented, funny, committed people, passionate people who are interested in what they’re doing.

And when I looked to the literature and the recent work around what kinds of teachers lead to more academic outcomes in schools, I found there’s a bit of a mismatch there of like, well, we want these kinds of people to be teachers, but we’re not really measuring that. 

What we are measuring in the research is academic achievement. And what we do find is high academic achievement amongst intending teachers does predict future improvements in student outcomes …

Would increasing pay be effective? How necessary is the social component? And how do you see this playing out on the ground, from a policy perspective?

What my research suggests is exactly what the research really has continued to affirm over the past few decades: There’s no silver bullet to getting more academic high achievers into teaching. 

What is particularly interesting about all of the work on offering financial incentives at career entry — scholarships, loan forgiveness or alternative certification pathways — all that research generally finds those policies to be effective, but some of the research finds that it’s only effective for individuals who are already interested in teaching. So if you’re interested in teaching, but you’re on the edge of, “Do I do teaching or not?” scholarships, loan forgiveness, all of that makes you more likely to want to become a teacher.

What that’s missing, though, is that whole other pool: people who aren’t interested in teaching because they haven’t been exposed to it as a career, because they don’t have the signals around them to encourage them to even consider teaching in the first place. And that’s a much larger pool of people who we really could have a lot of leverage on if we just had more structured pathways into exploring teaching as a career, encouraging people to be thinking about teaching, supporting them socially to want to become teachers …

Part 2: Students are exposed to messaging — both explicit and subtle — that teaching is not a desirable career. Harvard’s Zid Mancenido wants to fix that

Can you talk a little bit about what an example of that might look like?

It can be as simple as school systems or colleges collaborating to identify and raise the profile of alumni who have become teachers … Some of my participants talked about wanting to make sure that their degree was worth it. And so in elevating alumni who are teaching, who are in education pathways, is one way to say, “Actually, yeah, this is something that’s worth it because it gets these sorts of outcomes.”

We could also do things like elite colleges could be providing various summer internships or term time options where they are working in schools for course credit, trying to get students to explore what it looks and sounds like when you’re in a K –12 setting …

Or it can be sort of more broad: It can be things like school systems finding ways to go beyond the sort of teacher appreciation days and actually go into a, “What does it take in order to run a school system?” days where we really build up our collective understanding, peak behind the curtain of schools and really understand what does it take in order to create a healthy flourishing system? …

What I’m hearing you say is that by introducing that to a larger body of students, you’re perhaps opening the door for more students to become interested in this career path in the first place rather than just communicating with students who are already considering it.

That’s correct. Recently I’ve talked to some of my friends who are teachers, and I asked them questions like, “When was the last time that you said to one of your high-achieving students, ‘You should become a doctor’ or ‘Have you thought about becoming a lawyer’ or ‘Have you thought about going to this elite college?’” And every single one of them says, “I say that every time. It’s so important to be supporting your kids’ aspirations.”

And I go, “When was the last time you told one of your high-achieving students that they should consider becoming a teacher?” And so few of them say yes. That’s a small switch for teachers to make. Us turning around to high-achieving students and going, “Have you ever thought about becoming a teacher? I was a teacher once and it was a really incredible career that could potentially change the game.”

Well, one thing that strikes me there is then how much of that is about teacher satisfaction? 

Right.

So, are teachers not telling students to become teachers because they are not personally satisfied with their careers?

What I found really interesting in my research was that if you had parents who were teachers, the influence could go both ways. Some participants said, “My parents were teachers, and they lived such great lives and they got to do such important stuff. And so I want to become a teacher as well.” And then you have some people whose parents were teachers who said, “I watched my parents every day. I really learned through that what teaching was and I would never want to do that.” 

Some of my research was trying to find out how much of this was just people’s partial view of teaching. If they had seen a different part of teaching would their minds have changed? While I don’t have the data to suggest one way or another, what I do have is a lot of really strong participant beliefs that what they saw and observed were really pivotal in influencing how they ended up wanting to become a teacher or not …

It’s about creating the environment that allows for parents and teachers to want to encourage students to become teachers themselves. 

For this paper, you collected the stories of over 100 college seniors or recent graduates. Was there one perspective teacher’s story that stood out to you?

… Graham was a student who had gone to an urban charter school on the West Coast, and his mother was a teacher in that school. He had asked himself all the time while he was going through high school, “Why is my education like this and other people’s education like that?” 

He was really, really committed to education and had written about it in his college admissions essay, but then went to an elite college in the Northeast. [Once he was there] all of his friends who had said that they were [also] interested in education were voting with their feet and ended up majoring in business or economics or computer science. And when they would talk about their interest in education, they would always be doing it as a volunteering opportunity or an extracurricular.

When I probed him on why he was doing that, what were the underlying assumptions, he said to me that many of his other peers felt like — given the sorts of education they had — they couldn’t do work that was more technical or “front line” [like teaching]. They really wanted things that were more broad, high level or working in policy or in the strategic area. 

He reflected on how these codes that high achievers use — “front line,” “technical versus high-level policy having more of an impact” — were language games. They were delineations of occupational prestige and status that were masked otherwise as personal preference. 

And so it was a really interesting conversation that really illuminated how there’s no malice here, there’s no talking down about teaching. Everyone thinks teaching and education are really incredible, important careers. But in terms of the choices that we make in terms of the language that we use, it’s incredibly subtle but really streams people away from wanting to become teachers.

So where is he now? Did he end up in the classroom or did he stay in management consulting?

It was years later that he ended up going into the classroom, but there was a sort of personal realization about a number of all of these different things and a return to what really mattered to him.

And that’s what was also really exciting about undertaking this — that it told me that people can have these trajectories, but these trajectories aren’t path-dependent, these things can change. And our social environments can change and the support that we get to encourage people to want to become teachers can change and get people to make that choice in the end.

All isn’t lost, basically, when it comes to encouraging more people to become teachers.

The results of the benchmark federal exam, referred to as the “Nation’s Report Card,” revealed startling declines in student performance, including the first-ever drop in long-term math scores. Nine-year-olds, who have made steady progress since the test was first administered in the early 1970s, saw roughly twenty years of measured growth evaporate between 2020 and 2022. 

Using the results of state standardized tests, as well as private assessments like the MAP or iReady exams, a growing cadre of academics have offered evidence of K-12 learning deficits produced COVID’s disruption to school operations — including signs that Democratic-leaning states and districts, which were more likely to close schools longer, saw less instruction and steeper hits to achievement. But NAEP provides the first nationally representative data confirming those suspicions and charting the diverging effects on distinct student groups.

While average math and English scores fell for virtually all students, historically disadvantaged children — among them African Americans, Hispanics, the poor, and academically struggling students — generally saw larger drops, widening the gaps with their higher-scoring peers. 

Martin West is the academic dean and Henry Less Shattuck professor of education at the Harvard Graduate School of Education. As a member of the National Assessment Governing Board, an independent, nonpartisan body that sets policy for NAEP, he also has a unique perspective on the test and the data it generates.

In an email exchange Thursday with ����Ӱ��’s Kevin Mahnken, West spoke about the results, the possibility of growing educational inequity as a result of the pandemic, and public perception of schools in its wake. Asked whether the steep drop in performance could be remedied with time and more resources, his answer was stark.

“The honest answer is that we don’t know, as we’ve never seen a decline of this size and scope before,” West said.

The interview was edited for length and clarity.

����Ӱ��: The public reaction to these results has been huge — almost surprisingly so, given that prior studies have indicated significant learning deficits resulting from the pandemic. Do you think NAEP’s role as the authoritative national exam just makes these trends un-ignorable?

Martin West: It has certainly been gratifying, as a member of the National Assessment Governing Board, to see the strong reaction to the results. And I do think that reaction speaks to NAEP’s reputation as the authoritative source for tracking the achievement of American students over time. The NAEP Long-Term Trend, in particular, has remained essentially unchanged for more than fifty years. NAEP assessments are also the only tests that are routinely administered to samples that are truly representative of the nation as a whole. I think the latter factor in particular helps explain why NAEP results garner so much national media coverage. Reports based on state tests are inherently local stories. Reports based on interim assessment data leave room for doubts about whether they are truly representative. NAEP releases are national stories.

Do you think the steady trickle of bad news, whether from NAEP or other tests, is related to the diminished perceptions of public schools found in the EdNext poll, among others?

I think it is impossible to separate the role bad news about test scores has played in shaping public perceptions from the role played by factors like prolonged school closures that caused test scores to decline in the first place. So the bad news is definitely related to the diminished perceptions of school quality, but I don’t think we can say for sure that it has an independent effect.

It sounds pretty bad for students to be scoring at levels that were last seen 20 or more years ago. But what should we expect in terms of bounce-back achievement now that schools are essentially all offering in-person learning? In other words, while this is a very sharp one-time decline, is it likely to reset the learning trajectory for millions of kids permanently?

The honest answer is that we don’t know, as we’ve never seen a decline of this size and scope before. Recent reports based on interim assessments suggest that, as students resumed in-person instruction, they have generally demonstrated rates of achievement growth that were typical before the pandemic. That is encouraging as far as it goes, but it would not be enough to help the students whose educations have been disrupted return to where they would have been absent the pandemic.

This is one reason why I think it is critical to be clear about what we mean by recovery. Over the next decade, as students whose learning was not disrupted by the pandemic begin to move through the K-12 system, I’d expect NAEP results to revert back to pre-pandemic levels. We might then be tempted to say that the system as a whole has bounced back. But there are roughly 50 million students whose educations were disrupted — including two of my sons — and I would not want us to declare success unless we’ve also helped the specific students who were impacted make up lost ground. We have an obligation to help them experience accelerated rather than typical growth going forward.

On the other hand, these large scoring drops are presumably also interacting with the long-term stagnation or declines that we saw in last year’s release of long-term trends data from before the pandemic. If the pre-COVID situation was essentially one of weak growth, is it fair to say that the mere return to in-person learning won’t be enough to get students back on track?

That’s exactly right. From 2009 to 2019, we’ve seen the unfortunate combination of stagnant average scores and growing inequality between higher- and lower-achieving students. Today’s release confirmed that those lower-achieving students were also hardest hit by the pandemic. A return to business as usual would therefore only reinforce the pandemic’s unequal effects rather than offset them.

My impression is that the Long-Term Trend results since the early ’70s have essentially shown slowly shrinking performance gaps between students in different subgroups. But yesterday’s release indicated that the math disparity between white and African American students is now growing, and I believe Hispanic and students in the National School Lunch Program (a common metric of poverty) also experienced larger declines in math than white and non-NSLP students, respectively. How concerned should we be that COVID has not only led to general learning loss, but also hindered the progress of historically disadvantaged subgroups?

One legitimate (if clearly partial) success story of American education that is well documented by the Long-Term Trend NAEP is the gradual narrowing of achievement gaps between racial and ethnic groups. It is therefore jarring to see the math gap between Black and white students increase sharply in this year’s data. The other differences you note across groups were not large enough to be statistically significant, but they do point in the same direction of greater inequality. This is not necessarily surprising given what we know about the pandemic’s impact on historically disadvantaged subgroups generally, but it is certainly concerning.

Were you as surprised as I was by the reading scores, which didn’t show widening racial gaps between white, Hispanic, and African American students? Given what existing studies have shown about literacy setbacks during the pandemic, I was expecting a different result. It also seemed noteworthy that schools in cities, which obviously enroll disproportionate percentages of non-white students, didn’t see lower reading scores for nine-year-olds. How much faith should we put in these figures?

I agree that this pattern in reading is a bit unexpected, all things considered, but I find it hard to be too encouraged by results showing an equally large decline across these three racial groups. I’ll also be curious to see if this pattern is confirmed on the “main NAEP” results set for release this October. The lack of a decline for city schools is also a puzzle. Here, though, it is important to keep in mind that the NCES [National Center for Education Statistics, which administers the NAEP test] definition of “city” schools is not limited to large urban districts. 

For example, my own local district in Newton, Massachusetts, is classified as a “city” district despite the fact that most Boston-area residents would think of it as a suburb. The results for the 26 school districts that participate in the Trial Urban District Assessment and will be included in the main NAEP release will provide a clearer picture of what’s happened in the nation’s big-city schools.

Dismal releases from the National Assessment of Educational Progress (NAEP) — often referred to as the “nation’s report card” — have become a biannual tradition in recent years as academic progress first stalled, then eroded for both fourth and eighth graders. But today’s publication, tracking long-term academic trends for 9-year-olds from the 1970s to the present, includes the first federal assessment of how learning was affected by COVID-19.

The picture it offers is bleak. In a special data collection combining scores from early 2020, just before schools began to close, with additional results from the winter of 2022, the report shows average long-term math performance falling for the first time ever; in reading, scores saw the biggest drop in 30 years. And in another familiar development, the declines were much larger for students at lower performance levels, widening already-huge learning disparities between the country’s high- and low-achievers. 

The results somewhat mirror last fall’s release of scores for 13-year-olds, which also revealed unprecedented learning reversals on the long-term exam. But that data was only collected through the fall of 2019; the latest evidence shows further harm sustained by younger students in the following years. 

Peggy Carr, commissioner of the National Center for Education Statistics, said on a call with reporters that the “sobering” findings illustrated the learning losses inflicted by prolonged school closures and student dislocation. 

“It’s clear that COVID-19 shocked American education and stunned the academic growth of this age group of students,” Carr said. “We don’t make this statement lightly.”

Average math scores for 9-year-olds sank by a staggering seven points between 2020 and 2022, the only such decline since the long-term test was first administered in 1973. Average reading performance — generally by schooling than math, and therefore theoretically shielded from pandemic shock — fell by five points. 

Inevitably, that means that fewer students hit the test’s benchmark performance levels than two years ago. For math, the percentage of 9-year-olds scoring at 250 or above (defined as “numerical operations and basic problem solving”) fell from 44 percent of test takers to 37 percent this year; those scoring 200 or higher (“beginning skills and understanding”) fell from 86 percent to 80 percent; even the vast majority scoring at the most basic threshold of 150 (“simple arithmetic facts”) shrank slightly, from 98 percent to 97 percent, across the two testing periods.

No demographic subgroup saw gains on the test, but disparities existed in the rates of decline. For instance, math achievement for white 9-year-olds dropped by five points, but for their Hispanic and African American counterparts, the damage was even greater (eight points and 13 points, respectively). As a result, the math achievement gap between whites and African Americans increased by a statistically significant amount. 

In reading, scores for African Americans, Hispanics, and whites were all six points lower, leaving relative gaps unchanged. Scores for Asian students only fell by one point. 

Notably, the long-term trend assessment differs somewhat from the main NAEP test administered every two years. It follows student performance going back a half-century, and it is taken with a paper and pencil instead of digitally. For the most part, testing items are unchanged from the early 1970s, assessing more basic skills of literacy and computation than are generally seen on the main NAEP.

The broad trend-line has been positive over the life of the exam, and even in the most recent release, student scores on both subjects are far higher than when they were first measured. But Dan Goldhaber, a researcher and longtime observer of student performance, said it was striking to see that upward momentum evaporate so quickly.

“A bit of a hidden story in education, when you look at a swath of 40 or 50 years, is the progress that students have made — and the disproportionate progress that historically marginalized students have made,” said Goldhaber, the of the Center for Analysis of Longitudinal Data in Education Research (CALDER) at the American Institutes. “We’re seeing a lot of that very long-term progress completely erased over the course of a couple of years.”

‘Particularly bad’

One of the most consistent, and consistently worrying, findings of previous NAEP rounds has been the sharp disjunction of students at either end of the performance scale. For over a half-decade, high-scoring students have generally performed a point or two better with each iteration of the test — or at least stayed at the same level — while low-scoring students have seen their scores fall.

The phenomenon of growing outcome gaps is again apparent in the post-COVID results, though it takes a slightly different form. At all performance levels across both subjects, 9-year-olds experienced statistically significant declines in their scores; but even with the identical downward trajectory, struggling students lost so much ground that disparities still expanded.

In reading, 9-year-olds scoring at the 90th percentile of all test takers in 2022 lost two points compared with their predecessors in 2020. But students scoring far below the mean, 10th percentile fell by 10 points.

Consequently, the average reading gap between kids at the 90th versus the 10th percentile grew from 103 points to 110 points in just two years. In math,the divergence grew from 95 points to 105 points over the same period.

Goldhaber said that the trends visible in NAEP performance largely dovetailed with those using test scores from the MAP test, administered by the assessment group NWEA. In multiple data sources, he argued, it has become clear that the pandemic’s effects have been disproportionately negative for already struggling and disadvantaged children.

“It’s not just the drops, it’s where we’re seeing the drops in math and reading tests, and they’re disproportionately at the bottom of the test distribution,” he said. “So the pandemic is reversing a long-term trend of narrowing achievement gaps. That’s particularly bad, to my mind.”

The fact that losses are so heavily concentrated among the lowest-scoring segment of students may help explain what Goldhaber termed an “urgency gap”; neither states, school districts, or even families seemed driven to embrace the generational learning interventions — from dramatically lengthening the school year to implementing widespread one-to-one tutoring — that the scale of learning loss demands. As just one indicator, billions of dollars of federal COVID aid to schools remains unspent more than a year after it was first allocated.

That may change in the wake of the NAEP release. While previous studies have pointed to similar, and similarly inequitable, learning loss over the last few years by using data from the MAP and state standardized tests, the Nation’s Report Card is seen as the authoritative performance metric for American K-12 schools. As NCES Commissioner Carr noted, today’s release provides the first nationally representative results measuring achievement before and after the pandemic. Ninety-two percent of schools where the test was administered in 2020 were re-assessed earlier this year.

Tom Kane, an economist at the Harvard Graduate School of Education, agreed that NAEP scores definitively affirmed what prior studies have already demonstrated. More observers needed to study the magnitude of the loss, he added, because the proposed academic remedies in most of the country are “nowhere near enough” to combat it.

Kane analogized classroom learning to an industrial process — the conveyor belt slowed in 2020 and 2021, but has resumed functioning since at roughly the same rate as before the pandemic. But to make up for lost time, he argued, it would need to be sped even further.

“What we learned…is that the conveyor belt is back on, but at about the same old speed,” Kane said. “Somehow, we’ve got to figure out how to help students learn even more per year in the next few years, or these losses will become permanent. And that will be a tragedy.”

Along with collaborators from Dartmouth, the CALDER Center at the American Institutes for Research, and the nonprofit testing group NWEA, Kane released incorporating pre- and post-pandemic testing data from over 2 million students in 49 states. Its conclusion: Remote instruction was a “primary driver of widening achievement gaps” over the last several years, with schools serving poor and non-white students suffering some of the greatest setbacks. 

Overall, Kane and his co-authors found, high-poverty schools were more likely than others in the same district to stay remote throughout the 2020-21 school year; among all schools that stayed remote for longer, students at high-poverty schools showed much worse declines in math scores. And they calculated that some school districts would have to spend every dollar of their federal COVID relief money on academic recovery efforts to have any hope of making up the lost ground.

As Kane observed for the Atlantic, local education authorities are required to use only 20 percent of those funds on pandemic-specific remediation. And there is sufficient reason to doubt that even the most promising educational interventions, such as personalized tutoring, can be delivered at the necessary scale to reverse the damage inflicted by COVID. Even the Biden administration’s recently announced campaign to recruit 250,000 new tutors and mentors is at least several months away from being fully realized.

Kane, the faculty director of Harvard’s Center for Education Policy Research, has spent decades carefully evaluating the effectiveness of school improvement efforts. A Council of Economic Advisors staffer during the Clinton presidency, he has studied school accountability systems, teacher recruitment policies, and the effects of affirmative action throughout long stints in both academia and think tanks like the Brookings Institution. His research on teacher evaluation inspired a half-billion-dollar initiative launched by the Bill & Melinda Gates Foundation to lift classroom performance in large school districts around the country.

Now he’s hoping to work with state and district leaders to combat an educational disaster whose effects, he says, are still not well understood. While policymakers may now have a loose idea of the challenges facing educators and families, the policies they’re currently reaching for will likely prove inadequate as a solution.

“Once that sinks in, I think people will realize that more aggressive action is necessary,” Kane said. “In the absence of that, it’s hard to blame local folks for not taking more aggressive action because they have no way to know that what they’re planning is nowhere near enough.”

This interview has been edited for length and clarity.

����Ӱ��: How do your findings and research design differ from earlier studies that have looked at pandemic-related learning loss? I’m thinking specifically of last year’s study conducted by, among others, Brown University’s Emily Oster, which also pointed to really steep losses associated with the switch to virtual learning.

Thomas Kane: There are at least two ways that this paper is different. The first is that we’re able to estimate magnitudes of losses in a way that’s comparable to the effect sizes of [educational] interventions. In that [Oster] study, they can focus on changes in proficiency rates on state tests. Each state has its own cut score, so the magnitude of the changes in proficiency rates depends on whether that cut score is near the middle or near the tail of the test score distribution. If my cut score is near the middle, even a small decline in achievement can mean a big swing in proficiency. But if my cut score is at the tail, even a large decline in test scores can show up as a small change in the percentage of people who are proficient. 

So while that study could qualitatively describe what was happening — in areas that were remote for longer periods, proficiency rates declined — they really couldn’t characterize the magnitude of the decline in any way that was comparable to the effects sizes, which I think is critical. As we’ve argued, it’s not at all surprising that there were larger losses in places that were remote for longer periods. It’s the magnitude of the losses that’s startling.

This design also lets you make comparisons within districts, as well as between districts, right?

That’s another big difference between our paper and what’s out there now. The [Oster] paper was focused on district proficiency rates, and what they found was that districts with larger shares of minority students and high-poverty schools had larger losses. But it could have been, for instance, that the implementation of remote learning was just weaker in those districts — districts with a higher share of students in poverty may have seen bigger declines in achievement, but the losses could have been similar in the high- and low-poverty schools in those districts.

By being able to look within districts, we were able to test whether the number of weeks of remote instruction had disproportionate impacts on high-poverty schools and minority students in those districts. Our answer was pretty clearly yes, there were bigger losses. And it wasn’t just because the urban districts had a harder time implementing remote instruction; even within those districts, the higher-poverty schools lost more.

You used the word “startling” to describe the learning loss. Were you expecting to see effects of this size?

We went in without any clear expectations on the magnitude of the impacts we would see. The reason why I called it startling was because I know that there are very few educational interventions that have ever been shown to generate a .45 standard deviation impact [a common measure showing the difference in any population from the statistical mean; they can be loosely converted into other units, such as learning time or dollars spent] on achievement. Yet that’s the size of the loss that high-poverty schools that were remote for more than half the year sustained. So it was startling because when we compare the impact estimates of remote learning to the potential impact of the available interventions, it’s clear that there is no one thing that we could say, “If all districts did this and implemented it with fidelity, it would eliminate the gap in one year.” 

For instance: In a review of the pre-pandemic research, tutoring has been found to generate a gain of about .38 standard deviations. Well, you could provide a tutor to every single student in a high-poverty school that was remote for half the year and still not close the gap. You could get close, but you wouldn’t close that gap. And we know that districts are never going to be able to hire enough tutors to provide one to every student in a high-poverty school, let alone deliver that tutoring at the level of quality as these programs evaluated in the research. That’s why it was startling — not just because it conflicted with our prior expectations, but because when we saw it, we realized that we couldn’t come up with a long list of interventions that yield effects of this size.

So what can schools and districts realistically be expected to do in this situation? 

We can’t be thinking of this as a one-year catch-up. If we really are committed to making students whole and eliminating these losses, it’s going to be multiple years. There are other interventions that have been shown to have effects, it’s just that no single intervention gets you all the way. 

One example is double-dose math. There’s , and , that found that an extra period of math instruction over a whole year generates about .2 standard deviations. 

So more districts should probably be thinking about something like that, especially in high-poverty schools. But like tutoring, increased math instruction requires staff; you can’t double the number of math classes students take without increasing the number of math teachers. Again, districts should be considering doing some of that, but it will also have constraints on the scale they can implement. 

Another possibility, which a lot of districts are already planning for, is summer school. There are studies suggesting positive impacts of summer school. But [the effects are] small. The big challenge with summer school is getting kids to attend regularly, because it’s viewed as optional learning time. That’s not a reason not to scale up summer school, it’s just that we shouldn’t think that doubling or even tripling the percentage of kids going to summer school is going to close these gaps. It’s not. You get a learning gain of about .1 standard deviations — around five weeks of learning — based on the pre-pandemic research.

One option that really hasn’t gotten much serious consideration, largely because of political pushback from parents and teachers, is extending the school year. If we extended the school year by five weeks over the next two years, that would obviously cover 10 weeks of instruction. I recognize that teachers would have to be paid more for that time. In fact, they ought to be paid something like time-and-a-half. But that’s the kind of option that I hope will gain attention once people realize the inadequacy of the steps that they’re currently considering, like small increases in summer school or tutoring a small percentage of students. It’ll become apparent that that’s just not enough, though my fear is that it may not become apparent in time. Based on what I’m seeing, most districts are going to find that students are still lagging far behind when they take their state tests in May 2023. The danger is that if they only discover that then, and only start planning more ambitious recovery plans then, much of the federal money will have been spent already. That’s why we’re trying to get the message out about the scale of the declines, and the likely scope of the efforts required to close them, while there’s still time to act. 

Districts only need to spend 20 percent of their COVID relief funds mitigating learning loss. But you and your co-authors created a formula to determine the financial cost of reversing this academic harm, and in many cases, that figure would basically demand every dollar allocated by Washington.

We try to put the scale of the [learning] losses and the amount of aid that districts have received in the same scale. We report both as a share of an annual school district budget, which I think is a useful starting point for thinking about what it’s going to cost a district to recover. If a district has lost the equivalent of, say, 22 weeks of instruction as a result of being remote, and you’re asking what it’s going to cost to make up for that, the lower bound of the estimated cost would have to start with [the question], “What does it cost to provide 22 weeks of instruction in a typical school year?”

The answer would be whatever share of a district’s typical annual budget is spent over 22 weeks. In the paper, we use a 40-week year, under the assumption that salaries are paid over 40 calendar weeks instead of just 36 instructional weeks. And then we put the amount of federal aid that districts got on that same scale — say, what share of a typical year’s budget districts receive. We think that’s a useful starting point for people, and what they’d see is that in the high-poverty districts that were remote for more than half of 2021, the amount of aid they received is basically equivalent to — maybe a little more, but not much more than — the magnitude of their losses in terms of instructional weeks. That just means that, rather than spending the 20 percent minimum that was required in the American Rescue Plan, some districts should be thinking that they’ll need all of that aid for academic catch-up.

I have to say, this conversation is leaving me pretty pessimistic that some of this lost ground can ever be fully recovered. Without asking you to look into a crystal ball, is that concerning you as well?

Yes, but here’s a more hopeful spin: A friend of mine sent me a political ad for one of the gubernatorial candidates in Rhode Island, Helena Foulkes. She says, “I’m running for governor, and my top priority is restoring students’ achievement, and if I fail to restore achievement, I’m not going to run for reelection. Hold me accountable for whether we catch kids up.” 

I would hope more politicians take that pledge, and that the way to judge mayors and school board members and governors over the next couple of years is on whether they succeed in restoring students to their pre-pandemic levels of achievement. It would be that kind of accountability that would wake people up to the need for more aggressive action now. It’s one thing to read these reports about achievement losses nationally, but it’s another thing to see that your own schools, locally, followed exactly the pattern of this report. 

Most districts have seen the statistics from [the Oster paper] and know that their proficiency rates have declined by 10 or 15 percentage points. But that kind of statistic, as we’ve discussed, doesn’t really convey the severity. We’d like to provide districts with the tools to gauge the losses in the kinds of units — like standard deviations, or dollars spent, or weeks of instruction — that they could compare to the effect sizes of educational policies. That could make it easier for people to translate their local losses into a package of interventions of equivalent size. In , I tried to put both the learning loss and the intervention effects into instructional weeks rather than standard deviation units to make it easier.

I think that needs to happen. Local decision makers need to see the scale of their students’ losses in ways that are more readily comparable to the expected effect sizes of the interventions they have to choose from. Once that sinks in, I think people will realize that more aggressive action is necessary. In the absence of that, it’s hard to blame local folks for not taking more aggressive action because they have no way to know that what they’re planning is nowhere near enough. It certainly sounds impressive to say, “We’re going to double our summer school enrollment and provide a tutor to 5 percent of the students in our schools.” All of that would reflect more than the catch-up effort in a typical school year, but it’s only when you compare those to the effect sizes for those interventions, and the magnitude of their losses, that you realize that it’s nowhere near enough. So we’ve got to make that lack of proportionality clearer to local decision makers, and not just in these national reports.

Another recent study using MAP data found that U.S. students had sustained as much academic damage from school closures as kids in Louisiana suffered after Hurricane Katrina. But after the storm, the whole New Orleans school district was fundamentally restructured, such that it’s now mostly composed of charters. What do you think of more drastic attempts to change the organization of schools and districts? 

Here’s one reason why this challenge is greater — and it’s actually related to the situation in Boston. I think that if people were confident that a state takeover would produce the big improvements that are necessary in Boston, there would be political will. The problem is the uncertainty: “If we take this very difficult step, is it going to produce the results we’re hoping for?” 

If some district said, “We’re a high-poverty district, and we were remote for more than half of 2021. What should we do?” I could list a few things they should be trying, but I couldn’t point to a package that would definitely close the gap because it’s an unprecedented gap. There is one thing I think could provide the hope and ammunition that would generate political will: We could organize for the next few months around a set of interventions to be launched in the spring of 2023 and then find a few places that would be willing to try that package of things. If we could evaluate those and generate some results early in the summer of 2023, we could then say, “Here is a set of interventions that, if you implement them, it’ll get you a long way toward closing the gap.” And I think we’d have an easier time persuading people to use the political capital you need to invest in that.

So to anyone reading this interview: If there are districts or states that are willing to implement some really creative catch-up strategies next spring and want to contribute to an evidence base that the rest of the country can use, I want to work with you! Right now, there’s no package of efforts that I’d be confident would be enough to close the gap. Absent that, it’s no wonder that politicians aren’t willing to invest political capital in it. But if we had that, we could all get behind advocating for them. It would help everybody if a small set of districts would step forward and try to provide a model for the rest of the country to copy. 

The clock is ticking, and I think we’d have to do it next spring. I’m sure we could design a study and get results out quickly to people about the type of effort that would generate enough [learning] gains. But there shouldn’t be just one model we’re trying — there should be multiple approaches that we systematically try next spring, and ideally, one or two of them will prove to deliver the effects we need. And then we could organize advocacy around those.

So what comes next for your research in this area?

We’ve been working with a group of 14 districts that are giving us data on which kids they provided tutors to, which kids got double-dose math, and various other things over this past school year. We’ve been working with the NWEA data and hope to have a report out in August laying out the effect sizes that districts got for the interventions they attempted in 2021-22.

In a city renowned for its colleges and universities, Boston Public Schools earned its own acclaim in recent years as an innovative, fast-improving hub of K-12 excellence. Situated in the birthplace of American public education, and combining generous funding with a thriving charter school sector, the district was held up for over a decade as a model of urban education reform. 

But as the 2021-2022 school year draws to a close, those past accolades seem as distant as the days of Horace Mann. Amid plummeting enrollment, persistent achievement gaps, and a nasty COVID hangover, Boston faces perhaps the greatest educational crisis since its scarring experience with desegregation in the 1970s. And in the weeks to come, the city may lose more than its national luster.

In March, Massachusetts Education Commissioner Jeffrey Riley of the state of the district. Both local and national experts wondered openly whether the review, which follows a , was the first step toward a complete takeover of the region’s largest school district. In the months since, of bad press has done nothing to quiet speculation.

The audit, released Monday, provided the latest sign that state authorities are strongly considering action. Despite making improvements in a few areas, the reviewers found, “the district has failed to effectively serve its most vulnerable students, carry out basic operational functions, and address systemic barriers to providing an equitable, quality education.” The situation called for “immediate improvement,” they concluded.

The prospect of receivership (as takeovers are known locally) is hardly unprecedented in Massachusetts, which allows its education department greater latitude to reshape failing school districts than most state authorities elsewhere. But the structural problems facing Boston cast doubt on whether such an effort can be successful.

For three decades, the district has operated substantially under mayoral control, and newly elected Mayor Michelle Wu has already made clear her opposition to state intervention. Republican Gov. Charlie Baker — an education reform ally whose tenure has seen several takeovers — will soon be leaving office, likely to make way for a Democratic successor with sharply different views.

Wu told the that she met with Baker and Riley Friday and that they are still working on an agreement “that will set the district up for success.”

“A lot of what is in the review matches with what our school communities and administrators have been calling for, in how urgently we need to focus on BPS and our young people, and in the need for strong, effective leadership,” she said.

Cara Candal, a senior fellow at , said it was ambiguous whether Riley was leaning toward receivership or a somewhat less drastic approach. While significant obstacles existed, she said the recently completed review demonstrated that “kids aren’t learning, and many are unsafe in school.”

Candal, who calling a takeover Boston’s “best hope” for revival, said her takeaway was that things were “as bad as expected in some places and worse in others. In my opinion, the report underscores that the state needs to move with some urgency to provide BPS with the structures, support, and accountability necessary to effect change … There is a window for the state to act now, and I hope it will.” 

Ultimately, the audit called for “bold, student-centered decision-making and strong execution” to reverse what it described as the district’s “entrenched dysfunction.” What that means in practice is difficult to predict. The Massachusetts Board of Elementary and Secondary Education is expected to deal with the report’s findings at its regular meeting on Tuesday morning.

But Ross Wilson, executive director of the Boston-based Shah Family Foundation, said Massachusetts should consider multiple options for intervention instead of duplicating the takeovers of major districts that have taken place in other states.

“Our state and city have the opportunity to do things differently,” Wilson argued in an email. “We should think creatively, collaboratively, and with urgency about the support and accountability necessary to serve the students of Boston.”

‘A steady stream of negative reports’

Few share Wilson’s historical perspective on the highs and lows of Boston Public Schools. A former kindergarten teacher, school principal, and central office administrator, he finished his career with the district in 2017 as deputy superintendent.

That long tenure gave him an inside look at Boston’s ascent in the late-1990s and 2000s as a district known for continuity and rising performance. The schools were overseen for over a decade by Superintendent Tom Payzant, who placed and enjoyed a strong partnership with the city’s similarly long-serving mayor, Tom Menino. By the end of his tenure, Payzant was frequently named as one of America’s best schools chiefs, and the district the prestigious Broad Prize for excellence in urban education. As measured by the National Assessment of Academic Progress, the academic growth of Boston students that of students in other major districts during this time.

The momentum carried on for several years after Payzant’s departure but eventually began to stall. A major culprit was churn: Including interim appointments, Boston has named four superintendents since 2012. Fast turnover has also extended to the bureaucracy — between 2016 and 2019, the district, and less than 12 percent stayed in the same role — and even to the mayor-appointed school committee, which over the last few years.

Wilson remembered that the strategy for governing both traditional K-12 schools and their more autonomous counterparts (the district operates over 20 “pilot schools” that enjoy greater flexibility in hiring, setting budgets, and choosing curriculum) had “shifted from superintendent to superintendent,” leading to “overall confusion.”

The result of Commissioner Riley’s first review was a highly critical document that pointed to “staggering” rates of student absenteeism; in all, close to one-in-three Boston students attended schools that ranked in the bottom 10 percent across the state. In response, the city joined in a “memorandum of understanding” with Riley’s state education department in March 2020, pledging to turn around achievement in underperforming schools, diversify its workforce, and revamp its oft-troubled system of school transportation. 

But the memorandum went into effect at almost the exact same time that the city’s schools first closed due to COVID-19, not to reopen for fully in-person learning for over a year. As in most of the country, test scores tumbled dramatically during the pandemic. Since students returned to classes, however, Boston has also been plagued by constant bad press, including several of against school employees; at a K-8 school that the school committee voted to close; and that has left the district nearly 20 percent smaller than it when it won the Broad Prize. 

In February, Superintendent Brenda Cassellius that she would resign in June after three tumultuous years. In a letter to the school community, the Globe reported Monday, she vowed to push forward needed changes but acknowledged that “this work will require increasing staffing, operational support, and other resources, including a more robust collaboration with City departments, to ensure that we are prepared to meet all of our students’ needs.”

Paul Reville, a professor at the Harvard Graduate School of Education who formerly served as the Massachusetts Secretary of Education, said that the search for a new superintendent came at a distinctly unpropitious moment. 

“We’re trying to attract a new superintendent at a time when we’re on the heels of two superintendencies that did not end well,” observed Reville, who receivership. “We’re facing the threat of a state takeover, we’ve got a steady stream of negative reports on the performance of the school system, and the governance system is shifting. So you might be a superintendent working for a new boss in two years.”

Top on the list of responsibilities for the next superintendent will be dealing with a daunting set of problems laid out in the state audit. Among them:

• While one-in-five local students take part in special education, that area of services “remains in disarray” two years after the 2020 review found them to be sorely wanting. Education of English learners was also highlighted for particular criticism.
• Boston is not meeting minimal standards for the delivery of essential district services, including school transportation. Late or uncovered bus routes are “significantly disrupting education for tens of thousands of students each month,” the authors wrote.
• Even the grievances identified in the audit may understate the extent of the problems because of a “pattern of inaccurate or misleading data reporting by the district.” BPS officials inflated the number of buses arriving on time, inaccurately reported the number of school bathrooms it had renovated, and possibly displayed incorrect student enrollment and withdrawal data on its public website.

Skepticism on takeovers

But if the problems facing Boston are significant, it’s not clear that receivership is the remedy.

Takeovers are among the most contentious school improvement strategies available to states. Even when launched in cities where schools have struggled to serve students for many years, they often sideline elected boards and offend both teachers and families by abrogating local control. Some scholars contend that by alienating voters — disproportionately those of color in cities like Boston — from governance of their own institutions, takeovers do more civic harm than educational good. 

What’s more, evidence of their effectiveness is somewhat scant. A 2021 study of takeovers initiated in dozens of mid-sized school districts found that, on average, they yielded no positive outcomes on test scores; in fact, the disruption of the move led to further struggles in some communities.

Reville argued that the recent history of district takeovers suggested that most states lacked the capacity or the legal scope to pursue them effectively. 

“I think our legislation gives the state more tools and more power than is the case virtually anywhere else in the country, so if you got a chance to do it, it would be in Massachusetts,” he said. “Still and all, I think the evidence from past experience suggests more modest expectations about state takeover.”

Much of the Massachusetts debate will center on the existing takeovers launched over the last decade in the long-scuffling districts of Southbridge, Holyoke, and Lawrence. None of the three school systems have yet regained control over their school systems, and all still rank among the lowest-performing in the state. Still, initial test results included in the 2021 analysis found that reading test scores had improved somewhat in both Holyoke and Lawrence. Receivership in the latter city was personally overseen by none other than Riley, whose appointment as state schools commissioner was predicated partly on the results he achieved in Lawrence.

“Although nationally we don’t have great evidence that this is a key way to improve academic achievement, it does seem like Massachusetts has a stronger track record in this area than other states at using receivership toward the ends of improving achievement,” said study coauthor Beth Schueler, a professor of education and public policy at the University of Virginia.

Because of the relatively narrow time period under observation, that paper excluded the takeovers of schools in New Orleans and Newark, where student outcomes improved sizably while under state control. But in those cases, a principal tactic of improvement was the expansion of high-performing charter school networks, which came to enroll sizable portions of K-12 students across both cities. Boston is similarly home to in the country, but a statewide cap on new charter schools prevents their expansion.

“As much as I would love to say to Boston families, immediately, ‘We’re going to knock down district boundaries and make choice available to you,’ that’s not going to happen in Massachusetts,” the Pioneer Institute’s Candal said. “I think there are lessons to be learned, but we’re not going to be a Newark or a New Orleans because the other stakeholders in the state won’t allow it.”

A ticking electoral clock

The dynamics of receivership in Boston would differ from prior takeovers in at least one other aspect: Authority would be flowing away from a newly elevated leader with an unblemished record, and toward a state government that is headed for the exits.

Wu, both the first woman and first non-white person elected as Boston mayor, won the Democratic Party’s nomination in 2021 with for the district’s future. In office for just four months, she has already proposed her own “Green New Deal”: a $2 billion investment in school renovation and construction. With Superintendent Cassellius stepping down, she will soon help select BPS’s next leader, the most crucial decision facing the district in the coming months.

Wu’s outsize influence over local schools means that if receivership comes, it will be at the expense of a well-known and highly popular figure rather than the obscure members of a local school board. Wu has already demonstrated her awareness of that advantage by , alongside the head of the Boston Teachers Union, to warn against the possibility of receivership.

In a statement responding to the audit, Boston Teachers Union President Jessica Tang called the timing of the release “suspect, rushed, and ill-advised,” alleging that the state report was marred by unspecified factual errors.

“This is an opportunistic attempt to overcommit the state past the current governor’s tenure to a hostile, unhealthy and burdensome relationship with the city by bullying the new mayor into an untenable, undemocratic, and patronizing arrangement,” Tang said.

In response to the unified pushback, Schueler said she wondered how politics might influence a takeover’s effectiveness.

“Proponents of takeover often point to school board dysfunction as the source of all the problems. What do they see as the source of the problem in Boston, and is that problem going to go away with takeover? It’s not getting rid of the board in this case.”

Receivership is almost always dreaded in local communities, but in Boston, there is another wrinkle: Even while electing Wu last fall, voters also demanding a return to elected school board members. Such a move would also inevitably limit the powers of the new mayor, who has said she favors a hybrid committee including both elected and appointed members. 

Will Austin, a former charter school leader who now serves as the CEO of the nonprofit , argued that while popular opinion might be firmly set against the appointment of a state receiver, state law was unambiguous in delineating Commissioner Riley’s powers to act in struggling school districts — of which Boston is undeniably one. 

“The statute and regulations are clear and blunt,” Austin said. “ A vote by the Board of Elementary and Secondary Education decides this — nothing else.”

But the relevant actors also face a ticking clock. In November, the state’s deep-blue electorate will choose a new governor; it is widely expected that Gov. Baker, a two-term Republican, will be succeeded by a progressive Democrat cut approximately from Wu’s cloth. Whoever that person is — Attorney General Maura Healey appears to be — will have little interest in being accused of disenfranchising Wu and the voters of Boston. So while an opportunity exists to set a receivership in motion, it could disappear before long. 

In the meantime, the district continues its reemergence from the COVID era. With to be the next superintendent, Wu and the school committee could race to make a hire before the state reaches a consensus.

In response to the newly released review, Reville said the situation demanded close cooperation between Boston and the state.

“​​The report reiterates and describes problems that have persisted for a long time. The conversation needs to shift now from diagnosis to prescription. Neither the state nor the city is likely to be able to go it alone. The best chance for a remedy is a robust partnership between state and local leaders…and the political will to overcome resistance to change.”