A new analysis from NWEA, an assessment company, suggests that these children are experiencing learning disruptions even now. 

While kindergarten achievement levels in math and reading largely held steady during and since the pandemic, by first and second grade, students are performing below pre-pandemic averages, according to an of NWEA’s Map Growth assessment data from spring 2017 to spring 2025. In math, at least, first and second graders have shown slow, incremental progress. Gaps in reading achievement, however, seem stubbornly stalled. 

The performance dips in first and second grade are similar to those seen in older grades, said Megan Kuhfeld, director of growth modeling and data analytics at NWEA, who co-led the research. 

“The general pattern of stagnation and lack of recovery in reading is very similar in first and second grade as grades three to eight,” Kuhfeld said, adding that a slow recovery in math is also observed in the later grades. “It’s very parallel across, basically, all the grades except for kindergarten.”

So what’s happening to students as they matriculate from kindergarten to first grade to cause a performance drop?

“That’s the big mystery of the results,” Kuhfeld said.

She was willing to speculate about the cause, leaning on anecdotal evidence from kindergarten teachers and elementary school leaders. 

Chronic absenteeism rates in kindergarten, which are often higher than in any other grade before high school, may mean some students aren’t getting adequate instructional time, Kuhfeld offered, ultimately standing in the way of them grasping the foundational reading and math skills typically acquired in kindergarten.

And many kindergarten teachers have reported that students are showing up with more nascent social and emotional skills than their peers in prior years. They have less experience with important life skills such as sharing, cooperating and self-regulating. 

“Teachers are spending more time having to teach how to behave in a kindergarten classroom — that would normally be the purview of preschool teachers,” Kuhfeld said. “This time spent on behavioral management and behavioral regulation, cumulatively, could be affecting achievement.”

At Western Hills Primary School in Fort Worth, Texas, where students’ MAP Growth assessment results generally align with what NWEA has found nationally, principal Andrea Johnson said both factors could be at play. 

“We’re seeing kids who, if they don’t reach immediate success, we see them dysregulate,” said Johnson, whose school serves students in pre-K through first grade. “They struggle.”

At Western Hills Primary School in Texas, kindergarten and first grade performance in math and reading on NWEA’s Map Growth assessment generally mirror national trends. (Courtesy of Andrea Johnson)

She believes that may be a latent impact of the pandemic on these younger students. Many of them had extra time at home with parents and caregivers, when early care and education programs were closed. 

“They’re used to someone being close and someone solving their problems for them,” Johnson said. “We talk a lot about productive struggle. You’ve gotta let them do it. Give them that mentality, where they’ve gotta connect to that struggle.”

She has definitely seen high rates of absenteeism among students in pre-K and kindergarten, she added. 

“I think they think, ‘pre-K and kinder, they don’t really matter that much,’” Johnson said, adding that she often finds herself trying to communicate to families how crucial those years are for future learning and development.

Most measures of post-pandemic recovery have examined the impacts on students in later grades, making NWEA’s analysis a rare snapshot of students in grades K-2. 

Curriculum Associates, a curriculum and assessment provider, has also evaluated math and reading performance among students in the early grades, finding some similarities and key differences from NWEA’s results. 

NWEA’s Map Growth assessment and Curriculum Associates’ i-Ready Inform assessment are both widely used in U.S. schools, reaching a combined 19 million K-8 students. Both measure student achievement in math and reading, but they differ in approach.

Kristen Huff, head of measurement at Curriculum Associates, pointed out that these two assessments have distinct designs and methodologies — and that they are administered to different samples — which may account for variations in findings.

“From the big picture, we’re seeing the same thing,” Huff said. “Students today who were not in school — some were babies — when the pandemic hit are not performing at the same level as their pre-pandemic peers in either reading or math.”

But in a published in July 2025, Curriculum Associates actually found that students in kindergarten are seeing achievement level drops in both math and reading, and that declining math performance in the early grades is “more drastic” than in reading. 

At a high level, she said, both sets of findings send a similar message, which is that America’s children are not seeing the type of recovery needed to reach pre-pandemic achievement levels. 

“It opens up the question of what is happening,” Huff said. “We can no longer, in my opinion, say that that disrupted learning in 2020 and 2021 is the sole or primary cause of what we’re seeing. There is a larger, systemic issue — or issues — that are impacting this.”

For Rachael Garnick, a former first grade teacher, the results were a reminder of how tough it’s been for schools to recover from historic declines in learning since the pandemic. 

“The literacy scores are still abysmal and we should be displeased,” said Garnick, who heads the Pennsylvania Literacy Coalition. Made up of over 70 organizations, the group has pushed and state officials to fund and implement reading reform.

But despite the discouraging statewide results, she also sees districts, like in northeastern Pennsylvania and the Mohawk Area district, northwest of Pittsburgh, “trending in the right direction,” and demonstrating urgency over reading scores. Their attitude, she said, was “the opposite of ‘If it ain’t broke, don’t fix it.’ Instead, ‘It’s broke; we’ve got to fix it.’ ” 

on pandemic learning loss from NWEA, an assessment company, captured that combination of frustration and hope over the state of academic recovery. About a third of schools have reached pre-COVID performance levels in reading or math, and just 14% have recovered in both subjects. But even some that were hit the hardest, like high-poverty schools, have made impressive gains.

The report was just the latest collection of results pointing to a long road ahead for most schools. Last year’s National Assessment of Educational Progress scores showed students in the majority of states losing more ground, but included a few standouts with strong progress, like Louisiana in reading and Alabama in math. And state test scores tell a similar story: few have topped pre-COVID performance.

It’s not like experts didn’t predict a slow recovery. 

“If student performance improvement follows historical prepandemic trends, it could take decades for students to fully catch up,” researchers with McKinsey and Company, a consulting firm, .

Even the nation’s education chief isn’t expecting good news soon. 

“I would like to say that NAEP scores, when they come out again in January 2027, are going to show marked improvement,” Education Secretary Linda McMahon said in a recent K-12 Dive . “I don’t think they are.”

But Dan Goldhaber, director of the Center for Analysis of Longitudinal Data in Education Research, said it’s important to put NWEA data, and all measures of kids’ learning, in context.

“One of the reasons that we’re not seeing recovery and that the results aren’t better is because of what was happening in the decade ,” he said. “There was a slow degradation of academic achievement.”

Resisters and rebounders

Schools that were able to resist further declines during the pandemic are those that are more likely to be back on track, according to NWEA’s data, which represents five million students who took the MAP Growth tests through fall 2024. Such schools make up nearly three quarters of the recovered schools.

The Los Angeles-area is one example. 

Before the pandemic, the high-poverty, majority Latino district was already seeing gains on state assessments. When testing resumed in 2022, reading scores held steady. Math scores caught up the following year, and the district has continued to post gains ever since. 

Superintendent Darin Brawley highlighted a mix of academic routines, like a math problem of the day, weekly quizzes and challenging writing assignments, that the district continued despite the disruption of school closures. Teachers were encouraged to dial back their use of smart boards in the classroom and require students to keep math and language arts journals to improve retention. 

“Everything was being done on the smart board and kids weren’t notating anything,” Brawley said. “Certain things have to be worked out on paper.”

NWEA data also pointed to what the researchers call “rebounder” schools, those that saw significant drops in achievement but have been able to climb their way back. High-poverty schools are among those with impressive gains, but even districts seeing higher-than-ever performance still struggle to close wide achievement gaps.

“We’ve never had scores this high in English language arts or math,” said Buffy Roberts, associate superintendent of the Charleston County schools in South Carolina. “It’s been quite phenomenal.”

She was talking about , which, unlike NWEA and NAEP, aren’t comparable because states don’t all measure proficiency the same. But they can still reflect post-COVID trends if states haven’t changed their tests since 2019. 

South Carolina’s math test has remained constant. Results show that statewide, scores have nearly recovered. It’s a trend that NWEA noted as well, explaining that while schools “lost significant ground,” in math, many made “substantial gains afterward.”

In Charleston, 54% of students in grades three through eight met or exceeded expectations in math last year, up from 48% in 2019 and about 10 percentage points higher than the state average. The district also made the Harvard Center for Education Policy Research’s fully recovered districts in the nation last year.

Roberts pointed to a swift return to in-person instruction and high-dosage tutoring as some of the factors contributing to strong growth. But she said at the outset of the pandemic, leaders “knew there were some vulnerable groups” that would need “structures and support to mitigate some of that learning loss.”

The district’s , she explained, provided extra dollars to schools with high-poverty students even when the schools didn’t qualify for federal Title I funding. The schools used the funds for extra staff to reduce class sizes, incentives to increase attendance and mental health services.

But there’s still a lot of work to do. In fourth grade math, there’s a more than 50 percentage point gap between white and Black students, and students from wealthier families outscore students in poverty by 39 percentage points. 

“We agree that progress must be faster,” the district on Facebook after a conservative community group to the disparities. 

In an analysis of scores, Education Data Center researchers, led by Brown University’s Emily Oster, were hopeful about continued math recovery in 2026. Of the 32 states that have kept the same math test since before COVID, seven met or exceeded 2019 proficiency rates: Colorado, Georgia, Iowa, Mississippi, Missouri, Rhode Island and Tennessee.

But even if they didn’t, they all made some gains. Despite Pennsylvania’s decline in reading, for example, its performance in math is less than a percentage point from reaching the 2019 level. 

But the results in reading were less encouraging. Six out of 28 states have met or surpassed pre-pandemic performance. But several others, like Massachusetts, Minnesota and Oregon, remain well off that mark. 

Goldhaber, with CALDER, suggested that states haven’t seen improvement on tests because parents trust those scores less than the grades kids bring home on report cards and assignments. 

A recent reiterated that point. In a survey of over 2,000 parents, nearly three quarters said they believe grades more than tests when making decisions about their children’s learning. They’re also less likely to take action, like seeking out tutoring or other help for their child, when grades are good. 

The problem is that because of grade inflation, which was on the rise even before the pandemic, grades are a less accurate measure of how students are really doing. 

The results of that survey were no surprise to Bibb Hubbard, founder and CEO of Learning Heroes, a nonprofit that focuses on helping parents understand achievement data. She said she’s been “screaming from the rooftops for 10 years” that parents are about their kids’ performance. 

“Good grades do not equal grade level,” she said. “Parents are deeply engaged, but we can’t afford to leave them on the sidelines relying on grades alone. The stakes are too high.”

, released Tuesday by the nonprofit testing group NWEA, serve as more evidence of a trend that has stood out in earlier data: Students who were still in elementary school when the pandemic began are experiencing particularly worrisome setbacks as schools try to chart a path to academic recovery. Meanwhile, today’s elementary schoolers have nearly returned to the levels of learning last seen in 2019.

This summer, NWEA circulated scores from their widely used of math and reading. The results, which included the performance of nearly eight million American students from grades 3–8, revealed that today’s eighth graders were a full year behind 2019 learning in both subjects. By contrast, learning delays for third graders were only about one-quarter that size. 

In general, learning loss in science has been less significant than for core disciplines like math and reading because STEM instruction is comparatively limited in the early grades. In a 2018 survey, elementary teachers said they spent only 18 minutes each day focusing on science, compared with about an hour on math and 1.5 hours on reading. 

Susan Kowalski, an NWEA senior researcher who worked on the report, said the scores showed that students who experienced their foundational years of STEM instruction in 2020 and 2021 were now struggling to cope.

“If science is taught at all in elementary school, it’s in the fourth and fifth grades,” Kowalski reflected. “So in 2021, those were the kids who were hardest hit in science, and they are now seventh and eighth graders who have never really rebounded.”

The latest data is drawn from 621 public schools that consistently administered the MAP Growth Science test to the same grade levels between 2017 and 2024. This ongoing sample allowed the research team to measure students this spring against not only those in the pre-COVID period, but also during the initial phases of the pandemic, when tens of millions of students were receiving virtual instruction. 

Those results show that, by the spring of 2021, students across all tested grades had fallen significantly behind in science, with especially sizable learning losses mounting in grades 4 and 5. But elementary schoolers have since recovered the most ground compared with their same-age predecessors of three years ago, with learning gaps reduced by 50% for third graders, 82% for fifth graders, and 33% for sixth graders. Fourth graders, whose performance dropped the furthest during the early stages of the pandemic, have now fully returned to their 2019-era achievement levels in science.

But the gaps for older students — essentially, those who saw the biggest dips three years ago — have grown with time. In 2021, NWEA estimated that seventh and eighth graders would require 0.9 and 1.7 months of additional science instruction, respectively, to catch up to where similar students had been in 2019; by 2024, the projection for students in that grade had grown to 1.7 and 3.2 months of supplemental learning, respectively. 

In other words, kids’ whose initial encounters with science were thwarted by the COVID shock appear to be falling further behind, even as state and federal leaders have provided school districts with billions of dollars to lead recovery efforts. 

Heidi Schweingruber, the director of the National Academies of Sciences, Engineering, and Medicine’s , said that students’ halting progress in science could constrain their life chances in the future. Adolescents begin to develop aspirations for college and career only a few years after they receive their first lessons in science.

“If they’re missing that foundation and can’t follow a strong pathway into high school science, are we closing doors for them in terms of what they might consider after graduation?” Schweingruber asked. “Middle school is the time when kids are starting to develop an identity of who they want to be.”

As other testing data have indicated previously, the pandemic also significantly widened achievement gaps separating students along racial lines. While middle schoolers of all racial and ethnic backgrounds had fallen behind in science by the spring of 2021, Hispanic students lost more ground than their white classmates. Among eighth graders, Hispanics required a projected 3.4 months of academic recovery in 2021 and 6.3 months in 2024; whites in the same age cohort needed 0.5 months of recovery in 2021 and 1.6 months now.

Black students, who have mostly bounced back from the science losses sustained in 2020 and 2021, are still, on average, between 10 and 15 months behind the average achievement levels for children in their grades. Hispanic students, whose progress has only stalled further since 2021, are almost as badly off. 

Erika Shugart, CEO of the , said in a statement that she wasn’t surprised to see negative impacts concentrated among middle schoolers, though she added that persistent gaps in STEM instruction could prove economically and socially destructive in the long run.

“The U.S. is already facing significant challenges producing a STEM-ready workforce,” Shugart wrote. “Science literacy is crucial for making informed decisions about health, the environment, and technology. Falling behind in science education can impair individuals’ ability to engage with and understand complex issues, affecting personal and societal well-being.”

To combat lost science learning, NWEA’s authors recommended different strategies to curb chronic absenteeism, entice students to participate in summer learning opportunities, and weave science instruction into middle school reading instruction, which could improve performance in both subjects.

Kowalski said that, more than any particular approach, educators needed to embrace a “mentality shift” away from remediation and toward learning acceleration. A former high school physics instructor, she argued that schools can’t get their pupils back on track simply by offering them what they missed four years ago.

“They can’t succumb to low expectations and say, ‘These students are behind, so I need to slow down.’ It’s more like, ‘These students are behind, so I need to accelerate.’”

Released Tuesday, from over 7.7 million students who took the widely used MAP Growth tests from NWEA doesn’t bode well for teens entering high school this fall. Finishing 4^th grade when the pandemic hit, many students not only lost at least a year of in-person learning, but also transitioned to middle school during a chaotic period of teacher vacancies and rising absenteeism.  

The 2023-24 results reflect the last tests administered before federal COVID relief funds run out. Districts must allocate any remaining funds by the end of September — a cutoff that is expected to cause further disruption as districts eliminate staff and programs aimed at learning recovery.

Older students don’t make gains as quickly as younger kids and will have to work harder to catch up, the researchers said. At the same time, the effects of the pandemic “continue to reverberate” for children in the early elementary grades, many of whom missed out on preschool because of COVID. On average, students need at least four extra months of schooling to catch up.

“It’s not fun to continue to bring this bad news to the education community, and I certainly wish it was a brighter story to tell,” said Karyn Lewis, director of research and policy partnerships for NWEA. “It is pretty frustrating for us, and I’m sure very disheartening for folks on the ground that are still working very hard to help kids recover.”

Thus far, only two states, California and Colorado, have asked officials for extra time to spend the diminishing relief funds that remain, according to the U.S. Department of Education. That means the question for most leaders is how to keep paying for extra tutoring and staffing levels for students still learning below grade level — especially those belonging to groups that weren’t meeting expectations before the pandemic.

Relief money “made a difference, but it certainly did not eliminate the learning loss,” said Dan Goldhaber, director of the . A he authored showed that recovery linked to those dollars was small, in part because the federal government gave districts few restrictions on how to spend it. 

The American Rescue Plan’s requirement that districts devote 20% of the $122 billion toward reversing learning decline was “super loosey goosey in terms of what that actually meant, how it was measured and what programs counted,” he said.

Some districts that hired teaching assistants to give students additional practice in reading and math have now lost those positions. Dothan Preparatory Academy in Alabama, a seventh- and eighth-grade school, had several staff members who gave students “a few extra lessons” throughout the day based on their MAP scores, said Charles Longshore, an assistant principal. Now those positions are gone. 

He hopes a new sixth grade academy opening next month will better prepare kids for grade-level material. Two years ago, when he joined the Dothan City Schools, just north of the Florida Panhandle, he attended a districtwide administrator meeting where every school’s data was posted on the walls.  

He remembers looking at elementary school scores with “really low” student proficiency rates of roughly 20% to 30%; teachers have been trying to fill those gaps ever since. 

“We’re trying to go backwards to go forwards,” he said. “What third, fourth and fifth grade standards did you miss that are essential for your understanding of seventh and eighth grade standards?”

The NWEA results show achievement gaps continuing to widen. For example, Asian students are showing some growth, but made fewer gains in math last year than during the pre-COVID years. White, Black and Hispanic students, however, continue to lose ground. In both elementary and middle school, Hispanic students need the most additional instruction to reach pre-COVID levels, the data shows.

In reading, the gap between pre-pandemic growth and current trends widened by an average of 36%, compared with 18% in math. It’s possible, NWEA’s Lewis added, that districts focused extra recovery efforts on math because initial data on learning loss showed those declines to be the most severe. 

But that’s left many students without the reading skills to tackle harder books and vocabulary as they move into high school, said Rebecca Kockler, who leads Reading Reimagined, a project of the nonprofit . The organization is funding research to find which literacy strategies work with adolescents, who are easily turned off by books intended for young kids.

The pandemic, she said, only exacerbated a longstanding literacy problem for older students.

“About 30% of American high schoolers for 30 years have been proficient readers, and that really hasn’t changed,” said Kockler, a former Louisiana assistant superintendent who oversaw a redesign of the state’s reading program. “It’s always the hardest to move middle school reading results, and even some of the success we would see in fourth grade didn’t always carry up into middle school.”

School closures were especially hard on students with learning disabilities. Both of Tracy Compton’s daughters, who are entering fifth and seventh grade this fall, have dyslexia and didn’t receive services during the pandemic when they were in the Fairfax County Public Schools in Virginia. 

“The time they were learning to read was during the school board’s shutdown of schools,” she said. Under a , the Fairfax district still pays for makeup services with a private tutor. 

But Compton also moved to a Massachusetts district where she feels her girls are getting the support they need, like access to audio books and noise-canceling headphones during tests to help them focus. “They have made progress, but [have] not fully recovered,” she said.

She said too many parents don’t know their children are behind.

“They see the report card with A’s or whatever and think all is fine,” she said. “They don’t know where else to check and how to weigh things like standardized tests.”

That’s likely because different tests often tell different stories. The MAP results, for example, are worse than what many states have reported about student performance on their own assessments, which are used for accountability. 

Several states last year noted at least partial recovery, and a few showed students had even reached or were exceeding 2019 scores. Lewis explained that state tests measure the “blunt designation between proficient or not,” while MAP tests capture the full spectrum of student achievement levels during the school year.

Districts, particularly low-income districts that received the most funding, need to contend with the latest snapshots of students’ learning as they adjust to the end of federal relief funds, Goldhaber said. 

“How districts go about dealing with the fiscal cliff is going to have pretty significant consequences, particularly for the kids that were most impacted by the pandemic,” he said. If districts have to lay off staff — and newer teachers are the first to go — they should limit the impact on the neediest students. “They’ll be shuffling teachers within districts, and that shuffle itself is harmful for student achievement.”

As more time passes since the pandemic, Lewis added that school leaders might be tempted to stop comparing their students’ performance to pre-COVID levels, when states were in closing achievement gaps. 

“What keeps me up at night is this idea that these persistent achievement gaps are inevitable, that this is just how it’s going to be,” she said. “I don’t think that’s the case, but I do think it takes innovation and creative thinking … to get us out of this mess.”

Oh, and they’re fun.

That sophistication — and a bit of that fun — may soon be coming to school assessments.

Educators and developers are increasingly looking to the digital world of games and simulations to make tests more stealthy, playful and, they hope, useful. In the process, the new assessments may also push schools to become more creative.

“The idea is: Can assessment be more embedded?” said Y.J. Kim, an at the University of Wisconsin–Madison. “Can assessment be more exciting? Can assessment be more flexible?”

In November, NWEA, which publishes the widely used , unveiled a 3D digital assessment on the popular that tests how well middle-schoolers have learned Newton’s .

The game, called Distance Dash, requires two students to work together to launch vehicles of different sizes and payloads. The goal: Get both to the finish line in perfect sync.

Students pick a skateboard, a bike, a grocery cart or an automobile, load each with different items, then collaboratively fine-tune the forces placed on them. The whole time, the game covertly measures several objectives, including whether students understand the principles of acceleration and how to apply optimal force.

Tyler Matta, NWEA’s vice president of learning sciences engineering, said the assessment grew out of the , which require students to analyze and interpret data and understand patterns.

He said helping design it was a stretch for NWEA test makers, who hadn’t previously worked with game designers. “We got to see what goes into building educational games, which was all very novel for us. We learned a ton.”

The organization is working with developer , which has produced . 

“As an assessment, it’s important that you actually have the ability to fail,” explained Filament’s Kenny Green, the project’s producer. The data it generates — for instance, how many times students tried and what modifications they made — are all important for teachers to see. 

The new exam appears as Roblox, the popular gaming platform, moves further into schools. Last October, it said it’ll to expand educational experiences on its platform, two years after an initial $10 million outlay. 

Rebecca Kantar, Roblox’s head of education, said physics lends itself well to such collaborative simulations. Distance Dash, she said, is “representative of the kind of team-based problem solving real scientists do when they’re working through a physics problem in real life.” 

Another recent development: In 2022, the Organisation for Economic Co-operation and Development assessed creative thinking for 15-year-old students in more than 60 countries via the assessment, which boasts interactive items that allow students to submit drawings with a . 

The test also includes open-ended tasks with “no single solution but multiple correct responses,” organizers said. The first results are expected this year.

Advocates hope to someday make tests more personalized and, in many ways, indistinguishable from games, said Bo Stjerne Thomsen of the . “What we hope is that playfulness becomes a serious part of assessment,” he said.

Better still, more playful tests, he said, could open the door for schools to offer more creative, inquiry-based learning. 

He and others who are support the new tests don’t mince words: They envision a world where the kind of high-stakes, multiple-choice tests we all grew up with give way to assessments that for the first time allow teachers to capture a broader array of “non-cognitive qualities” such as teamwork and creativity, while keeping students focused on learning.

“Every time you try to pause an experience or stop a learning experience, it actually stops the engagement,” said Thomsen. It’s the same with play: “As soon as you start measuring play, the play stops.”

‘It’s about you engaging with someone else’

Tests can also be demotivating, even though they’re designed to help students show what they’ve learned, said Yigal Rosen, who led the creation of the PISA test.

He recalled interviewing fourth-graders who had taken NAEP science exams: At least one-third of the questions, according to students, were “super boring” and not engaging.

“They will skip them,” Rosen said. “They will just select ‘Whatever.’”

Now the chief academic officer at , the learning software company, Rosen recalled that when his team tweaked the NAEP test with a “playful version” that invited students to work together, he said, scores rose by 50%. “It’s no longer about you just responding to this dry prompt,” he said. “It’s about you engaging with someone else.”

When they think of playful assessments, most teachers probably think of digital tools like the popular learning platform , which allows teachers to create game show-like quizzes and polls that engage students on mobile phones and other devices. Louisa Rosenheck, Kahoot’s director of pedagogy, admitted that testing, for all its progress, is “still an underdeveloped, untapped area.” 

Digital tools like Kahoot that help teachers do informal assessments as they teach are helpful because they “feel more low-stakes” than traditional tests. “It’s very quick, it’s informative. You can get feedback very, very easily,” she said. “But the question types, the formats, often are still kind of discrete items.”

In that sense, she said, they don’t take advantage of what good games can do: Collect extensive data on students’ thinking and decision making — much more important indicators than whether they got the correct result. But that’s expensive, so many educational games simply assess how far a player gets and how many tasks or levels she completes.

‘Stealth assessment’

Researchers have been toying with the idea of more playful assessments for decades. Nearly 20 years ago, researcher began looking at ways to seamlessly weave tests directly into the fabric of instruction.

Shute devised the idea of “stealth assessment,” a system that discreetly tests students’ learning in interactive and immersive environments such as digital games. 

Aside from offering a less obtrusive way to measure learning, stealth assessment aimed to help with “flow,” the mental state in which a person is so engaged and exhilarated by a task that they forget they’re working. 

For most students, any exhilaration melts when test time nears.

“Assessment is inherently about power,” said the University of Wisconsin’s Kim. “Assessment is inherently about evidence and rules.”

By contrast, the new kinds of assessments empower students to challenge and question rules. In one proposed scenario, students in the PISA creativity test are asked to build a paper airplane, then come up with ideas to improve it.

In another, students design a “bicycle of the future,” suggesting three original improvements over standard bikes. Then they’re asked to tweak the design of a proposed anti-theft camera mounted on the bike. Finally, since the future bicycle is automatically powered, they must suggest “an original way to reuse or repurpose” the pedals.

“The idea should be original,” the test says, “in the sense that not many students would think of it.”

Kim has spent the past few years developing playful assessments for the classroom, originally with teachers, teacher trainees and game designers at MIT. Where Shute, her mentor at Florida State University, called it “stealth assessment,” Kim prefers the term “playful assessment.”

‘It’s a mind shift’

Kim has lately been testing something she calls the , a free, printable card game for teachers that Kim describes as “Charades meets Telephone” to teach the process of drawing conclusions from a chain of evidence.

In the game, players take on one of three roles: Performer, Observer or Interpreter. They can only see one of the other two players, and gameplay proceeds as the performer silently acts out, in three movements or less, what’s on a card. The observer takes notes on what she sees and determines how to tell the interpreter what she saw. 

Like many in the field, Kim said a big roadblock to more playful tests is that so many school systems use assessments for teacher evaluations. “At the end of the day, we are obsessed with the idea that ‘Assessment is score: score about performance and proficiency.’”

Meanwhile, for most educators, play “is not something that is productive,” she said. “So for teachers to kind of switch their mindset in terms of, ‘Assessment can be fun, and this is an assessment,’ it’s a mind shift.”

According to the latest from my colleagues at , COVID’s impacts continue to reverberate through the American school system. Researchers and analyzed test score data from approximately 6.7 million students in grades 3 to 8 in 20,000 public schools who took last academic year. They then compared the rates of growth for students throughout the 2022-23 school year against typical, pre-pandemic rates.

They found that, in nearly all grades, achievement gains last year fell short of pre-pandemic trends. Because students are behind where they were before the pandemic, they would need to make greater-than-ordinary progress to get back on track. NWEA data show that isn’t happening; over the course of the 2022-23 school year, older students’ movement toward full recovery stalled.

The graph below shows the results in reading (blue) and math (magenta). Third graders showed some sign of rebounding, meaning they had above-normal achievement gains, but students in grades 4 to 8 all gained less quickly last year than what had been the typical pace prior to the pandemic.

NWEA researchers now estimate that on average, students will require interventions and support equivalent to 4.1 months of additional schooling to catch up to pre-COVID levels in reading and 4.5 months in math. Middle schoolers are particularly far behind relative to where their older peers were performing just a few years ago — needing the equivalent of an extra 9.1 months of learning in math and 7.4 months in reading.

Progress for students of all races and ethnicities grew at paces that fell short of pre-COVID averages in 2022-23, but the problem is pronounced for historically underserved students. Given the disparate amounts of unfinished learning that remain as of spring 2023, NWEA researchers estimate that Hispanic and Black students still need an additional 6.4 and 6.2 months of math instruction, respectively, to get back on track.

At this time , NWEA data showed some signs for cautious optimism. But progress has stalled for many learners, suggesting the road to recovery may be even longer than expected. The recent corroborate just how far students have fallen behind in the wake of the COVID-19 pandemic. 

The question, of course, is why? While our national data can’t answer that, our partnerships with districts and states across the country provide some insight. Educators and leaders are doing all the right things — analyzing local data to understand which students need extra support and deploying evidence-based interventions to help. But that process is complex, and scaling up programs takes time. Indeed, the , and it may take years from the launch of even the highest-quality programs to see improvements in student test scores.

Given these realities, the clear message is that recovery efforts must continue with urgency. While the data is helpful in understanding recovery trends, education leaders must look to their own information about student progress to target support to the kids who are the furthest behind. States and districts will also need ongoing resources to create long-term, sustainable recovery plans that meet the scale of the challenge. 

Federal Elementary and Secondary School Emergency Relief funds have been essential in supporting initial , but there’s just one more school year before those funds expire. Federal policymakers have provided allowing the money to be spent after September 2024 (as long as it is obligated before then), but more is needed. Our data, and others, confirm that recovery will be a multi-year effort extending well past 2024. Federal and state policymakers need to provide sustained funding to enable school systems to rise to that challenge.

Ensuring that good data on student growth is available to parents, caregivers and communities is also an essential part of the recovery process. While educators see the ongoing effects of the pandemic in their classrooms every day, a recent found that over half of families believe COVID-19 had only a temporary effect on their child’s education. State and district leaders must take on this perception gap by engaging families directly. That starts by providing them with clear and timely information about their child’s progress and achievement based on grade-level standards. Families can’t help to address students’ learning gaps if they don’t know they exist. And it will take schools and communities working together for years to come.

Schools are heading in the right direction, but much more sustained work and investment will be needed to get all kids fully back on track. From these and data, it’s becoming increasingly clear that the depth and breadth of the crisis will demand a comprehensive and ongoing approach. Federal and state leaders must not lose patience and focus, and need to ensure that schools have the necessary resources and support to help students recover on the long road ahead. 

On average, students need four more months in school to catch up to pre-pandemic levels, according to the results from NWEA, a K-12 assessment provider. This fall’s ninth graders need far more — roughly a full extra school year. 

“There’s still a pretty big distance between the COVID and the pre-COVID trends,” said Karyn Lewis, director of NWEA’s Center for School and Student Progress. “We’re not doing anything to shrink that distance.”

Within the grim results, however, there was a spark of hope for the youngest students. First- through third-graders were the only students to make above-average gains. But that progress only returns them to “an already significantly inequitable state of academic achievement,” researchers said.

They described the data from 6.7 million students who took the organization’s MAP Growth tests last school year as “education’s long COVID.” The results come just weeks after the latest National Assessment of Educational Progress revealed sharp declines in reading and math for the nation’s 13-year-olds, continuing a downward trend that began a decade ago.

Students who experienced the greatest setbacks during the pandemic — the intended beneficiaries of billions of dollars in federal aid — have the most ground to make up. But time is running out to use the remaining relief funds for tutoring, expanded summer school programs and other recovery efforts. The extended emergency, Lewis said, has kept her up at night.

“I very much worry that these results will fan the flames of ‘Schools didn’t use these dollars wisely,’ and I don’t think that’s the case at all,” she said, noting that many are using proven teaching methods for students below grade level. “It’s just not enough. The dosage is not in line with the magnitude of the crisis.”

But Dan Goldhaber, director of the Center for Analysis of Longitudinal Data in Education Research at the American Institutes for Research, is more critical of efforts to reach students who have fallen the furthest behind. The data, he said, lines up with CALDER’s showing low participation in optional interventions like tutoring and summer programs.

“You’d never know that there had been a pandemic,” he said. He acknowledged of those who argue there’s too much emphasis on test scores, but added that such the types of courses students take and whether they’re on track to go to college. “The education ecosystem is not healthy.” 

The NWEA results show students who finished eighth grade this year need an additional 7.4 months of learning to reach pre-pandemic levels in reading and 9.1 months in math. Black and Hispanic students need even more extra instruction to get there, the data shows. 

Lewis compared pandemic learning loss to tornado damage. A hammer and nails, she said, might be sufficient to repair a lost shingle or a loose shutter. But if a garage is flattened or the roof is gone “a hammer and nail is not going to cut it,” she said. 

To Kimberly Radostits, last year’s Illinois Teacher of the Year, that’s an apt metaphor. Her house was leveled in 2015. She spent 18 months rebuilding and sees a parallel between recovering from that literal destruction and schools’ efforts to put the pandemic behind them.

“We’re back to normal, but everybody is just tired,” she said. “I do think there’s a burnout that comes with that.”

The NWEA results, she said, point to the need for more efforts to support ninth graders — like the mentoring program she co-founded in the Oregon Community Unit School District.

The program typically serves 15 students who struggle with missed assignments, poor grades and behavior problems in eighth grade. But Radostits said she would double that number if she could find additional teachers willing to volunteer as mentors. To fill some of the gaps this fall, the high school will replace one elective with an extra “skill-building class” in math and English for students who need it. 

“We need to be more flexible with our scheduling than ever before,” Radostits said. 

But some districts that have tried to add school days or reconfigure the traditional calendar have faced stiff opposition. The Los Angeles Unified School District last year had to move to holiday breaks after the teachers union fought a plan to embed them throughout the year. And an effort to implement a year-round calendar in the Richmond, Virginia, schools — to prevent further summer learning loss — was ultimately adopted after parents, teachers and board members pushed back.

Researchers studying the pace of recovery worry that schools aren’t always straightforward with parents about the long-term effects of the pandemic. 

“Schools never say to parents, ‘Hey I’m really worried that given these test scores or attendance patterns, I’m not sure that your kid’s going to go to college,’ ” Goldhaber said.

Oregon High School’s mentoring program begins “planting a seed” in eighth grade with parents whose children could benefit from the program, Radostits said. Using an that shows parents a dashboard of their child’s grades, attendance, discipline record and work completion often makes the difference.

“We can click a button that will fade every other student’s name on this database except for the one you are looking at,” she said. “That’s a powerful visual for parents.”

Some advocates say parents still get mixed messages from schools. Sonya Thomas, executive director of Nashville Propel, pointed to recent from district leaders about third graders’ performance on Tennessee’s reading test. Nashville Director of Schools Adrienne Battle said that just because students didn’t reach proficiency — required to advance to fourth grade — didn’t mean they failed or couldn’t read. 

“To me, that message was toxic,” Thomas said. “We have a literacy crisis that has gone on for decades.”

The organization is holding a six-week literacy institute this summer to help parents better understand the components of reading and their children’s performance.

‘It was shocking’

The fact that the gap between pre- and post-pandemic achievement for fourth through eighth graders grew larger this year instead of smaller was a hard pill to swallow, said NWEA’s Lewis. 

“There was a hope that a lot of lessons learned in 21-22 could be applied in 22-23,” she said. “We actually see the opposite. It was shocking to us.” 

In a summary of the results, she and Megan Kuhfeld, an NWEA senior research scientist, said the backslide was likely due to ongoing “behavioral, academic and staffing challenges.” Many districts are still reporting . 

They attribute the encouraging signs in first- through third-grades to younger students’ tendency to learn faster. Their scores were 4% above pre-COVID levels in reading and 2% above in math — largely echoing mid-year data released earlier this year from Amplify, a curriculum provider.

Lewis said perhaps “word got out” that younger students were missing basic skills and “that’s where schools have leaned in.”

Some districts have also heeded warnings from researchers about the most effective types of recovery efforts. In the 4,700-student William Penn School District, outside Philadelphia, students attending summer school get 90 minutes each of reading and math instruction followed by more traditional camp activities in the afternoon. 

“With rising third graders, we’re making sure they’ve mastered everything they were supposed to learn in second grade,” said Ed Dunn, curriculum supervisor. He still gets emails from parents asking if spots are open. About 1,100 students are enrolled this summer, up from 750 last year. 

The district considered offering students voluntary, online tutoring during the school year, but instead opted for organizing small groups for extra help during class.

“We want to make the most of the actual school day,” Dunn said. “Our students need too much support to just leave it to chance.”

In this episode of Class Disrupted,  Michael and Diane dig deep in analyzing the big acquisition of nonprofit testing provider NWEA in the assessment market by Houghton-Mifflin Harcourt, one of the largest curriculum players in the United States. They conclude that we should be skeptical that the acquisition will improve teaching and learning for students or that it will pay off as much as HMH might like. 

Listen to the episode below. A full transcript follows.

Diane Tavenner: Hey, Michael.

Horn: Hey, Diane. Wacky weather is continuing to roll out across the country it seems. And I hope speaking of that, you’ve all stayed dry enough in California, which frankly is not a sentiment that I normally need to express to you.

Tavenner: Well, Michael, the persistent rain has been a little bit strange, a little bit unfamiliar, but I will say, also welcome. Partly because we don’t have any personal complaints. We haven’t experienced significant destruction or risk and others have, so I want to be mindful of them.

But I will say, Michael, it does seem like our schools are never going to catch a break. Because with these storms have come power outages that have caused us to have to close several of our schools for several days, and honestly at this point we are kind of starting to wonder if we’re ever going to be able to consistently operate our schools with everyone in them.

Horn: It’s just insane to watch all this, Diane, and I’ve been following some of this with my friends in the Bay Area on Facebook and so forth.

But aside from weather and the normal question of our students in the schools themselves, which has been a recurring topic on a lot of our podcasts, I’ve been looking forward to talking with you because there was a major announcement in the world of assessments. And obviously assessments and accountability and the challenges they posed to the kinds of innovation that we both want to see are something you and I have discussed a lot on this show. But given the latest news, I’m just wondering if we can go much deeper on this topic.

I have a bunch of thoughts loaded up that I’m trying to sort out that are informed from the innovation side of my work on what this means for education. But I’ll be candid, I’m really excited to get on-the-ground perspective just to check my instincts on this.

Tavenner: Oh, you know how much I love geeking out on assessments. So I’m totally up for this. Before we dive in though, I’m pretty positive we’re thinking about the same story, but just to be sure. Are you talking about the one where Houghton Mifflin, or HMH, one of the three big textbook companies acquired NWEA, which as many of us know is a nonprofit that’s most known for what we call the MAP test. Which is basically an adaptive benchmark assessment and I think it’s used in about 10,000 school districts across the country and I will say personally I know used in a lot of charter schools.

Horn: Yep, that is the announcement that I’m talking about, Diana. It grabbed my attention.

Tavenner: All right. Well, mine too. I’m excited you’re bringing it up today. Because you know I’ve got a lot of feelings and thoughts about this topic in general and honestly this acquisition and so I’m excited to dive in. But it sounds like you have some real questions that we should unpack, so I’m curious about that. But before we do it, do you think I should give a bit of context and to help folks out?

Horn: Yeah, I think that makes sense, just so everyone’s on the same page before we dive in and geek out.

Tavenner: OK. Well, let me just, I’ll talk from my experience, and I was thinking, I think my first encounter with MAP, or NWEA, was probably around 2011-ish. If you remember, this is around the time of Summit’s redesign. It’s also a period of explosive growth for charter schools. And as a result, there were just a bunch of school leaders and philanthropists who were looking for ways to see if innovations, and I would say both charter innovations and school design innovations, were having an impact.

And interestingly, this is also the time period when Common Core is driving the full assessment conversation and what’s happening in the country. And so at least where we were in California, there was a bit of a space or pause on those big assessments while Common Core was coming into being. And so people were looking for things to fill the gaps during that time.

Anyhow, folks started using the MAP assessments. And a fun fact. Back in that timeframe, I remember giving the assessment to our 10th graders for the very first time and realizing that only a little over 2,000 10th graders in the entire country had taken that particular test.

I’m sure we’ll get into this as we start talking, but the number of people taking the test actually matters a lot given how it’s scored and what this test does. And certainly there were higher numbers in earlier grades. But the point is NWEA has grown pretty explosively over the last decade, at least from my perspective.

And so we fast forward to today, and NWEA is either a math or an ELA assessment. It’s administered three times a year — and I think that’s an important fact — by schools and teachers. It’s adaptive in that it’s on the computer and it feeds kids the next question based on how they did on the previous one.

And its purpose really is to show school year growth for our students and to give teachers and schools quote, “good information,” at the start of the year and the middle of the year about which students are on track to meet the grade level standards or to pass the end of the year assessment. And so conceptually, teachers and schools can make grouping decisions or curricular decisions or tutoring support decisions for students based upon that information.

But to be really clear, these aren’t really formative assessments that are embedded in curriculum itself. They’re not even tied to the curriculum. So they’re not driving instructional decisions and the absence of being connected to the curriculum…. Well, I’ll just say it as the company says it, is they really seek to be content-agnostic, and so there is a real distance there between what you’re doing in the classroom and these assessments.

Let me just give one additional piece of context, and that comes from the perspective of a parent. Because my child took these assessments three times a year for many, many years, and they don’t mean anything to students or families. I can’t sugarcoat it. There is literally nothing that happens for students or families as a result of these tests. The reports and results are basically nonsensical if you’re a student or a family and there’s just not any information in them that I can act on or use. And so in my experience, what happens is kids rightly start to ask the question of like, “Why am I taking this test so often? What is this for? What’s it doing?”

And as a school leader who was also that parent, I can tell you I spent a lot of my time trying to convince my own child and get him to convince his friends to do their very best on the test because it mattered for the school. And in reality it did because we were using those results for everything from charter renewal hearings to grant applications.

And so I’m going to stop there because that’s a lot, and you haven’t even introduced what you want to talk about yet, Michael. But as you can see, this is a juicy topic and I’m getting fired up already, so I’m super curious to hear what it’s provoked for you.

Horn: Well, I love this, Diane. I think it’s super useful context, and I love the passion and energy. I thought I would tap into it on your end, and so we have. But assessments more generally is something that I, as outside the classroom, as a non-educator, have really had to come up to speed over the last 15 years to better understand how different kinds of assessments can or can’t be useful. What their function is, what they are capable or incapable of telling you.

And I will say, I feel like I’m a constant work-in-progress on this, which is why I’m excited to check with you. But it is not really obvious to folks outside of the classrooms and schools oftentimes I think what an assessment says, and as a policymaker or someone like that making law about this stuff, it is very opaque, I suspect. That’s just further context.

But as I look at this acquisition, one of the stated reasons I think for it, and this is from the companies, not from me, but from the companies, I think the purpose of this assessment is that…. Look, you have this established assessment as a system that is used in lots of schools and districts across the country, and HMH now will basically be able to use those assessments to connect them with their curriculum and make personalized recommendations for each student.

The theory, I’m assuming, that they have at work is that they’ll be able to do what you suggest, or implied at least would be helpful. Which is that assessment should be used to actually be embedded in the curriculum and drive learning choices themselves, and in turn help bolster more personalized learning opportunities.

Now to be clear, they’ve been clear about this. Each set of products will still be sold separately and under their own brands. But I think a major rationale that they’re making is that the possibility now exists to combine these, to combine the assessments with the curriculum and buy them as a package, if you will.

Tavenner: I have to admit, Michael, I’m deeply skeptical. I’m really curious about the theory that suggests this is going to be a happy and productive coming together of these two very disparate things.

Horn: Yeah, well look, this is the theory or the theoretical reason for the rationale behind the acquisition. And why I assume as a nonprofit, NWEA probably said, “Look, this is for the good of our mission for the education sector.” I imagine that’s what they’re saying anyway.

But for the moment, I want to put aside one of the structural things that you mentioned that causes your skepticism, which is that these assessments aren’t, at present anyway, actually embedded in any curriculum or tied to any specific curriculum or given an on-demand way to drive student learning.

We’re definitely going to circle back to that. But I want to instead introduce a totally different theory on my end to analyze this move and its impact, and it’s called the theory of interdependence and modularity, and it’s one that we used in disrupting class as well as my most recent book, , and even in parts of . And it’s a theory that sort of combines engineering with business is the way I would think about it.

And as a refresher for those that don’t know and don’t geek out on the stuff routinely, the first half of the theory essentially says that when something is underperforming what customers or users need from it. And the way that two elements in the system interact are unpredictably interdependent, meaning the way one works and functions as dependent on the other one, the way it works and functions and vice versa. Then to make it good enough so that people will actually adopt it and use it, you have to do it in a proprietary structure and really have a proprietary design. And that’s because the two sides really need to be designed interactively.

So quick example from business, and then I’d love you to reflect on what that means for education. But way back when in the 1940s and ’50s when IBM came out with their mainframe computers, no one knew what a mainframe computer should look like or operator or anything. And so IBM just couldn’t exist as a standalone assembler of computers or even a seller.

They had to make every single part inside. The logic circuitry, the core memory, the operating system, assemble it, sell it, everything. Because each of those stages interacted in unpredictable ways with every other part of the process. And so if they had just thrown the operating system over the wall, if you will, to engineers on the other side assembling it, well, that would’ve impacted in unpredictable ways the performance of the fundamental computer and each side would’ve had to make unacceptable trade-offs. That just wouldn’t have been good enough and no one would’ve bought it basically, Diane.

Tavenner: OK, let me make sure I’m getting this before we go on, let me try to bring it into this context and see if I’m understanding it correctly. So in this case, we have curriculum, so textbooks, lessons, all of that, and a benchmark assessment, and they’re separate, they’re in separate organizations right now. But they’re interdependent because when you’re educating a child, you’re both assessing them and you’re doing this curriculum. Which is kind of confusing because curriculum has assessment in it as well, but we’ll leave that aside for a moment.

They’re interdependent, but they actually depend upon each other. But the key right now is it’s really unpredictable how that happens because, well, I guess I’ll just describe from my experience. Sometimes it often felt, like when we were giving NWEA, that the results of that were totally disconnected and unpredictable based on what we were seeing in classrooms and our knowledge of students and things like that. Is that, am I getting it?

Horn: Yeah, I think that’s exactly right, that there is certainly a dependence and in some systems it’s a well-understood dependence. It’s predictable. We can get into what that means later. But in many systems, it’s unpredictable.

You make a change to the curriculum, that should impact the assessments because otherwise you’re just not going to be getting reliable information and feedback on what a student actually has learned or knows now and can do with that information. And vice versa, if you make changes to the assessments, well that should mandate changes to the curriculum. And when they’re at arm’s length, they are not really feeding off each other.

And when you think about all the personalization that we want to see in the world, because students have different learning needs at different times, those interact in very unpredictable ways right now. Particularly as you get into, think about math or something like that and the sequence of learning and the dependence of different topics on other topics, etc., etc. These things weave together in very interdependent ways. They impact each other.

And so my sense is what you just said, which is that when you have the state of curriculum and assessment at arm’s length, separated from each other, they’re basically underperforming what schools need. Certainly Title 1 schools. I think you could make the argument in Lexington Public Schools where I am, that’s not the case. But in Title 1 schools in particular, those that serve lower income populations, for example, they really need these to be connected to help drive choices for the students on the ground, themselves.

And I think the evidence for this, Diane, actually, they don’t just have to take my word or maybe your word for this, it’s actually in the market itself. We’ve seen incredible traction and extraordinary growth of a different company over the past 15 years, which is this company called Curriculum Associates. And they make both the curriculum, the i-Ready curriculum, as well as the i-Ready assessments, and they use them as benchmark assessments, but they’re intricately tied to each other. And they essentially, in a closed-loop system, what you get on the assessment system determines what you do next in the curriculum and vice versa.

And interestingly enough, Curriculum Associates over the last 15 years has basically come from essentially nowhere to become one of the digital players that I would argue is disrupting much of the textbook market over that period of time. And I think it’s driven by the fact that their assessments and curriculum have been designed in an interdependent fashion.

And I think there’s further evidence for that in the struggles, at least relatively speaking, of a company called Renaissance Learning. And Renaissance, for those that don’t know, basically has historically offered the Star Assessments, another benchmark assessment instrument. And they’ve been sort of stuck, at least my sense is, with assessments where they’ve been trying to find partners, Diane, in the curricular space that they can align to, if you will, to provide information around.

But it’s been kind of kludgy because you’re integrating these curriculum that haven’t been built with the assessments in mind and vice versa, and you have a curriculum-agnostic assessment. And so they’ve acquired or partnered with providers like Nearpod or more to essentially have both sides of the equation.

But because they weren’t designed interactively, the feedback loop between them, or at least the theory would suggest … And full disclosure, I can’t make the judgments because I don’t use them. But my sense is that the theory suggests that the recommendations and integration of the two just wouldn’t be near as eloquent, if you will, as what Curriculum Associates offers because of the way they’ve built both sides of those items. And so to be clear, that means in a not good enough world where these things are underperforming, that favors interdependence.

Tavenner: Fascinating. What you’re sharing is bringing up two things from me, and so I want to just keep checking my understanding of this with you.

The first one is, it seems to me that maybe another example of this is the partnership between the College Board and Khan Academy. As folks began sort of pushing back on the SAT test as sort of these standalone assessments that are super high-stakes and they just felt like they’re disconnected and inequitable, the College Board partnered with Khan Academy to offer the learning curriculum part of the equation. The benefit there was it’s free, it’s to anyone who wants it. Bringing the equity piece. Is that a good example of this theory?

Horn: Yeah, and I think we’ll get into more how it is in a moment. But it’s sort of right, it’s this modular trying to build up into an integrated way, I think.

Tavenner: The second thing that comes up to me, which probably you’re like, “Of course it comes up to you,” is just what we did at Summit Learning. And it seems as you’re describing the theory, that this is what we did intuitively. We built, as you know, an entire system of learning and assessment to be interdependent and integrated from the very start. And we just thought it was the only thing that made sense. And I know we’re not alone in this realm, but there’s a reason that a lot of people don’t do it. There’s a whole bunch of reasons. That’s probably a whole other episode.

But it seems you’re offering evidence for those instincts. But I’m wondering about the modularity part of it, because we haven’t talked about that yet, and so how does it fit in?

Horn: Yeah, so I’m glad you asked because there’s a trade-off in this and there’s not one right place to be in this continuum. Because when you’re optimizing for raw performance by creating a really proprietary interdependent system, the trade-off is that you can’t get affordable customization. And this is a really important point, and it’s interesting actually because Summit Learning, you optimized for curricular modularity to create an interdependent system around parts that were underperforming, and so you made these sort of trade-offs.

And that’s the other piece of this, is that in practice, every system has some modular components and some interdependent ones, and you basically want to be interdependent where the performance isn’t good enough and modular where you’re overshooting so you can get that customization. And so in essence, what the theory says is that when you start to overshoot, that’s where you need that customization because people demand it, they want affordable customization. They want to be able to mix and match different parts and just pop them in, and to do that, you need modularity.

So the quick example, again, we’ll go to computers, is Dell computers, say, in the early 1990s. If folks remember, Apple, IBM, they had started to over-serve what people needed. They weren’t willing to pay for some of the improvements in that paradigm. And here comes Dell and they don’t make any of the parts inside of the computer and that means you can just jump on their website and be like, “I’d like this much memory, I’d like this kind of drive from Seagate, I’d like this kind of monitor, etc., etc.” And they fit together in very well understood ways and so Dell just quickly snaps the parts together and 24 hours later, you got shipped out an affordable customized computer.

And so I think you at Summit Learning know these desires very well. Because you chose to avoid purchasing on the open market a proprietary highly interdependent curriculum, and instead, as I said earlier, create something far more modular on that side because you wanted customized playlists and so forth for the students.

And interestingly enough — I’ll try to make the argument for you — I think you said that where the systems out there are really underperforming is in the integration or the interdependence, if you will, of the knowledge, skills and habits of success that are often broken apart and perhaps unfortunately so. And you said that’s where we really need the interdependence in the Summit Learning system.

Tavenner: I think that would be… 

Horn: Yeah, go ahead.

Tavenner: I think it’s all just clicking for me as you’re talking. We’ve never wanted or bought a textbook, because it’s so not customizable, and if you’ve ever been a teacher or know a teacher, you know that is the number one thing a teacher wants to do and needs to do, which is why those products don’t get used with fidelity. And so we are just, I think as practitioners so keenly aware of what actually happens on the ground and pragmatic about that, that that’s where we wanted customization. But super tight on the assessment part of it and the connection to it. And like you said, the connection of all the factors of learning, not just a single siloed content area, if you will.

Horn: Yeah, and I think that’s the big aha here, which is that you can’t actually just jump into a modular design. Everyone wants customization, it sounds great, but you can only move to it once the interfaces between the different components are predictable, specifiable and verifiable.

And I hear this all the time, Diane, in education circles, that we want an open system, we want an open this, we want an open that. As though open, in and of itself, is a good thing. And the only way though you can move to that kind of modularity is you really have to understand how each side works and functions and the impact of changes on one side with what it’ll do to the other.

And that means you have to be at the level of predictable causality. You need to be able to create specs and essence for exactly how those parts fit together at the interfaces. You need to be able to verify that the parts in fact meet the specs. And then in the early stages of creating something new, when it underperforms, that’s just really hard to do and it’s only when you get better and better and better, you start to over-serve people and you start to understand how these parts work.

Then you can start to shift to modularity because you have a better sense of how the overall system is interacting and the science of studying them essentially allows you to get more predictable. So a decade hence, maybe we could start to modularize Summit Learning, but we’re not there yet.

Tavenner: I’m having a reaction that I commonly have when we are talking, putting theory to practice, and often in my mind, I’m always putting it to our practice. And at first when you are describing the theory, my mind immediately goes to, oh no, we didn’t do that. We need to go back and do that. Oh no, we messed that up. We weren’t good innovators.

And then it’s funny, as we talk more and more, and I really calm myself down and think about it, I realize that a lot of what you’re saying, Michael, we do or did and that the theory really describes our behaviors. Which I think is good because you don’t want a theory that’s actually not describing what’s happening.

And I’ll be honest, the outcome of that work is strong. By designing a learning and assessment system from scratch that clearly prioritizes each and every student and incorporates the learning science and the full understanding of the needs and leverages technology, we really have created what I think is a very elegant and interdependent curriculum and assessment system that is highly modularized in the right places and thus customizable for the folks on the ground. Both students and teachers I would argue, and schools for that matter.

And as I say that, I’m cognizant that it sounds a little bit like bragging. But the point that I want to make is that it’s possible that what you and I are talking about, it’s not a fantasy, it’s not a future thing. It literally is happening right now in hundreds of schools across the country. And I just think that’s important because we always want to take this sort of third way future-looking perspective and what I don’t want this episode to sound like is just us criticizing a deal that happened without another solution. Because I do think there are other solutions and it doesn’t feel like we’ve compromised in the combined offering. And honestly, this acquisition feels like a compromise to me.

I suspect that a lot of people who care about seeing big rolled-up data and tracking and reporting on the percentage of kids in schools who are supposedly learning or not, they might be happy with this direction, Michael.

But for those of us who care about literally every single child and if they can read or solve problems and if every child’s motivated to come to school and knows how to learn, for those of us who think it’s unacceptable that we’re… School systems are unable to simply tell families if their child’s on track to be a skilled adult in areas that families care about in their life goals, and instead we send them these incomprehensible test scores in the mail months after a child took the test.

Well, look, I’m going to stop there because you could see I’m getting fired up again. But this is possible what we’re talking about, the theory says we should be doing, and I don’t think that this deal represents that.

Horn: And this is where I ultimately want to go as well, Diane. And I think there’s another issue as well underlying this, which is, it gets the multiple purposes that assessment serves. From driving learning to continuous improvement to accountability and transparency. And I think what’s also interesting on this is that the interdependent approach, frankly, it’s always going to struggle to serve a public accountability function in my mind in which we’re making judgements about the schools themselves, even as the theory shows why it’s the best way to optimize the teaching and learning.

And I’ll say that, again, to analyze Curriculum Associates for example, and with immense respect for the predictability of i-Ready assessments on what a student will score in a summative year-end assessment. But essentially, Diane, if the i-Ready assessments… I suspect some people are listening to us are just like, “Well, why wouldn’t we just use the i-Ready assessments to replace the summative year-end assessments. They’re curricular tied.”

Great end of story. But you can imagine that then there would be a really slippery slope that would come into play in which Curriculum Associates — And again, I’m not saying that they do this nor that these incentives exist right now, but in this alternative system I just described —  the incentive I think would exist for them to basically just give rosy assessments to student learning to show, “Hey, look, our i-Ready curriculum is really making a positive difference for students.” Because there’d be no check on the other end of it.

And then if they made those changes, which are no longer reflective of actual performance, now that means that the assessments would no longer be as useful for guiding learning for students and teachers. There’s a big trust issue here, in other words,

Tavenner: Michael, I experienced this in real life, where very few people actually take seriously the assessments built into Summit Learning as an outside valid way of measuring what we’re doing. I argue that they should because they can be completely valid and reliable in all of the things that we need. But there’s very little pickup there, and I suspect what’s underneath it is this trust issue that you’re surfacing right now.

Horn: Yeah, well, so I think this is where there’s another approach from the theory, which works frankly though only in a world of mastery learning that we’ve discussed in the past on this show in which students work toward mastery of each and every competency and time becomes the variable, not the student’s learning as the present system holds.

But that is if you have the interfaces clearly specified between curriculum and assessment, and that’s a big if, then you could have modular assessments created by a third party. So not the curriculum company itself. That an individual school or a district, or yes, even a state as Louisiana has, could select based on its alignment to their curriculum. And in that world, those assessments could both serve to inform instruction. They’re learning informative, but they’re also summative because the measures they’re giving are inherently of learning. Which is what your system does, just, to your point, it’s not third party, so people don’t trust it.

And the point being in that world, if I demonstrate mastery, I move on and that can be reflected and it’s more robust than a summative assessment because I can demonstrate it when I as a learner am ready to raise my hand in show mastery.

And so to come back to NWEA, this is the curve ball it seems to me that HMH is walking into here, and you’ve alluded to it several times. But just to say it from the theory perspective. On the one hand, by controlling the assessments, they theoretically, theoretically, different from theory, will have the opportunity to make them in a proprietary fashion alongside their curriculum and benefit from the same way Curriculum Associates has. But to do that, they have to do it from starting with assessments that are not curriculum aligned. That’s going to be a major lift.

And on the other hand, HMH can’t exactly serve the modular use case that I just outlined because it doesn’t have an array of assessments aligned to lots of different curricula that can be customized where the individual school or district or state or charter organization can select to drive learning. And it now owns both sides of the equation. So it’s sort of caught here, Diane.

Now I think there’s some silver lining here as well. There’s New Meridian, for example, a nonprofit. It exists as a newer third party provider seeking to offer modular baskets of assessments that you can pull off to match your curriculum. But I’m just not sure that HMH can make this jump as we’ve outlined.

So it’s possible, Diane, that it’ll be commercially successful as a sort of in-between solution, I guess in the market as the market maybe transitions over time. But I’m frankly not sure that that’s where the puck is going or frankly where it should go. And I really want to know how that lands on the ground for you.

Tavenner: Well, you probably have guessed that the idea of a commercially successful in-between solution isn’t landing very well with me. This is the thing that just really frustrates me about our sector and our work.

I agree with your assessment, Michael, and I think this is another example of what holds us back in education at the broader scale. We’ve got these two players who have a massive market share together. Well, I would call it massive, you might correct that, but it seems pretty big. And schools have to have curriculum and assessments and they’re under pressure to have something that tells them exactly where students are, especially coming out of COVID, so that they can help them direct resources to them. And just the expectations on teachers in schools continue to go through the roof around what they are expected to do.

And so tons of schools are going to use this joint offering, and in their mind they’re going to be told and sold on the idea that it’s better somehow and it’s going to suck for kids. I don’t know how else to say it.

It’s not better. It’s not good, and it’s not what our kids deserve and what’s possible. And what’s frustrating about that is it pulls the vision, the mind share, the resources, the energy away from what doesn’t suck, which is what we were describing before. Assessments that are embedded in project-based curriculum that’s authentically assessed in a way that dramatically improves engagement and learning and self-direction for students and job satisfaction and impact for teachers and gives them local control and all of the things that we care about that we know matter. And we have the capability to do that today, do something significantly more meaningful. And I just think that this pulls us away from that work. It pulls everything away from that work.

And I think it goes back to the conversation we had last episode, Michael, when we acknowledge that most people don’t even think about students as customers. And in my mind, this approach sort of epitomizes that. As I read the transcript of the interview with the two CEOs, there was literally not a single mention of this acquisition in terms of how it was valuable to students. They got down to teachers and said, “Teachers would have all this data and information.” Of course, the way I read that was like, “Wow, teachers have now a new expanded job.” And having had that information, know how hard it is to actually work with it.

So yeah, I’m going to stop again because I’m… But that’s how it’s landing with me honestly.

Horn: Well, no, I think that makes sense. But something you just said in terms of all the information we’re going to get and all this stuff and so forth. Let me step out of my innovator hat on that for a moment and talk about one other problem, in my mind at least, with NWEA assessments more broadly through an explicitly education lens. And this surrounds how so many of us want to shift to measuring student growth instead of point in time learning.

But the reality is that there are different ways to measure growth. And the public, and educators I think honestly, broadly speaking, don’t understand the difference. So when they see a growth measure, they don’t actually know what it’s telling them. And in this case, NWEA reports a norm reference growth measure as opposed to what would be called a criterion reference measure.

Essentially what that means is a norm reference basically evaluates you relative to other students like you at your “percentile,” quote unquote, of your learning. Whereas a criterion reference is against that yardstick of curriculum, but it has to be curricular aligned to be able to do that.

Now, as a result of that, frankly, the NWEA growth measure offers this incredibly false illusion where it basically, if I’m, say, a student in the fifth percentile of seventh grade or whatever else, then basically it’s going to compare my growth relative to other students in that same segment. And so I might grow one and a half years relative to those students. But that’s embedding all the lowered expectations and malaise of our education system in that growth measure.

When a parent hears one and a half years of growth, that’s not what they’re thinking. They’re thinking, “Oh, my kid grew from say the third grade to well into the fourth grade.” That’s what policymakers, I think here, that’s what I think educators and curriculum companies that use this stuff, I think that’s what they’re hearing too, and it’s just not what it’s measuring.

Plus, I think this gets back to what you said at the very beginning, and maybe this will start to wrap a bow around this. You mentioned that when you gave it to your 10th graders, only 2,000 kids in the country had used that. So now it’s driving a growth number relative to other students like them. I’m not going to be able to do the math that quickly in my head, but like 100 percentile points, whatever, it’s not a lot of kids in each percentile. You’re driving a growth measure off of that. That’s even less meaningful, I think, Diane.

Tavenner: Well, you’re opening the can of worms, and I know at this point we should be closing it, so I’ll be brief here. But I distinctly remember that conversation about those 2,000 kids. It stands in my mind a decade later because it was so clear to me, not initially, but when we really, really, really dug in to understand it, when the company rightfully said, “Well, there really aren’t kids like a lot of your kids.”

So you have to take these results with a grain of salt because there’s only 2,000 kids. And, I mean, Michael, we could do days on this, but let’s just start with the concept of, yeah, you’ve reinforced why these results are meaningless to parents and students because how can anyone in their right mind understand what you just said when you’re looking at a result or expect that that’s what you’re getting? You would just want something straightforward. Like, do I know this? Do I not? Am I good at this? Am I not? Yeah. Anyway.

But the second piece is this concept of bias. So we’re going to just compare you to people who are like at that moment in time, if you’re a quote “lower” performing student, what are the expectations? We know what our systems have in terms of expectations. We know what natural human inclination is.

I just think this is so destructive, and I’ll just share one last story about this. Part of those conversations with my son around this were so disturbing because he would come home and say, “The kids would just sort of share their raw score with each other and they would just stack rank the scores.” And what happened every time is quote the “smart kid, the super smart kid,” always got the top score.

So in my world where I’m caring about growth mindset and kids believing that hard work actually pays off, I was having to fight against what these results, the messages we’re sharing with kids, which was much more of a fixed mindset than a growth mindset. And we were doing battle against that in the context of the school because we were giving these tests, which I see the look on your face, and it just is like, it’s icky. There’s no other way around it.

Horn: It’s really crummy. It’s really crummy. All right, well, let’s not leave it there because I think there is hope on the horizon. And maybe Diane, frankly, as we see this acquisition, maybe it represents consolidation and often when you see consolidation in markets, it creates room for the disruptive plays underneath to gain market share. And so I’m hopeful that we will move to content aligned assessments and have some third party players working hard to align to the different curriculums out there so you can select the right ones for you and move to this world where it’s both driving learning, but also giving parents and students the information that they want about their growth. So I’m hopeful we’ll get there, but I don’t think this acquisition is the step.

Tavenner: A perfect place to leave it, Michael. And before we sign off, I would love to hear what you’re reading, listening to, or thinking about outside of our day-to-day? Add to my list?

Horn: Yeah, you bet. Well, so Sal Khan sends a book every few years it seems to folks who are on his advisory board. And so I got the one in the mail from him, which is by Epictetus. And the interpretation that he sent is by Sharon LaBelle. And I read it chunked, like a few lessons at a time over the last several weeks. And it was really helpful, Diane, at just resetting my own sense of how I think about my own life lived and what we can control and focusing on that. And so I really enjoyed it and was appreciative that Sal made the time to send it. What’s on yours?

Tavenner: Well, that’s awesome. I appreciate that. I have a fun one this week. So as we’re about to head to India, folks have been hearing about that for a long time. We watched a movie that’s on Netflix, it’s called RRR. It’s getting some positive press in the U.S. It is a Bollywood movie out of India. The hard part is it’s three hours, but I will say it is a fun three hours. It was joyful and interesting. It’s packed with all sorts of over the top action and it is over the top in many ways. But we deeply enjoyed it. And I think on the sort of interesting part of it, potentially quite controversial, but it really is offering a different narrative about the drive in India to be their own independent nation and how they engaged and interfaced with the British. And the normal narrative we all have is surrounded by Gandhi and a peaceful approach and there’s some folks who want to have a more fight struggle narrative. And so that is presented here for better or worse.

Horn: Super interesting. That’s a major political current right there, I know. And so I can’t wait to hear what you’ll learn from it as you come back if you choose to report on that. But for now, we’ll leave it there. Thanks for engaging with me on this conversation. And for all of you listening, we’ll see you next time on Class Disrupted.

Michael B. Horn strives to create a world in which all individuals can build their passions and fulfill their potential through his writing, speaking, and work with a portfolio of education organizations. He is the author of several books, including the award-winning and the recently released . He is also the cofounder of the Clayton Christensen Institute, a nonprofit think tank.

Diane Tavenner is CEO of Summit Public Schools and co-founder of the Summit Learning Program. She is a life-long educator, innovator, and the author of

As U.S. students in August and September began their fourth school year under the shroud of COVID-19, researchers from NWEA, the nonprofit behind the widely used MAP Growth assessment, took an early look at their achievement. The data suggest that gaps between pre- and post-pandemic performance have been slowly shrinking. 

Among the biggest contributors, for reasons researchers don’t quite understand: In 2022, the typical “summer slide” didn’t slide quite as much, giving kids a small advantage as the school year began.

The new findings are “evidence of resiliency on the part of students,” said Karyn Lewis, director of NWEA’s Center for School and Progress. “I think we’re seeing some buoyancy in terms of students’ achievement levels. That’s a testament to simply getting back in the classroom, being reconnected to their peers and their teachers.”

But in a more sobering finding, NWEA found that the youngest students in the study — third-graders who were kindergarteners when the pandemic closed their schools — showed the largest reading achievement gaps and the least “rebounding” from previous tests.

In reading, third-graders reduced their widest achievement gaps by just 10%, far less than other groups. By contrast, sixth-graders’ gaps shrank by 38%. The research found similarly small reductions for third-graders in math.

“That suggests to me it was really detrimental for those kiddos to be doing Zoom school in kindergarten to pick up on some of those foundational reading skills,” said Lewis. “And then for kids that return to the classroom in first grade, imagine trying to learn phonics with your teacher wearing a mask.”

These young students’ reading improvement was slower than their math improvement, researchers found. And they estimate that it will take them at least five years to fully recover from the pandemic in both reading and math, longer than nearly any other group studied except current eighth-graders. Given the five-year time horizon, many of those students may never fully get up to speed in either subject by the time they finish high school, they warn. 

The new findings add to data NWEA released last month that showed achievement losses in the 2021-2022 school year disproportionately affected low-performing students, whose skills have languished since the pandemic, unlike that of high-performing students. At the time, Lewis told ����Ӱ��, “If we think of the range of test scores, we see that the ceiling has remained stable. But the bottom has dropped out.” 

Meanwhile, the new study found, the highest-performing 10% actually made more progress toward academic mastery than would have been expected absent COVID.

“In study after study we are seeing clear evidence that while some students are getting back on track quickly, far too many are not,” said of the Center on Reinventing Public Education (CRPE) at Arizona State University. “We’re also seeing clear evidence that schools and districts are really struggling to meet student needs.” 

Federal ESSER funds, slated to help schools recover from the pandemic, are due to run out over the next three years — and schools must figure out how they’re going to spend the money by September 2024. 

“I think this is a group we really need to keep our eye on and make sure that elementary teachers that are serving them now actually know how to help them catch up,” NWEA’s Lewis said. “You can imagine the trickle-down effect it has if these kids continue to struggle in reading.”

In addition to hurting students who had just transitioned to kindergarten, the pandemic deeply affected those who were even younger — from newborns to four-year olds, said Michelle Kang, CEO of the National Association for the Education of Young Children. As they prepare for elementary school, those students are going to need supportive teachers, she added, who “help them build on their resilience and thrive in academic and social emotional contexts.”

W. Steven Barnett, senior co-director and founder of the National Institute for Early Education Research, said it makes sense that the youngest students have struggled in reading. “If you think about where kids learn math and reading, the home environment has a bigger impact on reading than math. So the extent to which the home environment was also negatively impacted by the pandemic, you’d expect that to have a bigger impact on reading, a bigger impact on younger kids.”

While he’s encouraged by the summer findings, Barnett noted that a lot of federal COVID relief funding remains unspent, so it’s difficult to draw a direct line between federal aid and improvements in achievement. But he said schools should not waste time figuring out how to spend their relief money going forward, whether it’s for summer programs or intensive tutoring. “There are a bunch of tutoring programs that have been proven highly effective that schools could adopt that identify the kids who are behind,” he said.

A number of programs offer one-on-one instruction and don’t require licensed teachers for implementation. Actually, Barnett said, they offer training for volunteer and paraprofessional tutors. He also suggested that student-teachers could step in to fill a tutoring void in a tight teacher labor market. 

CRPE’s Lake said the new data suggests one clear conclusion: “Now is the time to shift course. We must start acting like this is a national emergency and bring new solutions to the table. We cannot continue to ignore the mounting evidence that we are failing to give this generation of students a solid foundation for the future.”

A ‘dampened’ summer slide

NWEA’s Lewis said the overall upward progress among the students studied is in part due to what she called a “less-worse slide” over the summer, compared to typical years.

And while it might seem logical to attribute that progress to intensive, ESSER-supported summer learning efforts, like Barnett, Lewis cautioned against trying to find “a thread of causality” in the findings. 

Actually, she said, other research suggests that many of the available summer remediation efforts “were probably not taken advantage of to the extent that would really lead to the results that we’re seeing here.”

But Lewis said it’s important to put both the progress and challenges in perspective. Those who would call the nation’s third-graders “a lost generation” — as well as those who don’t think the ongoing gaps are a big deal — are both missing the larger point.

“Like most things, I think somewhere in the middle is where we need to find ourselves,” she said. What may be most important is “shining light on which kids have been harmed the most” and serving them in a less standardized, more personalized way — for instance, through a “layered approach” to tutoring, summer programming, and other strategies. 

“There will be no silver bullet,” Lewis said.

The second — that the already broad range of academic levels within classrooms would yawn wider — has now been confirmed.

Before the pandemic, the average fifth-grade class had students whose learning spanned seven grade levels. But in May 2020, NWEA, the nonprofit behind the widely used MAP Growth assessment, predicted that as the lowest-scoring students would fall two years behind. Because the exams are not given before third grade, the researchers warned then that their findings might not reflect students whose levels of understanding are even lower. 

finds this range has increased by 5% to 10%, with the losses disproportionately affecting low-performing students. While all pupils made less progress than they would have without COVID-19’s interruptions to instruction, children who were already struggling academically have fallen much further behind. 

The findings mirror the recently released dismal results of the National Assessment of Educational Progress, which showed that, collectively, U.S. students lost two decades of academic growth. That exam, too, showed widening gaps between affluent, advantaged students and low-income children, those learning English and students with disabilities. 

Because the MAP is administered at least twice a year, researchers were also able to more precisely identify when learning stalled, and how quickly — and for whom — it rebounded. Across the board, drops in math and reading occurred during the 2020-21 school year, with high achievers losing less ground than their struggling peers. 

In 2021-22, those who already performed in the highest 10% of students made more progress toward academic mastery than would have been expected absent COVID, while the lowest performers have languished. 

“If we think of the range of test scores, we see that the ceiling has remained stable,” says Karyn Lewis, director of NWEA’s Center for School and Progress. “But the bottom has dropped out.” 

To arrive at the findings, researchers crunched two sets of numbers. They compared scores from 8 million students in grades 3 to 8 from 2019 through 2022 with those of students who were tested in the three academic years before the pandemic. They also examined data on 1 million students who, pre-pandemic, scored in the top and the bottom 10%.

A secondary concern surfaced by the data is that the already inequitable process of identifying which students receive gifted and talented services and access to other higher-rigor academic opportunities will skew further in favor of affluent and white children, who backslid the least in the pandemic. 

The widening array of mastery levels has profound implications for teachers. Pre-pandemic, most struggled to reach students at varying grade levels. As it became clear that NWEA’s first “COVID slide” warnings were, if anything, understated, debate arose about whether to continue grade-level instruction or drop back and remediate students who were far behind. Some researchers favor a strategy known as acceleration — teaching all students age-appropriate material while using data to identify specific missing skills. 

To reach all children in a single class effectively, their teachers now must understand not just the academic standards for their grade, the researchers suggest, but the material for one to two grade levels above and below.

“I do think it begs the question of what do we expect each individual teacher to be able to do,” says Lewis. “I think we have to be honest that there’s a limit to teachers’ ability to be all things to all students.”

Dismal releases from the National Assessment of Educational Progress (NAEP) — often referred to as the “nation’s report card” — have become a biannual tradition in recent years as academic progress first stalled, then eroded for both fourth and eighth graders. But today’s publication, tracking long-term academic trends for 9-year-olds from the 1970s to the present, includes the first federal assessment of how learning was affected by COVID-19.

The picture it offers is bleak. In a special data collection combining scores from early 2020, just before schools began to close, with additional results from the winter of 2022, the report shows average long-term math performance falling for the first time ever; in reading, scores saw the biggest drop in 30 years. And in another familiar development, the declines were much larger for students at lower performance levels, widening already-huge learning disparities between the country’s high- and low-achievers. 

The results somewhat mirror last fall’s release of scores for 13-year-olds, which also revealed unprecedented learning reversals on the long-term exam. But that data was only collected through the fall of 2019; the latest evidence shows further harm sustained by younger students in the following years. 

Peggy Carr, commissioner of the National Center for Education Statistics, said on a call with reporters that the “sobering” findings illustrated the learning losses inflicted by prolonged school closures and student dislocation. 

“It’s clear that COVID-19 shocked American education and stunned the academic growth of this age group of students,” Carr said. “We don’t make this statement lightly.”

Average math scores for 9-year-olds sank by a staggering seven points between 2020 and 2022, the only such decline since the long-term test was first administered in 1973. Average reading performance — generally by schooling than math, and therefore theoretically shielded from pandemic shock — fell by five points. 

Inevitably, that means that fewer students hit the test’s benchmark performance levels than two years ago. For math, the percentage of 9-year-olds scoring at 250 or above (defined as “numerical operations and basic problem solving”) fell from 44 percent of test takers to 37 percent this year; those scoring 200 or higher (“beginning skills and understanding”) fell from 86 percent to 80 percent; even the vast majority scoring at the most basic threshold of 150 (“simple arithmetic facts”) shrank slightly, from 98 percent to 97 percent, across the two testing periods.

No demographic subgroup saw gains on the test, but disparities existed in the rates of decline. For instance, math achievement for white 9-year-olds dropped by five points, but for their Hispanic and African American counterparts, the damage was even greater (eight points and 13 points, respectively). As a result, the math achievement gap between whites and African Americans increased by a statistically significant amount. 

In reading, scores for African Americans, Hispanics, and whites were all six points lower, leaving relative gaps unchanged. Scores for Asian students only fell by one point. 

Notably, the long-term trend assessment differs somewhat from the main NAEP test administered every two years. It follows student performance going back a half-century, and it is taken with a paper and pencil instead of digitally. For the most part, testing items are unchanged from the early 1970s, assessing more basic skills of literacy and computation than are generally seen on the main NAEP.

The broad trend-line has been positive over the life of the exam, and even in the most recent release, student scores on both subjects are far higher than when they were first measured. But Dan Goldhaber, a researcher and longtime observer of student performance, said it was striking to see that upward momentum evaporate so quickly.

“A bit of a hidden story in education, when you look at a swath of 40 or 50 years, is the progress that students have made — and the disproportionate progress that historically marginalized students have made,” said Goldhaber, the of the Center for Analysis of Longitudinal Data in Education Research (CALDER) at the American Institutes. “We’re seeing a lot of that very long-term progress completely erased over the course of a couple of years.”

‘Particularly bad’

One of the most consistent, and consistently worrying, findings of previous NAEP rounds has been the sharp disjunction of students at either end of the performance scale. For over a half-decade, high-scoring students have generally performed a point or two better with each iteration of the test — or at least stayed at the same level — while low-scoring students have seen their scores fall.

The phenomenon of growing outcome gaps is again apparent in the post-COVID results, though it takes a slightly different form. At all performance levels across both subjects, 9-year-olds experienced statistically significant declines in their scores; but even with the identical downward trajectory, struggling students lost so much ground that disparities still expanded.

In reading, 9-year-olds scoring at the 90th percentile of all test takers in 2022 lost two points compared with their predecessors in 2020. But students scoring far below the mean, 10th percentile fell by 10 points.

Consequently, the average reading gap between kids at the 90th versus the 10th percentile grew from 103 points to 110 points in just two years. In math,the divergence grew from 95 points to 105 points over the same period.

Goldhaber said that the trends visible in NAEP performance largely dovetailed with those using test scores from the MAP test, administered by the assessment group NWEA. In multiple data sources, he argued, it has become clear that the pandemic’s effects have been disproportionately negative for already struggling and disadvantaged children.

“It’s not just the drops, it’s where we’re seeing the drops in math and reading tests, and they’re disproportionately at the bottom of the test distribution,” he said. “So the pandemic is reversing a long-term trend of narrowing achievement gaps. That’s particularly bad, to my mind.”

The fact that losses are so heavily concentrated among the lowest-scoring segment of students may help explain what Goldhaber termed an “urgency gap”; neither states, school districts, or even families seemed driven to embrace the generational learning interventions — from dramatically lengthening the school year to implementing widespread one-to-one tutoring — that the scale of learning loss demands. As just one indicator, billions of dollars of federal COVID aid to schools remains unspent more than a year after it was first allocated.

That may change in the wake of the NAEP release. While previous studies have pointed to similar, and similarly inequitable, learning loss over the last few years by using data from the MAP and state standardized tests, the Nation’s Report Card is seen as the authoritative performance metric for American K-12 schools. As NCES Commissioner Carr noted, today’s release provides the first nationally representative results measuring achievement before and after the pandemic. Ninety-two percent of schools where the test was administered in 2020 were re-assessed earlier this year.

Tom Kane, an economist at the Harvard Graduate School of Education, agreed that NAEP scores definitively affirmed what prior studies have already demonstrated. More observers needed to study the magnitude of the loss, he added, because the proposed academic remedies in most of the country are “nowhere near enough” to combat it.

Kane analogized classroom learning to an industrial process — the conveyor belt slowed in 2020 and 2021, but has resumed functioning since at roughly the same rate as before the pandemic. But to make up for lost time, he argued, it would need to be sped even further.

“What we learned…is that the conveyor belt is back on, but at about the same old speed,” Kane said. “Somehow, we’ve got to figure out how to help students learn even more per year in the next few years, or these losses will become permanent. And that will be a tragedy.”

Overall, the findings — from 8.3 million students in 25,000 schools — “point to a long road to recovery still ahead,” from nonprofit testing provider NWEA.

With the fourth pandemic school year approaching, the report underscores the need for urgent action to address learning loss, the authors wrote. MAP test data shows that it could still take three to five years — in the middle grades, more than five — to return to pre-pandemic performance. 

“It was a big question mark,” Karyn Lewis, director of NWEA’s Center for School and Student Progress, said about what to expect this year. “We were really hopeful at the beginning, but it was much more difficult than we thought it would be.”

Compared to a pre-COVID sample of students who took the assessments, reading scores are 2 to 4 percentile points lower and math scores remain 5 to 10 points lower. High-poverty schools, as well as Black, Hispanic, and indigenous students, are having a harder time bouncing back, the data shows. And, as other assessment experts have noted, reaching pre-COVID levels in math and reading does nothing to address long-standing achievement gaps.

‘Leaning in to those grades’

But NWEA’s findings also point to some hopeful patterns that weren’t present a year ago. The pace of recovery between fall and spring picked up compared to the 2020-21 school year. 

Performance in math, which declined the most last year, rebounded more than reading. And the gap between pre-pandemic and 2022 performance is smaller for younger students — whose scores declined the most in earlier results — than it is for those in the middle grades. 

Lewis said she and other researchers “hit home” on the negative effects of school closures on younger students and it “could be that schools heard that message and have been leaning into those grades specifically.”

She added that there’s evidence that “it was more challenging for younger students to learn in the absence of a classroom environment” and that maybe their growth has accelerated because of the return to school.

The results should help target students for additional instruction this fall, Lewis said. She added that while students typically experience a so-called “summer slide,” past research on summer learning loss is less reliable this year because of districts’ myriad tutoring efforts and other extra academic support, due to relief funds. 

The spring 2021 scores drew some districts to change how they organized students for teaching in reading and math this past school year. In School District 81, which serves Chicago’s Schiller Park neighborhood, Superintendent Kimberly Boryszewski said the needs were so great that what “used to be the exception had become the rule.”

Many students were two years below grade level, while some were still advanced. Teachers split students up into several groups to provide more individualized teaching at their level for the entire day. In one school, less than half of the second graders were meeting expectations at the beginning of the year, but by spring, 75 percent had made it.

“It tells me that model works for us,” Boryszewski said. “[Teachers] needed me to say, ‘Put the curriculum on the shelf and let’s teach the kids’ and to give them permission not to feel like they had to be on a certain page on a certain day.”

State results are mixed

The MAP results are meant to help teachers make decisions about which students need more support, but not to hold schools accountable for student performance. Such ratings will return to state report cards this year for the first time since the start of the pandemic. 

Parents and educators should also pay close attention to state test results because they reflect state standards, said Scott Marion, executive director of the nonprofit Center for Assessment. As with NWEA trends, state data so far presents of recovery. 

In , the percentage of third graders reaching level 3 — or satisfactory — did not change from last year and is still below 2019’s 58%. The results in English language arts across other grades are similar.

In math, students made gains over last year, but remain below . Across grades three through five, 62% of students reached level 3 in 2019, 52% in 2021 and 57% in 2022.

Texas officials point to additional funding for “learning acceleration” and legislation providing 30 hours of extra instruction for students below grade level as reasons why are meeting and mastering expectations in reading. 

But according to on the data from the Texas Education Agency, roughly 10% fewer tests were taken in 2022 than in 2019 and there was an increase in students changing schools — factors that “may have contributed to the increases in proficiency rates.” If students with disabilities or English learners were among those not tested, for example, that could skew the results higher. 

Tennessee was among the first to release this year’s state testing data, which shows that students are in reading than in math. 

The early state results, Marion said, show there are no easy answers to increasing performance for all students.

“If we know how to accelerate student learning at scale, why do we see these massive gaps that have persisted and gotten worse?” he asked. “If we really knew how to do this, were we just holding back?”

As the United States approaches the end of another school year marked irrevocably by COVID, the paper’s findings are merely the latest to show substantial academic reversals resulting from lengthy school closures, uneven execution of virtual learning, and the effects of countless social and economic upheavals over the last two years.

Study co-author Megan Kuhfeld, a senior research analyst at the nonprofit research group NWEA, declared herself “surprised in some ways that the results were a lot bigger” than those following the seminal natural disaster of recent American history. But while the “concentration of disrupted schooling” immediately following Katrina was especially intense, Kuhfeld argued, most affected children were relocated and enrolled in new schools within a matter of months. 

“If you think about how prolonged the disruptions were, it does make sense to me that we saw larger drops than we did during the hurricane,” she said. 

The study, by Brown University’s Annenberg Institute for School Reform, examines student performance on NWEA’s Measures of Academic Progress (MAP) test, which assesses growth in both reading and math. The test is typically administered in participating K-12 schools three times during the school year, and results include demographic information detailing student age, gender and race or ethnicity.

All told, the study’s sample comprised roughly 5.4 million students in grades 3–8 across a swath of 12,000 elementary and middle schools; those schools represented between 12 percent and 15 percent of all American schools in each grade. To illustrate the learning trajectory of these students during the pandemic, Kuhfeld and her collaborators tracked the scores of same-age test-takers during the fall of 2019, 2020, and 2021.

As expected, the arrow pointed downward over those three academic years. Average math scores dropped by a range of 3.3 points to 4.5 points (sixth graders and eighth graders, respectively)during the period under examination, while reading scores dropped between 1.5 and three points (eighth graders and third graders, respectively).

To put those figures in perspective, the authors followed up with , which compared the negative impact captured on the MAP tests to the positive growth catalyzed by various education policy interventions in separate studies. The pandemic-related hit to math scores among elementary schoolers between 2019 and 2021 was nearly three times as large as the benefit that similar students have received from a specialized course of summer instruction in the subject. Score slippage for middle schoolers was larger in magnitude than the boost that comparable children received from high-dosage tutoring, one of the most effective education interventions that has ever been measured. 

Mike Magee, who recently stepped down as CEO of the education advocacy group Chiefs for Change, called the study “yet another indication that an alarming number of students have fallen far behind during the pandemic.” He called for a range of remedies — including extending the school day, offering higher-quality instructional materials, and deploying more tutors — to arrest the downward momentum in achievement.

“Covid has exacerbated inequities and widened gaps,” Magee wrote in an email. “If schools do not have an uncompromising focus on academics, the lasting consequences will be catastrophic — and Black and brown children, and those who live in poverty, will hurt the most.”

Indeed, the study also detected widening disparities in academic performance between students of different levels of socioeconomic advantage. In just two years, the already sizable gap in reading performance between students enrolled at relatively high-poverty schools (those in which over 75 percent of students are eligible to receive free or reduced-price lunch) and relatively low-poverty schools (those in which fewer than 25 percent are eligible) expanded by 15 percent. The gap in math performance grew by 20 percent.

Longtime education observer Tom Loveless, the former director of Brookings’s Brown Center on Education Policy, was struck by the sheer toll exacted by COVID’s disruption to learning, but also the smaller stories within the learning loss.

“What’s interesting is that the losses from the pandemic seem to be the opposite of the learning gains registered over the past three decades,” Loveless wrote in an email. “Since the early 1990s, greater gains were made by younger kids than older kids, and math gains outpaced gains in reading. Now — at least in early studies of pandemic effects — we are seeing greater losses by younger students compared to older ones and a larger setback in math compared to reading.”

The explanation for the subject-specific differences in MAP scores is still unclear, though Loveless added that math performance may simply be “more sensitive to schooling” — and therefore vulnerable to decline when schools shut down. Evidence for that theory can be found in a curious time lag: Even after the sudden impact of school closures in the spring of 2020, reading scores measured that autumn were still only modestly affected, sinking much further over the course of the 2020-21 school year. But significant declines in math scores were apparent in fall 2020 MAP performance.

Kuhfeld agreed that previous research has shown that math learning “seems to be more sensitive to time in school,” which could account for the steeper backsliding in that subject. But she added that the MAP results can’t capture the lost growth among early readers in the first and second grades, who were just beginning to develop the foundational skills of literacy when COVID hit.

“Those kids who were first-graders when the pandemic started are now in third grade,” Kuhfeld observed. “There’s probably going to be a cumulative effect where kids who missed out on those formative, learning-to-read experiences will probably see even larger negative effects on reading than we’ve observed so far.”

Further research will additionally clarify the extent to which the educational consequences of the pandemic differed among students in different learning environments, she added. One study — circulated last fall by a team of academics including Brown economist Emily Oster — has already found that learning loss was much greater in school districts that kept remote schooling for longer. Kuhfeld argued that there was likely to be “huge variation” based on the time children spent in physical classrooms.

“It was probably worse for certain groups that either didn’t have access to in-person instruction, or maybe had access but chose not to go for health or other reasons,” she said. “The kind of scary part is knowing that this was the average effect; there were certainly a number of students who had larger drops than what we’re reporting.”

Christopher Ochoa of McAllen, Texas, has loved mathematics since he was a young child, his interest fueled by summer-time math camps and trips to Space Center Houston.

The high school senior’s strong work ethic helped him manage his ADHD, dyslexia, and sensory overload well enough to earn stellar marks and gain entry to Texas A&M University.

But during the pandemic, both his grades and his academic confidence plummeted.

“When you’re in the classroom, you can ask a question, go to the whiteboard with your teacher and he’ll work through it with you,” the 18-year-old said. “Now, when you ask a question, you have to unmute your mic and you can’t see the teacher face to face or make eye contact. It’s just not the same. There isn’t that physical interaction.”

Teachers say pandemic-related setbacks in mathematics will linger well into the coming school year, especially for students who suffered the most during shutdowns. Unable to peer over their students’ shoulders and correct their work, math teachers lost the ability to offer on-the-spot tutorials. The results showed: A November NWEA of fall 2020 test scores for nearly 4.4 million children in grades 3 through 8 found they lagged 5 to 10 percentage points in math compared to students in the prior year.

Chase Nordengren, a senior research scientist at NWEA, said math-related learning loss in the upper elementary and middle school grades has been stark, mirroring what happened to students who suffered through .

He’s unsurprised by the timing: It’s in the later elementary years that students tackle computational arithmetic and conceptual thinking — including fractions and ratios — for the first time. “It’s not hard to imagine those skills are more difficult to teach remotely,” he said.

Matthew Reames, a fifth grade math teacher at Daniel Morgan Intermediate School in Winchester, Virginia, said distance learning has hindered some students’ ability to build their skills.

“The way the standards are set up, the kids really need to master every single thing in that grade to be successful the next year,” Reames said. “Even if you are missing one little thing, it might amplify itself later on.”

Other subjects, including science, don’t face the same hurdles, he said.

“If you don’t understand the water cycle, that won’t put you at a disadvantage with magnets,” he said. “You might have gaps in what you know but it is not a specific skill. Math is a whole lot of skills and concepts that build on themselves from year to year. There is not really anything you can leave out. And that’s true whether you are in a pandemic or not. Each year, we look for these gaps.”

And there are other challenges, too, as the pandemic lingers and even worsens in some areas. Trevor Doyle, a middle school math teacher in Camino, California, remains on a hybrid schedule a year into the crisis. Students attend school in the mornings or afternoons for just three hours, four days a week, with Mondays reserved for planning and meetings. It’s simply not enough time to keep them on track, he said.

Doyle is not new to the profession. He’s taught mathematics for 18 years, including three at Folsom State Prison. But nothing could prepare him for the challenges brought on by distance learning.

He believes 80 percent of his students will have made up for what they lost during the next school year.

But he’s less optimistic about the remaining 20 percent. It could take that group twice as long because the school could not offer on-site remediation during shutdowns. Prior to the pandemic, struggling students were invited to school an hour early and for another 45 minutes after the last bell rang — in addition to working through lunch.

Now, a sizable portion of his students are floundering, including those who struggled in school prior to the shutdowns and whose parents are unable to hold them accountable for online coursework.

“It’s that middle-of-the-road kid who needs that extra push, who is the furthest behind,” he said.

Summer school might help, Doyle said. President Biden’s American Rescue Plan requires states to invest at least $1.2 billion in summer enrichment programs. Doyle’s own district saw a fourfold increase in such spending, and is planning an intensive summer program focused almost solely on academics.

But he does not expect it will make up for all that was lost.

“A lot of teachers are expecting to reteach material at the beginning of next year,” Doyle said.

Yvonne Calderon said it’s not only those children from disadvantaged backgrounds who are struggling during the pandemic. Even more well-off students have lost ground. Calderon teaches 7th grade mathematics at an affluent private school in Tempe, Arizona, one that has been in-person most of the school year.

All of her students have experienced at least some trauma related to the pandemic, she said. Several told her they suffer from anxiety and many are having difficulty retaining what they’ve been taught.

She’s found herself reteaching concepts these students should have mastered years ago.

“Despite wealth, these children still face challenges within their own families,” Calderon said. “I have years of experience in teaching math and I have never seen this before.”

Danilsa Fernandez teaches middle- and high school algebra at City College Academy of the Arts in New York City. Dubbed a master teacher by the non-profit Math for America, a New York City-based group that supports educators and improves retention, she stayed on task for much of the school year until a pandemic-related closure in mid-March.

But even after her students returned, she had reason to revisit concepts she’d taught before: The transition back to in-person learning allowed her to see more of her students’ work, which reflected their inability to master key concepts.

“Mistakes that were easily hidden behind a computer screen were now in full display,” she said. “I decided it would benefit the students to re-discover some topics without rushing through the material.”

Eighth grader Jaslyn Ovalles is one of Fernandez’ students. She struggled mightily with online learning: Her grades dropped sharply across every subject.

“It’s hard when you are at home because you are in your own environment,” said Jaslyn, 13, who spent part of each day looking after her four younger siblings. “Not all of your teachers require you to turn on your camera, so you can use your phone, watch TV while you’re sitting in your bed or on the couch. It’s easy to fall asleep.”

Once she returned to campus, Jaslyn’s grade shot up almost immediately.

“If I could just get my grades up in science, I’d be passing all my classes,” she said.

Nordengren, of NWEA, advises teachers to take the time to understand each child’s abilities and be sure not to waste precious time reteaching concepts they have already mastered or skipping ahead to topics for which they are unprepared.

He said parents can help, too, by incorporating math in their everyday life — grocery store check-out lines can provide a great opportunity to consider addition, subtraction, percentages and other, more complex topics — and by not speaking negatively about the subject in front of their children.

“If you have a parent that says, ‘I’m not a math person,’ or ‘We are not math people,’ that will put that deficit mindset into a kid’s brain,” Nordengren said.

Ochoa never had that problem, until now. He’ll spend his summer working with a tutor trying to undo it.

And while he was thrilled to gain entry to Texas A&M, he made a last-minute switch to the University of Oklahoma, a smaller school that would offer him far more academic support, something he values after such a difficult year.

“Whatever I missed in online learning, I can learn there and be better prepared,” he said. “I’d rather be ahead than behind.”