“The bigger, the better, because the children depend on one another,” said Principal Susan Engdall. In a Montessori classroom, “the teacher is sparse, so children have got to be creative and figure things out.” 

It’s a philosophy that not only teaches kids to solve problems, but fosters stronger reading and memory skills by the end of kindergarten than other models of early education, according to from the University of Virginia and the American Institutes for Research. The first nationwide study of public Montessori programs shows that they also achieve more positive outcomes at a lower price tag, mostly due to those larger class sizes. Over the three-year span, public Montessori programs cost $13,127 less than traditional preschool and kindergarten programs, the study found.

Angeline Lillard, a psychology professor at the University of Virginia and lead author of the report, attributed the findings to Dr. Maria Montessori’s theory on how children naturally learn through imitation, choice and the use of that teach practical skills and academic concepts.

“This is a method that a really brilliant Italian physician made up by watching children,” she said. “She studied them in free environments and said, ‘What are they like and how can we help them?’ ” 

The findings add more complexity to a long-running debate over whether the benefits of early childhood education fade out over time. Some studies show that children who don’t attend preschool often catch up to those who did, leading whether such programs are wise public investments. A found that students who attended Tennessee’s pre-K even had lower test scores in elementary school than those who didn’t participate.

In the new study, the results were particularly strong among children from lower-income families, but Lillard stressed that only programs that stay true to Montessori principles are likely to see such positive results. 

“I see all these schools that claim to have Montessori when what they offer is just a shadow of it or ‘Montessori toys’ sold on the web,” she said. “I expect most of the folks implementing ‘Montessomething’ are also trying to help children, but without [taking] time to understand the model.”

‘For all children’

Publicly funded programs make Montessori education, long preferred by wealthy families who can afford , more accessible to low-income and working class parents. They include charter schools in a network of Montessori microschools called Wildflower, and district programs like Milwaukee’s seven Montessori schools. The district was among the first, over 50 years ago, to offer Montessori in the public sector.

One of the district’s “passion points” is ensuring that Montessori is not “only for certain kinds of people, but for all children,” said Abigail Rausch, the district’s Montessori coordinator. 

Rae Johnson, whose son is now 16, said she could never have afforded a private program in Milwaukee as a single parent working at Starbucks and picking up freelance writing assignments. But Montessori seemed like a good fit for Elijah.

“He always marched to his own beat,” she said. “I knew that traditional school just was not going to work for how he operated.” 

At first, Johnson didn’t understand Montessori’s emphasis on “practical life” skills, like pouring water without spilling or cutting with a knife. At 5, he would come home with a loaf of bread he baked at school.

“I’m like ‘This is what you did all day?’ But then he would be like ‘Oh mom, can we bake?’”she said. “That turned into a math lesson, like ‘OK, if you want to make a cake, let’s do some fractions.’ ”

The Montessori model is among the curricula used in 11 state-funded pre-K programs, according to the . Students traditionally enter Montessori at age 3, but most state-funded pre-K programs begin at age 4. That means districts often face the challenge of paying for the extra year.

The Phoenix Elementary district, which recently because of , began charging $500 per month this year for 3-year-olds entering Shaw Montessori because funds supporting the program were “needed elsewhere,” said Engdall, the principal. The waitlist to get in dropped to zero, but at town hall meetings, she heard requests from parents for things like more field trips and hands-on learning that “already encompass” what Montessori offers, she said. She expects demand to bounce back.

In addition to allowing children more freedom in the classroom, the Montessori method is in sync with the , Lillard said. Classrooms emphasize phonics, and their materials, like , make learning letter sounds and sight words a more concrete activity. In the study, students who won a spot in a public Montessori program through a lottery had “significantly higher scores” on a standardized reading test than those who didn’t get in. 

Montessori students also performed better on an that asked them to do the opposite of what the researcher said. If the adult told them to touch their head, they were supposed to touch their toes. 

Lillard speculated that the results for Montessori students might have been even stronger if the researchers hadn’t started the study the year after the pandemic, an unprecedented disruption that led to in children’s development. Because students were at home in 2020, they didn’t have an opportunity to interact in person and learn from older peers. 

“COVID impacted all classrooms, but it might have had especially strong impacts for multi-aged, peer-learning models,” she said. 

Classrooms don’t have duplicate copies of the same materials, so children, Rausch said, have to practice patience and negotiation if another child is already busy with something they want to use. “How do you plan your day? How do you communicate with someone else? You don’t just grab it out of their hand,” she said. “We’re teaching these really complex skills to 3-year-olds.”

In the study, Montessori students scored higher on a test of understanding other children’s perspectives than those who didn’t attend. But kids who went to more traditional preschools, or stayed home, were a little better at getting classmates to share. 

The fact that social-emotional learning programs are common in public schools, and likely teach topics like sharing, could account for the slight difference between the two groups, Lillard said.

On another test, non-Montessori kids were more likely to keep working on a difficult puzzle when Montessori children gave up — a finding that surprised the researchers. Montessori teachers encourage students to stick with a challenging task until they master it. 

‘A high payoff’

Overall, the results back up earlier research on public Montessori, like in South Carolina that found higher growth in math and reading among Montessori students than among those in traditional schools.

But like all studies, this one has limitations. Comparing kids who did and did not win a seat through a lottery isn’t the strongest research design. Families who apply don’t necessarily represent all families; more tend to be white and financially better off.

“It may be that there were other features of the schools that parents found desirable,” said Steve Barnett, senior co-director of the National Institute for Early Education Research. He also questioned whether high absenteeism following the pandemic could have affected the results for either Montessori students or kids in the control group. 

He still thinks the results are promising, and said even non-Montessori programs could adopt multi-age classrooms that include 5-year-olds. But what the field needs is more evidence that the benefits last beyond kindergarten, he said. 

“None of this is to suggest we should ignore or discount the results, only to be cautious,”​​ he said. “Certainly, Montessori deserves more attention. There would be a high payoff to additional rigorous research.”

With roughly 160 students — nearly two-thirds of them low-income — the rural Charleston County school recorded well below district and state averages. One-third of students were chronically absent, and school climate was ranked low by teachers.

“The school was a failing school, and it had been a failing school for a few years,” Smalls said.

Smalls’s first year coincided with the school’s acceptance into the University of Virginia Partnership for Leaders in Education, a program that helps improve low-performing schools through administrator training and professional development. 

Frierson Elementary is one of three schools that transformed from struggling to succeeding because of the turnaround program. After one year, the school went from an unsatisfactory to excellent rating, the in the state’s report card system. 

Since 2004, the partnership has worked with more than 900 schools from 33 states. Roughly half achieve double-digit gains in reading, math or both, within three years of starting the program.

For two to three years, administrators receive professional development at the university and coaches visit their schools to help brainstorm ways to improve academic achievement, attendance and culture. Districts must apply and, if selected, pay roughly $90,000 for program costs.

Leighann Lenti, the program’s chief of partnership, said the key to transforming a low-performing is to work with district and building administrators to make systemic changes that will lead to improved student outcomes.

“They’re given a chance to think about the design and the decisions they’re making in their buildings and in their school district,” Lenti said. “[They] think about their highest priorities and the root cause of what hasn’t worked, so they can solve those problems differently — not just keep doing the same things over and over — and see tangible results for kids.”

A 2016 found that 20 Ohio schools that participated in the program saw statistically significant academic improvement that persisted even two years after completion. 

The program focuses on four areas of school improvement: system leadership, support and accountability, talent management and instructional infrastructure. 

During the first year, University of Virginia staff work with district and school leaders to develop a plan for their school. They try to find root causes for low performance and create goals that are revised every 90 days.

Administrators at Schoolfield Elementary in Danville, Virginia, started the program before the 2023-24 school year and finished in January. Principal Kelsie Hubbard and her colleagues created a 90-day plan with three main areas of focus: professional learning, classroom instruction and teaching strategies.

Educators began professional development twice a week to make sure instruction and activities matched existing rigorous academic standards. They also worked to ensure students were being taught the same way in every classroom, so they didn’t have to relearn strategies if they changed grades or teachers.

“Coming out of COVID, we were seeing a lot of our students performing below grade level, and so a trend we started to notice is that our instruction was not meeting the rigor of the standards,” Hubbard said.”We were teaching lower level because we were assuming that students needed that intensive intervention. … But we were holding and keeping them further and further behind.”

At the end of the program, Schoolfield — a building of 500 students, with 85% low-income — improved its from 68% in 2023 to 78% in 2024. Math proficiency went from 68% to 73%.

Similar gains were observed in Alabama’s Florence City Schools, a district of 4,500 students that recently finished the program. Three of its lowest-performing elementary schools that participated all reported improvements in reading, math and chronic absenteeism.

Superintendent Jimmy Shaw said principals met with reading and math teachers to brainstorm why academic scores were lacking. 

For example, they found in Weeden Elementary that third graders had a hard time with geometry and other math topics while taking state assessments. Teachers began to give 10-minute mini-lessons daily to help students master specific skills.

“It’s been beautiful work to be able to build the capacity of our leaders and our research teams. To us, that’s what it’s about,” Shaw said. “It’s not about having some dynamic leader, but it’s about building the capacity of a group of adults who can understand system structures and processes to be able to attack a problem.”

Smalls’ 90-day plan for Frierson Elementary began with a list of goals such as improving school climate by training educators and ensuring they got enough classroom time to teach the? curriculum. She also delivered a “state of the school” address for families to explain Frierson’s unsatisfactory rating and what steps were being taken to fix it. Teachers hosted literacy and math nights to get parents more involved in their child’s learning.

“I felt like I created an environment, a climate or a culture where everybody was valued and everybody was seen as a leader,” Smalls said. “[The program] is very effective. It is very self-provoking, very reflective, very action-based and action-oriented. I really believe in it.”

At the University of Virginia’s Partnership for Leaders in Education (UVA-PLE) Program, where I was the chief support officer, talent management was one of four core levers to create district conditions for schools to significantly improve. Our approach, which the . identified as having significant evidence of impact on school performance, suggests that our focus on increasing the number of highly effective teachers in schools was critical.

As districts prepare for 2023-24 staffing, here are five strategies that should be explored as a part of reimagining human resources practices and approaches to improving student outcomes — especially for underserved children.

Implement a strategic staffing approach like Opportunity Culture. , developed by the nonprofit , was launched in the Charlotte-Mecklenburg Schools in 2012 as part of the district’s turnaround initiative, . This approach includes several innovative models, such as multi-classroom leaders, in which teachers who have had success with improving student performance lead a small team of educators for substantially higher pay. The leader teaches students for a portion of each day; guides lesson planning, data analysis, instructional changes and small-group tutoring assignments; coaches and works directly with team members in the classroom; and models great teaching. Among the :

• Graduation rate increased from 54% to 86%, reducing the gap between the 10 L.I.F.T. schools and the district overall from 20 percentage points to less than 5.

• in L.I.F.T. schools were met or exceeded 80% of the time, a rate equal to the district’s average and higher than the state’s.

• 94% teacher retention. After Opportunity Culture was implemented, the number of teacher vacancies dropped from at the start of the school year to five.

Leverage a housing concierge concept. At UVA, our program was anchored by a partnership between the Curry School of Education and the Darden Business School. We encouraged districts to consider adopting a concierge approach, as the business sector does, to smooth the way for new staff trying to settle into a new district or city — by helping to arrange housing, register for utilities and cable, and identify options for day care and banking.

Develop lead teachers. When I was chief innovation officer for Cincinnati Public Schools, it earned the distinction as the state’s top-performing urban district, with a “B” report card rating, in part because of its career ladder for educators. The model provides stipends to teachers who go through a credentialing process, which involves earning excellent evaluations and undergoing advanced training. Underperforming schools can select these highly trained educators to work in classrooms and be teacher leaders in their building.

Provide recruitment and retention stipends. As chief turnaround officer (deputy superintendent) for the Georgia State Board of Education, I partnered with the state’s General Assembly to create a pilot program that would give chronically underperforming schools $5,000 per teacher from the state to recruit and retain highly effective educators. Teachers would receive ongoing training to benefit both their students and the school overall. Each school could receive funds for up to five teachers. To ensure some skin in the game, the legislation required each district to contribute $2,500 per teacher, meaning participating educators could receive an extra $7,500 per year. A slightly amended approach was signed into law.

Recruit talent early, as major college athletic programs do. Major college football and basketball programs identify potential student-athletes as early as possible — sometimes even in elementary school — and do recruitment and outreach until these kids accept a scholarship. Using that model, districts can use in-classroom training, student teaching and other opportunities to build early relationships with college students. This could connect to a district’s that guides graduating seniors into the teaching field in hopes they will come back to teach in a local school.

These strategies alone will not provide marginalized students with the conditions and support they need. And, of course, no two districts or communities are exactly alike. But as leaders determine their own talent strategies, reimagining talent practices should be an important part of any broad transformation effort.

In a city renowned for its colleges and universities, Boston Public Schools earned its own acclaim in recent years as an innovative, fast-improving hub of K-12 excellence. Situated in the birthplace of American public education, and combining generous funding with a thriving charter school sector, the district was held up for over a decade as a model of urban education reform. 

But as the 2021-2022 school year draws to a close, those past accolades seem as distant as the days of Horace Mann. Amid plummeting enrollment, persistent achievement gaps, and a nasty COVID hangover, Boston faces perhaps the greatest educational crisis since its scarring experience with desegregation in the 1970s. And in the weeks to come, the city may lose more than its national luster.

In March, Massachusetts Education Commissioner Jeffrey Riley of the state of the district. Both local and national experts wondered openly whether the review, which follows a , was the first step toward a complete takeover of the region’s largest school district. In the months since, of bad press has done nothing to quiet speculation.

The audit, released Monday, provided the latest sign that state authorities are strongly considering action. Despite making improvements in a few areas, the reviewers found, “the district has failed to effectively serve its most vulnerable students, carry out basic operational functions, and address systemic barriers to providing an equitable, quality education.” The situation called for “immediate improvement,” they concluded.

The prospect of receivership (as takeovers are known locally) is hardly unprecedented in Massachusetts, which allows its education department greater latitude to reshape failing school districts than most state authorities elsewhere. But the structural problems facing Boston cast doubt on whether such an effort can be successful.

For three decades, the district has operated substantially under mayoral control, and newly elected Mayor Michelle Wu has already made clear her opposition to state intervention. Republican Gov. Charlie Baker — an education reform ally whose tenure has seen several takeovers — will soon be leaving office, likely to make way for a Democratic successor with sharply different views.

Wu told the that she met with Baker and Riley Friday and that they are still working on an agreement “that will set the district up for success.”

“A lot of what is in the review matches with what our school communities and administrators have been calling for, in how urgently we need to focus on BPS and our young people, and in the need for strong, effective leadership,” she said.

Cara Candal, a senior fellow at , said it was ambiguous whether Riley was leaning toward receivership or a somewhat less drastic approach. While significant obstacles existed, she said the recently completed review demonstrated that “kids aren’t learning, and many are unsafe in school.”

Candal, who calling a takeover Boston’s “best hope” for revival, said her takeaway was that things were “as bad as expected in some places and worse in others. In my opinion, the report underscores that the state needs to move with some urgency to provide BPS with the structures, support, and accountability necessary to effect change … There is a window for the state to act now, and I hope it will.” 

Ultimately, the audit called for “bold, student-centered decision-making and strong execution” to reverse what it described as the district’s “entrenched dysfunction.” What that means in practice is difficult to predict. The Massachusetts Board of Elementary and Secondary Education is expected to deal with the report’s findings at its regular meeting on Tuesday morning.

But Ross Wilson, executive director of the Boston-based Shah Family Foundation, said Massachusetts should consider multiple options for intervention instead of duplicating the takeovers of major districts that have taken place in other states.

“Our state and city have the opportunity to do things differently,” Wilson argued in an email. “We should think creatively, collaboratively, and with urgency about the support and accountability necessary to serve the students of Boston.”

‘A steady stream of negative reports’

Few share Wilson’s historical perspective on the highs and lows of Boston Public Schools. A former kindergarten teacher, school principal, and central office administrator, he finished his career with the district in 2017 as deputy superintendent.

That long tenure gave him an inside look at Boston’s ascent in the late-1990s and 2000s as a district known for continuity and rising performance. The schools were overseen for over a decade by Superintendent Tom Payzant, who placed and enjoyed a strong partnership with the city’s similarly long-serving mayor, Tom Menino. By the end of his tenure, Payzant was frequently named as one of America’s best schools chiefs, and the district the prestigious Broad Prize for excellence in urban education. As measured by the National Assessment of Academic Progress, the academic growth of Boston students that of students in other major districts during this time.

The momentum carried on for several years after Payzant’s departure but eventually began to stall. A major culprit was churn: Including interim appointments, Boston has named four superintendents since 2012. Fast turnover has also extended to the bureaucracy — between 2016 and 2019, the district, and less than 12 percent stayed in the same role — and even to the mayor-appointed school committee, which over the last few years.

Wilson remembered that the strategy for governing both traditional K-12 schools and their more autonomous counterparts (the district operates over 20 “pilot schools” that enjoy greater flexibility in hiring, setting budgets, and choosing curriculum) had “shifted from superintendent to superintendent,” leading to “overall confusion.”

The result of Commissioner Riley’s first review was a highly critical document that pointed to “staggering” rates of student absenteeism; in all, close to one-in-three Boston students attended schools that ranked in the bottom 10 percent across the state. In response, the city joined in a “memorandum of understanding” with Riley’s state education department in March 2020, pledging to turn around achievement in underperforming schools, diversify its workforce, and revamp its oft-troubled system of school transportation. 

But the memorandum went into effect at almost the exact same time that the city’s schools first closed due to COVID-19, not to reopen for fully in-person learning for over a year. As in most of the country, test scores tumbled dramatically during the pandemic. Since students returned to classes, however, Boston has also been plagued by constant bad press, including several of against school employees; at a K-8 school that the school committee voted to close; and that has left the district nearly 20 percent smaller than it when it won the Broad Prize. 

In February, Superintendent Brenda Cassellius that she would resign in June after three tumultuous years. In a letter to the school community, the Globe reported Monday, she vowed to push forward needed changes but acknowledged that “this work will require increasing staffing, operational support, and other resources, including a more robust collaboration with City departments, to ensure that we are prepared to meet all of our students’ needs.”

Paul Reville, a professor at the Harvard Graduate School of Education who formerly served as the Massachusetts Secretary of Education, said that the search for a new superintendent came at a distinctly unpropitious moment. 

“We’re trying to attract a new superintendent at a time when we’re on the heels of two superintendencies that did not end well,” observed Reville, who receivership. “We’re facing the threat of a state takeover, we’ve got a steady stream of negative reports on the performance of the school system, and the governance system is shifting. So you might be a superintendent working for a new boss in two years.”

Top on the list of responsibilities for the next superintendent will be dealing with a daunting set of problems laid out in the state audit. Among them:

• While one-in-five local students take part in special education, that area of services “remains in disarray” two years after the 2020 review found them to be sorely wanting. Education of English learners was also highlighted for particular criticism.
• Boston is not meeting minimal standards for the delivery of essential district services, including school transportation. Late or uncovered bus routes are “significantly disrupting education for tens of thousands of students each month,” the authors wrote.
• Even the grievances identified in the audit may understate the extent of the problems because of a “pattern of inaccurate or misleading data reporting by the district.” BPS officials inflated the number of buses arriving on time, inaccurately reported the number of school bathrooms it had renovated, and possibly displayed incorrect student enrollment and withdrawal data on its public website.

Skepticism on takeovers

But if the problems facing Boston are significant, it’s not clear that receivership is the remedy.

Takeovers are among the most contentious school improvement strategies available to states. Even when launched in cities where schools have struggled to serve students for many years, they often sideline elected boards and offend both teachers and families by abrogating local control. Some scholars contend that by alienating voters — disproportionately those of color in cities like Boston — from governance of their own institutions, takeovers do more civic harm than educational good. 

What’s more, evidence of their effectiveness is somewhat scant. A 2021 study of takeovers initiated in dozens of mid-sized school districts found that, on average, they yielded no positive outcomes on test scores; in fact, the disruption of the move led to further struggles in some communities.

Reville argued that the recent history of district takeovers suggested that most states lacked the capacity or the legal scope to pursue them effectively. 

“I think our legislation gives the state more tools and more power than is the case virtually anywhere else in the country, so if you got a chance to do it, it would be in Massachusetts,” he said. “Still and all, I think the evidence from past experience suggests more modest expectations about state takeover.”

Much of the Massachusetts debate will center on the existing takeovers launched over the last decade in the long-scuffling districts of Southbridge, Holyoke, and Lawrence. None of the three school systems have yet regained control over their school systems, and all still rank among the lowest-performing in the state. Still, initial test results included in the 2021 analysis found that reading test scores had improved somewhat in both Holyoke and Lawrence. Receivership in the latter city was personally overseen by none other than Riley, whose appointment as state schools commissioner was predicated partly on the results he achieved in Lawrence.

“Although nationally we don’t have great evidence that this is a key way to improve academic achievement, it does seem like Massachusetts has a stronger track record in this area than other states at using receivership toward the ends of improving achievement,” said study coauthor Beth Schueler, a professor of education and public policy at the University of Virginia.

Because of the relatively narrow time period under observation, that paper excluded the takeovers of schools in New Orleans and Newark, where student outcomes improved sizably while under state control. But in those cases, a principal tactic of improvement was the expansion of high-performing charter school networks, which came to enroll sizable portions of K-12 students across both cities. Boston is similarly home to in the country, but a statewide cap on new charter schools prevents their expansion.

“As much as I would love to say to Boston families, immediately, ‘We’re going to knock down district boundaries and make choice available to you,’ that’s not going to happen in Massachusetts,” the Pioneer Institute’s Candal said. “I think there are lessons to be learned, but we’re not going to be a Newark or a New Orleans because the other stakeholders in the state won’t allow it.”

A ticking electoral clock

The dynamics of receivership in Boston would differ from prior takeovers in at least one other aspect: Authority would be flowing away from a newly elevated leader with an unblemished record, and toward a state government that is headed for the exits.

Wu, both the first woman and first non-white person elected as Boston mayor, won the Democratic Party’s nomination in 2021 with for the district’s future. In office for just four months, she has already proposed her own “Green New Deal”: a $2 billion investment in school renovation and construction. With Superintendent Cassellius stepping down, she will soon help select BPS’s next leader, the most crucial decision facing the district in the coming months.

Wu’s outsize influence over local schools means that if receivership comes, it will be at the expense of a well-known and highly popular figure rather than the obscure members of a local school board. Wu has already demonstrated her awareness of that advantage by , alongside the head of the Boston Teachers Union, to warn against the possibility of receivership.

In a statement responding to the audit, Boston Teachers Union President Jessica Tang called the timing of the release “suspect, rushed, and ill-advised,” alleging that the state report was marred by unspecified factual errors.

“This is an opportunistic attempt to overcommit the state past the current governor’s tenure to a hostile, unhealthy and burdensome relationship with the city by bullying the new mayor into an untenable, undemocratic, and patronizing arrangement,” Tang said.

In response to the unified pushback, Schueler said she wondered how politics might influence a takeover’s effectiveness.

“Proponents of takeover often point to school board dysfunction as the source of all the problems. What do they see as the source of the problem in Boston, and is that problem going to go away with takeover? It’s not getting rid of the board in this case.”

Receivership is almost always dreaded in local communities, but in Boston, there is another wrinkle: Even while electing Wu last fall, voters also demanding a return to elected school board members. Such a move would also inevitably limit the powers of the new mayor, who has said she favors a hybrid committee including both elected and appointed members. 

Will Austin, a former charter school leader who now serves as the CEO of the nonprofit , argued that while popular opinion might be firmly set against the appointment of a state receiver, state law was unambiguous in delineating Commissioner Riley’s powers to act in struggling school districts — of which Boston is undeniably one. 

“The statute and regulations are clear and blunt,” Austin said. “ A vote by the Board of Elementary and Secondary Education decides this — nothing else.”

But the relevant actors also face a ticking clock. In November, the state’s deep-blue electorate will choose a new governor; it is widely expected that Gov. Baker, a two-term Republican, will be succeeded by a progressive Democrat cut approximately from Wu’s cloth. Whoever that person is — Attorney General Maura Healey appears to be — will have little interest in being accused of disenfranchising Wu and the voters of Boston. So while an opportunity exists to set a receivership in motion, it could disappear before long. 

In the meantime, the district continues its reemergence from the COVID era. With to be the next superintendent, Wu and the school committee could race to make a hire before the state reaches a consensus.

In response to the newly released review, Reville said the situation demanded close cooperation between Boston and the state.

“​​The report reiterates and describes problems that have persisted for a long time. The conversation needs to shift now from diagnosis to prescription. Neither the state nor the city is likely to be able to go it alone. The best chance for a remedy is a robust partnership between state and local leaders…and the political will to overcome resistance to change.”

is one of the first thorough examinations of classroom observation — the common method of using an evaluator, such as a school principal, to watch and rate a teacher’s work with pupils — across an entire state. Its findings may cast doubt on the efficacy and fairness of the practice not only in Tennessee, but also the huge number of states that also place the in-person reviews at the heart of their federally mandated teacher evaluation systems.

Study co-author Jason Grissom, a professor of public policy at Vanderbilt University, said that distortions in teacher evaluations — which were especially large in observations of male instructors relative to females — held significant sway over decisions on retention, firing, and promotion. Biased scores could undermine states’ ability to raise teacher performance and offer a better education to students, he added. 

“If we’re not collecting accurate information, it’s going to disrupt the feedback that’s supposed to be a big way that evaluation can drive improvement,” Grissom said. “And it can treat people unfairly, which can undermine the capacity of the system to improve schools.”

The study, conducted by Grissom and University of Virginia professor Brendan Bartanen, focused on Tennessee as an example of an evaluation framework that has long since reached maturity, with standards-based performance rubrics and observers who are trained to follow specific procedures in rating teachers. One of the original winners of the Obama administration’s Race to the Top school reform initiative, the state first rolled out its system in 2011. In-person appraisals represent the largest single element in each teacher’s overall performance score, alongside student test scores and other factors. 

To isolate the possible role of bias in ratings, the researchers accessed detailed administrative data on Tennessee teacher demographics, locations, and work experience. Next, they poured over information from over 460,000 classroom observations between the 2011–12 and 2018–19 school years. Teachers in the state typically undergo between two and five observations each year, and the overwhelming majority are rated on 19 indicators of instruction, environment, and planning. On each metric, subjects are measured on a scale of one (“significantly below expectations”) to five (“significantly above expectations”). 

Across all years, male teachers scored approximately .18 points lower than females on average on the 1–5 scale, while African Americans scored approximately .09 points lower than whites. Black male teachers, faced with two possible sources of bias, were the lowest-scoring group, rated about half of a standard deviation lower than their white female counterparts, the highest-scoring. Black women scored slightly higher than white men. While ratings for all groups crept upward over time, gaps between categories remained roughly the same throughout.

The racial and gender disparities shrank somewhat, but did not disappear, when Grissom and Bartanen controlled for factors such as teacher experience, educational attainment (whether or not they had gained a master’s or PhD), and student test performance. In other words, even when comparing similarly credentialed teachers whose pupils achieved at about the same level, white and female teachers were rated higher.

As a way of demonstrating the effects of these gaps, the researchers theoretically “credited” African American and male teachers with the points that they evidently lost due to bias during their classroom observations; ultimately, 9 percent of all male teachers would have ascended to the next threshold on the five-point measurement scale, including one-third of all males rated at Level One and nearly one-quarter of males rated at Level Two. 

The difference in those grades, especially at the lower margins of teacher performance, could mean everything to a given educator, Grissom argued.

“The difference between a Level-One and a Level-Two [grade] is very likely the difference between you getting to come back to your school next year or not,” he said. “The difference between Level Two and Level Three might be the difference between you being on probationary or non-probationary status. So the magnitude is large in that sense.”

Exploring possible explanations for the trends, the authors discovered that the racial gap, while smaller, was perhaps more explicable: Black teachers were more likely than white teachers in their own schools to be assigned students who had previously achieved at lower levels and were more likely to be absent from school. They also received modestly higher grades from same-race observers than from white observers, and experienced larger score gaps in schools that employed fewer African American teachers. 

The explanation for the difference between genders was murkier, though it could stem from the fact that men are more likely to teach subjects (such as career and technical education) and at grade levels (particularly high school) that tend to see lower classroom observation scores on average.

The results somewhat echo those of earlier research focusing on the Measures of Effective Teaching Project, a teacher evaluation initiative funded by the Bill & Melinda Gates Foundation. , two groups of teachers were more likely to be graded lower on a set of low-stakes classroom observations: Men, and those who worked in classrooms with higher concentrations of low-performing students and students of color. A authored by researchers at Brown University also found that low-achieving students are disproportionately likely to be assigned to non-white and novice teachers.

Grissom added that his own prior investigations have suggested that school leaders rely heavily on classroom observations as a kind of “eye test” to help form judgments on personnel decisions. 

“One of the really stark findings is that principals really emphasize what they’re seeing in observation,” he said. “That’s the real information that’s useful, and in their own minds, they down-weight other information for various reasons.”

New research being released today aims to change that by focusing explicitly on the effects of in-school mentoring. The , circulated as a working paper through Brown University’s Annenberg Institute for School Reform, finds that high school students with mentors tend to earn better grades, stay in school longer, and make more money than peers who are otherwise similar to them. Unfortunately, the lower-income students who seem to benefit the most from mentoring at school are also the least likely to receive it.

Get stories like this delivered straight to your inbox. Sign up for ����Ӱ�� Newsletter

The paper builds on that has detected significant benefits from providing mentors to kids. But that work has usually looked at structured and well-known programs like Big Brothers Big Sisters, which draws together adults and children who are both expressly looking to establish connections. Matthew Kraft, an economics professor at Brown and one of the study’s authors, said that the webs of “natural mentoring” in school environments represent a much more common phenomenon that needs to be investigated in its own right.

“Natural mentoring — when students and adults in school buildings develop relationships that go beyond the formal role of the teacher in the classroom or a coach on the athletic field — happens far more frequently than the ways in which we offer formal mentoring,” Kraft said. “So we need to understand the degree to which that matters for kids, where it’s happening, and where it’s not happening.”

But Kraft and his co-authors, University of Virginia psychologist Noelle Hurd and Anneberg research analyst Alex Bolves, faced a problem. Natural mentoring is, if not random, organic and difficult to replicate: You can’t design a research trial that will offer identical doses of care and attention to kids in schools and then compare them with a control group.

To help overcome those issues, the team turned to a huge data set, the National Longitudinal Study of Adolescent to Adult Health. Commonly called “Add Health,” the project was launched in the 1994-95 academic year to track a nationally representative sample of over 20,000 middle- and high-schoolers as they aged into early adulthood. Waves of in-home interviews with participants have revealed countless details of their home and social lives over nearly three decades, including their relationships with adults. All told, over 15 percent identified a teacher, coach, or school counselor as an important mentor, with 80 percent of those saying that their mentorship persisted past high school.

Determining the effects of all that mentoring required the researchers to use a variety of statistical methods. They studied the academic records of students from before and after they connected with their mentor; examined similar pairs of adolescents including 1,213 twins and triplets, 1,378 students who named one another as best friends, and 548 students who engaged in romantic relationships with one another; and they controlled for a host of demographic factors including race, gender, disability and immigration status, family structure and household income.

In the end, the data pointed to a clear, wide-ranging set of benefits resulting from mentorship. Students with mentors gained between .06 and .48 points of grade point average, were between 18 and 35 percent less likely to fail a course, and were 10 to 25 percentage points more likely to attend college. Turning to workplace outcomes, the authors estimate that mentorship may boost the annual earnings of students by between $1,780 and $5,337. Those effects compare favorably to some of the most effective education interventions that have been studied, including high-quality pre-K and lower class sizes.

Kraft cautioned that these associations between in-school mentoring and improved short- and long-term circumstances should not be regarded as clear causal evidence. But they offered the “most robust empirical evidence to date” of the importance of school-based mentoring, he said — and they fall in line with existing evidence, both from formal mentoring programs and the lived experience of many people.

“None of our methods are gold standard, and we can’t definitively say without a doubt that natural mentoring causes the outcomes we observe to improve,” Kraft said. “However, we are able to leverage multiple approaches to account for the biases we think might be present. And across all the approaches, we can’t make what appear to be the benefits of natural mentoring go away.”

‘Size of these effects is amazing’

Unfortunately, the Add Health data was equally clear that not all K-12 students benefit to the same extent from strong relationships with adults at school. Roughly 15 percent of white participants and 20 percent of Asian -American participants said they had experienced in-school mentoring; roughly 12 percent of African -American and Latino males, and about 10 percent of African -American and Latino females, said the same.

Class was also a noteworthy factor: Over 17 percent of students from more affluent families reported the existence of an in-school mentor, compared with just 12.5 percent of students from less affluent families.The divergence is especially damaging because the apparent effects of mentoring, including reduced course failures and greater college attendance, are significantly larger for children of lower socioeconomic status.

In this aspect, the study’s findings closely coincide with those of , this one examining a more formal mentoring system in Germany. That experiment looked at over 300 high schoolers from 10 cities who were paired with university undergraduates through a program called Rock Your Life! The younger students were drawn from schools in each city’s lower academic track, making them much less likely to attend college. But after years of collecting data, researchers found that receiving mentoring had delivered substantial improvements to their math grades, social skills and declared willingness to attain a workplace apprenticeship.

Ludger Woessmann, a professor of economics at the University of Munich and one of the study’s co-authors, told ����Ӱ�� that those positive effects accrued almost exclusively to poorer students; in contrast with Kraft’s work, which found the benefits of mentoring to be universal, if weighted somewhat toward the economically disadvantaged, the German experiment showed that more affluent participants received almost no benefits.

“The size of these effects is amazing,” Woessmann said. “It’s somewhat hard to quantify exactly what they mean, but they are huge. And I think it’s a very gratifying result because we really see in all these dimensions — some of these are subjective things, but school grades come from official data — they are really improved, big time. So what we learn is that the life outcomes [of disadvantaged students] are malleable.”