Spotlight On Evidence: Investing in Innovative Tutoring Approaches To Boost Student Achievement

The passing of the American Rescue Plan (ARP) in 2021 infused $125 billion into public education across the U.S., presenting school districts with an unprecedented opportunity to make significant investments in addressing pandemic-related learning loss. Estimates indicate that about $700 million of this funding went to the adoption and scale of tutoring. These investments were based largely on rigorous evidence finding that high-impact tutoring—defined as students spending substantial time each week receiving targeted curriculum-aligned instruction while building strong tutor-student relationships—had sizeable, positive impacts on student achievement, with consistently large effects on students from families with low incomes. Although learning gains have not fully recovered from the pandemic, there is evidence that trends in achievement are pointing upward and federal ARP money likely did boost students’ academic achievement. Collectively, these findings point to the potential value of high-impact tutoring for supporting learning recovery at scale.

As districts spend their final ARP dollars, they must now figure out the best way to sustain high-impact tutoring going forward. Although doing so may be critical to academic recovery, there are key challenges with scaling high-impact tutoring, including the substantial cost per student, the need to deploy a high-quality tutoring workforce, and the ability to effectively allocate resources to the students most in need of support.

Below, we examine the evidence across these three areas:

Reducing costs through hybrid tutoring models

The bulk of the research on tutoring to date has focused on traditional, in-person supports that offer tutoring at least four times per week, for approximately 30 minutes per day, in groups of four or fewer students to one tutor. This type of model, though robust, costs districts approximately $1,200-$2,500 per student on average.

In 2017, Overdeck Family Foundation’s Innovative Schools portfolio began funding Saga Education, an evidence-based, personalized tutoring model that, in its original structure, embedded the best practices of high-impact tutoring by matching two students with a tutor four times a week. Recently published results from two randomized controlled trials (RCT) of Saga’s in-person tutoring program from 2013-15 found that tutoring provided by Saga improved participating ninth and tenth graders’ math achievement by about .28 SDs, or about 1.5 years of learning. Tutoring also increased math grades for these students by 0.52 points on a traditional GPA scale and decreased the proportion of failed math courses by about nine percentage points. However, at approximately $3,500 per student (in 2013-15 dollars), this type of investment may not be sustainable in many school districts.

To understand whether more cost-effective models could potentially deliver comparable results, Overdeck Family Foundation and Arnold Ventures provided a grant to the University of Chicago Education Lab in 2018 to study Saga’s new hybrid tutoring approach. In this model, students work in groups of four—two students work with an in-person tutor while two work on aligned independent computer-assisted learning, alternating every other day. The research team again used an RCT to compare math achievement for students randomly assigned to this hybrid tutoring approach versus those who continued with business-as-usual instruction. The study reported impacts of approximately 0.23 SDs, or about a year of learning, on math achievement for participating ninth graders. At approximately $2,250 per student (in 2018-19 dollars), this less expensive hybrid model highlights the potential of blended strategies to support learning in a more cost-effective manner for districts.

Utilizing virtual tutoring to access and deploy a strong tutoring workforce

As tutoring providers seek to test more cost-effective models, one of the most significant issues they face is how best to identify, train, and deploy effective tutors—an even greater challenge in rural areas and regions with fewer college-educated education professionals.

Virtual tutoring models, which attempt to approximate the in-person tutoring experience via a video platform like Zoom, may be effective for addressing these staffing challenges by offering a central cadre of tutors who can work with students anywhere at the time of day that works best for the school. Although virtual models are very new and still largely focused on R&D, there is emerging evidence that they can be effective, though impacts are typically much smaller than in-person approaches. For example, in a recent RCT of BookNook, a virtual tutoring platform focused on reading, a research team from Columbia University found small but positive impacts of 0.05 SDs, or about three to six weeks of learning depending on grade, on a widely-used, broad assessment of language and literacy skills for students in first to fourth grade. Impacts were slightly larger, 0.08 SDs, for the group of students who participated in at least 10 tutoring sessions.

Overdeck Family Foundation has explored the potential of virtual tutoring with investments in organizations like OnYourMark, a program that provides virtual literacy tutoring to 2,600 students per year in kindergarten through second grade, making high-impact tutoring more accessible in places like Louisiana and Texas at approximately $1,600 per student. In 2022, researchers at Stanford University conducted an RCT of OnYourMark’s virtual program comparing literacy achievement at the end of the school year for students in kindergarten through second grade randomly assigned to receive 1:1 virtual tutoring, 2:1 virtual tutoring, or business-as-usual instruction. OnYourMark’s 1:1 virtual tutoring improved one measure of participating students’ foundational literacy skills by about .12 to .16 SDs, or roughly one to two months of learning depending on grade level. The 2:1 approach, while less costly, did not measurably improve student learning outcomes in this particular study and neither the 1:1 nor the 2:1 approach had impacts on a second, broader measure of language and literacy skills examined by the research team.

These findings highlight the potential for virtual tutoring models to help boost key foundational skills that are critical for becoming a stronger reader. Yet, more work is needed to identify approaches that can yield impacts that come closer to traditional, in-person tutoring and meaningfully address the need for learning recovery. Continued investment in research for early-stage organizations like OnYourMark is critical to building better and more actionable evidence to achieve that goal.

Harnessing the power of AI to effectively allocate tutoring supports

A final constraint faced by tutoring providers is their ability to ensure tutoring is deployed as efficiently as possible, while maintaining impact and cost-effectiveness. There is some evidence finding that tutoring has larger impacts on students who start the school year with the lowest levels of achievement. As such, many tutoring programs target the lowest achieving students or those identified by teachers as most in need of support. Yet, other work has found that all students can benefit from tutoring in most settings assuming the content is aligned with their level of learning.

Determining the best way to implement a dynamic approach—wherein tutors are monitoring student learning and providing support as needed—has largely eluded the field. However, the introduction of AI technology has begun to make this idea seem more feasible. Products like Khanmigo are testing ways to create student-directed chatbots that act as personal tutors, but there is currently limited evidence of impact on student learning outcomes.

In contrast, there are emerging opportunities to meld human tutoring with AI to help tutors identify which students to support and when. New to the Innovative Schools portfolio is PLUS tutoring, developed by Carnegie Mellon University, which operates during regular math periods as students work with a school’s chosen math software. PLUS’s human tutors use a dashboard—the PLUS Toolkit—which connects to a school’s math software and provides tutors real-time and historical information on which students are progressing as expected, struggling, or not engaging. Using these data, tutors can virtually support the students who most need their guidance, allocating time more efficiently while providing personalized learning support. Tutors are also upskilled with PLUS Training, an AI-powered tutor training program that aims to enhance tutor knowledge on relevant topics.

Although there is not yet experimental evidence on the PLUS model, which currently costs approximately $460 per student per year, there is some emerging descriptive research that PLUS can help accelerate learning. For example, a recent propensity score matching analysis comparing students with access to PLUS to those in business-as-usual settings found that PLUS students made gains of 6.8 points on the NWEA math assessment across the school year compared to 3.6 points for students receiving typical instruction and no tutoring. Across a series of quasi-experimental studies, researchers also found evidence of increased efficiency: PLUS was associated with reductions in student:tutor ratios and increases in the amount of time students spent with their tutors. With our Foundation’s support, PLUS will continue to strengthen its evidence base with a future focus on using experimental methods to estimate the impact of its approach on achievement across settings.

What’s next

As the funding cliff approaches and educators navigate tough decisions about what is best for students and how to allocate limited resources, our team remains committed to investing in the development and scale of evidence-based, high-impact tutoring solutions that cost-effectively yield substantial gains in student achievement. To continue learning more about progress with these approaches, we recently approved a grant of $500,000 to the University of Chicago Education Lab and MDRC to support the Personalized Learning Initiative, which studies the effects of high-impact tutoring in eight school districts across the country, with a specific focus on understanding what works, for which students, under what circumstances, and at what cost.

In the months ahead, we are eager to deepen our understanding of the potential of innovation, including hybrid models and AI, to reduce the costs of providing high-quality personalized learning opportunities to students while maintaining relative impact. Arriving there will take a collective investment in evidence-building from public and private funding sources. Doing so now is the best step forward to ensure states, districts, and schools have the information they need to access the most effective solutions for their students.

To access a full list of recent research funded by Overdeck Family Foundation, we invite you to explore our Research Repository.

Watch “Spotlight On Evidence: The State of Tutoring” to learn more.

Thank you to Lina Eroh, Pete Lavorini, and Brittany Sullivan for your contributions to this article.

Header image courtesy of OnYourMark & Accelerate