What We’re Learning: New Research From Our Grantees Around Implementation and Dosage

One of our guiding principles at Overdeck Family Foundation is the power of high-quality research to drive evidence-based decision-making and, ultimately, to improve educational outcomes for all children. We’re committed to advancing the sector’s knowledge of effective programs and models, which is why we fund rigorous and actionable research across our portfolios and support our grantees in strengthening their evidence base.

Year-round, we update our Research Repository to reflect the latest publications from our grantees, highlighting key lessons that have the potential to help inform critical decisions affecting districts, schools, teachers, and students.

Here, we describe the latest findings from three grantees, which underscore the role of dosage and implementation fidelity in student impact.

The role of implementation and dosage in driving impact with Zearn

Zearn Math is a top-rated math platform that serves as a digital supplement to classroom instruction with independent lessons for students, instructional materials, student performance insights, and training and implementation support for educators.

A recently published randomized controlled trial (RCT) conducted by RAND estimated the impacts of Zearn Math in third through fifth grade over two years. Sixty-four schools in Texas serving high proportions of economically disadvantaged, Hispanic, English-learner, and below-proficient students were randomly assigned to either receive access to Zearn Math with training for leaders and teachers or to continue with business-as-usual instruction (and asked not to use Zearn). As a result, there were small positive, but not statistically significant, effects of Zearn Math on the Texas standardized math achievement test (STAAR). However, exploratory analyses revealed a statistically significant impact of 0.11 standard deviations—equivalent to about four percentile points—of Zearn Math on the MAP, an adaptive assessment of math achievement used by the district. Notably, there was a large difference in usage of Zearn between year one and year two of the study period, suggesting that more research is needed to understand if Zearn could have larger effects when used at the recommended level of three grade-level lessons per week.

A separate quasi-experimental study conducted by Johns Hopkins University examined the impact of Zearn Math Supplemental on students’ math achievement when paired with dedicated implementation support. Researchers used a quasi-experimental method called propensity score matching to estimate the impacts of Zearn Math Supplemental on fourth through eighth graders’ math achievement across three Louisiana school districts. This study explored the impacts of using the Zearn Math Supplemental relative to not using Zearn (Comparison Group 1); using Zearn Supplemental with and without implementation support (i.e., professional development, training, coaching; Comparison Group 2); and comparing Zearn Supplemental users with “intended” Zearn users (Comparison Group 3).

Impact analyses showed a statistically significant effect size of 0.20 SD on the LEAP mathematics assessment for Zearn Supplemental users compared to non-Zearn user students (Comparison Group 1). Students using Zearn Supplemental with implementation support had a statistically significant effect of 0.23 SD on the LEAP mathematics assessment compared to students in Zearn districts without support (Comparison Group 2). Exploratory, non-causal analyses revealed that students who completed more on-grade-level lessons showed higher math achievement at the end of the year, regardless of their treatment group. Taken together, these two studies point to the importance of implementation and dosage when using and evaluating programs like Zearn.

What it takes to boost student learning through high-dosage tutoring

The Personalized Learning Initiative (PLI) at The University of Chicago Education Lab aims to deliver high-dosage tutoring to students across the country using a series of RCTs to test the impacts of different approaches on students’ academic achievement.

Recent interim evidence released from SY 2023-24 highlights the potential power of tutoring for improving outcomes for students. Across four years, PLI researchers randomized over 27,000 students in grades K-12 across eight state, district, and charter education agencies nationwide to either receive math and reading tutoring through evidence-based high-dosage tutoring (HDT), more innovative and lower-cost tutoring through “sustainable” high-dosage tutoring models (SHDT), or business as usual.

Researchers found that both HDT and SHDT were effective overall, with pooled analyses showing the effect of participating in tutoring as statistically significant and ranging from 0.06-0.09 SD, or approximately one to two months of additional learning. Tutoring impacts were robust across models, with lower-cost models ($1,200 per student) proving as effective as higher-cost models ($2,000 per student), and virtual tutoring proving as effective as in-person tutoring.

These impacts, however, were much smaller than effects reported in earlier meta-analyses of high-dosage tutoring. Researchers theorized that students in the larger study did not receive enough tutoring to realize the expected impact seen in earlier work. To that end, the team also released site-specific results from a New Mexico district, which saw the highest levels of tutoring participation across all districts in the study—and it appears that these high dosage levels paid off. Students randomly assigned to tutoring learned approximately 38 percent more math, equivalent to about 3.5 extra months of learning, over the year than those who did not receive tutoring. Together, these two studies offer compelling evidence that high-dosage tutoring can meaningfully accelerate student learning, but dosage—an area our foundation continues to focus on—is critical to achieving sizable impacts on student learning.

How SEEDS professional development can strengthen early learning

SEEDS of Learning (SEEDS) is an evidence-based professional development (PD) program that prepares early childhood educators to help children develop the social-emotional, language, and emergent literacy skills they need to be kindergarten-ready.

A newly published RCT examined the impact of the SEEDS PD program on students’ oral language, literacy, and social-emotional skills at the end of Pre-K. Understanding that new programs require learning time to reach full implementation, the study was designed to test differences in impacts in study year one (SEEDS Implementation Year 1) and study year two (SEEDS Implementation Year 2), where SEEDS Implementation Year 2 reflected a more mature implementation of the program. Findings suggest that the impact of SEEDS strengthened over time, with the second year and more mature program conditions showing more improvements in children’s outcomes than one year of implementation.

While the new program had modest but statistically significant effects on letter names, rhyming, alliteration, and letter sounds, the second-year and mature program conditions demonstrated consistently strong gains in these skills, particularly for letter sounds and alliteration. Picture naming showed a significant improvement only in the mature program contrast, while executive functioning did not exhibit meaningful changes across any condition. These findings suggest that full implementation and extended experience with the SEEDS program may yield greater benefits.

Looking ahead and investing in research to drive impact

Our 2025 research strategy is focused on expanding field-building research, building grantee evidence, and disseminating our findings with transparency. The three studies highlighted above, along with the many others newly added to our Research Repository quarterly, illustrate the importance of investing in rigorous research. By funding studies like these, we’re not only helping nonprofits strengthen their models, but also identifying key areas for further investment and generating knowledge to inform decision-making. For more updates on Overdeck Family Foundation’s latest research news, subscribe to our newsletter.

Thank you to Jessica Siegel, a research fellow at Overdeck Family Foundation, for authoring this blog post. 

Header image courtesy of Zearn