Back to K-12 High-Quality Instructional Materials

Evaluation of Zearn Supplemental with Dedicated Implementation Support

Principal Investigator

Nathan Storey – Johns Hopkins University School of Education

Project Description

This study used propensity score matching to understand the impacts of using the Zearn Math Supplemental relative to not using Zearn (Comparison Group 1); using Zearn Supplemental with and without implementation support (i.e., professional development, training, coaching; Comparison Group 2); and comparing Zearn Supplemental users with “intended” Zearn users (Comparison Group 3). Multi-level propensity score matching was conducted in two rounds, with schools first matched within districts and then students within schools matched across districts.

Research Questions

  • What is the impact of using Zearn Supplemental with dedicated implementation support on student mathematics achievement, as measured by the LEAP assessment and relative to students who do not receive any form of Zearn tutoring?
  • What is the impact of using Zearn Supplemental with dedicated implementation support on student mathematics achievement, as measured by the LEAP assessment and relative to students who receive standard Zearn tutoring (not the special models used in the three target districts)?
  • How do the impacts of Zearn Supplemental with dedicated implementation support vary by student characteristics (e.g., grade, race, ethnicity)?
  • What is the effect of Zearn Supplemental with dedicated implementation support on student outcomes for students who received the full treatment (defined as 90 lessons)?
  • To what extent is greater fidelity of implementation (average weekly Zearn usage) associated with better student outcomes?

Key Findings

Quantitative impact analyses showed a statistically significant effect size of 0.20 SD on the LEAP mathematics assessment for Zearn Supplemental users compared to non-Zearn user students (Comparison Group 1). Students using Zearn Supplemental with implementation support had a statistically significant effect of 0.23 SD on the LEAP mathematics assessment compared to students in Zearn districts without support (Comparison Group 2). There were no differential effects by student characteristics in analyses related to Comparison Group 1 or 2. A positive but not statistically significant impact (0.09 SD) was seen in Comparison Group 3 between Zearn Supplemental users and Zearn-identified intended users, with larger effects for Black, Latino, and White students than students from other racial/ethnic groups. Exploratory analyses related to usage revealed that students who completed more on-grade-level lessons showed higher math achievement at endline, regardless of their treatment group.

Study Citation

Storey, N. S., & Neitzel, A. J. (2025). Evaluation of Zearn Supplemental with dedicated implementation support. Center for Research and Reform in Education, The Johns Hopkins University.

The Key Findings above were reproduced from the published report and do not necessarily reflect interpretation of Overdeck Family Foundation staff.

Publication
Year

2025

Status

Completed

Study Type

Quasi-Experimental

Grantee

Zearn

Full Report
Back to K-12 High-Quality Instructional Materials

More from K-12 High-Quality Instructional Materials

Completed

Efficacy of Zearn Math Over Two Years in Grades 3 to 5: An Experiment in Texas

This randomized controlled trial (RCT) examined the impacts of Zearn, a software platform for mathematics learning, on math assessment scores in a sample of 64 schools in an urban Texas district over two years.

Completed

Efficacy Analysis of Zearn Math: Findings from Implementation in Louisiana

This is a matching study with retrospective data that aimed to estimate impacts of using Zearn Math—a computer-based math platform that provides individualized supports to students—at the recommended dosage on students’ math performance.

Completed

Effects of a Quill.org Intervention on Paragraph Revision

This is a randomized controlled trial (RCT) of Quill, a free, open-source digital platform that provides research-based writing instruction to students.

Completed

State of the Instructional Materials Market: Use of Aligned Materials in 2022

This is a descriptive study that sought to understand the degree to which school districts used high-quality instructional materials in 2022, as well as the factors associated with adoption of these materials.

Completed

Catching Up and Moving Forward: Accelerating Math Learning for Every Student

This is a retrospective correlational study of more than five million instances of student math performance based on data from the 600,000 students in first through seventh grade who (i) used the Zearn math learning digital platform during the 2020-21 and 2021-22 school years and (ii) had multiple instances of repeated “struggle” with math content during the two-year study period, as defined by instances in which a student repeatedly answered a math question incorrectly following in-the-moment online support.

Completed

Efficacy Analysis of Zearn Math in DC Public Schools

This is a set of retrospective correlational and quasi-experimental studies of student math performance based on data from students in grades one through six who used the Zearn Math learning digital platform during the 2018-19, 2020-21, and/or 2021-22 school years.

In Progress

Evaluating the Impact and Implementation of the Fishtank English Language Arts Curriculum

This study evaluates the impact of the Fishtank ELA curriculum when implemented at-scale and examines how implementation quality relates to student outcomes.

In Progress

Evaluating the Impact of Teaching Lab’s Professional Learning Program on Teacher Practice and Student Achievement

This study evaluates the impact of Teaching Lab’s teacher professional learning program on teacher practice and students’ reading achievement.

In Progress

Evaluating the Implementation and Impact of High-Quality Instructional Materials Across States

This multi-state study examines how the quality of implementation of high-quality instructional materials (HQIM) is associated with student achievement and what drives strong implementation at scale.

Completed

Evaluation of NBA Math Hoops: Impacts on Middle School Students’ Math Outcomes and Engagement

This randomized controlled trial (RCT) evaluates the impact of NBA Math Hoops, a basketball-themed board game, on middle school students’ math achievement, engagement, and attitudes toward math.

Completed

Connection, Trust, and Learning: Student Attendance in the Middle and High School Grades Following the COVID-19 Pandemic

This study examines how student absenteeism changed in Chicago Public Schools following the COVID-19 pandemic and explores how attendance relates to academic outcomes and school climate.

Completed

Evaluating Opportunity Culture: Improving Student Achievement Through Strategic Staffing in West Texas

This four-year mixed-methods evaluation examines the effects of Opportunity Culture in Ector County ISD on student math and reading achievement.

Back to K-12 High-Quality Instructional Materials
Young boy sits in desk at school

Courtesy of TalkingPoints

Follow us

Sign up for our newsletter.

Subscribe to Overdeck Family Foundation’s newsletter to stay up-to-date on our latest news and resources.