Principal Investigator
Philip Oreopoulos – University of Toronto
Project Description
This paper includes two studies: 1) a descriptive pilot study conducted in Nashville examining the feasibility of Khoaching with Khan, a teacher-coaching approach to supplement the implementation of Khan Academy (a computer-assisted learning platform); and 2) an RCT in Arlington testing the impact of Khoaching with Khan—delivered to math teachers over an academic year—on elementary and middle school students’ math achievement. The study conducted quasi-experimental follow-up analyses to explore whether impacts varied depending on average classroom practice time. Researchers used third through eighth grade students’ math standardized test scores to measure achievement outcomes.
Research Questions
- What is the effectiveness of a program designed to encourage greater mastery learning through technology and proactive, continuous teacher support?
Key Findings
The Nashville study findings found that students spent about 19 minutes practicing on Khan Academy during the first time of a data review and average practice time on KA was 19 minutes on the first day of the review and about 38 minutes total across a week of data reviewed. These pilot findings highlighted significant variation in dosage and usage across participating classrooms. Arlington RCT findings revealed that, across the full sample, there were null effects of Khoaching with Khan on student math achievement. Subgroup analyses did find positive, statistically significant impacts of about .12 SDs on students in third through sixth grade. Impacts were null and negative in direction among seventh and eighth-graders. Follow-up work did suggest that impacts were more likely to be positive for students who practiced at least 50 minutes per week. The authors hypothesize that seventh and eighth graders did not benefit from Khan Academy because the large majority of them did not reach that practice threshold.
Study Citation
Oreopoulos, P., Gibbs, C., Jensen, M., & Price, J. (2024). Teaching teachers to use computer assisted learning effectively: Experimental and quasi-experimental evidence (No. w32388). National Bureau of Economic Research. Retrieved from NBER: https://www.nber.org/system/files/working_papers/w32388/w32388.pdf
The Key Findings above were reproduced from the published report and do not necessarily reflect interpretation of Overdeck Family Foundation staff.
