Grantee: National Student Support Accelerator
Study Type: RCT
Principal Investigator: Susanna Loeb – Stanford University, National Student Support Accelerator
Project Description: This study uses an RCT to estimate the impacts of Tutor CoPilot—a novel human-AI approach that leverages a model of expert thinking to provide guidance to tutors as they tutor—on tutoring quality and students’ mastery of content knowledge. The study involved 900 tutors and 1,800 K-12 students from historically under-served communities. All students in this study received tutoring but the research team randomly assigned about half of the tutors to receive access to Tutor CoPilot in their tutoring sessions and the half to tutor as normal with no additional support.
Key Findings:The study found that students working on mathematics with tutors randomly assigned to have access to Tutor CoPilot were four percentage points more likely to master topics. Notably, students of lower-rated tutors experienced the greatest benefit, improving mastery by nine p.p. relative to the control group. Tutor CoPilot costs only $20 per tutor annually, based on the tutors’ usage during the study. Exploratory analysis of 550,000+ messages using classifiers to identify pedagogical strategies revealed that tutors with access to Tutor CoPilot were more likely to use strategies that fostered student understanding (e.g., asking guiding questions) and less likely to give away the answer to the student, aligning with high-quality teaching practices. Tutor interviews qualitatively highlight how Tutor CoPilot’s guidance helps them to respond to student needs, though tutors flagged common issues in Tutor CoPilot, such as generating suggestions that were not grade-level appropriate.
Study Citation: Wang, R. E., Ribeiro, A. T., Robinson, C. D., Loeb, S., & Demszky, D. (2024). Tutor CoPilot: A human-AI approach for scaling real-time expertise. arXiv preprint arXiv:2410.03017.
Full report here.
The Key Findings above were reproduced from the published report and do not necessarily reflect interpretation of Overdeck Family Foundation staff.
Social-Emotional Learning
Impact of PowerMyLearning's Social Emotional Learning Program: 2021-2022 Mid-Year and School Year Analysis
This is a pre-post study of PowerMyLearning’s Social Emotional Learning (SEL) program—Nurture Student Growth Through Social Emotional Learning—which serves teachers, students, and families with children in first through sixth grade with the goals of addressing challenging behaviors and supporting family wellbeing.
Read more