2019 Research Year in Review

It’s hard to believe that 2019 is drawing to a close, especially since it feels like we wrote our mid-year research update just a few weeks ago. As we shared in that post, since 2013, research has been a central component of our foundation’s grantmaking and comprises approximately 30% of our annual distribution. 

This year, Overdeck Family Foundation funded four new studies and renewed 18 grants totaling $6,140,722 in research funding. This is an increase of two new studies, 11 renewals, and $4,135,040 since our mid-year overview. We’ve also seen nine studies conclude since June. 

As we shared in July, we fund research not only to uncover innovative solutions that improve children’s lives, but also to evaluate our programmatic grants to ensure they’re having the best possible impact on the lives of children, families, and teachers.

Updates on 2019 Research Projects 

Since June, we’ve funded two new studies and 12 renewals, totaling $4,673,162. These grants range from $25,000 to $1,000,000. We also have interim updates/findings to share as well as the completion of a few studies. 

Below is an overview of the new research we’ve added to our grantmaking portfolio since June:

• Education Northwest: Research on how evidence-based afterschool STEM programs are structured to reach students both regionally and nationally and to support the afterschool field with improvements to training and implementation quality at scale.
• Harvard University: Opportunity Insights: American Opportunity Study (AOS): The AOS is an effort led by Raj Chetty’s team at Opportunity Insights that aims to increase the utility of historical census data by recovering and digitizing key fields from microfilmed decennial manuscripts from the 1960-1990 censuses. The team will eventually construct a full longitudinal dataset that spans 1940-2020. With 1940, 2000, 2010, and soon-to-be 2020 data already digitized, the bulk of this effort will focus on building the necessary 1960-90 bridge between the 1940 and 2000 censuses.
• SRI Mobile Messaging Research: In partnership with Packard Foundation, conduct a cluster RCT of four leading mobile messaging programs: Bright by Text, ParentsTogether, Tips by Text, and Too Small to Fail, to better understand the extent to which mobile messaging can increase parents’ responsive caregiving with young children.
• Teach for America: A planning grant to Teach for America will allow the organization to develop and refine an approach to measure alumni impact on systems change. The planning grant includes funding for a landscape analysis, a convening with external research partners to determine the appropriate evaluation approaches going forward, and a peer-reviewed white paper to be shared with the field. The responses from the field to the white paper will inform the development of the research plan for Teach for America’s eventual RCT on alumni impact.
• University of Oregon’s Center for Translational Neuroscience: A new grant to Dr. Phil Fisher’s team at the University of Oregon’s Center for Translational Neuroscience will support the expansion of the IMPACT (Integrated Measurement, Program Assessment, and Collaboration Tools) Measures Repository and related technical assistance. The goal of the grant is to improve organizations’ ability to quantify the impact, as well as the early childhood field’s use of precise and standard measures. We expect guidance from the Center to reach an estimated 200 organizations serving approximately two million families with young children.

Below is an overview of the renewal research we’ve added to our grantmaking portfolio since June:

• Crazy 8s Coach Efficacy Study Phase II: A renewal grant to WestEd will focus on analyzing the coach and program efficacy of Crazy 8s, Bedtime Math’s afterschool club. The goal is to identify and implement strategies for program improvement that would increase the consistency of student and coach experience, ultimately resulting in higher math interest for 130,000 K-5 students.
• Harvard University: Opportunity Insights: A renewal grant to support the continued study of economic and intergenerational mobility, with a focus on bridging connections between research and local policy decisions. This year’s research will dive deeper into the impacts of higher education institutions admissions on mobility, as well as examine how personal networks and social capital shape opportunities for upward mobility.
• JPAL: A renewal grant to JPAL to support the RCT design and planning for year two EdTech initiative winners ST Math and BookNook. The planning grant includes support on evaluation design, implementation, and stakeholder engagement.
• KnowledgeWorks Student-Centered Learning Research Collaborative: Renewal grant to complete two studies focused on effective student-centered learning models and approaches.
• The New Teacher Project: A renewal grant funding an RCT studying the effectiveness of promising program training elements.
• PERTS: A renewal grant to support the Engagement Project, which provides teachers with the ongoing support they need to motivate and engage their students. This year, our grant will focus on a few critical aspects of their work, including supporting PERTS to develop more compelling evidence for the engagement’s effect on downstream outcomes (e.g., attendance, GPA).
• Teachers College Columbia CTSC Cartwheel Study: A renewal grant that will focus on the following three questions: 
  • In what ways does ‘Fun Factor’ support teachers in teaching math content? 
  • What does the implementation of ‘Fun Factor’ activities look like in classrooms? 
  • What do teachers report that students learned through this structure?
• University of Chicago and University of Washington: A renewal grant to support  Susan Levine of the University of Chicago and Andy Meltzoff of the University of Washington I-LABS in a collaboration to validate a battery of tests that evaluate children’s math attitudes (math anxiety, implicit/explicit bias, stereotype threat), and study the relationship between children’s math attitudes and those of their parents.
• University of Washington I-LABS: A renewal grant to the Institute for Learning and Brain Sciences (I-LABS) to continue generating and measuring evidence-based parenting practices that improve the quality and quantity of parent-child interactions. The goal is to disseminate practices to at least 600,000 families, accelerating child language development and building larger vocabularies starting in infanthood. 

Three of our grantees also have updates and interim findings:

• Harvard University Opportunity Insights: Initial results are available for the  Creating Moves to Opportunity (CMTO) pilot study in Seattle and King County, WA. Based on a sample of 273 families, the CMTO program has so far increased the share of families who lease units in high-opportunity neighborhoods by 40%. This result demonstrated that low-income families did not concentrate in lower-opportunity areas only out of preference for those communities, but also due to barriers that prevent them from moving to higher-opportunity areas. The research was released in early August with coverage in the New York Times, NPR, and Vox, among others.
• NewSchools Venture Fund: In partnership with Transforming Education, NewSchools Venture Fund released the latest findings from their ongoing research on expanded definitions of student success. This report is informed by work inside 40 district and charter schools in 16 states that served nearly 12,000 students during the 2017-18 school year. One key finding: Growth mindset and perceptions of school safety are once again the indicators most strongly associated with academic performance in the social-emotional and culture/climate categories.
• University of Washington I-LABS: In an ongoing longitudinal study, interim analysis shows that short-term, infrequent coaching (two 45-minute sessions when children were six and ten months old) improved parent language behaviors with their infants, which improved child language, even when measured months after the parental intervention ended. Children’s language outcomes at every age were significantly higher in the intervention group than the control group, and the difference appears to increase over time. Previous I-LABS research has demonstrated a positive predictive relationship between similar measures of parent linguistic environment and infant language development at 11 & 14 months predicting child vocabulary at 24 months. Early vocabulary is strongly predictive of later academic skills.

And nine of our grantees have concluded their research. Here’s what we’ve learned from those studies:

1. Bellwether Education Partners completed a report on how to address learning gaps while helping students attain grade-level knowledge. The report, titled “Insights from Ongoing Work to Accelerate Outcomes for Students with Learning Gaps,” was funded by the Instructional Materials Funders Group, an informal community of funders that includes Carnegie Corporation of New York, Chan Zuckerberg Initiative, Bill and Melinda Gates Foundation, William and Flora Hewlett Foundation, W. K. Kellogg Foundation, Overdeck Family Foundation, Robin Hood Learning and Technology Fund, and the Charles and Lynn Schusterman Family Foundation. The goal was to create a more-in-depth, shared understanding of ways in which personalization can be paired with a commitment to standards-aligned materials to accelerate learning.

Based on Bellwether’s analysis of the existing evidence and learning science, interviews with over 50 stakeholders, and an examination of 14 model schools, they hypothesized that there exists a way to help students who are behind to get back on track and gain grade-level knowledge. This evidence-based approach, called rigorous differentiation, would provide students instruction grounded in high-quality materials alongside differentiated support. In practice, students experiencing rigorous differentiation would: 

• Have equal access to grade-level work; 
• See the coherence across different materials and learning experiences;
• Be in an environment that fosters engagement and agency; and
• Have a caring relationship with their teacher, with frequent 1:1 and small group learning opportunities. 

We believe this report is a significant first step in figuring out how to combine the benefits of high-quality instructional materials and personalized learning. As such, we are eager to continue an authentic dialogue with communities, curriculum providers, educators, funders, and researchers around a ‘both-and’ versus an ‘either-or’ vision. 

2. Christensen Institute released a report detailing innovative practices happening at schools across the country. Titled, “A View from the Canopy: Building Collective Knowledge on Schooll Innovation,” the report describes the consequences of knowledge silos on school innovation and suggests an approach for aggregating innovative methods and designs. Some hypotheses raised by the report include:

• Rural schools may be facing barriers to innovation—or are innovating in ways that don’t reflect national trends—and may benefit from targeted support and investment
• Students in schools serving predominantly Black populations may not be getting the same opportunities for learner agency and social-emotional skill development
• Experiential learning and competency-based models are facing barriers to scale in schools serving low-income students and students of color
• Efforts to redefine student success could be playing out differently depending on whether school models are designed to help marginalized students
• Schools identified “learner agency” and “SEL” most frequently as components of their models, but implementation lags their commitment to these approaches—or isn’t being codified and captured coherently

The Canopy work is a crucial step toward addressing information siloes by rethinking both where to source information about school innovation, as well as how to structure data to improve collective knowledge. The next steps include enhancing participation by nominating more organizations to partake in the study, further developing the tagging system, and digging deeper into the hypotheses through additional research both within and beyond Canopy data.

3. An RCT of 600 classrooms in grades three-to-five compared the use of Engineering is Elementary to another elementary engineering curriculum intervention. Students generally self-reported, becoming more positive in their interests and attitudes after participation. There was no treatment difference on scales where students rate the value of engineering to themselves and society. Still, students in the treatment group showed less gender bias after the EiE curriculum intervention than the control, and less interest in becoming an engineer. Females were more positive than males on all scales. Time spent on the engineering unit, the percent completion of student journals, teacher experience in general and with teaching engineering, and the class average quality of student responses in journals all predicted more positive outcomes. 

4. A seven-year longitudinal quasi-experimental study on FIRST participants showed positive impacts on STEM-related interests and attitudes five years after they entered the program. The study found increased interest in STEM, involvement in STEM-related activities, STEM identity, STEM knowledge, and interest in STEM careers. 

• Participants from all three FIRST programs in the study (FIRST LEGO League, FIRST Tech Challenge, and FIRST Robotics Competition) and all major population groups and community types show positive impacts. FIRST team members are two-to-three times more likely to show gains on STEM-related measures than comparison students.
• Impacts on STEM attitudes and interests continue to be significantly higher for girls in FIRST than those for boys. 
• FIRST’s impacts persist in college. Among students in their first and second year of college, FIRST alumni continue to show significantly higher gains on STEM-related attitudes than comparison students, report significantly higher interest in majoring in computer science, engineering, and robotics, are more likely to declare a major in computer science, engineering, or a STEM-related field, are 2.3 times more likely to take an engineering course in their freshman year, and girls are 3.4 times more likely to take engineering courses.

5. A study conducted by KnowledgeWorks and RAND on competency-based learning in the RSU2 district in Maine found no positive effects on student outcomes. All student outcomes were negative, and most were statistically significant. Other findings that emerged were:

• As opposed to consistently adverse effects in the fall-to-spring analyses, fall-to-fall results showed a mix of positive and negative effects, most of which were not statistically significant
• The district experienced relatively consistent growth in fall scores over time, particularly in mathematics
• There was some evidence suggesting that high schools may have produced more positive effects than elementary and middle schools
• RSU2 graduation rates were comparable to statewide rates, with both groups experiencing upward trends, and RSU2 outperforming the state average most years 
• SAT data yielded positive findings, showing the subset of RSU2 students with reported scores to have slightly surpassed state averages in the two years where data is complete

We know from this study and others that competency-based learning builds student agency and ownership, as well as social-emotional skills that are critical to student success. On the other hand, a negative versus null effect points to some of the implementation challenges that come with this approach, including ensuring kids are getting enough “at-bats” at mastering grade-level standards. 

6. Through JPAL’s EdTech initiative, which launched in 2017, eleven education technology research projects are now complete. In 2019, a randomized experiment evaluated the impact of “My Student’s Team,” an intervention that proactively invites parent-nominated adults who have pre-existing relationships with students (e.g., family members, neighbors, coaches) to support students’ education. The preliminary results show no significant average effect on traditional academic outcomes. However, exploratory analyses suggest that My Student’s Team may have a more substantial impact on reducing absences for students with below-median pre-intervention GPA and students with above-median pre-intervention absences. 

7. In partnership with Glass Frog Analytics, we embarked on learning more about the impact of residency and differential staffing programs, in particular, the following research questions.  

• Residency programs RQ1: What is the added value of hosting residency personnel in a classroom, as measured by improved teacher effectiveness scores (TESes) during the year of the residency?      
• Differential staffing programs RQ2: Are teachers who worked with DS programs before becoming teachers more effective (as measured by student academic outcomes) than teachers who took more traditional routes to become teachers? 

Glass Frog completed research on the ancillary benefits for these staffing program partners; all NCTR-affiliated programs were anonymized for the study. The report focuses on programs’ impacts on outcomes that are secondary to their primary outcome goals, which, in the case of residency programs, includes training future teachers and, in the case of DS programs, includes improving student outcomes. 

• Residency programs: In two programs that follow a very similar residency model—Res Ed and City Teach—host teachers with a resident in the classroom have higher TESes compared to similar teachers who do not host residents. In Teacher Prep, which follows a different residency model, we see no significant difference for host teachers compared to comparison teachers.  
• Differential Staffing programs: Using students’ state test scores as a proxy of teacher effectiveness, we find no evidence that DS alumni teachers are more effective than a matched sample of comparison teachers. The results do not change when we analyze the pooled data for all teachers and subjects together. However, differential staffing did have an impact on the retention of these teachers. Based on alumni who responded to a survey, approximately 30 percent of DS alumni become teachers. Across the three cohorts of teachers included in the analysis, DS teachers have higher retention rates compared to the matched comparison group of teachers, with the difference being statistically significant. In the pooled sample, the retention rate of DS alumni teachers is higher than that of the comparison teachers by almost six percentage points, and the difference is highly statistically significant. 

8. DCPS LEAP’s (LEarning together to Advance our Practice) implementation study has brought a more nuanced understanding of the many ways the LEAP experience can differ across teachers. In the study, the University of Virginia team examines the evolving design features of IMPACT, associated descriptive changes in the teacher workforce, and corresponding causal effects of incentives on teacher attrition and performance under this sophisticated and redesigned system. The study is now complete with papers forthcoming in Education Finance and Policy. 

9. Alder Graduate School of Education conducted a literature review to synthesize past research on best practices in the student- and teacher candidate-centered co-teaching. In the fall of 2018, as part of its OFF-funded research, it observed mentor-resident teaching pairs. It interviewed them about their use of co-teaching practices and the perceived level efficacy of these practices for resident and student learning. It also surveyed a broader selection of mentors to map the landscape of co-teaching practices in the organization and to hear their perspectives about which co-teaching methods might be most beneficial for student and resident learning. Alder GSE also conducted interviews with residency directors to gain their insight on the student- and resident-centered co-teaching. As a result of these analyses, Alder GSE concluded that the cycles of co-planning, co-instruction, and co-debriefing should occur in the day-to-day experience of a co-teaching relationship.

Alder then designed and tested a pilot intervention in the spring of 2019 based on the theories that were surfaced in the fall. In this strand of work, Alder GSE studied resident-led, small group instruction in elementary mathematics, a co-teaching model that lowers the teacher/student ratio. A multilevel model of case study residents’ K12 student data demonstrated the potential for this model for student learning, as measured by formative assessment scores. Observations of the residents’ instruction also showed promising content-area teaching practices (for example, supporting students’ procedural fluency) as well as new focus areas for resident training (for example, encouraging students’ self-monitoring).

We are excited for what’s to come next year as we have many new studies underway as we continue to expand our research grantmaking in 2020.

Want to Apply for Research Funding? 

If you would like to submit a research idea that fits within our continuum, click here and select Research Proposal in the dropdown menu. Please use 200 words or less to describe your idea. We are always open to new ideas and welcome your input, but please note that it is rare for unsolicited research ideas to result in funding. If we are interested in learning more about your idea, you will hear from us within 30 days.