Evaluations of Education Interventions and Programs
Improving Communication of Education Research
Over the last 20 years, education research has focused on conducting high-quality causal studies to help school decision makers implement the best interventions, curricula, and practices and communicating them through mechanisms such as the Institute of Educations Sciences’ What Works Clearinghouse. In a 2022 report, the National Academies of Sciences, Engineering, and Medicine (NASEM) recommended a new type of research, “Knowledge Mobilization,” to study the connections between education research and schools. IPR statistician Elizabeth Tipton participated in the committee that developed NASEM recommendations, and she and former IPR graduate research assistant Kaitlyn Fitzgerald, now at Azusa Pacific University, present a three-part framework for the new field in the Journal of Research on Educational Effectiveness. Using their own field of statistical research as an example, they ask, “How should statistical evidence be reported and conveyed to facilitate evidence-based decision making by education practitioners and policymakers?” The three facets of the framework are (1) normative, examining the norms researchers assume and embed in their findings; (2) descriptive, understanding how educators and policymakers reason about evidence; and (3) prescriptive, developing and evaluating communication to enable decision makers to better use evidence. Fitzgerald and Tipton emphasize the disconnect between the message researchers send and what decision makers receive, which is also influenced by their outlook and understanding, as well as the particular situation. The researchers argue for creating an integrated science that focuses on translating and disseminating scientific findings incorporating data visualization and human-computer interaction.
Improving Immigrant Parents' English with a Two-Generation Program
In 2012, the Community Action Project of Tulsa County (CAP Tulsa) created a family-focused English as a Second Language (ESL) program for parents and children in their Head Start program. Immigrant parents seeking to improve their English language skills were enrolled in intensive, free ESL courses with a child-centered curriculum, while their young children attended Head Start. In the first experimental evaluation of this two-generation program, IPR researchers Teresa Eckrich Sommer, Lauren Tighe, Terri Sabol, Lindsay Chase-Lansdale, and their colleagues assess the effects on parents’ English language, parent engagement, and psychological wellbeing, as well as the children’s language and cognitive skills after one year. The randomized control trial of 197 parent-child pairs from Spanish- (89%; mostly from Mexico) and Zomi-speaking (11%; from Myanmar) families, published in Applied Developmental Science, finds that parents in the program reported higher English reading skills and engagement of their child’s teacher compared to parents not in the program. Parents who began the program with lower English proficiency reported more benefits–more positive parenting skills and lower levels of psychological distress. Parents with greater English skills, however, reported fewer benefits–more parenting stress and psychological distress after one year. The researchers suggest that parents with advanced English skills may become more aware of being treated differently, which may lead to distress. No effects on children’s language and cognition were detected, but all children in CAP Tulsa’s Head Start program scored highly on cognitive assessments. The authors emphasize that early investment in immigrant families with young children may have benefits for both parents and children and a long-term study is needed. They recommend creating a national demonstration program to explore how two-generation ESL programs may function in other Head Start contexts.
Making Space for Black and Latinx Parents to Engage in Their Child’s Learning
Educational programs outside of traditional school hours can offer parents a different way to engage in their child’s learning. In International Journal of Child-Computer Interaction, learning sciences scholar and IPR associate Nichole Pinkard and her colleagues analyze the Digital Youth Divas (DYD) program, which provides youth educational programming, to understand how caregivers navigate out-of-school (OST) learning programs, relationships in these program, and what supports are necessary for them to be involved. DYD is a youth educational program the researchers run providing activities around STEAM (science, technology, engineering, art, and mathematics) for Black and Latina girls in fifth through eighth grades. During the 2021–22 program year, 16 girls met twice a week during the school year to work on activities such as cooking, digital design, rocket building, and block-based coding. The researchers interviewed ten families to understand their experiences in the program, how they manage their child’s activities and interests, and how they navigate OST learning programs. Parents discussed a desire to give their child an inclusive learning environment, how much autonomy to give their daughter in choosing the program, and their appreciation for the opportunities to engage with their daughter through DYD. The researchers argue that to engage in justice-orientated approaches to computer education, OTS must be designed to meet the needs and desires of families in the community, make space for multiple parenting styles, and give parents ownership and opportunities to participate. Pinkard is Alice Hamilton Professor of Learning Sciences.
Race, Academic Achievement, and Inequitable Motivational Payoff
Academic success is strongly associated with high motivation, but despite being similarly motivated, Black, Latinx, and Indigenous (BLI) students often receive lower grades than their White, Asian, and Asian American peers. In Nature Human Behaviour, a research team led by Northwestern PhD student David M. Silverman, with Northwestern PhD student and IPR graduate research assistant Josiah Rosario, and IPR social psychologist Mesmin Destin examine the motivational payoff of students enrolled in university and high school math courses across three studies. The first two studies analyzed patterns of motivation among 2,766 university math students. They were asked about a variety of aspects of motivation, including their expectations for success, their perceived barriers to success, and their beliefs about the nature, usefulness, and meaning of academic skills. In the third study, the researchers used similar data from 5,527 tenth graders and their teachers. The studies show that even with the same levels and patterns of academic motivation, BLI students received math grades 9% lower than their White, Asian, and Asian-American peers. The researchers also found that teachers' lower expectations of BLI students' potential to succeed in school were linked to BLI students receiving significantly lower math grades than similarly motivated non-BLI students. These studies shift attention away from reducing racial and ethnic disparities in academic achievement solely by targeting individual students' motivations, to a more nuanced understanding of the relationship between motivation and achievement, with an emphasis on the social forces that hinder educational equity.
Affordable 'High-Dosage' Tutoring in High Schools
Can in-school tutoring be effective and affordable on a large scale? In the American Economic Review, IPR economist Jonathan Guryan and his colleagues explore intensive, in-school tutoring as a cost-effective intervention for high-school students who are behind academically. The researchers conducted two randomized controlled trials in Chicago Public Schools in 2013–14 and 2014–15 of Saga Education’s tutoring program, involving 2,633 and 2,645 ninth- and tenth-grade students, respectively. The program, which involves each tutor working with two students at a time for a full class period every day in addition to their regular math class, is integrated into the school day and uses paraprofessionals instead of full-time teachers, making it less costly. In the first trial, participating in the tutoring program caused student learning to increase by the equivalent of more than an additional year of learning, and in the second trial, tutoring generated even larger test score gains. Tutoring also raised grades and graduation rates for all students, who were also less likely to fail math and other courses. The cost for these beneficial outcomes is comparable to many successful early childhood programs, suggesting that effective tutoring may be affordable and scalable in traditional high schools. The research was supported by the National Institutes of Health, MacArthur Foundation, and Arnold Ventures. Guryan is Lawyer Taylor Professor of Education and Social Policy.