Evaluation and Accountability in Education
Improving Communication of Education Research
Over the last 20 years, education research has focused on conducting high-quality causal studies to help school decision makers implement the best interventions, curricula, and practices and communicating them through mechanisms such as the Institute of Educations Sciences’ What Works Clearinghouse. In a 2022 report, the National Academies of Sciences, Engineering, and Medicine (NASEM) recommended a new type of research, “Knowledge Mobilization,” to study the connections between education research and schools. IPR statistician Elizabeth Tipton participated in the committee that developed NASEM recommendations, and she and former IPR graduate research assistant Kaitlyn Fitzgerald, now at Azusa Pacific University, present a three-part framework for the new field in the Journal of Research on Educational Effectiveness. Using their own field of statistical research as an example, they ask, “How should statistical evidence be reported and conveyed to facilitate evidence-based decision making by education practitioners and policymakers?” The three facets of the framework are (1) normative, examining the norms researchers assume and embed in their findings; (2) descriptive, understanding how educators and policymakers reason about evidence; and (3) prescriptive, developing and evaluating communication to enable decision makers to better use evidence. Fitzgerald and Tipton emphasize the disconnect between the message researchers send and what decision makers receive, which is also influenced by their outlook and understanding, as well as the particular situation. The researchers argue for creating an integrated science that focuses on translating and disseminating scientific findings incorporating data visualization and human-computer interaction.
Improving Immigrant Parents' English with a Two-Generation Program
In 2012, the Community Action Project of Tulsa County (CAP Tulsa) created a family-focused English as a Second Language (ESL) program for parents and children in their Head Start program. Immigrant parents seeking to improve their English language skills were enrolled in intensive, free ESL courses with a child-centered curriculum, while their young children attended Head Start. In the first experimental evaluation of this two-generation program, IPR researchers Teresa Eckrich Sommer, Lauren Tighe, Terri Sabol, Lindsay Chase-Lansdale, and their colleagues assess the effects on parents’ English language, parent engagement, and psychological wellbeing, as well as the children’s language and cognitive skills after one year. The randomized control trial of 197 parent-child pairs from Spanish- (89%; mostly from Mexico) and Zomi-speaking (11%; from Myanmar) families, published in Applied Developmental Science, finds that parents in the program reported higher English reading skills and engagement of their child’s teacher compared to parents not in the program. Parents who began the program with lower English proficiency reported more benefits–more positive parenting skills and lower levels of psychological distress. Parents with greater English skills, however, reported fewer benefits–more parenting stress and psychological distress after one year. The researchers suggest that parents with advanced English skills may become more aware of being treated differently, which may lead to distress. No effects on children’s language and cognition were detected, but all children in CAP Tulsa’s Head Start program scored highly on cognitive assessments. The authors emphasize that early investment in immigrant families with young children may have benefits for both parents and children and a long-term study is needed. They recommend creating a national demonstration program to explore how two-generation ESL programs may function in other Head Start contexts.
Making Space for Black and Latinx Parents to Engage in Their Child’s Learning
Educational programs outside of traditional school hours can offer parents a different way to engage in their child’s learning. In International Journal of Child-Computer Interaction, learning sciences scholar and IPR associate Nichole Pinkard and her colleagues analyze the Digital Youth Divas (DYD) program, which provides youth educational programming, to understand how caregivers navigate out-of-school (OST) learning programs, relationships in these program, and what supports are necessary for them to be involved. DYD is a youth educational program the researchers run providing activities around STEAM (science, technology, engineering, art, and mathematics) for Black and Latina girls in fifth through eighth grades. During the 2021–22 program year, 16 girls met twice a week during the school year to work on activities such as cooking, digital design, rocket building, and block-based coding. The researchers interviewed ten families to understand their experiences in the program, how they manage their child’s activities and interests, and how they navigate OST learning programs. Parents discussed a desire to give their child an inclusive learning environment, how much autonomy to give their daughter in choosing the program, and their appreciation for the opportunities to engage with their daughter through DYD. The researchers argue that to engage in justice-orientated approaches to computer education, OTS must be designed to meet the needs and desires of families in the community, make space for multiple parenting styles, and give parents ownership and opportunities to participate. Pinkard is Alice Hamilton Professor of Learning Sciences.
Race, Academic Achievement, and Inequitable Motivational Payoff
Academic success is strongly associated with high motivation, but despite being similarly motivated, Black, Latinx, and Indigenous (BLI) students often receive lower grades than their White, Asian, and Asian American peers. In Nature Human Behaviour, a research team led by Northwestern PhD student David M. Silverman, with Northwestern PhD student and IPR graduate research assistant Josiah Rosario, and IPR social psychologist Mesmin Destin examine the motivational payoff of students enrolled in university and high school math courses across three studies. The first two studies analyzed patterns of motivation among 2,766 university math students. They were asked about a variety of aspects of motivation, including their expectations for success, their perceived barriers to success, and their beliefs about the nature, usefulness, and meaning of academic skills. In the third study, the researchers used similar data from 5,527 tenth graders and their teachers. The studies show that even with the same levels and patterns of academic motivation, BLI students received math grades 9% lower than their White, Asian, and Asian-American peers. The researchers also found that teachers' lower expectations of BLI students' potential to succeed in school were linked to BLI students receiving significantly lower math grades than similarly motivated non-BLI students. These studies shift attention away from reducing racial and ethnic disparities in academic achievement solely by targeting individual students' motivations, to a more nuanced understanding of the relationship between motivation and achievement, with an emphasis on the social forces that hinder educational equity.
Affordable 'High-Dosage' Tutoring in High Schools
Can in-school tutoring be effective and affordable on a large scale? In the American Economic Review, IPR economist Jonathan Guryan and his colleagues explore intensive, in-school tutoring as a cost-effective intervention for high-school students who are behind academically. The researchers conducted two randomized controlled trials in Chicago Public Schools in 2013–14 and 2014–15 of Saga Education’s tutoring program, involving 2,633 and 2,645 ninth- and tenth-grade students, respectively. The program, which involves each tutor working with two students at a time for a full class period every day in addition to their regular math class, is integrated into the school day and uses paraprofessionals instead of full-time teachers, making it less costly. In the first trial, participating in the tutoring program caused student learning to increase by the equivalent of more than an additional year of learning, and in the second trial, tutoring generated even larger test score gains. Tutoring also raised grades and graduation rates for all students, who were also less likely to fail math and other courses. The cost for these beneficial outcomes is comparable to many successful early childhood programs, suggesting that effective tutoring may be affordable and scalable in traditional high schools. The research was supported by the National Institutes of Health, MacArthur Foundation, and Arnold Ventures. Guryan is Lawyer Taylor Professor of Education and Social Policy.
Strong Versus Weak Commitments to Save for Education in Uganda
Commitment devices that limit people’s behavior in the present, such as savings accounts with restrictions, may help them achieve long-term goals. In a working paper, economist and IPR associate Dean Karlan and Leigh Linden of the University of Texas at Austin test whether a strong versus a weaker commitment device helps Ugandan children and their families save, spend more on educational expenses, and achieve higher test scores. In coordination with a local non-profit, Private Education Development Network (PEDN), the researchers randomly assigned students in 136 primary schools to one of three groups to save for educational expenses: a strong commitment savings account where funds were available at the end of the term and could only be use on educational items with a voucher, a weak commitment savings account where funds could be withdrawn at the end of the term, but were available in cash to spend on anything, and a control group. At the end of each trimester, students could use their vouchers or cash to purchase school supplies at a fair. Halfway through the program, half of the parents in the treatment groups attended a workshop explaining the value of saving the money early enough to impact their child’s behavior. The other half attended the workshop too late to influence their behavior. The researchers find that when their parents attended the workshop, children save more under a weaker commitment savings account than a stricter commitment, and they spend the money on school supplies. Children in this group also had better academic scores, suggesting that financial constraints play an important role in academic outcomes.
Relations with Parents in the Era of Test-Based Accountability
The era of test-based accountability has placed pressure on urban schools. In a study published in Urban Education, IPR sociologist Simone Ispa-Landa and Jordan Conwell of the University of Wisconsin-Madison analyze Chicago Public School principals’ relationships with parents. They conducted an analysis of 166 interviews done by education professor and IPR associate James Spillane and IPR faculty adjunct Michelle Reininger between 2009 and 2012 from a longitudinal study of 26 Chicago Public School principals. All of the principals were new to their position. Nine of the principals identified as White, eight as Black, eight as Latinx, and one as multiracial. The questions focused on principals’ relationships with school stakeholders, including parents, whom they frequently discussed in relation to students’ test scores. The researchers find that boosting test scores was central to the principals’ conception of urban school leadership and that the pressure to raise scores were visible in CPS principals’ self-reported views of and relations with parents. Some principals pursued strategies of banning parents from involvement in academic matters, seeking to disconnect home and school by isolating students from their homes, while others thought schools would have better test scores if parents were more involved in their children’s education. The researchers write that thinking of “principals in urban schools as street-level bureaucrats and parents as clients of the urban school bureaucracy offers a way to understand why test-based accountability pressures were so visible in principals’ views of and relations with their schools’ parents.” Spillane is Spencer T. and Ann W. Olin Professor in Learning and Organizational Change.