INSTITUTE FOR POLICY RESEARCH

Building an ‘Evidence-Based Ecosystem’

With new STEPP Center, IPR’s top statistical minds aim to boost policymaking

STEPP Center co-founders Larry Hedges and Elizabeth Tipton.

Between 1890 and 1990, the average life span of a human nearly doubled, rising from 38 to more than 70 years.

How did this happen? It was not because doctors became smarter or better observers, or cared more about their patients, noted IPR’s Larry Hedges, one of the world’s pre-eminent educational researchers and statisticians. Rather, it was because doctors turned away from letting “magical thinking” and their personal experiences with a few patients rule their decisions—instead turning toward scientific methods that established the systematic collection and analysis of data on the many.

Hedges charges this same challenge faces education, and other social science domains, today.

“The function of social science is to help people understand which things they know are absolutely true,” Hedges said at the 2019 Yidan Prize conference. During the event, he announced the establishment of the Statistics for Evidence-Based Policy and Practice, or STEPP, Center at IPR.

In launching STEPP, he and co-director and fellow IPR statistician Elizabeth Tipton (PhD 2011) aim not just to address today’s pressing issues, but to develop the methodological tools to solve tomorrow’s problems across many fields. When Hedges received the 2018 Yidan Prize for Education Research with its attendant $3.9 million, he saw the opportunity to ensure that work will continue at IPR for decades to come. 

The STEPP Center believes in “a scientific approach to transform the social world and a practice-driven approach to advance science,” and says its goal is to “improve lives through methodological innovation in research.” Tipton says interdisciplinary cooperation will be key to STEPP’s overall contribution to the policy sphere.

“We want to hear from policymakers and practitioners about what their questions are, and then we want to think about whether social scientists have adequate methods to answer those questions,” Tipton said. “We are interested in improving connections between research and practice.”

To propel such discussions, the STEPP Center has launched a lecture series, featuring talks by up-and-coming and established methodologists. The series started in September with Blake McShane of Northwestern’s Kellogg School of Management, followed by and Wendy Tam Cho of the University of Illinois at Urbana-Champaign, who spoke about methods of testing replicability and statistical analysis of gerrymandering, respectively.

Hedges and Tipton have a well-established history of collaborating and contributing to the broader project of making sure all social science research is as methodologically sound as possible. Hedges is a founding father of meta-analysis, a method for aggregating the results of multiple studies of the same research question, and Tipton is an expert on issues of sample selection and generalizability in randomized trials. More recently, Tipton has developed “The Generalizer,” a web tool to help researchers more easily recruit for and conduct randomized trials in schools.

STEPP’s interdisciplinary nature will carry on IPR’s tradition of methodological innovation and rigor across its other research areas in that vein, including research on framing and replication by IPR political scientist James Druckman and meta-analytical research by IPR psychologist Alice Eagly.

IPR’s spirit of collaboration is reflected in STEPP’s evolution from the former Q-Center, which provided foundational work in statistics and methodology. Building on that foundation, the center’s goal is to bolster the institute’s methods for generating, synthesizing, and translating evidence.

The center also seeks to help foster an “evidence-based ecosystem,” Hedges said, marked by the rise of research clearinghouses such as What Works Clearinghouse for education or the Cochrane Collaboration for medicine. These entities also seek to democratize access to research and knowledge that is often locked away behind journal paywalls. Perhaps more importantly, they seek to translate such evidence for practitioners and policymakers, Hedges notes. Such translation efforts, however, require collaboration between academics and practitioners to break apart difficult concepts like “treatment impacts” and “heterogeneity of results” and render them into easily accessible language. STEPP will seek to set such conversations in motion.

Tipton compared STEPP’s work going forward to the core role that statistics plays in science, providing the framework through which researchers can provide sound policy analysis. When stakeholders across the board are vexed by gaps between research and practice, it might be because researchers simply do not yet have the tools available to ask their questions properly—and that is where the STEPP Center comes in. 

“We're kind of like these hidden engineers in the background creating methods for people to analyze data. Once they have these tools, they can do new types of analyses, which turns into telling us something new about the world,” Tipton said. “[Researchers] are like pilots, wanting to explore new areas, yet limited because their airplane can’t fly as far, or as fast, as they’d like, because nobody has developed the necessary engine part.”

“Our goal is to develop those necessary parts and to build ‘airplanes’ that can fly,” she said. 

Larry Hedges is Board of Trustees Professor of Statistics and Education and Social Policy.  Elizabeth Tipton is associate professor of statistics. Both are IPR fellows.

Photo of Larry Hedges by Rob Hart and photo of Elizabeth Tipton by Monika Wnuk. 

Published: October 29, 2019.