INSTITUTE FOR POLICY RESEARCH

Which Networks Are Most Effective at Improving Student Achievement?

Study shows 'collective impact' framework no more effective than others; simplifies ‘formula’ for achieving social impact

 

Children playing in Centennial Park Playground in Howard County, MD.

How can organizations in different sectors work together to improve educational outcomes? A Northwestern study shows city and nonprofit leaders can achieve social impact by using a variety of frameworks that are just as, if not more, effective than the commonly used "collective impact" framework.

The Network for Nonprofit and Social Impact (NNSI), led by communication studies scholar and IPR associate Michelle Shumate, conducted the three-year study that evaluated the effectiveness of 26 education networks’ “collective impact” in improving education outcomes in their communities.

"This research simplifies the formula for achieving social impact in ways that should be a relief to municipal and nonprofit leaders,” Shumate said.

According to NNSI’s previous research, more than one-third of U.S. communities use a collective impact framework to coordinate collaboration between agencies and organizations. The framework focuses on five key conditions: a common agenda, a shared measurement system, mutually reinforcing activities, continuous communication, and backbone support organizations.

The study points to several key takeaways that could change the way organizations work together to improve student achievement. Most notably, the researchers find that greater adherence to the five initial conditions of collective impact does not result in a more significant social impact.

NNSI researchers used a novel, mixed-methods approach that combined educational outcome data and qualitative interviews.

“If you are a small community with some resources, you can bring together organizations to offer a new set of missing programs to improve educational outcomes and get there,” Shumate said. “However, if your community has higher rates of poverty or is larger, you need to invest in systems for learning and system alignment.”

She continued, “In other words, your job is to help organizations improve their existing programs and services and for organizations to offer them in a coordinated fashion.”

Three research reports detail the study’s results, with a fourth on navigating network change forthcoming:

• Report 1: Networks That Create a Social Impact
• Report 2: Equity and Empowerment in Education Networks
• Report 3: Effective Data Practices Support Learning and Systems Alignment

Key findings include:

• Achieving social impact is rare and time-consuming. Out of the 26 networks, the NNSI researchers find conclusive evidence that only eight networks’ activities led to social impact—as measured by higher fourth- and eighth-grade reading scores and high school graduation rates—and all eight were at least three years old.
• Greater adherence to initial collective impact tenets does not result in a more significant social impact, as measured in this research. Only one of the eight networks with evidence of social impact followed the collective impact model. Networks adopted a variety of different models; this suggests more than one way to achieve social impact when using improved student achievement as a measure.

• NNSI pinpoints two network designs that are almost as likely as a collective impact framework to result in social impact.
  • To make a social impact, some networks combined learning-based mechanisms that focus on improving service quality and systems-alignment, where members coordinate services to explore service gaps. These networks use robust data models to identify both gaps in services and improve existing programs’ quality.
  • In communities above the national poverty line, networks using distributed governance models and a project model of change significantly improved students’ outcomes in their community. Network members shared power and responsibility in distributed governance. A project logic of change creates social impact by delivering new programs from the network's joint activity.

Find out more about the research here.

Michelle Shumate is Delaney Family University Research Professor, director of the Network for Nonprofit and Social Impact (NNSI), and an IPR faculty associate.

Photo credit: Howard County Parks and Recreation.

Published: April 28, 2021.