What principles can help guide public leaders—whether policymakers or public managers—in their use of evidence-based policy to improve results? Howard White (@HowardNWhite) of the Campbell Collaboration joins us to share four fundamental principles:
- Use the right evidence to answer the right question. Different types of evidence — e.g., monitoring, process evaluation, impact evaluation and systematic reviews — all can produce useful information for decision makers. But each type of evidence should not be used to answer questions that are beyond its usefulness.
- Don’t rely on single studies. When possible, leaders should avoid making important funding decisions based on single studies, especially those done in one site. That’s because the findings from one study are often different from those of further studies. The best approach is to use systematic reviews (where they exist), meaning syntheses of multiple high-quality studies.
- Context matters for transferring evidence. Why do findings from one study often not replicate in another? A key reason is that context matters. For example, when a home visiting program found to be effective in the U.S. was tested in Britain, it produced no impact. Why? Likely it was the different context: Britain already provides services to low-income parents that are quite similar to the home visiting program in the U.S. It is why leaders should test out, with rigorous evaluation, programs and initiatives in their own setting, particularly if previous research was conducted in a different context.
- Evidence-based policy is not a blueprint (aka cookie cutter) approach. This is a way of summarizing the previous two principles. While architects can take the blueprints for one building and build the same building elsewhere, and chefs can take a recipe from one restaurant and cook it in another, public leaders need to be careful when applying research from one place or setting to another.
Dr. White is the Chief Executive Officer of the Campbell Collaboration, a nonprofit best known for its use of systematic reviews to help policymakers and others make well-informed decisions. Previously he was the founding Executive Director of the International Initiative for Impact Evaluation (3ie) and led the impact evaluation program at the World Bank’s Independent Evaluation Group.