As we enter the final days of preparation for the What Works Global Summit in Mexico City next week, I still worry about whether this is time and money well spent. I do believe that research on what works is one of the best buys in development. Conferences can be worthwhile if they further the evidence agenda. Will we do that, and what will CEDIL’s contribution be?

On 14 October, CEDIL will be hosting a full-day pre-conference workshop – ‘Frontiers in evaluating causality’ – where leaders in the field will present CEDIL’s work from the proving grounds of impact evaluation. I’m looking forward to introducing the full spectrum of our work to an international audience and getting critical reflections from the audience.

The challenges CEDIL was set up to address are still with us. Impact evaluation is not a new field of endeavour. By 2017, I’d been involved with The Campbell Collaboration and 3ie, the International Initiative for Impact Evaluation, which commissioned hundreds of impact evaluations and systematic reviews across the world. The utility of these studies can’t be overstated, not least in demonstrating that rigorous impact evaluations can be applied to social sector interventions.

Bridging the gap

Still, these evaluations didn’t consider more complex interventions or packages of interventions being implemented in low-income, fragile or conflict-affected countries, particularly by the UK’s Department for International Development (DFID). In Afghanistan, for example, DFID’s support encompasses a range of projects to curb corruption, improve markets, reduce violence and provide health, education and other services in fragile, conflict-affected contexts. The question ‘What works?’ for such a set of interventions can’t just be answered with a single randomised controlled trial on any one of these interventions in isolation. It’s the same problem with generalised budget support, debt relief and multidisciplinary issues like climate change. Donors and policymakers are asking us how different approaches interact. CEDIL was established to provide evaluations that credibly address the impact question for such programmes.

Recognising the gaps in thematic/geographical coverage, methodology, research synthesis and research uptake – CEDIL was set up with a five-year grant from DFID, with a mandate to develop and test evaluative methods that would be directly relevant to policymakers: specifically, DFID. Following a pre-inception phase to inform these evaluation gaps and set out a research agenda, we produced a number of inception papers exploring complex interventions, mid-range theory and evidence into use. We also put out a call for proposals for grants with a total value of £10 million for these three programmes of work, and we are currently in the process of finalising our grantees, some of which we will be discussing at WWGS 2019.

Complex interventions

Governments and donors often structure development interventions as ‘packages’ to address complex challenges that should be tackled simultaneously. This means that evaluation designs need to be complex, too. We hope that proposals for this programme of work – evaluations, secondary data analysis/retrospective evaluations and evidence synthesis – will advance our understanding of how and why combinations of interventions work. It’s worth pointing out that CEDIL is looking for complex interventions at work, and not necessarily complex settings.

Mid-range theory

Researchers and policymakers alike have long sought the middle ground between project-level theories of change and ‘grand’ social science theories. Middle-range theories aim for transferability by explaining processes, mechanisms and behaviours in terms that are general enough to be applied to multiple contexts.

It’s known, for example, that if a child is deprived of proper nutrition in the first 100 days of its life that child will suffer irreversible cognitive disadvantages. What’s less commonly known is that the same applies to stimulation. Unfortunately parents, globally, often don’t think it’s worth talking to babies. But it’s been found that parent training classes on stimulation make a difference – whether they’re held in China, Burkina Faso or Thailand. This is a great example of a mid-range theory being tested across contexts, yet such methods are unknown and underutilised. CEDIL therefore looked for proposals that included large impact evaluations that develop middle-range theories (using either primary or secondary data) to explain intervention outcomes and test them in a specific impact evaluation.

Evidence into use

Going beyond anecdotal evidence and ‘best practice’, we looked for research proposing to test different approaches of promoting research uptake, synthesise existing evidence or produce guidelines for policymakers. This will help them effectively use evidence from multiple sources to make decisions.

Watch this space

At CEDIL, we’re trying hard to ensure that the burgeoning field of impact evaluation keeps pace with changes in the sector of international development so it can be truly useful to the people and communities meant to benefit. So far, we’ve been asking ‘What works?’ in areas where it’s proven difficult to actually answer that question. Next week, the global impact evaluation community is coming together to take stock. We may not have all the answers yet, but we’re happy to help map out areas of research. The need for innovation remains, and in a future round of funding, CEDIL will explore novel approaches to push the frontiers of impact evaluation. Come and tell us what you think.