2. Enhancing evidence transferability
Assessing the extent to which existing evidence is applicable to issues encountered in specific contexts is a key challenge faced by policymakers in using evidence for decision-making. Strong evidence that an intervention worked in one context is not a guarantee that it will work somewhere else. A wide range of inescapable variations in context and implementation can influence the effectiveness of any intervention design.
To address this challenge, CEDIL is exploring methods of using evaluation and evidence synthesis in the development of ‘middle-range theories’. While traditional impact evaluations and systematic reviews can provide evidence on whether interventions had an effect, a middle-range theory uses evidence to describe why and how interventions generated the outcomes that were observed. As such, they should provide a much more useful tool for deciding whether and how an intervention should be implemented, as policymakers can assess whether their context has the necessary conditions and whether intervention processes can be implemented in the way that is needed to ensure effectiveness.
As an example, a team from Innovations for Poverty Action will collect data for a seven-year follow-up to three youth skills programmes implemented in Uganda. Through a combination of methods, including machine learning, the project will identify how different skills curricula influenced the economic opportunities for youth. Another project, consisting of a collaboration between the Universities of North Carolina, Malawi, Ghana and Zambia, combines several sources of secondary data to understand how cash transfers influence psychological states and household behavioural responses. The research should provide better insights into the circumstances in which cash transfers can enable households to graduate out of poverty.
The evidence synthesis projects will find innovative ways to allow evidence to inform our understanding of how types of interventions work, not just whether or not they do. For example, the Education Endowment Foundation will lead a project to expand its Teaching and Learning Toolkit to include data from education interventions in Cameroon, Chad, Nigeria and Niger. Evidence will be analysed to understand why the effectiveness of intervention types varies between contexts.
Another evidence synthesis project will use novel Bayesian methods to identify the relevant importance of different components of Teaching at the Right Level programmes, a popular type of education intervention. A third synthesis project, led by the London School of Hygiene and Tropical Medicine, will deploy several systematic synthesis methods to develop a generalisable theory of how interventions generate demand for contraception among adolescents in low- and middle-income countries.