Dorgham Abusalim | 28th April, 2022
In a blog post first posted on the FHI 360 blog, Dorgham Abusalim provides an overview of the discussions on the final day of the CEDIL 2022 conference.
On Friday, March 25, the Centre of Excellence for Development Impact and Learning (CEDIL) closed its 2022 conference with a panel session titled Transferring evidence
between contexts. CEDIL’s four-day long conference convened researchers and experts from around the world to discuss strengthening evidence use during the pandemic and beyond.
CEDIL’s Research Director, Dr. Howard White, chaired the panel and began with opening remarks, followed by presentations by Prof. Chris Bonell, Professor of Public Health Sociology and Head of Department of Social & Environmental Health Research at the London School of Hygiene & Tropical Medicine (United Kingdom); Sheree (Bennett) Kullenberg, Senior Advisor for research and evidence at International Rescue Committee (Germany); and Dr. Patrick Okwen, Team Lead at eBASE Africa (Cameroon). The presentations were followed by remarks by Dr. Annette Brown, FHI 360’s Principal Economist and Acting Head of Strategy, in her role as a panel discussant.
The COVID-19 pandemic continues to present challenges that require expeditious access to high quality evidence at both global and local scales. Thus, the topic of research and evidence transferability across humanitarian and development interventions and contexts remains especially critical to informing policy responses.
With this ongoing demand for evidence in mind, the panelists explored the conditions and methods necessary for transferring evidence between different contexts, and the role of theory, evidence base, and systematic reviews.
Prof. Bonell, a member of the British Independent Scientific Pandemic Insights Group on Behaviours (SPI-B), discussed the role of theory in the transferability of evidence during his experience developing behavior change communications for adherence to the COVID-19 lockdown in March, 2020, in the United Kingdom. Given that little information was available about the virus at the time, the group utilized existing literature and theories to propose whole-population messaging for local dissemination by the British government. This approach included the COM-B Framework, moral foundation and social marketing, psychology of social influences, and social cognitive theories.
The theories above were “the means by which evidence from other contexts informed our interventions in a novel and urgent context,” said Prof. Bonell. Although the SPI-B group assessed the face validity of the theories, rather than their evidence base, “this approach resonates with increasing interest in evaluations informing broader policy,” he added.
Ms. Kullenberg highlighted IRC’s overarching approach to transferring evidence between contexts. A key component of her presentation was a demonstration of IRC’s Outcomes and Evidence Framework tool, established in 2016, which delivers key information on programmatic interventions and outcomes in the context of available evidence. The goal was to provide “an assessment of the evidence, the strength of the evidence, and the quality of the evidence around these interventions that are linked to the outcome of concern,” she explained.
The tool has since evolved to a prototype of “Evidence Cards” that share and bring evidence together by incorporating “impact findings, both qualitative and quantitative,” including a “confidence level” indicator by IRC based on the quality and inferences that can be made from the available evidence, Ms. Kullenberg added.
Dr. Okwen discussed his experience utilizing meta-analysis to predict the transferability of evidence from global contexts to local decision-making using a mathematical model based on his work in education programs in the Lake Chad Basin. A key component of this approach is engaging with stakeholders to assess evidence transferability relevance, cost, complexity, impact, and feasibility. This step is followed by a mathematical scoring of transferability using five measurements: transferable; transferable with minor modification; transferable with moderate modification; transferable with major modification; and non-transferable.
Dr. Brown drew on her own FHI 360 work conducting a rapid systematic review on social norms while sharing key points about theories, research and evidence base, theory of change, and outcomes relevant to evidence transferability.
“The more complex the program we’re designing, the more important it is we have evidence that applies to that type of program,” said Dr. Brown. “We need to make sure that we have evidence that is showing a change in one thing leads to a change something else; and finally, when we’re using middle level theories, we have evidence supporting those theories from the same intervention in one study,” she added.
Dr. Brown echoed the input of fellow panelists, noting the role of theory in transferring evidence, operationalizing quantitative and qualitative evidence in interventions, and the use of meta-analysis. She also discussed how the conversation about evidence transferability is a key element of the growing focus on locally led development, which is informing new ways of making decisions about the design and implementation of programs. Thus, it is critical to keep in mind “the challenge of how we can keep bringing evidence into these news ways of making decisions,” she noted.
Watch the panel below learn more about CEDIL’s conference sessions here.