Generating and using evidence during a global crisis: what can we learn from the humanitarian sector?

Generating and using evidence during a global crisis: what can we learn from the humanitarian sector?

Marcella Vigneri | 12 March 2021

In September 2020, CEDIL opened its new webinar series with a thought-provoking talk between Ruth Hill – Chief Economist at the Centre for Disaster protection, and Kevin Watkins – CEO at Save the Children UK.  The event was a great opportunity to ask our colleagues to talk about their work as scholars and as promoters of evidence use in disaster prevention and humanitarian settings. What lessons can be learned and transferred in response to the ongoing health, economic and food crisis of Covid-19? 

First: using evidence in situations of great uncertainty is particularly helpful to respond to the collateral damage of a crisis on vulnerable people.  Evidence helps us better understand how an economic and social crisis triggered by any major humanitarian shock plays out in unanticipated ways in the lives of vulnerable people.  With Covid-19, no one predicted that the lockdown measures put in place to mitigate the spread of the virus would have catastrophic effects across developing countries in education, disrupting health systems, and breaking down transport systems and markets. It has been challenging to estimate the combined impact of the lockdown and the economic downturn caused by the pandemic. A recent paper on the indirect effects of Covid-19 argues that the pandemic will affect the health of women and children in ways that are immediately observable, and through unknown causal pathways. Fewer mothers using health facilities and the disruption of vaccination services are just two obvious examples.  Kevin referred to Steve Collins’ 2006 contribution on the effectiveness of community-based therapeutic care programmes as clearly affordable and cost-effective approaches to respond to a disruption in routine health care services and to a decrease in food access (both unavoidable outcome of shocks).  If cases of those most at risk are identified earlier on in communities – and treatment is available – this is clearly the most cost-effective way to avoid a devastating increase in child and maternal deaths.  

Second: more evidence helps to move from reaction to readiness. Waiting for needs to emerge has proven time and again as an inadequate strategy to mitigate the impact of disasters, no matter the nature of the crisis. The time is ripe for moving away from the medieval funding model based on begging bowls, where many poor countries appeal directly to the international community for support after disasters. As individuals, communities, local and national governments, international agencies, and NGOs continue playing the part of the beggar, benefactors are likely to remain committed to digging into their pockets to share their coins with those who clearly are facing hardship, but their coins may run out before the job is done or may be misallocated. There are, however, options available that have not yet been adequately taken up.  One is to ensure the continued accumulation of robust evidence that is reliable, accessible, and used.  Howard White has advocated for some time now the need to build a global evidence architecture that will show what works and where from the large body of evaluation research that remains mostly unused. Evidence is an important lens in decision-making, and in humanitarian settings it becomes even more valuable as a good-enough version of the evidence available to be used before important decisions are made. There are at least two reasons why using evidence is even more important in a humanitarian crisis and before disaster strikes. One is that the resources are very limited relative to needs, so evidence helps to better target resources at the right time when households are most in need of them.  The second reason is that evidence is important to understand how we need to change the way we respond to disasters: if we are asking people to do things differently, we must be prepared for an ‘evidence-making’ approach to handle strategically the relationship between new evidence of ‘unprecedented’ events and rapid policy decision making. The imperative to act swiftly in the absence of empirical data during the times of an emergency is crucial and requires other ways of generating and synthesising evidence to forecast in real time the possible effects of new interventions. 

Third: we need more evidence on the impact of disaster response. We can now do better at estimating the probability of people descending into poverty in the immediate aftermath of a crisis outbreak: innovations in data collection can help get better timely data to evaluate the impact of a disaster (big data and machine learning  have become smart tools  to measure impact rapidly). Similarly, it is now possible to standardise methods for data collection to ensure that when data is collected at a time of crisis it is possible to understand precisely what it is representative of and how to benchmark it. Evaluation work can take advantage of these tools to increase the number of evaluations of disaster response. A recent example is an evaluation of a WFP anticipatory action pilot in Bangladesh, that shows that providing a small amount of cash to households in advance of predicted flooding offers some protection to households as they go through a severe disaster. There are currently very few examples of disaster response being evaluated, yet rigorous impact evaluation is increasingly possible

Fourth: the resurging importance of older debates around evidence use for better finance allocation. Why is good evidence still not used to inform policy? How do we gain more policy traction?  Evidence gaps may drive the ineffective allocation of money when donors are not aware of priority areas in public health that are under researched. There are areas (e.g. child malaria) where lots of evidence is produced and used that accordingly become donors’ preoccupation territories.  But the flip side of this is that areas that donors neglect tend to get neglected also in evaluation (e.g. child pneumonia), therefore leading to an inefficient policy response in this area during a humanitarian crisis. In the aftermath of Covid-19, escalating infections rates and moral panic triggered a rush into the ventilator market for supplying intensive care units (ICUs).  But most poor people in rural settings have no chance to get anywhere close to ICUs, whereas basic oxygen systems could be spared instead to help save hundreds of thousands of lives. Evidence gap maps have become a tool for better funding decisions; for example the International Rescue Committee has created evidence maps  as part of their outcomes and evidence framework, and use these maps as a practical resource to standardize how they design and implement their programmes on the ground

Finally, is the evidence community responsible for not communicating the best evidence in a way that can generate policy traction? Evidence use to inform policymaking remains alarmingly low: the anti-vaccine movement is clearly an antithesis to evidence-informed policymaking. Evidence needs to be translated and communicated as actionable and feasible recommendations. We need more initiatives like the ongoing work of Evidence Aid to make the latest research in humanitarian settings available and discoverable when needed. Good, reliable, and timely evidence is what benefactors need to know who the neediest are to ensure they are the ones receiving the most benefits- not least when time is of essence and decisions need to be made quickly based on limited information. Can we see the fourth wave of the evidence revolution coming on at this time of global crisis when accurate, concise and unbiased synthesis of the available evidence is one of the most valuable contributions the research community can offer to decision-makers?

Marcella Vigneri is Research Fellow in CEDIL’s research directorate. This blog was written with inputs from Ruth Hill and Kevin Watkins.

Photo Credit: Dominic Chavez, World Bank

Leave a Reply