3 December 2021

Evidence and gap maps (EGMs) have grown in popularity as an important component of evidence architecture for evidence-informed policy and practice in recent years. The Uganda Country Evaluation Map – the first country EGM – identifies potential knowledge gaps and opportunities for synthesizing existing evidence for Ugandan policymakers to use. It includes all policy domains that affect the welfare of the country’s people, as well as all evaluation types, including formative, process, outcome, and impact evaluations. CEDIL interviewed two of the map’s authors to get their perspectives on why this EGM is unique in many ways. Here are highlights from our Q&A.

CEDIL: Where did the idea of a Uganda Country Evaluation Map come from?

Howard White: I was putting together a presentation for the Uganda Evaluation Association that included a slide about impact evaluations in Uganda. Before showing that slide, I asked the audience how many randomised controlled trials (RCTs) of development interventions they thought there were in Uganda. Several people guessed zero, with the highest guess being ten. In fact, the 3ie Evidence Portal at the time showed that there were over 100 RCTs and over 200 impact evaluations. What I took from that was that people were simply unaware of where they could find evaluation evidence and the magnitude of evaluation work that had been carried out. That evening, I did some more searching to look for process evaluations. I found that there were a similarly high number of process evaluations out there, implying that there were likely to be more than 500 evaluations of development interventions in Uganda.

CEDIL: That is a great story which shows that we may have access to a lot of evidence but may just not be aware of it. How many evaluations did you find through the process of developing the Uganda map?

Caroline Otike: The preliminary search produced approximately 500 studies. However, when we finished the map, it had 621 evaluations of Ugandan development interventions published since 2000. The map made the evaluations discoverable and accessible via the hyperlinks to studies. The majority of these studies are almost certainly unknown to potential users because they may be unfamiliar with the evaluation repositories we searched.

CEDIL: The purpose of all EGMs is to make evidence accessible. What makes this one unique?

Howard White: The Uganda Country Evaluation Map was an important innovation in a few ways. It is the first comprehensive country-level map across all sectors. Secondly, it was one of the first maps which included both effectiveness studies and process evaluations. And, finally, and perhaps most importantly, the map was co-produced by the Africa Centre for Systematic Reviews and Knowledge Translation at Makerere University’s College of Health Sciences, with the Office of the Prime Minister, Uganda, the Campbell Collaboration and 3ie. This model of a collaboration with a local research team and a central government policy agency, is something Campbell has now adopted for other country maps.

CEDIL: Did this research team model help in promoting the use of the map?  What are some of the striking findings of this map?

Caroline Otike: Yes, it surely did. The fact that the map was created in collaboration with the Office of the Prime Minister (OPM) meant that it has a direct line into informing policy. In consultation with our colleagues from government, the categorization of interventions in this map was in line with Uganda’s Vision 2040.  And yes, the map has already been used in several ways. OPM has used it to develop an evaluation agenda for the health sector.  It has also been used to identify recent relevant studies across the different Sustainable Development Goals to inform Uganda’s Voluntary National Review of the implementation of the goals. More generally, the studies in the map have become a source for scoping studies in specific areas. To me, the map’s most important finding is that there are numerous opportunities to synthesize national studies in order to learn sector-specific lessons, and the map serves as a starting point for such research to be conducted in order to inform policy and programme design.

Howard White: The most striking finding to me was the low share of studies including Ugandan team members, and an even smaller number of studies that were led by Ugandan authors. There has been enormous growth in the Ugandan evaluation community in recent years, thanks in large part to the work of entities like the Uganda Evaluation Association, the support they have received from OPM, and the establishment of the graduate diploma in evaluation at the Uganda Management Institute. Evaluation funders should be drawing on this local expertise. Hopefully the next edition of the map will be able to highlight a new trend in this direction.

CEDIL: That brings us to our final question. Are you planning to update this map?

Caroline Otike: I am glad you asked about an updated map. The search for the existing map was carried out in 2018. If there are any agencies who can support us updating the map, please get in touch with me at the Africa Centre for Systematic Reviews and Knowledge Translation. The map is a great resource, whose value will be preserved by regular updates.

Caroline Otike is a research fellow at the Africa Centre for Systematic Reviews and Knowledge Translation at Makerere University’s College of Health Sciences. You can contact her at carolotike@gmail.com or afcen.makchs@gmail.com

Howard White is the CEO of Campbell Collaboration and is CEDIL’s Research Director.

Photo credit: ILRI/Erika Chenais