CEDIL - Centre for Evaluation Lecture Series

CEDIL – Centre for Evaluation Lecture Series

The Centre of Excellence for Development Impact and Learning (CEDIL) and the Centre for Evaluation host a lecture series addressing methods and innovation in primary studies.

Download the CEDIL CfE-Lecture-Series Calendar. Most lectures will be live-streamed, recorded, and posted on this site later.



Next Lecture

 



Evidence for Action in New Settings: The Importance of Middle-Level Theory

Date :  Wednesday 5th June 2019 Time :  12:45 – 14:00 GMT Location :  Keppel Street, London School of Hygiene & Tropical Medicine Room :  LG09 Speaker :  Nancy Cartwright 

 

About the lecture

For predicting intervention outcomes in a new setting you need a context-local causal model of what is expected to happen there. A theory of change (ToC) for the intervention is a starting point. ToCs are ‘middle-level theories’: they aim for some, but not universal, general applicability. These are typically ‘arrows-and-variables’ models depicting what steps should occur in sequence but not the interactive factors necessary at each step, nor possible interrupters/defeaters.  For policy prediction these theories need context-local thickening. This requires an understanding of how each step produces the next, which in turn calls for middle-level theory of a different kind:  the middle-level principles (mechanisms) that govern that production. The context-local model allows better prediction of whether an intervention can work there, what it would take for it to do, what side effects might be and whether all this is affordable and acceptable in the context. The lecture will be streamed online, and will be available to an external audience on the day of the lecture at this link

 

About Nancy Cartwright

Nancy  FAcSS is Professor of Philosophy at Durham University and a Distinguished Professor at the University of California, San Diego (UCSD). In the first half of her career at Stanford University she specialised in the philosophy of the natural sciences, especially physics; in the second half, at the London School of Economics and now Durham and UCSD, she has specialised in philosophy and methodology of the social sciences with special attention to economics. Her current research focusses on objectivity and evidence, especially for evidence-based policy.


Previous lectures are listed below

 


Designing evaluations to inform action in new settings

Date :  Wednesday 22nd May

Time :  12:45 – 14:00 GMT

Location :  London

Speaker :  Calum Davey

 

 

About the lecture

This presentation will be based on a CEDIL inception report. The report drew on the perspectives of more than five academic disciplines — from epidemiology to philosophy — and reviewed a diverse range of literature on the task of ‘learning for elsewhere’, addressing the questions: what is learned in evaluations of complex interventions that is useful for future decision making, and how can this be improved? Suggested answers all involved theory, begging questions about which setting theories apply, and how to know quickly. The notion of context-centred interventions challenged the sentiment that learning ‘what works?’ or even ‘how does it work?’ helps when in fact approaches to knowing ‘why is outcome occurring?’ would be more useful.  A copy of the slides from this presentation can be found by clicking on this link

 

About Calum Davey

Calum is an Assistant Professor at the London School of Hygiene and Tropical Medicine. He has worked on evaluations in the health and education sectors. He has been involved with CEDIL from the start, and was the lead author on one of the pre-inception reports and one inception report. He currently splits his time between research on evaluation methods and the PENDA programme, a £7m DFID-funded research programme on disability-inclusive development. 

Learning and Adapting in Development Practice

Date :  Wednesday 15th May Location :  London Speaker :  Patrick Ward

 

 

 

About the lecture

The growth of the global evidence base has provided opportunities to accelerate development through the systematic sharing of evidence of ‘what works’. Context matters however, and the implementation of development programmes requires the ability to learn from and respond to successes and failures on a short time scale and in the face of limited data. This lecture will explore factors influencing the extent to which development programmes are able to adapt in the light of evidence and learning. It will draw from the practical experience of monitoring and evaluating development programmes and supporting government statistics across a range of sectors in developing countries. To watch the livestream, please click on this link To see the slides, please click on this link

About Patrick Ward

Patrick is CEDIL Programme Director and has overall responsibility for the delivery of the programme, working closely with the Research Director as a member of the CEDIL Directorate. Patrick Ward is Director of Oxford Policy Management’s Statistics, Evidence and Accountability programme. He has more than 20 years’ experience in leading work in the monitoring and evaluation of development for both donor-financed programmes and government systems, particularly in the social sectors.

 


The Need for Using Theory to Consider the Transferability of Interventions

Date :  Wednesday 8th May 2019 Location :  London Speaker :  Professor Chris Bonell

 

 

 


About the lecture

This lecture explores ways that the transferability of interventions to new settings might be modelled statistically and the role of theory in considering the question of transfer. The lecture draws on preliminary results of the realist trial of the Learning Together whole-school health programme in the UK. Unfortunately, technical error has meant the first part of the lecture was not recorded, but the remaining can be found by clicking this link Slides from the lecture can be downloaded by clicking on this link

 

 

 

 

About Chris Bonell

Chris is Head of the Department of Public Health, Environments and Society, and Professor of Public Health Sociology at the London School of Hygiene and Tropical Medicine.  Prior to joining LSHTM, Chri was Professor of Sociology and Social Policy at University College London, and Professor of Sociology & Social Intervention at the University of Oxford.  His main areas of research are on adolescent health, sexual health, substance use and social exclusion and health, as well as in research methodology. Chris Bonell portrait


Using RCTS to Evaluate Social Interventions: Have We Got It Right?

Date :  Wednesday 27th March 2019 Location : London Speaker :  Professor Charlotte Watts

 

 

 

 

 

 

About the lecture

Randomised controlled trials (RCTs) provide the gold standard method to obtain evidence of intervention impact. Historically, the approach was developed to assess the impact of clinical interventions. Increasingly however, RCTs – both individual and cluster – are being used to assess a broad range of behavioural and social interventions. Although some argue that using randomised designs are not appropriate for evaluating social and community based interventions, we disagree. Whilst there may be challenges (as there often are with clinical interventions), randomisation and the use of control populations should always be considered, as this gives the most robust measure of effect size. But this doesn’t mean that we have everything right. Drawing upon examples from intervention research on HIV, as part of the STRIVE research programme, and on violence against women, as part of the LSHTM Gender, Violence and Health Centre, the presentation will discuss whether it is appropriate to apply all of the standards and ‘rules’, without consideration of the potential implications for the feasibility, forms and applicability of evidence generated. To watch the livestream, please click on this link

 

 

 

 

About Charlotte Watts

Charlotte is Chief Scientific Adviser to the UK Department of International Development (DFID).  In this role she is Director of the Research and Evidence Division and Head of the Science and Engineering Profession for DFID. She is currently seconded to DFID from the London School of Hygiene and Tropical Medicine, where she is Professor of Social and Mathematical Epidemiology, and where she she founded the Social and Mathematical Epidemiology Group.

 

 

 

 

 


When context is the barrier: Evaluating programmes during political turmoil

 

Date :  Wednesday 6th March 2019 Location :  London Speaker :  Dr Joanna Busza

 

 

About the lecture

 

The importance of “context” in evaluation is increasingly recognised, especially when considering how complex interventions might be scaled-up, adapted or replicated for new settings. Most process evaluation frameworks include documenting contextual characteristics deemed relevant to the intervention’s implementation, such as the structure and function of health systems, cultural norms and practices, and existing laws or policies. The MRC guidelines for process evaluations, for example, suggest using existing theory to identify a priori the factors likely to facilitate or hinder successful implementation of intervention components. This seminar will focus on challenges to design, implementation and evaluation of community-based health programmes when context – at its broadest level – changes in abrupt, unpredictable ways. I will share examples from research in Cambodia, Ethiopia and Zimbabwe, including both negative and positive consequences of dramatic political or policy changes, and discuss implications for completing and interpreting the affected studies.  Livestream can be accessedby following this link: Livestream

 

About Joanna Busza

Joanna is a social demographer with over 25 years’ experience in community-based public health research, with a focus on sexual, reproductive, and maternal health. Her expertise lies in qualitative and mixed methods studies.  Currently she is Director of the LSHTM Centre for Evaluation and her research primarily involves process evaluations of complex interventions in the fields of HIV prevention and treatment, safe labour migration, and adolescent sexual health in both Ethiopia and Zimbabwe.  

 

 


Evidence Standards and justifiable evidence claims

 

Date :  Wednesday 6th February 2019 Location :  London Speaker :  Professor David Gough

 

 

About the lecture

 

In developing findings and conclusions from their studies, researchers are making ‘evidence claims’. We therefore need to consider what criteria are used to make and justify such claims. This presentation will consider the use of evidence standards to make evidence claims in relation to primary research, reviews of research (making statements about the nature of an evidence base), and guidance and recommendation informed by research. The aim is to go beyond testing the trustworthiness (quality appraisal) of individual studies to discuss the ways in which evidence standards are used to make evidence claims to inform decisions in policy, practice, and personal decision making. The live audio and slide show recoding can be found by following this link: Live recording of lecture. Please note that recording started early, so please fast forward to 17 minutes. The slides can be downloaded here: Slide show

About David Gough

David is Professor of Evidence Informed Policy and Practice and the Director of the EPPI-Centre in the Social Science Research Unit (SSRU) at UCL. His early research was on child welfare at the University of Glasgow and at Japan Women’s University. Since moving to SSRU in 1998, his work has focused on methods of systematic mapping and synthesis and  ‘research on research use’. Recent work include the Wellcome Trust funded ‘Science of Using Science’ and the ESRC funded ‘UK What Works Centres: Aims, Methods and Contexts’. He is lead editor of Introduction to Systematic Reviews (2nd Edition) and Systematic Reviews and Research, both Sage Publications. David Gough close up

 


Stakeholder Engagement for Development Impact Evaluation and Evidence Synthesis

 

Date :  Wednesday 23 January 2019 Location :  London Speaker :  Professor Sandy Oliver

 

 

About the lecture

 

This lecture explores methods for engaging stakeholders in making decisions for international aid and social development in the presence and absence of relevant research. It draws on empirical evidence about engaging stakeholders in the generation and use of evidence, taking into account political analysis, social psychology and systems thinking. It finds that the suitability of methods for engagement depends largely on the confidence that can be placed in knowledge about the specific context, and knowledge from elsewhere that seems theoretically or statistically transferable. When decisions are about generating new knowledge, the suitability of methods for engagement depends largely on whether the purpose is to generate knowledge for a specific context or for more generalizable use and, at the outset, the confidence and consensus underpinning the key concepts of interest. The Lecture will be available to view live at the link below. Live Recording

About Sandy Oliver

Sandy is Professor of Public Policy at UCL Institute of Education. For thirty years her interests have focused on the interaction between researchers and people making decisions in their professional and personal lives, largely through the conduct of systematic reviews. She is a member of the Board of the Campbell Collaboration and Cochrane editor with their Consumers and Communication Review Group. Her recent contributions to research synthesis methods come from working with the UK Department for International Development and the Alliance for Health Policy and Systems Research at WHO to build capacity in systematic reviewing in developing countries.

Sandy Oliver


To boldly go where no evaluator has gone before: the CEDIL evaluation agenda

 

Date :  Wednesday 12th December 2018    Location: London Speaker :  Dr Edoardo Masset

 

About the lecture

 

In this lecture I will introduce the newly established Centre of Excellence on Development Impact and Learning (CEDIL). CEDIL was established by the UK Department for International Development to develop new evaluation methods and to commission evaluation and synthesis studies in neglected areas of international development. Over its inception phase CEDIL identified key methodological evaluation challenges to address and priority thematic areas. The talk will illustrate CEDIL’s ambitious evaluation agenda over the next 5 years, and will be followed by Q&A and discussion. The Lecture will be available to view live at the link below. Live Recording

About Edoardo Masset

Before joining CEDIL, Edoardo was Deputy Director and head of the London office of the International Initiative for Impact Evaluation, which he joined after working for seven years as a Research Fellow at the Institute of Development Studies at the University of Sussex. Edoardo is an agricultural and development economist with extensive experience conducting impact evaluations, researching development interventions and consulting for a variety of institutions, including the World Bank. Edoardo also has experience of research synthesis work through a number of systematic reviews on a wide range of topics such as nutrition, health insurance and cost-effectiveness. His core research interests include rural development, child nutrition, poverty and inequality, and the analysis of household surveys. Edoardo Massett Deputy Director CEDIL


Intervention to foster early childhood development evaluation, sustainability and scalability

 

Date :  Wednesday 28th November 2018 Location: London Speaker :  Professor Orazio Attanasio

 

About the lecture

 

Early Childhood Interventions have recently received much attention.  The consensus is that ECD interventions can ‘work’ and be very effective and important.  The new challenges however are: (i) understand how interventions work and how they obtain the observed effects, through which channels, at what age, and so on, and (ii) how to scale up effective interventions. The answer to the second question is related to the answer of the first.  Orazio will present some concrete examples of these issues in this lecture. Watch the recording here

About Orazio Attanasio

Orazio is the Research Director of IFS and one of the Directors of the ESRC Centre for the Microeconomic Analysis of Public Policy (CPP) and co-directs the Centre for the Evaluation of Development Policies (EDePo).  Orazio is a Professor at UCL, and a Research Fellow at the Centre for Economic and Policy Research.  In 2001 he was elected Fellow of the Econometric Society and in 2004 he was elected as a Fellow of the British Academy and is currently President of the European Economic Association.


Uncertainty and its consequences in social policy evaluation and evidence-based decision making

 

Date :  Wednesday 31st October 2018 Location :  London  Speaker :  Doctor Matthew Jukes  

 

About the lecture

 

The methodologies of RCTs and systematic reviews imply a high level of rigor in evidence-based decision-making. When these standards are not met, how should decision-makers act? When a clear body of evidence is not available, there is a risk that action is delayed or that action is taken without optimal use of the existing evidence. This paper addresses the following question: what level of certainty is required for which kinds of decisions? We argue that decisions should be based on considerations of both the uncertainty and the consequences of all possible outcomes. We present a framework for making decisions on partial evidence that has implications for the generation of evidence too. More systematic analysis of uncertainty and its consequences can improve approaches to decision-making and to the generation of evidence. Watch the lecture here  Download the lecture slides here LIDC podcast on Innovative approaches to evaluation and evidence synthesis Please also see his blog post on this topic by following the link here

About Matthew Jukes

Matthew is a Fellow and Senior Education Evaluation Specialist at RTI International.  He has two decades of academic and professional experience in evaluating education projects and is contributing to projects in Malawi and Tanzania aimed at improving the quality of pre-primary and primary education in those countries.  He has also applied his research to work with the World Bank, UNAIDS, UNESCO, USAID and  Save the Children.


Using mid-level theory to understand behaviour change examples from health and evidence-based policy

 

Date :  Thursday 30th August 2018 Location :  New Delhi, India Speaker :  Doctor Howard White

 

About the lecture

Mid-level (or mid-range) theory rests between a project-level theory of change and grand theory. The specification and testing of mid-level theories help support the generalizability and transferability of study findings. For example, in economics, the operation of the price mechanism to balance supply and demand is a grand theory. An agricultural fertilizer subsidy programme would have a project-level theory which partly draws on the theory of supply and demand (lowering price increases demand). A mid-level theory could be developed related to the use of price subsidies, of which the fertilizer programme would be a specific application. This talk will adopt the transtheoretical model of behaviour change to apply mid-level theory to the analysis of two sets of interventions: the adoption of health behaviour, and promoting evidence-based policy change. Watch the lecture here (currently in several parts) Download the lecture slides here

About Howard White

Dr White is the CEO of the Campbell Collaboration and the Research Director for CEDIL and this presentation is based on work undertaken as part of CEDIL.


Development impact attribution: Mental models and methods in 'mixed marriage' evaluations

 

Date :  Wednesday 18th July 2018 Location :  London Speaker :  Professor James Copestake

 

About the lecture

The marriage metaphor will be used to explore collaboration that spans academic traditions and disciplines, researchers and managers, public and private sector agencies.  The idea of mental models will be used to explore the ontological, epistemological, contractual and socio-political tensions created by formalised evaluative practice.  It will focus particularly on experience with mixing qualitative impact evaluation with other approaches to generating evidence, and learning and legitimising public action.  It will draw on case studies from the garment industry, medical training, housing micro-finance and agriculture spanning three continents. Watch the recorded lecture here

About James Copestake

Prof Copestake is a professor of international development at the University of Bath.  In addition to recent work on the Qualitative Impact Protocol, his recent research has addressed contested perceptions of well-being in Peru, financial inclusion and micro-finance in India.  He has also researched the relationship between social policy and development studies, and the use of challenge funds in aid management. James Copestake portrait


Representing theories of change: Technical challenges and evaluation consequences

 

Date :  Wednesday 30th May 2018 Location :  London Speaker :  Doctor Rick Davies

 

About the lecture

This lecture will summarise the main points of a paper of the same name.  That paper looks at the technical issues associated with the representation of Theories of Change and the implications of design choices for the evaluability of those theories.  The focus is on the description of connections between events, rather than the events themselves because this is seen as a widespread design weakness.  Using examples and evidence from a range of internet sources, six structural problems are described along with their consequences for evaluation.  The paper then outlines six different ways of addressing these problems which could be used by programme designers and evaluators.  These solutions range from simple to follow advice on designing more adequate diagrams; to the use of specialist software for the manipulation of much more complex static and dynamic network models.  The paper concludes with some caution, speculating on why the design problems are so endemic but also pointing a way forward.  Three strands of work are identified that CEDIL and DfID could invest in to develop solutions identified in the paper. Watch the recorded lecture here

About Rick Davies

Dr Davies is an independent Monitoring and Evaluation consultant and is based in Cambridge, UK.  He has managed the MandE NEWS website and email lists since 1997.

Rick Davies portrait


The four waves of the evidence revolution: Progress and challenges in evidence-based policy and practice

 

Date :  Wednesday 11th April 2018 Location :  London Speaker :  Doctor Howard White  

 

About the lecture

The evidence movement has rolled out in four waves since the 1990s : the results agenda, the rise of RCTs, systematic reviews, and developing an evidence architecture.  This revolution is uneven across sectors and countries and is an unfinished revolution.  Drawing on experiences from around the world, this talk will provide a historical overview of the evidence movement and the challenges it faces.  Response from these challenges will be considered, including those offered by the work of CEDIL. Watch the recorded lecture here

About Howard White

Dr White is the CEO of the Campbell Collaboration and the Research Director for CEDIL and this presentation is based on work undertaken as part of CEDIL.