Enhancing evaluation use: rubric methodology in programme evaluation

Increasing evidence use

Sten te Vogt

Mekong Economics Myanmar Ltd

University of Tennessee Knoxville

February 2020 to January 2021

£72,195.97

Exploratory project

Myanmar

This study aims to look at how program evaluation rubrics influence evaluative thinking among stakeholders in Myanmar, and how to build evaluation capacity to increase the use of evaluation findings.

The proposed questions relate to both research on methods (i.e. How do different stakeholders view the credibility of different methodological approaches?) and research on context (i.e. What contextual factors alter the evaluation? How do they affect evaluation methods?) to help evaluators select appropriate practices and advance evaluation use among stakeholders.

The study will use a randomised experiment to investigate the impact of program evaluation rubrics on evaluative thinking and judgements.

Myanmar participants with a university degree from Yangon will be recruited to participate in the study under the guise of seeking community involvement in evaluating a social programme. Each participant will be randomised into one of two conditions: groups with rubrics and without rubrics. Participant demographic information will be collected along with previous evaluation experience.

The survey construction will use exploratory focus groups to study the significance and meaning of information credibility and decision transparency.

The study can provide insights on how rubrics work in different contexts and how the motivational, attitudinal and relational aspects of using rubrics can inform evaluative judgements.

• CEDIL Design Paper 11: Enhancing evaluation use: An experimental assessment of rubric methodology on collective judgements in Myanmar