Enhancing evaluation use: rubric methodology in programme evaluation

Programme of work

Increasing evidence use

Principal investigator(s)

Sten te Vogt

Host institution

Mekong Economics Myanmar Ltd

Other institutions

University of Tennessee Knoxville

Dates

February 2020 to January 2021

Project type

Exploratory project

Country/ies

Myanmar

Research question

This study aims to look at how program evaluation rubrics influence evaluative thinking among stakeholders in Myanmar, and how to build evaluation capacity to increase the use of evaluation findings.

The proposed questions relate to both research on methods (i.e. How do different stakeholders view the credibility of different methodological approaches?) and research on context (i.e. What contextual factors alter the evaluation? How do they affect evaluation methods?) to help evaluators select appropriate practices and advance evaluation use among stakeholders.

Research design

The study will use a randomised experiment to investigate the impact of program evaluation rubrics on evaluative thinking and judgements.

Myanmar participants with a university degree from Yangon will be recruited to participate in the study under the guise of seeking community involvement in evaluating a social programme. Each participant will be randomised into one of two conditions: groups with rubrics and without rubrics. Participant demographic information will be collected along with previous evaluation experience.

Data source

The survey construction will use exploratory focus groups to study the significance and meaning of information credibility and decision transparency.

Policy relevance

The study can provide insights on how rubrics work in different contexts and how the motivational, attitudinal and relational aspects of using rubrics can inform evaluative judgements.