Interviews

Semi-structured interviews will be conducted with internal and external stakeholders at the global and country levels. The sampling strategy for interviews will be ‘purposive’ in that it will target a sample of informants who are likely to have diverse insights into the evaluation questions from a range of useful perspectives.

Interview questions will be structured around the Primary Evaluation Questions and Guiding Questions, with conversations guided by open-ended questions. Interview data will be coded against the question framework and analysed for consensus, common views, emerging themes and divergent perspectives.

Outcome Harvesting

Outcome harvesting is a method to collect or ‘harvest’ outcomes – that is, the actual outcomes of an activity, not just the intended outcomes. In contrast to evaluations that focus exclusively on logframes and linear program logic, outcome harvesting is concerned with better understanding ‘how change happened’ rather than how it was planned to happen. In this sense outcome harvesting is a participatory approach, whereby participants identify change (outcomes) and work backwards to analyse how the change happened, with the goal of better understanding how the intervention contributed to the change. This approach is particularly useful when the cause and effect’ relationships are unclear or incomplete. Outcome harvesting is also useful for formative evaluations because it focusses on actual outcomes and what has actually contributed to those outcomes, rather than testing that all inputs and outputs have been undertaken and assuming they have contributed to change.

For the evaluation, outcome harvesting workshops will be conducted at the global level via ‘Policy Case Study Forums’ and in two deep dive case study countries. During each workshop, outcome statements will be developed of actual outcomes that have been observed. Workshop participants will then discuss how these statements can be tested and analysed to better understand what role EITI ‘inputs’ played in contributing to the outcomes. This contribution analysis includes participants providing advice on how to identify means of verification such as documents, media sources, policies and legislation and parliamentary records.

Most Significant Change

Most significant change (MSC) is an inductive method that helps to identify how change happens and how it is experienced. Participants will be asked to tell a story explaining the most significant change (intended or unintended) that they have experienced or observed that they attribute at least in part to EITI. Stories will then be analysed in a participatory sense-making workshop where a panel reviews the stories and identifies impacts, consequences (both intended and unintended) and ‘impact pathways’. This approach will allow data to be collected and analysed in a way that reveals complexity, interconnectedness and non-linear change. The approach also offers an opportunity for mainstreaming the evaluation learnings through including stakeholders in the approach to analysis.

For the MSC stories the evaluation will be adopting a ‘digital storytelling approach’ – where participants from all country case study countries will be invited to contribute their change story in a video format via an online platform. The participatory sense-making workshop will then take place at either the country or global level, whatever is most practical. This approach will also generate stories for the Evaluation’s ‘video report’