I was in Kenya last week undertaking some field evaluation work for the Darwin Initiative. This blog post is intended to shed some light on what we do during these field trips, what the value is (and to whom) and how we use the material we produce.
The Darwin Initiative has an overarching M&E framework which includes all the standard things such as a Theory of Change and Programme Logframe. In essence what this boils down to is a set of objectives, indicators and assumptions for the programme. Our job, as the M&E contractor is to measure the collective progress of Darwin funded projects against these objectives and test assumptions.
This sounds straightforward until you consider the diversity of projects funded under the Darwin Initiative – both in subject matter and in geography. Each project is collecting and presenting evidence in a myriad of different forms and somehow we need to make sense of it all. Here I’ll talk through what the fieldwork element of our work focuses on.
It serves a number of purposes depending on whether the projects evaluated are ongoing or closed. For ongoing projects the purpose for field evaluation includes:
- the verification of results
- obtaining an independent perspective on the projects
- troubleshooting and supporting projects to reorient themselves
For closed projects fieldwork is more centred around the full Darwin programme than the individual project. The purpose of these activities includes:
- analysis of evidence to meet a specific programme objective i.e. gender or traditional knowledge etc.
- an attempt to understand what the legacy is of Darwin funding i.e. what happened next
- an attempt to understand the collective impact of multiple projects
We tend to restrict fieldwork to 2 distinct periods – halfway through a project (a Mid Term Review) and after a project has finished (a Closed Project Evaluation).
Most of the time we have 5-7 days with each project depending on logistics. For all projects visited we are driven by what the project has defined as being their measures of success i.e. the project Application Form. This includes the logframe, workplans, methods, team composition etc. We additionally have a Terms of Reference that defines what questions we are looking to answer through the fieldwork. This week my terms of reference centres around 3 questions:
- What biodiversity benefits has the project achieved?
- What has the contribution of the project been to poverty alleviation? Poverty should be considered in the context of the MDG’s.
- What factors have governed the project organisation’s capacity to collect appropriate evidence?
The methods we use are partially dictated by the types of evidence available from the projects. In most cases however we will also use semi-structured key informant interviews with the project implementers, key stakeholders and beneficiaries of the projects.
We may also undertake focus group sessions that may use methods such as participatory ranking, participatory pairwise ranking, and Most Significant Change.
For most ongoing projects we generally conduct a Theory of Change session at some point during the evaluation. This is a useful tool for an evaluator to better understand what the project team consider success to look like, what assumptions this success hinges on, what measures might be available to better understand this success and who the various actors are that influence this success. As well as being useful for the evaluator, this is often a useful exercise for the project team to undergo as it allows them to reflect on their progress and what tasks they must undertake in order to best achieve success. I will talk more about Theory of Change and its uses in a subsequent blog post.
The final outputs of this field evaluation is of course a report. This report is shared with Defra and DFID for approval before being published on the Darwin Initiative website. You can see some examples here.
We also develop a shorter learning note which is intended to draw out lessons for the wider Darwin community, which you can also see here.
This is rarely the end point of these reviews since they often result in recommendations for both the projects and the programme. We may therefore use this material to refine our systems and processes within the Darwin Initiative to better ensure impact such as the refining the application forms. Additionally we use this material to help improve understanding within the wider Darwin Initiative community, through a variety of forums including this blog!
I hope this blog post is helpful in understanding what the purpose of these field evaluation visits are. We have at least 2 Mid Term Reviews to conduct later this year and a Closed Project Evaluation in September, hopefully in Kyrgyzstan, so watch this space!