Insights and personal musings from the world of biodiversity conservation and development. For more info on the Darwin Initiative see https://www.gov.uk/government/groups/the-darwin-initiative
Part 1 in this series highlighted the importance of ecosystem differences. In part 2 I discuss some of my recent thoughts on the use of income as an indicator of poverty alleviation…
On 21st October we held an experts meeting in conjunction with Flora and Fauna International and International Institute for Environment and Development about what constitutes good evidence in the context of conservation and poverty alleviation projects. Part of the discussions focussed on how evidence fits within broader project monitoring and evaluation (M&E). Since I’ve been working on the Darwin Initiative we’ve had lots of discussions about strengthening project M&E, so the workshop provided lots of food for thought.
A recent Darwin Initiative briefing paper highlights that indicators are an essential component of any effective M&E system; they provide information to monitor performance, measure achievement and demonstrate accountability. For the Darwin Initiative, indicators are part of the framework for collecting evidence to show how projects have contributed towards poverty and biodiversity.
Kenya 20-011 Shortages of water means people travelling far distances to fetch water Credit J Bett
Kenya 20-011 Aweer community women representatives prioritizing livelihood options. Credit Nickson Orwa
On the poverty side of things, projects tend to select income as an indicator of poverty alleviation. The thematic review has revealed 2 issues that are worth considering:
Increasing household income is difficult to achievewithin 3 years (average timeframe of a Darwin Initiative project). It can also be difficult (or expensive) to measure. Also, income may increase but the household is still living in poverty. Linked to this, income does not capture wider wellbeing benefits which are often more tangible in 3 years. For example a recent evaluation in Kenya revealed that one of the benefits the beneficiaries of a project thought were most important was empowerment in particular recognising people’s access rights and supporting people to manage resources. This demonstrates that income is not always an appropriate or relevant indicator for projects to use.
Increasing household income may increase environmental degradation and/or have unintended social consequences. Case studies have shown that raising incomes can increase pressure on natural resources, for example if people use income to buy and graze more livestock. At the same time, household income may not benefit all members of the household, if for example it is spent on alcohol or gambling. In such circumstances additional consumption indicators may be needed to monitor how household income is being spent. Participatory methods can provide a useful way to understand how household distribution of income and monitor changes over time.
So when designing projects and selecting indicators, here’s a couple of questions that may be worth thinking about:
Do you are have any experiences trying to measure income, wellbeing or assets that you can share?
What do you think are good indicators?
Do you have any examples of good indicators?
We’d be keen to hear from you, so let us know what you think.
Part of my job working on the thematic review has been looking at what we can learn from the Darwin Initiative about the relationships between poverty and biodiversity. This has involved reviewing a lot of project documents and reports, talking to project leaders and collecting evidence through focus group discussions in Kenya. At the moment, we are still making sense of all of the data and writing up the final outputs of this thematic review. Whilst this is all being finalised, there are some lessons that I wanted to share. I don’t think they are necessarily ground-breaking, but can be easily forgotten when trying to design projects that address the dual objectives of biodiversity conservation and poverty alleviation. My plan is to write about one lesson at a time, so this is the first instalment.
Sudanese fisherman Credit Equipe Cousteau
When conducting research we often look for trends or lessons that can be scaled up or generalised across different contexts. Such pursuit can generate valuable insights into the poverty and biodiversity, but at the same time the quest for generalizable trends and lessons must not cloud the lessons we can learn from identifying differences.
…..Differences are important
One of the key success factors in the Kenya projects that we visited as part of the recent evaluation was the speed at which fish stocks were recovering once community-managed closed fishing areas had been implemented. In around 6 months, local fishing communities were noticing differences in both the number of fish, the size of fish, and the diversity of fish species. This observation was also supported by ecological surveys carried out by projects, with support from members of the local communities trained to conduct low-cost ecological surveys.
Fishing in Rodrigues Credit Uni of Newcastle
Edge of Bwindi Impenetrable Forest Protected area, Uganda Credit M Harrison
Yet, one of the reoccurring challenges raised by organisations involved in forest projects is that it takes a long time to see or measure changes in biodiversity, even if restricted forest access has been introduced. Forest projects also describe how the time and technical expertise needed to measure changes is often beyond the scope of a 3 year project.
Contrasting these two examples shows how ecosystem characteristics and dynamics influence the rate at which an ecosystem can recover and therefore what kinds of changes a project can expect to see.
Identifying these differences is not to say that we can’t learn lessons. In fact, I think that there’s a high potential for learning. One option could be to better target events, workshops, newsletters, and learning notes on particular ecosystems and then facilitate a dialogue to identify any reoccurring lessons across ecosystems.
It may seem obvious to point out that the characteristics of marine and forest ecosystems are distinctly different. But this can be overlooked when designing projects and setting ambitious targets when the competition for funding is high. However, projects should be realistic about the types of change they can expect to see and carefully select ecosystem-appropriate indicators in order to capture such change.
This is the first in a series of blogs about lesson-learning, so follow the blog to be notified about the next instalment.
In the meantime send me your thoughts and experiences too – jami-dixon@ltsi.co.uk
Community dancing in Papua New Guinea Credit J Sawyer
I was in recently in Kenya with Lesley, another member of the Darwin team at LTS, conducting field based monitoring and evaluation (M&E). Although I have been to Kenya before, and travelled throughout East Africa, it was my first time to be at the Kenyan coast. The idea was that as well as supporting Lesley and finding out more about M&E in the Darwin Initiative, we could also use some of the findings for the poverty thematic review that we’re working on.
I was in Kenya for 8 days in total, and mainly focused on a mid-term review of one project. As Lesley’s previous blog post mentioned, the main purpose of these mid-term reviews is:
the verification of results
obtaining an independent perspective on the projects
troubleshooting and supporting projects to reorient themselves
In her blog, Lesley talked generally about what we do and why, so following on from that, I thought that it might be interesting to focus on the methodology that we used during the fieldwork. We often think that it’s the findings of M&E that are important, but I think that there’s lots of interesting lessons that we can learn from having open methodological discussions.
A methodology is basically a system of methods that are used in a particular way or area of study. I particularly enjoy developing methodologies, as I think that they help you think practically about how you are going to conduct a piece of research, however big or small it is. For this particular field-based evaluation we decided to use a range of methods to capture different aspects of the project and to suit the different people we were targeting. For consistency, we used the same approach across the different geographical areas the project was operating in.
This methodology, informed by the broader evaluation questions, guided the fieldwork:
Identifying indicators of coral reef health
I was in Kenya specifically to look at the poverty aspects of a marine project. Little did I know that as well as talking to various people the project had been working with, including local communities, local fisheries officials, this would also involve some snorkelling. As someone who isn’t a particular fan of sea creatures (including fish), I was a little bit apprehensive about the snorkelling part. Luckily, I was with Lesley (a marine expert) and representatives of the community (local experts) who made me feel extremely comfortable, whilst at the same time educating me on indicators of marine ecosystem health. Luckily the majority of the time was spent talking to people, so I felt much more in my comfort zone.
Semi-structured interviews with key informants
Semi-structured interviews are an established tool in conducting evaluations. We used specific evaluation questions to develop a set of questions to act as an interview guide. Questions were left open to encourage participants to elaborate on their responses and explore why respondents were giving particular answers. Such an approach also enabled us to probe on particular issues, whilst at the same time allowing participants to lead the conversations.
We started out with semi-structured interviews with project staff to give us a better idea of how the project worked in practice. A couple of days into the evaluation we also talked to a range of local fisheries officials to understand their level of engagement with the project and their perceptions of how they felt the project was contributing.
Semi-structured focus groups with community members
We followed a similar semi-structured process for focus group discussions with community members. Initially, we held meetings with a couple of community groups to verify information about the project, such as when it started, what the main activities were, and also try to understand how both beneficiaries and non-beneficiaries viewed the project. We also used these discussions to identify what the main benefits and challenges had been so far. In some communities we split the community members into smaller groups to encourage participation.
Participatory ranking
Participatory ranking is a commonly used methodology to better understand the range of views. It is a ‘mixed methods’ approach that generates a rich picture of the participant’s views that can be quantified and compared within and between groups, and act as points of discussion for the collection of qualitative information.
Building on what we’d found out in the community meetings, we developed a participatory ranking exercise. Each individual was given three ‘votes’ to identify which, for them, were the greatest benefits the project had brought them. When participants had completed the ranking exercise, we recorded the voting and then asked a series of questions to help us understand why people had voted for certain things.
Theory of change mapping
We concluded the trip with a final meeting with the project team to share what we’d observed in field and also obtain their input into building a theory of change for the project. The idea of building a theory of change was daunting at first, but after a while the staff got to grips with the process and were able to talk animatedly about how they envisaged the project, how this linked to their activities and identify the associated assumptions.
We selected this particular range of methods because we felt they best suited the questions we were asking and the people we were targeting. This methodology provided us with a systematic way of conducting M&E in this context. Of course this is just one approach, and there are a whole range of methods and other participatory tools that we could have used. Was this the best approach? Well that’s open to debate, so let us know what you think.
Want to know more about our findings? Then follow the blog for updates and keep your eyes peeled on twitter as I will be discussing them in future blogs. @Darwin_Defra
Today (22nd May) is International Day for Biological Diversity 2015. This year’s theme, ‘Biodiversity for Sustainable Development’, reflects a broader shift in thinking about biodiversity (the variety of life on earth) and human development as interconnected issues.
Historically, such issues have often been tackled separately, leading to trade-offs between economic, social, and environmental concerns. But the world’s poor, particularly in rural areas, depend on biological resources for many of their basic needs, including clean water, food, fuel, medicine, shelter and transportation. Sometimes efforts to conserve biodiversity have excluded and displaced local communities, denying them access to land and other resources, and therefore undermining poverty reduction efforts. These kinds of examples justify the need for thinking about the links between poverty and biodiversity.
Reducing poverty (often defined as a lack of well-being) isn’t just about providing a source of income, but also includes issues such as health, governance, gender and strengthening people’s capacity to act. Efforts to conserve biodiversity are increasingly tackling such issues too, for example by building capacity through education and training, changing governance structures, and developing financing mechanisms to ensure the equitable distribution of biodiversity costs and benefits.
I’m currently grappling with the complexities of these inter linkages in a review of the Darwin Initiative that I’m leading on. More specifically, we’re looking at how this UK-funded grants scheme has contributed to biodiversity and poverty alleviation.
The Darwin Initiative is a practical example of a programme trying to support both biodiversity and sustainable development. Launched in 1992 to align with the Convention on Biological Diversity, the Darwin Initiative was solely funded by Defra (the Department for Environment and Rural Affairs) until 2010. It is now jointly funded by Defra, the Department for International Development, and the Foreign and Commonwealth Office, with poverty alleviation as an explicit objective. So what progress has been made?
Measuring impact is very high on funders’ agendas at the moment. Partly because it was included in the Research Excellence Framework (2014), which recently assessed the quality of research in UK higher education institutions. But more importantly for development, it provides a way of understanding the difference international aid makes to people’s lives and helps to ensure the best results possible are obtained from the money spent.
However, achieving and measuring impacts on biodiversity and poverty alleviation within the average project timeframe of three years is challenging. That said, one of the questions that keeps coming up is, yes, poverty and biodiversity are interconnected, butis addressing them simultaneously the most effective approach?
At this stage, the evidence from the Darwin Initiative review isn’t finalised, making it difficult to directly quantify and measure ‘what works’. However, it is clear that Darwin Initiative projects use a range of different approaches to contribute to biodiversity and poverty. Some focus on poverty reduction as a means to promote biodiversity, such as the Royal Society for the Protection of Birds’ work in and around the Gola rainforest in Sierra Leone. While others focus on biodiversity benefits for poverty reduction, for example Royal Botanic Gardens, Kew’s project on organic pesticides in Mali. Increasingly projects are integrating approaches to achieve dual objectives, for example, one project in Uganda has demonstrated that building capacity for local communities to conserve forest resources can have biodiversity benefits and local economic and social benefits both in the short and longer term.
Unfortunately it’s not clear which approaches are effective and under what conditions, and the wider literature also offers little evidence; highlighting that we’ve got lots more to learn.
There is an argument that issues of poverty and biodiversity should be tackled separately as they require different approaches and areas of expertise. However, this tends to result in significant trade-offs, with environmental concerns marginalised or excluded. Some feel the best approach is to be aware of these potential trade-offs and find compromises so that the process of achieving multiple objectives can be transparent and balanced.
It’s also important not to discount the potential for synergies in addressing both issues simultaneously. The Darwin Initiative is funding a number of projects exploring the potential for these ‘win-wins’ (for example ZSL’s work with communities on livelihoods and coastal protection in the Philippines and Mozambique). Harnessing the potential for these ’win-win’ situations is both a challenge and opportunity for achieving sustainable development.
The results of the Darwin Initiative review will be coming out in the next few months and will hopefully provide insights into how we can measure the linkages between biodiversity and poverty, and provide examples of when it has been possible to achieve benefits to both.
Over the past week I’ve been going through lots of Darwin Initiative project reports for the review that I’m leading on, which aims to identify how Darwin Initiative projects have contributed to / are contributing to biodiversity conservation and poverty alleviation (read more about this here). I’ve really enjoyed reading the reports and finding out about the range of projects that have been funded, but at times I’ve also found it quite difficult to work out what the projects have actually achieved.
Collecting evidence during fieldwork in Uganda, 2013. Photo credit: M. Wieland
I found it really confusing that reports could contain lots of information, for example about what activities had been undertaken and the partners that they’ve worked with, but when it came to extracting evidence to demonstrate the impact of these activities on poverty and biodiversity, there were lots of gaps.
A lot of the evidence provided in the reports focuses on outputs, for example:
how many workshops have been held,
how many maps have been made,
how many plans have been developed, and
how many publications have resulted from Darwin projects.
All useful information, but is has left me wondering if this is sufficient evidence to demonstrate that projects are having (or likely to have) positive impacts on biodiversity conservation and poverty alleviation?
Talking about what, if anything, has changed potentially requires going beyond reporting numbers of plans, maps, workshops and publications. But is this possible given that projects tend to last around 3 years? Is it realistic to expect changes in biodiversity and/or poverty in this time? If not, does that meant that Darwin Initiative projects are having limited impacts on biodiversity conservation and poverty alleviation?
The good news is that when I discussed these questions with others in the Darwin Initiative team, it became apparent that projects are creating change and do have evidence, but they don’t always include it in their reports. This had me puzzled – why wouldn’t you demonstrate your achievements, i.e. provide evidence, in the reports? I am now interested in exploring the idea that projects have evidence, but that isn’t always documented or reported on. This interest has inspired me to think (and read) about it, and I keep coming back to the same question – what is evidence and what kind of evidence could Darwin Initiative projects provide?
Demonstrating the result of training farmers in Fiji, 2012. Photo credit: Birdlife Fiji
What is evidence?
According to dictionary definitions, information refers to
“knowledge or facts about someone or something”[1], whereas evidence is “the available body of facts or information indicating whether a belief or proposition is true or valid”[2]
Evidence is basically information that proves something, i.e. it contains facts, reasons, proof, confirmation, or verification to demonstrate that something holds true. But how do you decide what information qualifies as evidence?
The first thought that springs to mind is that evidence requires numbers, statistics and/or graphs. These may not always be appropriate for Darwin Initiative projects. Raymond et al., (2010) demonstrate that evidence can be generated in different ways, including formal science (from experiments etc.) to informal locally-held knowledge (from talking to people). It can be produced by looking at a variety of sources, for example from secondary data in peer reviewed journal articles and grey literature, or through collecting primary data using observations and research tools, or a combination thereof. Evidence can be collected and analysed using a range of both quantitative methods (i.e. objective measurements and their statistical, mathematical, or numerical analysis), such as meta-analysis, economic data, and experiments, and qualitative methods (i.e. methods that tend to generate words as data for analysis to provide an in-depth understanding into a social situation) including interviews, focus group discussions, and observations.
Communities participating in a focus group in Uganda, 2015. Photo credit: L. King
The point that I’m trying to highlight is that there are different types of evidence and lots of ways that evidence can be produced. There is generally little agreement on what is the ‘best approach’ in terms of demonstrating a project’s impact on biodiversity conservation and poverty alleviation. More often than not, the approach and the suitability of evidence will depend on the purpose or why it’s being collected/presented and what’s appropriate to the context.
An important first step in selecting an approach is to be aware that there are different approaches each with their own strengths and weaknesses. For example, in the context of poverty alleviation, some changes cannot be easily counted or quantified making it difficult to produce graphs, tables and statistics (Sandbrook, 2013). In this context pictures, videos, and quotes can also provide useful evidence.
I think it would be good to see Darwin Initiative projects make use of these different options available. Providing more guidance to projects on what constitutes evidence and how it can be used effectively may enable projects to better communicate their achievements (and challenges) and demonstrate their impacts on poverty and biodiversity.
What do you think? Could the guidance be stronger on how to produce evidence of your progress and/or impact in terms of poverty and biodiversity? What form could that guidance take – workshops, webinars, YouTube videos, guidance booklets?
Reaching a wide audience – talking on TV in Myanmar, 2012. Photo credit: P. Bates
References for further reading
Raymond, C. M., Fazey, I., Reed, M. S., Stringer, L. C., Robinson, G. M., & Evely, A. C. (2010). Integrating local and scientific knowledge for environmental management. Journal of Environmental Management, 91(8), 1766–77. doi:10.1016/j.jenvman.2010.03.023
Sandbrook, C. (2013). Biodiversity, Ecosystem Services and Poverty Alleviation : What constitutes good evidence ?, (10), 14.
The Darwin Initiative Blog 