01 December 2010

The Sorry State of M&E in Agriculture: Can People Centred Approaches Help?

The context for Monitoring and Evaluation (M&E) is rapidly changing. Two overlapping drivers are challenging M&E: impact and value for money on the one hand, learning and adapting on the other. My sense is that M&E as we currently know it is threadbare. It does not attract enough investment. It is viewed as an enabler of compliance rather than of competence. When it is done well, it is done to satisfy donors, not intended beneficiaries.

A recent IDS Bulletin that I co-edited with Yvonne Pinto, David Bonbright and Johanna Lindstrom has a paper in it that concludes that agriculture is no different and may even be one of the worst offenders. M&E in agriculture is woeful. Why is M&E so weak? The paper provides some evidence to back up the assertions and argues that investment and interest is low because the multiple benefits of good M&E are not identified and when they are, they cannot be captured. The fact that so much M&E goes on undercover allows this situation to persist.

What can be done? In a paper available here, we suggest a new type of M&E is needed, one that is people centred. People centred in the sense that it focuses on wellbeing outcomes, and in the sense that it asks people about what they need and what they think is working. What are some of the components of this approach? It has three.

First, it balances multiple accountabilities through greater participation in programme design and in programme evaluation. The literature on the impacts of these approaches has grown in the past 10 years and shows more successes than failures.

Second, it focuses on enhancing organisational incentives for learning. What needs to change for organisations to engage in single and double loop learning? Beneficiary feedback systems represent one such incentive change, and new donor requirements would provide another.

The third feature of this people-centred M&E is that it seeks to build wider learning about M&E, its users and its providers. The semi-closed nature of M&E is killing learning about what works. We need to find ways to let more light into the system.

ALINe, a collaboration between IDS, Keystone Accountability and the Bill and Melinda Gates Foundation is one new contribution to the construction of this different view of M&E. It builds on other initiatives such as ILAC and Farmer First Revisited. It is helping farmers get engaged in theory of change discussions, it is evaluating farmer feedback mechanisms in terms of process outcomes and development and wellbeing outcomes, it is analysing organisational incentives for M&E use and it is seeking to open up the M&E sector by promoting the open access of data and new research. We would welcome additional partners.

I believe that M&E in agriculture has to be improved -- this paper has some ideas about how to do it. There are lots of other very good papers in the Bulletin on this and similar issues.

If M&E in agriculture is not improved then we will have wasted the political opportunity represented by the current high interest in food and agriculture. We will have no excuses when the budget axe is eventually aimed at food and agriculture and we will have failed to meet our obligations to the current and future generations of hungry and malnourished people.


Anonymous said...

I work in community conservation not agriculture, but here I see various problems with M&E:

- Local staff are not always big fans of constructive criticism or capable of self-analysis. For M&E to work properly project staff should be active, willing participants, who want to find ways to improve.

- M&E is either highly technical (requiring sophisticated data analysis techniques) or highly subjective. For different reasons these both tend to lean towards a high reliance on expensive, international consultants.

- M&E can be really difficult to get right. You can collect a lot of data which don't really tell you very much.

- Thus M&E can be really expensive, hence it is only natural for project managers to favour spending money on delivering other outputs over the uncertainty of M&E. I'm not anti-M&E but reckon I might have fallen into this trap in my time.

- Finally, the nature of academic publishing and the tendency of any project implementing agency to emphasise successes means that failures are rarely fully documented or explored. Maybe if they were we would have a few less dud projects out there, tho I think that has more to the incentives on donors to push money out the door than anything else.


Owen Barder said...

"The semi-closed nature of M&E is killing learning about what works. We need to find ways to let more light into the system."

Lawrence - I thought this was a little ironic, when you are posting about articles in a development journal which is only open to paid subscribers, even though most of your funding comes from the public purse. Perhaps you could let a little light into the system?


Lawrence said...

Owen, you are right to call me on this. The paper is uploaded on Scribd now. The Bulletin has to cover its costs...it becomes free after a year (I think) and we have lots of deals with hundreds of Southern University Libraries to make it free to access right from day one. Best, L.

Nick Perkins said...

Hi Owen

Thanks for the comment.
The Bulletin is open access on a 4 year rolling window - which means that articles published before 2007are freely availabe online at the moment. (Next year this will shift to 2008 and so on.) We also regularly make selected articles within the current year freely available where we believe they can make a significant impact on a live debate. In addition the Bulletin is freely available from date of publishing to several thousand institutions in the global south through philanthropic initiatives we have joined like HANARI and AGORA. IDS also has an exchange programme with selected institutions which means that they too receive free copies of the Bulletin. The truth is that while we recieve funding from the public purse for specific research projects, it does not cover core functions like our publications programme and certainly not the Bulletin so we are forced to think of business models that deliver public goods sustainably. Nick

Anonymous said...

I'm no expert in M&E, so I'm not able to comment on proposals for improving these instruments / processes in agriculture... But I would question the interests behind the collaboration of the Bill and Melinda Gates Foundation in this IDS project.

This Foundation acts against the interests of small-scale farmers, actively promoting the use of genetically-engineered (GE) seed and agrochemicals (particularly those of Monsanto, in whose shares the Foundation invested over U$23 million in 2010).

To quote a recent Vía Campesina declaration (http://viacampesina.org/en/index.php?option=com_content&view=article&id=943:la-via-campesina-denounces-gates-foundation-purchase-of-monsanto-company-shares&catid=49:stop-transnational-corporations&Itemid=76): "The Bill & Melinda Gates Foundation was founded in 1994 by Microsoft founder William H. Gates, and today exerts a hegemonic influence on global agricultural development policy. The Foundation channels hundreds of millions of dollars into projects that encourage peasants and farmers to use Monsanto’s genetically-engineered (GE) seed and agrochemicals. In August the Bill & Melinda Gates Foundation Trust, which manages the $33.5 billion asset trust endowment that funds the Foundation’s philanthropic projects (and to which Bill & Melinda are trustees) disclosed that it purchased 500,000 shares of Monsanto shares for just over $23 million.(1) According to Dena Hoff, a diversified family farmer in Glendive, Montana and North American coordinator of La Via Campesina, “The Bill & Melinda Gates Foundation Trust’s purchase of Monsanto shares indicates that the Gates Foundation’s interest in promoting the company’s seed is less about philanthropy than about profit-making. The Foundation is helping to open new markets for Monsanto, which is already the largest seed company in the world.”"

(please continue below)

Celia said...

(continued from above...)

If we believe that M&E – or indeed international development and its related projects in general – should be people centred, then the voices, struggles and alternatives of the excluded (such as small-scale famers struggling for land reform and against transnational agribusiness) need to be given priority. And if this is to become a reality, and if we are working to truly transform the world in which we live (including taking the control of agriculture out of the hands of a few transnational corporations – such as Monsanto – whose ‘raison d’être’ is profit making through the exploitation of people and the environment, rather than reducing hunger), then I believe we need to be wiser about who we chose to collaborate with. It is not a Foundation who actively supports agribusiness that will change the lives of peasant farmers around the world, with or without improved M&E…

Celia Alldridge
MA Gender and Development
Volunteer for 4 years with the Landless Rural Workers Movement (MST), Brazil

Lawrence said...

Hi there and thanks for these comments

...a couple of comments have questioned why we would have worked with the Foundation on this. Well, it was the first time we had worked with them and we wanted to see how serious they were about giving small scale farmers priority.

The Monsanto story broke 3 months ago,and I think its a shame that the Foundation has not resonded to the question in the Guardian about why they made the investments.

But the real story is that agricultural investments, made by a wide range of agencies and bodies, are very poorly assessed. My guess is that this will be turned around if farmers can be supported to have real agency in influencing upstream and downstream choices.

Joshua Smith said...

Nice to read such reviews. Take into consideration auto insurance by zip code from top insurers.