21 January 2012

Do Indices Change Anything?

With so many indexes out there, whether they change behaviour in those collecting, reporting or reading them seems like an obvious question to ask. Thirty minutes on Google Scholar does not reveal anything.

I have indices on the brain this week. On Monday I was at the Danish Ministry of Foreign Affairs making a presentation on hunger, food security and agriculture. This is going to be a new priority for the new Minister for International Development. I presented the Hunger Reduction Commitment Index (HRCI) that an IDS team has constructed with support from Irish Aid. The HRCI shows Denmark at number 1 in the donor countries. This went down well, but raised the question: if Denmark is doing so well, why the new priority?

Then on Tues-Weds there was a meeting at FAO on measuring hunger outcomes, using various indices--my IDS colleagues Edoardo Masset and Stephen Devereux were in attendance. I hope some progress has been made in measuring hunger properly since the last such meeting in 2002 and that even more progress will be made in the next 10 years.

Then on Thursday we had the Global Think Tank Index from the University of Pennsylvania. The index places IDS at 8th in the world in terms of think tanks on international development (2nd in UK behind ODI) and 2nd in the UK in all topics for a University affiliated think tank (although we are independently governed). Of course I would like to be higher than 8th, but last year we weren't even on the list (because no one thought to nominate us, presumably thinking that someone else would --you cannot nominee yourself) so that is an improvement.

Finally on Thursday and Friday we hosted a DFID-IDS Learning Event on Nutrition Governance, showcasing 6 country case studies. Each case study looked at the incentives, barriers and tradeoffs behind the structures and frameworks that are supposed to help coordinate nutrition actions across sectors and across levels of government. We noted the vast difference in governance effectiveness behind the "good governance" scores of the WHO Landscape indicators which track whether such structures are in place.

In all of these examples, the index in question has proven to be a lightning rod for discussion. Sometimes this can be a distraction, but most times it serves as a kick off point for discussion and debate about the index, but also about the issue.

Nevertheless, it does seem that there is a need for rigorous evaluations of indices used--do they spur effective action? This is something that IDS will work with Irish Aid and DFID on in evaluating the HRCI and a related but distinct nutrition commitment index.

If you know of any studies we can draw inspiration from, please let me know. Thanks.
Enhanced by Zemanta


Richard Manning said...


In a way, I was aiming at the same target with my 2009 report for DIIS and DFID on 'Do Indicators Change Behaviour? The Case of the MDGs'.


It's the unspoken logic of the whole post-2015 debate and of the Sarkozy Commission that the way things are counted can indeed change behaviour. I found in 2008/09 that while the MDGs had had a large impact on discourse it was less obvious that they had affected donor behaviour and still less behaviour of actors at country level. Three years on, I am inclined to be a little more positive, but testing the hypothesis is hard (I noted that there is an MDG on HIV/AIDS and that funding for AIDS had risen sharply - but in particular from the Bush Administration, which basically objected to the MDG construct).

So better evidence in this area would be really useful.

Austin said...

Indices as far as I know are descriptive points for scaling a particular behavior. It is what the evaluator does to the results of the measurement using indices that will dictate whether there will be a change in behavior. For instance, a company measures the compliance of employees in wearing their ID. Aside from identification purposes, it doesn't serve any purpose. As such, most employees will not use it. That being the result of the study, the company used custom lanyards to promote wearing of the ID. This behavior is also observable with students.

Kerena Tosetti said...

Indices are like a collection of measuring sticks. In teaching, these are an aggregate of qualities that you would like to see before proceeding to the next level of teaching.