27 March 2013

The new IDS-ITAD Centre for Development Impact

This week IDS and ITAD launched our new joint initiative, the Centre for Development Impact (CDI). 

CDI is a response to the growing interest in impact evaluation in development and a rather cookie-cutter approach to it. 

CDI will be driven by a need to make sure context, questions and capacities drive choice of methods.  It is also dedicated to expanding the tool box of methods, critically evaluating established and new methods as we go along.  

The current focus is on:
  • Exploring a wide range of evaluation designs and methods, including complexity theory, systems thinking and different approaches to causal inference, in order to understand what works, for whom, why, and in what context.
  • Designing appropriate methodologies for evaluating complex interventions in challenging contexts, such as: interventions in emergent, dynamic and uncertain situations (such as conflict), those that are multi-sectoral in nature (like nutrition or empowerment), and those not readily amenable to counterfactual construction (such as value chains).
  • Better understanding the realities of the evaluation process, such as when evaluations are conducted under severe resource (or other) constraints, the politics of evaluation, and the use of evidence.
CDI will aim to do this by: (1) Doing evaluations that provide scope for innovation in a particular domain; (2) Doing research on evaluations, including meta-research and meta-evaluations; and, (3) Developing and delivering training and learning programmes.

The CDI launch was at this week's Impact, Innovation and Learning event at IDS where we are live-streaming keynote speakers and panel sessions.

Part of the launch of the new centre also includes the launch of a new Practice Paper series – the first of which is Cathy Shutt and Rosie McGee’s Improving the Evaluability of INGO Empowerment and Accountability Programmes.  

Also happening is the ‘release’ of the Ghana Millennium Villages IE project Initial Design Document – which IDS is involved in and which has gone through a number of iterations and peer review processes.

We aim to connect CDI with the Guardian’s Global Development Professionals Hub on Impact and Effectiveness and other monitoring, learning and evaluation communities of practice. 

I hope you will find the work of CDI interesting and useful.

25 March 2013

What matters most? Evidence from 84 participatory studies with those living with extreme poverty and marginalisation

Participate is a DFID-supported initiative, co-convened by Beyond 2015 and IDS.

Amongst other things, The initiative seeks to bring the perspectives of those in poverty into the decision making process around the post 2015 settlement.

The initiative has been going for about 5 months. They have just published a synthesis report of 84 existing participatory studies from 107 countries. (They are also doing primary research in 14 or so countries). The aim is to feed these perspectives into the post 2015 process, with a key focus on the upcoming UN High Level Panel meeting in Bali.

Each of the 84 studies is focused on different experiences of poverty and exclusion and they use a variety of different participatory methods.  The methodology could be clearer (a methods paper is promised) but the studies are selected based on (1) participatory methodological credibility, (2) whether the research is embedded in a longer term relationship, (3) whether the research is multi country and multi-site to avoid one-offs and (4) the research is with groups that are marginal, vulnerable or excluded. 

The study's findings:
  1. Barriers to access for the very poorest need to be challenged directly in order to ensure equity and bring people out of extreme povertyInterventions that improve the lives of the poor frequently fail to benefit the very poor. Implications:
    • improve information about rights and services
    • work to identify and reduce hidden costs to access 
    • work hard to address institutional discrimination
  2. Rapid change, insecurity and uncertainty increasingly characterise the environments that the very poorest live in. Design processes, services and infrastructure which have the capacity to adapt to rapid change and help enhance the ability of the poorest to respond to risk and protect their rights. Implication:
    • rates of change are accelerating; development agencies-domestic and international need to get better at operating in dynamic contexts
  3. Promote better governance based on values of accountability, trust, access to informationresponseiveness and effectiveness-values that can best be achieved through citizen participation and influence in decision making.  Poor governance is widely seen as a major contributor to poverty. Implications:
    • the rules of the game really matter--formal and informal, national and subnational
  4. Address social inequalities: this means dismantling intolerances and prejudices that discriminate, marginalise and exclude at all levels and in all settings. Inequalities are persistent and perpetuate exclusion at all levels of development. Implication:
    • inequalities lead to hopelessness and negative self perceptions--these are rarely taken into consideration in development policy 
  5. In order for development to be sustainable, policies must address the root causes of poverty and marginalisation. Current policies and approaches can lead to unintended and perverse effects, which make exclusion and poverty worse. Implication: 
    • sustainable development means investing in long term relationships that build local ownership and local knowledge
  6. Participatory development should be at the heart of interventions implemented through the new framework. Development that is sustainable requires meaningful participation that leads to strong local ownership. 
    • how development occurs is just as important as what occurs vital--it builds confidence, dignity and a recognition of abilities
The study's key recommendations for the Post 2015 Framework

  • Aim for the eradication of extreme poverty and reduction in inequalities.
  • Strengthen the individual and collective capacities of people living in greatest poverty and marginalisation.
  • Participation should be prioritised throughout the framework. 
My reflections
  • it is not surprising that when you survey a set of participatory studies that the conclusion is that participatory development to be at the heart of sustainable development
  • but the value of the participatory approach is clear as it shines a light on:
    • a focus on the how --it is just as important as the what and why
    • the psychosocial aspects of development that we spend so much time on in the Western world are given scant attention in the developing world
    • the accelerating speed of change is typically an opportunity for people with resources but for people with fewer resources, it makes managing those resources more difficult
    • the institutions of the state often codify (sometimes deliberately) discrimination that only the discriminated against can discern
    • the realisation that access to information is not something only the rest of us crave
    • the costs to participation are significant and hidden to the rest of us 
I am biased, but well done to Participate.  I really look forward to the primary field work to what value it adds over this synthesis work. 

23 March 2013

The Late Great Nevin Scrimshaw

Last month Nevin Scrimshaw passed away. He was 95.

He was a giant, perhaps THE giant, in the field of nutrition.  His achievements are myriad.

What I admired about him:

* his ability to marry research and action (he invented post weaning foods that were healthy and used local foods--such as Incaparina);

*  his commitment to founding and building institutions (INCAP in Central America, the nutrition department at MIT, the Food and Nutrition Bulletin journal at the UN University),

* his intellectual drive (he is credited with the acceptance of the idea that nutrition and infection interact)

* his lack of dogma (I didn't already know what he was going to say when he spoke up in a meeting).

Here are a couple of nice pieces on his life--one from the New York Times and one from Alok Bhargava, a highly respected economist working in this field, writing for Economics and Human Biology.

Most lovely of all--Nevin's was not an ego that had to be handled with extreme care. He was a gentleman and always had time for a young whippersnapper.

He was an inspiration to millions and he will be missed.

21 March 2013

Where should nutrition position itself in the post 2015 settlement?

Last week I was at an EC/Scaling Up Nutrition meeting presenting on the above topic. The SUN meeting was timed to coincide with the announcement of a sensible new EU nutrition policy focused on stunting reduction.

My take was that in a world where the High Level Panel has received nearly a 100 proposals for different Goals (including one on plankton--as a measure of the health of oceans), that nutrition was not going to get a stand alone goal.

I suggested the following:
  • A separate nutrition goal, while attractive to nutrition community, is a nonstarter
  • Replacing underweight with stunting is a step forward, but not nearly enough
  • Bracket hunger and nutrition together as equal--but different– partners
  • Beyond this vertical “home” for nutrition, create a “horizontal” nutrition goal with nutrition indicators strategically placed throughout the different goals
  • Number of nutrition specific indicators in new framework likely to be small, but worth pushing for 5-6
  • Important to push for nutrition-sensitive indicators too
The powerpoints are here.

15 March 2013

The consequences of undernutrition and the price of a short skinny latte

Here is a link to an article I did for the Guardian this week on the consequences of undernutrition (one of their readers had requested it).

Incidentally, I'm at the Scaling Up Nutrition meeting in Brussels right now. One of the discussions is on some work that is looking at costed country plans for scaling up nutrition in 16 countries.

The costs are coming out at $4-5/per person per year. The estimates are rough, preliminary and subject to change, but this is about the same price as a short skinny latte at your local European and North American coffee shop.

The price of one short skinny latte per year to prevent a large proportion of babies and infants from becoming short and skinny. Do I sense a campaign coming on? 

14 March 2013

To MOOC or not to MOOC? That is the question (and no, it won't be on the test)

MOOC is not yet a verb (surely only a matter of time). Rather it is an acronym for Massive Open Online Courses.

These are courses, offered free, with numbers of viewers in the tens of thousands. They are delivered typically by video and have online discussion boards and blogs to critique, expand and analyse. Platforms for MOOCs include Coursera, FutureLearn and edX.

We were lucky enough to have Jonathan Kydd, an IDS alumnus and now Dean of International Programmes at the University of London, come to talk to us this week about MOOCs.

They have a number of advantages for participants, including:

1. They are usually high quality (who wants to put their mediocre stuff out there for all to see?). Coursera's bold claim is "Take the World's Best Courses, Online, For Free"
2. They are made for social media-they can be embedded, tweeted, and shared easily --and then discussed.  They are thought to lead to the building of knowledge networks
3. They can be participatory--with lots of feedback given on provided presentations and new additions to the learning materials provided by participants
4. They are a relatively low cost way (they are not free if they are using your time) of checking out a potential University or set of faculty
5. They help participants fit their learning in around other parts of their lives

They are not a substitute for a face to face set of interactions on a campus, but they are a complement (pre or post a traditional degree) and possibly an alternative. There are high drop out rates, but then the number of participants taking a final test for a certificate is very high compared to a traditional course.

Why on earth would universities or faculty want to give this stuff away for free?

1. It profiles the academics and the Universities. Look at what we do and how good we are at it.  This is helpful to raise awareness and attract students, faculty and funding opportunities. Some observers think that the number of views of your MOOC might become an important element of the way academics and universities are evaluated for promotion and funding. (Would a MOOC viewed by 100,000 people contribute well to a REF Impact score?)
2. It raises the pedagogic standards--within the university (so that's how she/he does it!)
3. It helps spread ideas and viewpoints from the host university (meme anyone?)
4. It creates a wider learning network for your students who are perhaps also taking this course as one in 20 courses for a full time accredited degree
5. It is a way of getting in on the ground floor of something that might end up being really big (a disruptive technology for higher education, as some say).

There are sceptics (see this Financial Times article from this week). Points include:

  • MOOCS do not lead to greater employability -- still the key motive for first degrees
  • technology/bandwidth is too big a barrier in many countries
  • it is just froth--we will have forgotten about it in 18 months
  • mass online=dumbed down material through new suppliers coming in with cheaper material
  • it does not democratise learning, it concentrates content in the hands of the powerful
  • it takes too many resources to set up a course (some estimate £25k)
  • it perpetuates an American way of viewing the world (because so many of the first movers are American elite universities)
  • they will not make money
  • they will lead to stealing of intellectual property and rampant plagiarism

All valid concerns, but all--in principle-manageable, it seems to me. Indeed, many have managed it. And Open University have been delivering related models for a long time now (they are behind FutureLearn).

There is even a growing research programme around MOOCS (see an interesting example here).

Higher Education is changing fast (see an IPPR report published this week too). Like any new technology, MOOCs surely have the potential to be used for good teaching and learning. 

Are they being used for teaching and learning in development? Have you MOOCed yet?  

12 March 2013

ATNI: A new metric of how pro-nutrition the big food and beverage companies actually are

I am a fan of indices.  Admittedly there is not much research to show whether they do or do not change anything.  But in the absence of such evidence, my priors are that they do.

So I really welcome the Access to Nutrition Index from GAIN, supported by the Bill and Melinda Gates Foundation and the Wellcome Trust, and released today.  The research was done by MSCI Environmental, Societal and Governance research, a company that guides investors who place a special emphasis on these kinds of dimensions.

ATNI ranks 25 major food and beverage (non-alcoholic) companies on a range of indicator categories (weights of each category give in brackets):

  • Governance (12.5%) Corporate strategy, governance and management
  • Products (25%) Formulation of appropriate products
  • Accessibility (20%) Delivery of affordable, available 
products
  • Marketing (20%) Responsible marketing policies, 
compliance and spending
  • Lifestyles (2.5%) Support for healthy diets and active lifestyles

  • Labeling (15%) Informative labeling and appropriate use of health and nutrition claims

  • Engagement (5%) Engagement with policymakers and other stakeholders

Findings:

1. On a 10 point scale the majority of companies scored below 3.  The top 3 companies (Danone, Unilever and Nestle) scored between 6 and 7.  Clearly room for massive improvement.

2.  The companies that are breast milk substitute manufacturers are designated by a red flag in the rankings. Three of  5 these companies score in the top 8. The report commends Danone and Nestle for their relatively high overall mark, but it says their "reported lack of compliance with the International Code of Marketing of Breast Milk remains a significant concern".

3.  Companies’ practices often do not measure up to their commitments. Broadly, companies’ scores on nutrition strategy and governance were higher than their scores on product formulation, accessibility, and marketing. And within each of these areas, their level of implementation lagged behind their stated commitments.

4.   Many companies are not very transparent about their nutrition practices. This lack of disclosure limits the ability to understand the full scope of companies’ nutrition-related efforts.

Reactions?

1. The methodology seems sound (but only a preliminary view--I have only had the report for a day).   The methodology process has tried to be inclusive.  The scoring has to rely on a lot of self reported data, but tries hard to get independent verification. I like the weighting which gives greater emphasis to what companies actually do, rather than what they say. I also like the fact that companies who do not cooperate or are not transparent with their data receive lower scores.  

2. Some would say that it is difficult to reconcile breast milk substitute (BMS) companies such as Danone getting high overall scores given violations of the Code.  I also think this is incongruous--one could argue that violations at this stage are so consequential that they negate whatever is done thereafter.  I would give this much higher weight in later versions although I realise the technical difficulties given that essentially there is a mixing of apples and oranges.  Perhaps there should be a separate ranking for BMS firms. 

3. ATNI constructs 2 sub indices: one on overweight and obesity and one on undernutrition.  While the rankings of companies does not change much, the scores for  undernutrition are much lower.  The report argues that this is because companies' undernutrition efforts are more philanthropic and not yet incorporated into core business. This accords with my own non-systematic experience and remains a massive challenge.

4. What features of the companies drive rankings?  Size? Nationality? Market share? Corporate governance?  History?  We only have 25 observations so statistical analysis is challenging, but some correlation work might throw up some interesting hypotheses to test in future case-studies.   

5.  Are there conflicts of interest?  GAIN works with many of these companies as do MSCI.  I am not too concerned about this, because the process seems transparent and the funding is from two third parties who would suffer greatly if they were seen to fund anything other than a process that aimed to be unbiased.  Many organisations also receive funding from governments to rank government performance (e.g. CGD, ODI and IDS) and so this is a wider challenge. I should note that I am a member of GAIN's Partnership Council, so I also have a potential conflict of interest. 

6. Finally, will this index change anything? Do any indices change anything?  The people who generate these indices tend to say they are highlighting best practice, and not naming and shaming.  Personally I think there is nothing wrong with doing both.  IDS is developing the Hunger and Nutrition Commitment Index (HANCI) which assesses the commitments of high burden countries to hunger and malnutrition reduction.  We are thinking hard about how to evaluate the indicator in terms of change that it might facilitate and we will be using regression methods and country case studies to do this assessment. 

So while not perfect, and while it will no doubt improve in later versions, I welcome ATNI and congratulate GAIN, Gates, Wellcome and MSCI for doing something that is not easy--technically or politically.   

11 March 2013

At the Kumbh Mela: A Holy dip for worldly aspirations or nirvana?


In response to the New Spiritualism blog of last week, here is a guest blog from Biraj Swain.

The Kumbh Mela drew to a close yesterday. And official figures claim 5.5 million took a holy dip[1] on its last day. It has been dubbed as the largest religious congregation on earth and it has lived up to its reputation in every possible way. The 55-day religious extravaganza in Allahabad, India, is estimated to have attracted over 130 million pilgrims over the duration of the festival. And it is a global logistical challenge for any million-plus city handling those numbers without any disease outbreak. Barring the unfortunate stampede on 11th February, it has been a successful event and now the administrators can take a breather!

I marvel at the number of urban professionals flocking to the festival. And most of them are in their thirties too (me included). It isn’t bragging points, or just spiritual adventurism. Is this a recent phenomena or are there demographic data-sets of pilgrim profile and disaggregations to show us the trend? The over-present 24X7 news media beaming images of Kumbh mela into our living rooms could be one factor but not the only one. Mid-career blues? Is Kumbh also the career counselling destination where spiritualism opens our inner eyes to our Myers-Briggs[2] self-scores / management aptitude? Why is Kumbh becoming the destination for so many working professionals in these times of recession and high food inflation?

Any conversation reveals that not everyone goes to Kumbh to get emancipation from the worldly cycles of life and death. Nirvana is not on their mind while taking those holy dips in chilly waters. Many go for purely worldly desires, professional aspirations, career goals, family well-being et al (me included again). That we are living in a country of high economic growth and persistently high levels of hunger and malnutrition is the perfect enigma spark to seek out the Gods. The job loss in agriculture and non-agriculture sector has been huge as per the recent Economic Survey and casts the long shadow of uncertainty and powerlessness. Only faith can help and prayers can redeem say the Gurus.

The contradiction of seeing the super-efficient state (by Indian standards) perform without a hitch and organise the mammoth festival with all its challenges, logisitic and administrative while it fails so miserably in its everyday routine administration and deliverance of welfare is also another enigma on display. One cannot escape the attack of rationality in the face of such contradictions while invoking the inner spiritual power.

At the confluence of the three holiest rivers of Hindu mythology (Ganga, Yamuna and the invisible Saraswati), as the pilgrims stream in for their last holy dip in this Maha Kumbh and the wait for the next Kumbh after 12 years begin, the benevolence of the state is also in full display. 

Those pilgrims who follow the ritual of Kalpvaas (taking a single meal over an elongated period as per Hindu calendar to achieve maximum piety), the food, fuel and essentials are supplied by the state. The Public Distribution System, India’s principal food subsidy programme is also alive and kicking with over 120 shops open in the festival area alone.

The debated cash transfer scheme hasn’t yet made an appearance, unless, we consider the offerings from pilgrims to the priests as the ultimate cash transfer. 

Faith, development, politics, economy and the state do get inter-twined in more ways than one but rationality and enquiry also raise questions again and again.

As Lawrence Haddad[3] in an earlier blog discussed the rise of new spiritualism, one is left wondering, new, old or just spiritualism? For many, Kumbh Mela is also about our current state of being in the times when there is withdrawal of the state in many facets and its over bearing presence in many more. Attempting to make sense of that contradiction with a holy dip at the Maha Kumbh............
-Biraj Swain[4]


[4] Biraj Swain is an independent researcher working on the inter-sectionality of food, nutrition and agriculture policy and the citizens-state interface. She is also an over-enthusiastic pilgrim and her pilgrimages aren’t restricted to Hindu festivals alone.

07 March 2013

Finance and Regulation for Good Nutrition: Two Recent NGO Reports

The nutrition community is, I think, uniquely blessed with some very good NGO research and evidence gathering. Perhaps the NGO work is filling a gap that the UN and research organisations have inadvertently created, I don’t know, but I am continually impressed by their work.

Two recent reports are good examples of the value they add.

First, there is a report by ACF International on the financing of nutrition. It is a short Information Note on Aid for Nutrition. It builds on some work that IDS and others contributed to. The thing I really like about this note is the exceptional clarity it brings to a topic that is not a natural fit with the nutrition community. The crux of the note is a table ranking 5 different financing mechanisms (nutritional impact bonds, matched funds, financial transactions tax, high fat, sugar and salt tax and solidarity lottery against undernutrition) by 5 different criteria (additionality, predictability, feasibility, economic efficiency and nutritional impact). The 5 financing mechanisms are clearly explained (and that is not easy), the criteria are the right ones, and the ratings, while essentially guesses based on a weak evidence base, are intelligent guesses, transparently explained.

The second report is by Save the Children UK on Superfood for Babies: Overcoming barriers to breastfeeding. In terms of the list of priorities for the London pre-8 meeting on nutrition on June 8, breastfeeding has to be near the top. Failure to exclusively breastfeed is an invitation to increase the relative risk of death due to diarrhea and pneumonia by 10-15 times. The Save report describes exclusive breastfeeding for the first 6 months of a baby’s life as the closest thing to a silver bullet in the fight against malnutrition and newborn deaths. Like all silver bullet analogies, if only it were that simple. As the report itself describes there are many barriers to exclusive breastfeeding.

The most interesting chapter in the Save report was the one on the conflict of interest the infant formula industry faces: it’s main competitor, breast milk, is free and superior and so there is a commercial interest in reducing its provision. The question is how to incentivise countervailing behaviour? The International Code of Short of Marketing of Breast milk Substitutes adopted by the UN’s World Health Assembly, is not legally binding, although it is increasingly being enshrined in national law (Kenya, Vietnam and South Africa have all done so in 2012). When governments have the capacity to hold powerful multinationals to account, the Code is clearly very helpful. But the power imbalances in resources and information make enforcement, even of legislation, a constant challenge and there is constant testing of boundaries by the companies and documentation of frequent violations. The descriptions of the violations are the best I have seen because they try to find evidence from a number of sources and they highlight good practice. And solutions are proposed—for example combining the standards of the Code in a FTSE4GOOD-type ethical investment index approach.

In this year of Nutrition2013 it feels good to be working in an environment where our civil society is not just advocating for more to be done, but they are thoughtfully highlighting problems and proposing solutions.

05 March 2013

Climate change: why is learning and acting so hard?


Today IDS welcomed 80 participants to a learning event on how we learn and act on that learning in climate policy. 

The event was organised in partnership with the GEF Evaluation Office, CDKN, and the Climate Change, Agriculture and Food Security (CCAFS) programme of the CGIAR. 

So what makes learning so difficult in the climate change and development space?

  1. The often long wave nature of the gap between impact and action. As Sam Knight writes in Prospect this month, this tempted us to try hope, then denial, then delay. He says now it is time to face reality (although what will actually force us to do this is not clear to me). Clearly it is a challenge to communicate urgency to those who fail to see it. Intergenerational justice is not exactly a vote winner, yet.
  2. The multidisciplinary and multisectoral nature of climate change fragments alliances or prevents alliances forming in the first place. Effective knowledge generation for action requires alliances across science and social science, those who hold expert and local knowledge and across public and private actors.  Not to mention multilateralism. The high coordination demands require lots of effort. 
  3. There is a lot is at stake beyond climate change.  Powerful business interests are challenged.  This makes the whole debate even more politicised. 
  4. There is not that much evidence in the social science space--the methodological challenges are huge in relatively slow moving processes and the construction of counterfactuals is always model based and models are always contested. This is a problem for evidence-based culture ‘what works, where and for whom’?  As one of my IDS colleagues said to me recently, this means we need to rely on ‘what is, what has been and what is likely to be’ – but of course this is also highly contested
What about the challenges of acting on knowledge?
  1. Is the learning and evidence actionable? Policymakers need help from the researchers and vice versa.  What does it mean to a policymaker to make resilience more pro-poor? What questions are key from the policy person's perspective?
  2. Is implementation capacity too fragmented to act on knowledge? My climate change colleagues at IDS have noted the fragmentation across communities in disaster risk reduction, climate change adaptation and social protection.  Different but highly related fields that do not readily act in unison. 
  3. Is action being picked up by M&E systems? What is caused by climate change and what is not?  M&E is challenging enough--in climate change it needs to be even better.  
  4. Can the evidence be framed in ways that reflect capacity and political opportunity? How can we be political about the use of evidence and knowledge without being political about its generation?  A generic question, but critical in climate change and development, it seems to me. 
So, it is really appropriate to reflect on how we are and aren’t learning and the barriers and enablers to effective action.

The learning event very much reflects the USP of the climate and development work of IDS: convening broad networks to support these alliances, a focus on the political economy of climate change action and a focus on the politics of evidence--whose knowledge counts?