21 December 2012

UK and Australian Attitudes to Aid: A tale of two countries

"It's a Policy Knockout" shouts a Prospect magazine article by Peter Kellner, president of YouGov, the polling firm, in the January 2013 edition.

YouGov conducted a poll for Prospect on the policy choices the UK public most want.

Instead of ranking the 16 options as a list, the 16 were set up in a Champions League style knockout tournament.

"Cut overseas aid" was one of the 16 options put forward by YouGov. In the round of sixteen, Cut ODA was more popular than "tax £2m properties". In the quarterfinals it was more popular than "end all immigration" (quite an feat--see the AusAID story below).  In the semis it was more popular than "improve the NHS" (our national health service, which could do with some improving). So "cut overseas aid" made it to the final, but was beaten by "crack down on firms that avoid tax" which was the most popular of all policy choices (at least ODA is more popular than Google and Amazon--hang on, that's not saying much).

As the article notes, the options that were more popular not only reflected their innate desirability to those polled but also the perceptions of the feasibility of politiicans actually doing something about them.

And while--apologies to Chelsea fans--cup competitions like the Champions League have a lot of luck attached to them, the results above deserve serious attention.

In the same edition of the Prospect there are two letters relating to articles in previous editions. One letter is from former DFID Secretary of State Clare Short, defending the MDGs against an "attack" from Clare Lockhart. Another letter is from Andrew Christensen agreeing with an article by Ian Birrell on "aid being a poor answer to poverty". As we are on the verge of a step increase in the UK aid budget (at least in terms of the percent of GNI) this is perhaps not surprising. As usual, the evidence base is absent from most of these arguments (although not in Clare Short's letter).

I would really like to see a big picture view of what all the systemic reviews from the past 3 years have done to support or undermine the "aid effectiveness" picture. Can someone work with DFID, 3ie and AusAID to tell us?

And even if the evidence was clear and rigorous, it would still be contested--we need to rethink how we communicate when aid is and is not effective.

Talking of AusAID, Australians are also angry about ODA. But not because it is too high, rather because it has been "cut".

I put cut in quotes because technically the aid budget is intact (although an increase for next FY was postponed back in May) but rather because a chunk of it has been reallocated. Specifically 375m Australian dollars (out of about 5 billion) of ODA has just been allocated to meet the needs of refugees to Australia, in Australia. The opposition are somewhat gleefully describing Australia as the third largest recipient of AusAID, behind Indonesia and PNG. Backbenchers on the centre left are not happy either.

The Government is claiming that this use of ODA is in line with OECD DAC definitions. And indeed it is, the DAC stating

"Assistance to refugees in developing countries is reportable as ODA. Temporary assistance to refugees from developing countries arriving in donor countries is reportable as ODA during the first 12 months of stay, and all costs associated with eventual repatriation to the developing country of origin are also reportable."

The Australian NGO community is not happy (see a joint editorial from World Vision Australia and Caritas Australia) and the centre right opposition is making the most of it (the Government can't keep its word and made cynical promises on aid to secure the Security Council seat etc.). And while Australian aid has been growing rapidly, as a percent of GNI is it still in the bottom half of the DAC league table. 

My only comment on this is the irony of it all. While aid is so political in the countries of origin, we treat it as apolitical in the countries of destination. 

Apparently something magically happens to it during the transfer.   

19 December 2012

What do we want? Nutrition sensitive agriculture! How do we incentivise it? No clue

The past month has seen three important publications on nutrition-sensitive agriculture.

First, a paper from Anna Herforth, Andrew Jones and Per Pinstrup Andersen, commissioned by the World Bank. This paper develops 8 "Guiding Principles for Operational Investments" (pdf) for prioritising nutrition in agriculture and rural development. There are many sensible things in here (not surprising given the calibre of the authors) and it is a good menu of what to do if you want to improve the nutritional impact of your agricultural interventions: include a nutrition objective, target nutritionally vulnerable groups, invest in women, improve food safety and the nutritional quality of food, minimize water borne disease hazards, improve nutrition education, policies, taxes and subsidies to incentivise certain strategies, increase capacity to identify and seize multisectoral opportunities for nutrition and better impact evaluation.

Second, a paper by Noreen Mucha for the Bread for the World Institute (pdf). This paper is on nutrition sensitive development more generally. It talks about the importance of identifying pathways to nutritional impact, and the importance of setting objectives, and then monitoring them over time. The paper has a useful collection of definitions of nutrition sensitive development and calls for global normative body or bodies to define nutrition sensitive development to avoid the old wine in new bottles syndrome.

The third paper is by Robert Paarlberg, and is an evaluation of the IFPRI 2020 Conference in 2011 on Leveraging Agriculture for Improving Nutrition and Health in Delhi (pdf). The report tries to assess the impact of the conference on individuals, institutions and discourse using participant surveys, personal interviews and web searches. For individuals the conference had a small impact on their views--it had a bigger impact on motivation and empowerment, at least in the short term. At the institutional level the findings were that NGOs and the private sector were not much affected, because they are already comfortable working across sectors. The institutions that found it most useful were the agricultural ones, looking for entry points. The discourse was measured by links on websites to nutrition-agriculture issues and these were much higher immediately after the conference and even over one year later (of course there could be other factors at play). 

These 3 papers are all valuable. But none of them focus on the incentives for agricultural and nutrition professionals or organisations to engage with each other. For the IFPRI conference 75 % of conferees surveyed were already convinced of the need for a cross-sectional approach--what about the other 25% and, more importantly perhaps, the ones who did not attend? The Herforth et. al. paper assumes there is a willingness of some agricultural professionals to focus on nutrition, but says very little about what to do to incentivise more of them to want to engage with nutrition. The Mucha paper says that "experts agree that reducing maternal and child undernutrition will require nutrition sensitive actions", but what will drive the actions? The Paarlberg paper notes that the IFPRI conference had the least traction within governments where undernutrition is high. This reflects their reality: few degrees of freedom and weak incentives. 

It is important to know what to tell policymakers when they ask "what can I do?" But I would argue it is more important to (1) know how to get more of them asking the question in the first place and then (2) understand the incentives and barriers to getting any subsequent policies implemented across sectors. 

So what are the incentives? At a policy level taxes and subsidies might have a role to play. Donors can always incentivise short term collaboration. And in the longer term surely multisectoral training in nutrition and agriculture --whether accredited or not-- is key. 

But the best way to find out about the incentives and barriers is to do some market analysis: ask the agriculturalists and nutritionists to identify the barriers and what they think would help overcome them. That is a study I would like to see. 

18 December 2012

Evidence: Is the pendulum swinging back to the centre?

The past 4-5 years have seen a flurry of Randomised Controlled Trials (RCTs) and Systematic Reviews come into the international development space.

In general, I think this has been a good thing. It certainly was odd that there were so few of each prior to 2008. 

However, pursued in an unthinking way, we know these methods can be unhelpful. 

RCTs which spend a lot of money to figure out if something works in one place at one time do not tell us much about why they work and whether they will work elsewhere. 

And Systematic Reviews tend to look at only one outcome of an intervention, even if multiple outcomes have been explored by those studies.

A mechanistic application of these processes can lead to a kind of multicollinearity in the evidence.  Multicollinearity is an econometric term for lines fit to data when all the data are highly correlated. This makes the estimates very unstable (think of a table top being balanced on 4 legs which are all situated in a line, i.e. correlated). 

Is the pendulum swinging back to the centre? Based on very unsystematic and uncontrolled evidence, my intuition tells me "yes".

In the past few months 2 examples have come my way.

The first is a paper by Chris Bonell et. al. 2012 in Social Science and Medicine which argues for Realist RCTs: that is, RCTs which have many arms, which focus on mid level programme variables, operate in several sites, and draw on complementary qualitative analysis. The authors argue that the extra costs involved in multiple answer are more than recovered by the resources not spent on single answer RCTs. I agree with all this, but having been involved in many funder-researcher negotiations, the outcome frequently seen is a stripped down RCT and so I am under no illusions about how difficult this is to do. 

The second is a paper lead authored by my IDS colleague Michael Loevinsohn and others, which is a "re-review" of a systematic review by Waddington and Snilstveit  on the effectiveness of water, sanitation and hygiene interventions on diarrhoea prevention. They review a subset of the papers in the original review which described local context and which interventions allowed for greater agency of individuals in how they use or apply the interventions. While finding no fault with the original study, the authors find that the protocol used led to many impact pathways and additional insights being missed. They argue that this is because systematic review protocols tend to narrow down the terrain along disciplinary, outcome and intervention lines, presumably sometimes to make the study manageable, but perhaps often just out of habit. If you want to request a copy, I suggest you contact m.loevinsohn@ids.ac.uk
I found these two papers to be interesting twists on two approaches that have become quickly entrenched in development evidence generation. 

I like the two papers because they do not throw the baby out with the bathwater--they recognise the usefulness of the two approaches--and they try (successfully in my view) to adapt them for the complex world we live in.

15 December 2012

Foreign Policy Top Global Thinkers: Relevance for Development?

Every year Foreign Policy (FP) produces its list of top 100 global thinkers. OK, so FP says that "policy making is more or less the opposite of thinking" nevertheless many of the thinkers caught my attention. They fall into two categories: the insiders and the refreshers. 

The insiders are already part of the development firmament and are making huge contributions: Esther Duflo (poverty experiments and impacts), Bill and Melinda Gates, Joyce Banda (new Malawian President, taking on vested interests to reduce poverty), Ngozi Okonjo Iweala (Nigerian Finance Minister, taking on vested interests to help more oil income go to poverty reduction), Bjorn Lomborg (Copenhagen Consensus etc,), Sri Mulyanai Indrawati (former Indonesian Finance Minister, now Managing Director at the World Bank, for fighting Indonesian corruption), Nitish Kumar, Chief Minister of Bihar, for "turning around India's poorest state" and Daron Acemoglu and James Robinson on Why Nations Fail (extractive institutions which are used by those in power to create poverty for many and wealth for themselves).

But it is the refreshers who I found most interesting. The refreshers are those who would not frame themselves within the development community, but whose work may have nonobvious but potentially huge and relevance for development. They include:

  • Shai Reshef, Founder of University of the People in the US (tuition-free, first rate university education to anyone who speaks English and has an internet connection) and Daphne Koller and Andrew Ng founders of Coursera (1 million students online from 196 countries, linked to 30 universities). Will quality tertiary education access become a reality for hundreds of millions in this way?
  • Michael Sandel on the moral limits of markets. The more we commodify everything and put a price on anything, the more we generate inequality (the rich can now buy them) and the more we devalue the social value of what we commodify. We are surely going through a period when we are thinking hard about what markets can and cannot do and what they should and should not do. Sandel's thinking should will help.
  • Kiyoshi Kurokawa, author of a report commissioned by the Japanese government on the meltdown at Fukushima. The report boldly criticised groupthink and collusion. The report says that Japanese culture is particularly vulnerable to this, but I have seen plenty of it in international development (e.g. the risks of resilience bandwaggoning). Orthodoxy creeps up on us and we must always be open to having it challenged.
  • Ruchir Sharma on Breakout Nations. The emerging markets of the last decade will not be most likely to drive growth in the next one--we need to look to the next breakouts: Philippines, Indonesia, Turkey, Poland, Thailand, Sri Lanka and Nigeria. Do the BRICS already need re-pointing?
  • Ricken Patel, co founder of Avaaz, a transnational activism movement with over 16 million members, supporting gay rights, evacuate journalists, break prostitution rings in international hotel chains, and coordinating assistance in a guerilla war. Why isn't there something like this for hunger reduction?
  • John Coates, exposing how biology affects Wall Street--as profits mount, so does testosterone and irrational exuberance (and you know the rest). Solution? Hire more women and older men on the trading floor and delink massive bonuses from short term profits. Does something similar happen in development agencies on spending targets and promotions?

The insiders are good value, but perhaps the refreshers can be even more transformational.

12 December 2012

The MDGs: Where Does Nutrition Fit?

At a recent nutrition meeting I realised that I don’t yet have a clear idea of where nutrition should fit into the next set of development goals.

I also realised that the wider nutrition community has not had this discussion either.

As many critical decisions will be made in the next 6 months, we need to get our act together.

So what are the options?

A preliminary set might look something like this:

1. Business as usual.

That is, the underweight indicator in the poverty and hunger MDG. This seems unsatisfactory given the ambiguity attached to underweight—improvements in it do not necessarily track healthy growth (e.g. overweight but short kids).

2. Nutrition as a separate MDG.

There is probably not enough political space for this, given that the MDG set is already health-heavy. But if there were space would this be a good thing? It would probably draw more resources to nutrition (that is what the health MDGs did, by most accounts) and heaven knows nutrition needs that. It would also be a goal that could be embraced by rich and poor countries alike, thus unifying the under and over nutrition sides of the coin and generating a truly global goal, leading the way on other goals that will have to be global. A battery of indicators would be used: stunting, wasting and a healthy range for underweight (young children) and body mass index (adolescents and adults). The World Health Assembly might be supportive of this given the stunting target they recently announced.

3. More nutrition indicators throughout the MDGs, but no MDG on nutrition.

This seems more politically feasible, but maybe less desirable for the reasons given above. If it were an option, what would go where? Stunting is a marker of chronic undernutrition, but it is also a marker for poverty and deprivation in general. It could be used as an indicator for MDG1, with the hope that its existence will bring nutrition interventions into the poverty frame as they are a proven way of moving a stunting MDG1 indicator in sustainable ways that which generate high benefit cost ratios. Did this happen with the underweight indicator? I don’t think so, but I could be wrong. But even if I am right, things might be different now with the energy of the SUN movement. Wasting could be used as an indicator of child ill-health—we know that kids with severe acute wasting are many times more likely to die as kids without. Ironically this could help the treatment of SAM be better rooted in the health sector (it does not have much traction there). Diet diversity could be an indicator of food security (quantity, indirectly and quality, directly) and of agricultural productivity (via income effects and via improved physical access to food where markets don’t work well). If we could measure resource use and ecosystem services, we could begin to think about sustainable diets.

There could be combinations of 1 and 3 and 2 and 3.

There are probably other options out there.

And then there is the case of targets and timelines: should the aspiration be to end undernutrition? To halt the increase in overnutrition? To halve both rates?

What do you think?

30 November 2012

Why Poverty? Why Malnutrition? BBC Podcast

As part of the Why Poverty? series, I was asked to do a talk for BBC Radio on why malnutrition exists and what we can do to help overcome it.

Here is the link to the podcast and here is a link to a related article. I hope it is useful. Best, Lawrence

27 November 2012

Sustainable Diets: What are they? Why should we care?

Yesterday I attended a workshop on Sustainable Diets hosted by the Daniel Carasso Foundation, Bioversity International (a CGIAR centre) and FAO.

What is a sustainable diet? A diet that is healthy, affordable, environmentally sustainable and culturally acceptable. That is a tall order. It is easy to think of diets that are affordable but unhealthy, or diets that are healthy but environmentally unsustainable, or diets that are environmentally sustainable but culturally unacceptable. But all together?  Its important that we try to do this--we have to operate more consciously in a resource constrained world.

The exam topic I was set by the organisers was to say something useful on metrics. My powerpoints are here.

We had an interesting presentation by Jennie Macdiarmid from Rowett Research Institute at Aberdeen University on the Livewell project in partnership with the World Wildlife Fund—on what an affordable Scottish diet looks like--one that meets dietary minimum requirements, is culturally acceptable (i.e. looks like actual Scottish diets) and, crucially, meets greenhouse gas emission targets.
It turns out that what is healthy is not the same as what is environmentally sustainable. In other words it is not all about reducing meat consumption. The results were generated by a relatively straightforward linear programming model. The innovation is incorporating emission data and limits into the model.

My presentation was about how it was essential to get some metrics on sustainable diets. This would force us to be specific, to make choices about what is in and what is out, to identify and assess tradeoffs and importantly, to tell us what difference it would make to policy choices and decisions.

I outlined a few related pieces of work that might be borrowed from: the Multidimensional Poverty Index, the Sarkozy Commission report, the Global Hunger Index, and the Hunger and Nutrition Commitment Index (HANCI) that IDS will release in January. Ultimately we want to know if equally healthy and affordable diets have different environmental footprints.

There are a number of technical challenges:

• Getting data on food consumption, on the nutrient composition of local foods, and on the environmental and resource use implications of the use of different foods in the diet. (We need a Global Database on Food Consumption at FAO, similar to the Global Database on Child Growth and Malnutrition at WHO--Graziano da Silva please note!).

• Getting data on government and private sector commitment to sustainable diets (important because this is an inherently “whole of government” affair).

• How to combine the data? Perhaps (1) via a nonlinear programme (or some variant) that generates diets that can serve as a starting point for discussions in the kitchen and in government meeting rooms, (2) in an index or (3) in terms of “agricultural growth that reduces hunger by x for z levels of input use”?

The technical tradeoffs are important but the preference or social welfare tradeoffs that consumers and policymakers are prepared to make are equally important. They reflect values, tradition, history, politics and culture. To get at these tradeoffs we could use behavioural economics to test preferences through constructed situations, choices, and experiments—with different groups of consumers and different groups of policymakers to get a range of “who’s values count?”

On the policy research side, the options seem to be:

• Find positive examples of sustainable diets and try to work out whether they have anything in common and whether policy played any role—deliberately or inadvertently

• Identify relationships that may serve as entry points for policy: e.g. prices, information, social norms, women’s power in decision making

• Evaluate existing interventions that aim to promote sustainable diets (there aren’t many to evaluate)

• Evaluate existing interventions that inadvertently have a positive or negative impacts on sustainable diets (e.g. the US farm subsidies to corn syrup production, the Common Agricultural Policy)

I found the workshop interesting—I learned a lot and met people outside of my usual circles.

I also got to see the first Daniel Carraso Foundation Award get presented to Jessica Fanzo, a worthy recipient. She has managed to combine bench work in molecular nutrition with community nutrition and policy research. She has worked with environmentalists, engineers, economists, agriculturalists, and even political scientists (e.g. Andres Mejia Acosta at IDS). Well done Jess.

22 November 2012

A policymaker's confessions: between knowledge based approaches and political vigour

We were treated to a fascinating Sussex Development Lecture yesterday from Hege Hertzberg, Political Director for Development in the Norwegian Ministry of Foreign Affairs.

Hege is one of those people who can analyse like a researcher, think like a politician and operate effectively as a top civil servant. Quite a combination.

The themes of her talk?

(1) policy is usually evidence based--but often the evidence is selected to be consistent with the message. This is quite common because evidence is often contradictory, fractured and certainly not neutral--not when it comes to much social science. (I would add that systematic reviews can help to locate a centre of gravity, but they often serve to highlight heterogeneity and sometimes exacerbate it by comparing apples and oranges in an uncritical way).

(2) most research papers are not useful in policymaking because there are too many caveats and clauses and in any case they are too long. Part of this is because the issues are complex and part is because researchers don't like to stick their neck out (there are weak incentives to do so, let's face it). Researchers should not stop striving for academic journals--but if they want their work to be used they need to go beyond their comfort zone and talk about their findings, recommendations and warnings in ways that non-researchers can understand.

(3) her most effective way of learning about new research is by attending workshops and meetings and by talking to researchers, developing working relationships with them over the years and by reading blogs (2 minute videos?-- depends).

(4) Hege used the MDGs to illustrate some of her points, describing them as a masterpiece of policymaking.  Pointing out that the 8 goals (7 before they remembered to add in Environment) were agreed in non-inclusive way, and that this was accepted because no-one really thought they would be important. She said that is certainly not the case now. She also wondered what the MDGs would look like if they had been evidence based rather than policy message based.  Would they have been better?

(5) She said she thought it was unlikely to be politically acceptable to national leaders to meet at the September 2015 UN General Assembly just to announce more of the same on the MDGs. She said national politicians would want to announce something bold. She felt (as I do) that the goals have to generate a clear and accountable obligation for all countries. For example on zero hunger, there were obligations of rich, emerging and poor countries to do something. For rich countries for example, relax the "best before" deadlines on packaged foods so that less food gets wasted (and, I might add, reform the CAP!).

I won't share with you her 6 confessions, but I did think drawing up a list of my own confessions would be a good topic for a future blog.

It is no surprise to me that Hege was so insightful and provocative--I have been on several panels with her before. 

But then, she started her career in nutrition and we know how provocative those types can be.

15 November 2012

Jan Pronk: There is no "post conflict"

At the Hans Singer Memorial Lecture last night (a joint initiative of IDS and the German Development Institute in Bonn), Jan Pronk gave us his views on peace building and development.

Hans Singer (picture left) was co-generator of the Singer-Prebisch hypothesis about the gains from trade being skewed away from primary goods exporters due to declines in commodity prices and Redistribution with Growth which he developed with Richard Jolly which argued against  the conceptual and policy separation of growth and distribution so popular in the 70s (and today for some).

Hans passed away in 2006 and this series in his honour fluctuates between the 2 places he spent most of his academic life--at IDS in Brighton and in Bonn.

Jan Pronk (picture right) was a fitting person to give the 4th lecture in the series. He was the Minister for Development Cooperation in the Netherlands in the 1970s and again in the 1990s. He held senior positions in UNCTAD, was Assistant Secretary General of the UN, the UN Special Envoy on Sustainable Development and Kofi Annan's Special Representative to Sudan in 2005-6. He one of the few development economists who understands politics--in theory and practice. I could go on--his Wikipedia entry runs to 8 pages!

Pronk made some key points:

  • get rid of the term "post-conflict" there is always conflict in change, and development is change--what we need to worry about is preventing and containing conflict escalation
  • peace keeping by military forces runs many risks--particularly the entry of foreign interests to inadvertently or deliberately undermine the whole peace process
  • enduring peace processes are home grown, go with the grain of development and cannot be imported
  • the data that show conflicts to be declining and that support analyses that conclude greed rather than grievance drives conflict, use definitions of conflict (death on a battlefield) that are too narrow
  • peace keeping, peace making, and peace building don't happen in sequence --we need to be integrating them and practising them simultaneously

He concluded that peace building is complex and needs to take a comprehensive view, it needs to employ caution and wisdom and it is OK if it slow and does not oversell expectations.

These are refrains we hear frequently in development, and we are struggling with many of the same issues (not terribly successfully it should be said).

What I missed from his talk were the implications for external actors of his view of peace building. Nevertheless a fascinating and provocative presentation. Hans would have loved it.

Watch out for it on the IDS website.

13 November 2012

Agricultural saviours: young, senior and dead

Three items of note today, all--in one way or another--about agricultural development and saviours.

First up, a new IDS Bulletin by my colleague Jim Sumberg and his collaborator Kate Wellard on "Young People and Agriculture in Africa".  Written mostly by African collaborators, the 9 papers take on this "problem". What is the problem you say? Well, Sumberg and collaborators state that the problem is that we keep zeroing in on the wrong problem. The issue is framed either as "agriculture is the saviour of young people" or "young people as the saviour of agriculture". The real questions, they argue, are: (1) what are the determinants of  young people's interest in and success at exploiting the agri-food opportunity space? and (2) and what national and international forces shape that space?

They argue, convincingly I think, for more of a life course approach--pushes and pulls into agriculture may be temporary or permanent, strategic or tactical, but the lifecourse provides a useful way of thinking about livelihood strategies in 3-D. It is remarkable how limited the evidence base is in this area--in the last 12 years there have only been 63 papers published in journals on African youth and agriculture. This is strange given that most poor families, in on away or another, still rely on agriculture for work, income, low food prices and food in the market. There are no saviours, only opportunities to be taken, strategically or otherwise. We need to know more about what the evidence says about how we can enable young people to seize those barriers.

Next up, an altogether different type of saviour, Meles Zenawi, the President of Ethiopia who passed away in August. My colleague Jeremy Lind has written an IDS Rapid Response Brief: After Meles on the implications for Ethiopia's development. Lind writes "Meles built up a complex web of relationships that conjoined domestic political forces with foreign investors, leading the country towards impressive rates of growth and substantial achievement of some development indicators. Under his rule Ethiopia’s national image began a slow transformation from famine-plagued nation to a fast-growing country which was at the heart of a new global realpolitik in Africa. The challenge now is whether Ethiopia’s institutions, dominated at all levels by a single party, can transition to greater pluralism and, if so, will this enable the country to approach middle-income status by 2025 – a much-vaunted goal of the late Prime Minister."  

Lind makes several points: (1) under Meles, Ethiopia's image has been transformed from basket case to one of strong economic growth, including investments in agriculture, (2) but there are real questions about whether this growth is having a sufficiently strong impact on poverty, (3) in addition there are questions about how long the implicit social contract of increased growth for reduced freedoms will last, (4) the new President Hailemariam Dessalegn is from Wolaita – a part of the Southern Nations, Nationalities and People’s Region (SNNPR), but is not a member of the Tigray People's Liberation Front. Despite this appointment, Meles' power base is still firmly in control of key sectors of the Government. So is the recipe for middle income status "more of the same" or is it via increasing plurality? My guess is the former--for as long as it all holds together.  

Finally, we have Gordon Conway's new book, One Billion Hungry: Can We Feed the World?  The answer? We can feed the world, if we recognise, acknowledge or do 24 different things, things that are eloquently outlined in the book.  The 24 are entirely sensible, but given the political weakness of hunger leaders it would be a major achievement if any one of the 24 things actually happened (e.g. one of them is the Doha Round is completed with a satisfactory outcome for the least developed countries) let alone a major number of them.  So it is a long list and a non-prioritised, non sequenced, non political one at that.   There are few clues as to how we make political leaders, citizens and businesses wake up to the immorality, bad economics and bad politics of persistent hunger.  For example, "political economy" only comes up once in the index and "political stability" one other time.  This is a fantastically comprehensive book, written by a true authority in the field, but it does not really give us enough clues about how we wake people up about hunger.

The real saviours for hunger reduction are the everyday citizens who act up and pressure their governments to act--we have to find tangible ways to support their efforts.

08 November 2012

Is it wrong to cut UK aid to India?

Today’s announcement that the UK is to end financial aid to India by 2015 will re-ignite the debate about aid to middle income developing countries. 

It's a difficult one. 

Is India rich? No--its GDP/capita is a third of China's and a sixth of Brazil.  India is still a very poor country.  You won't see it so much if you go to Delhi or Mumbai but go one hour out of town and you will be shocked. 

Is India using its domestic resources as well as it can for poverty reduction? No, but it is trying to get it right--hence the debate over the massive National Food Security Bill. 

Why should a country with a space programme get aid?  As I have said before the space programme is as much about weather and land quality mapping as about anything else. 

Is UK aid "peanuts" for India as a Indian Minister said in February?  In absolute terms, yes.  But it is an invaluable source of experimentation, piloting, access to knowledge, and risk taking. 

What if the Indian government does not want aid?  Well, obviously it could easily say "no thanks". 

For every poor country, not just India, the future is in domestic resource mobilisation, not aid. 

Just try telling that to the millions of Indian mothers trying to keep their babies alive.

Maybe it's not so difficult.

07 November 2012

The US Election: 6 things we learned

I happened to be in Washington DC on election day (we are holding a Washington IDS Alumni meeting this evening).

Here are 6 things I learned:

1.  The Republican Party need a massive change in strategy to remain competitive. Republicans have now lost 5 of the last 6 popular votes.  In 1988 George Bush senior won 60% of the white vote and won over 400 electoral votes 270 is the target).  Last night Mitt Romney won 60% of the white vote but only scraped over 200 electoral votes.  This is because whites now only constitute 72% of the electorate.  The Republicans have to figure out how to become a national party--a party that appeals to all groups.

2.  This is no status quo election.  One of the commentators said a theme of the election is "I spent $6 billion and all I got was the status quo".  The election did cost a staggering $6 billion, but this is not status quo. Yes, there is no change in control of the Presidency, Senate and House.  But much like in Bill Clinton's second term, the Republicans and Democrats now have to make an effort to reach across the aisle to get things done for America. 

3.  Obama's big signature policies are now very unlikely to be turned back in 2016.  Affordable health care will now take root.  No new wars are on the horizon.  The deficit will not just be reduced through cuts--there will be some tax increases for those who can afford more.

4.  This was a disaster for the Tea Party. Their extreme Senate candidates got beaten in states where they should have been competitive.  In all the CNN, MSNBC and Fox coverage I caught there was no sign of Tea Party members.  No Michelle Bachmann, no Sarah Palin, no Amy Kremer.

5.  Mitt Romney was a weak candidate.  President Obama was a sitting duck.  High unemployment, a net 14 percent of the electorate thinking the country was heading in the wrong direction, healthcare that still has a net negative favourability rating.  And yet this was a decisive result for the President.  In the end, Romney was wooden, had shifting positions on everything, and just came across as fake.  Governor Chris Christie would have wiped the floor with the President.

6. Obama's victory is good for international development.  IDS' US development partners I talked to in the past few days were very anxious about a Romney victory.  President Obama is seen as an internationalist, a multilateralist, someone who understand the need to engage, someone who is serious about rights and who understands soft power.  And in the wake of "Superstorm" Sandy, some also have hopes for some much needed US leadership on climate change in Obama's second term.

02 November 2012

Hunger and Malnutrition: Then and Now

For those of you who grew up listening to mix tapes (for the under 30s read "playlists"), the IDS Virtual Bulletin is for you.  

The idea is to mix and match articles by theme, spanning the years. 

The latest edition is on Hunger and Malnutrition and has articles from the last 30 years from the likes of:

Michael Lipton, Margaret Buchanan-Smith, Mona Sharma, Paul Howe, Jeremy Swift, Richard Longhurst, Simon Maxwell, Ian Scoones, Stephen Devereux, John Thompson, Biraj Swain, Geoff Tansey and Harsh Mander.  

It was compiled by our Editorial team of Alison Norwood and Gary Edwards.  All the articles are available online and free of charge. 

Here is the Foreword from me. 

The first article in this virtual IDS Bulletin is by Michael Lipton and dates from 1982. In that year the WHO stunting rate for children of preschool age in sub-Saharan Africa was 39 per cent. In 2012 the rate is still 39 per cent. The FAO hunger numbers paint a slightly less depressing picture for that region – the percentage of the population that were hungry in 1980 was 38 per cent and 30 years later it is 27 per cent. 

Like the numbers, the issues in this Bulletin remain remarkably constant – investing in smallholder agriculture, linking relief and development, dealing with seasonality, turning economic growth into child growth, ensuring technology is hunger-reducing – joined by innovations such as the growing right to food movement. 

But there is hope. The picture has improved much more dramatically in East Asia and Latin America, driven by various combinations of income growth and strategic public spending on health, social welfare and agriculture. Looking forward there is cause for optimism. In the high burden regions, economic growth is relatively strong and the food price increases of the last five years have put these issues higher on the political agenda. 

As Brazil, Peru, Honduras, Vietnam and Ghana show, the most precious and potent ingredient in ending hunger and malnutrition is political leadership. 

All of us have to push our politicians to be leaders of this cause. This virtual Bulletin is one such contribution to doing just that.

01 November 2012

Global Development Goals: End of the Beginning or the Beginning of the End?

Yesterday I gave evidence to the UK's International Development Select Committee, the body of MP's that, among other things, holds DFID to account. The Committee is conducting an Inquiry into what should come after the Millennium Development Goals (MDGs) which currently end in 2015, I was on a panel with Richard Morgan, Senior Advisor at UNICEF and Eveline Herfkens (former Minister of Development Cooperation, the Netherlands).

IDS provided 5 submissions to the Inquiry. The one from the ESRC STEPS Centre, coordinated by IDS (Melissa Leach, Ian Scoones) and the Science Policy Research Unit at the University of Sussex (Andy Stirling, Adrian Ely) focused on politics, inclusiveness and power, with a special emphasis on how to reconcile environmental and development objectives. Noshua Watson at IDS delivered a submission on financing for the next set of goals. Andy Sumner (now at Kings) submitted on the new geography of poverty.  The "Participate: Knowledge from the Margins" team (IDS and Beyond 2015) presented on the need to listen to those who are normally not heard, and argued strongly for citizen-led accountability mechanisms to help legitimise the next set of goals.

It is good to see the various dialogue processes intensifying--it's about time. The end of 2015 is approaching fast and there are no guarantees that the world will have decided what to do with the Goals by then.
If the MDGs were the end of the beginning will the post-2015 discussion mark the beginning of the end of Goals? I hope not, and it need not, but high level political leadership is in short supply.

My written testimony is below, keying in on some of the exam questions set to us by Malcolm Bruce and his Committee. Here is a related powerpoint to AusAID.

1. Lessons learned from the adoption of the International Development Targets and the Millennium Development Goals: in particular how effective has the MDG process been to date?

The current MDG framework has provided a basis around which a broad international consensus has been built and that has concentrated global attention and resources on addressing some of the most pressing development outcomes, outcomes that if dealt with will save and improve people's lives.

The MDG process is thought to have had the following effects on donors and recipient countries:
It has strengthened the view that if support for aid is to be sustained, measurable progress must be shown in areas that the public in donor countries view as desirable. Recognition of the MDG framework within traditional donor countries has been highly variable. It has been good in the Nordic countries, yet much less visible elsewhere.

It is thought to have (a) increased Overseas Development Assistance (ODA) and (b) directed a greater share of it to Sub Saharan Africa.

There has been more of an impact on the aid discourse than on resource allocation and there is little evidence of the impact on national policies in developing countries.

2. The coverage of future goals: should they be for developing countries only or should progress be monitored in all countries

All countries should be bound by at least some of the goals. A new framework needs to recognise the changes that have taken place in the world since the inception of the existing MDGs in 2000. Notions of 'developed' and 'developing' nations are now outmoded and aid is no longer the main source of development finance. Remittances, taxes, foreign direct investment (FDI) and private foundations all play an increasingly significant role. A more integrated approach to development is required, with more cross cutting policy responses and improved cooperation across all development actors.

Some of the goals would not make sense in the richer countries (e.g. $1.25 or $2 a day poverty rates) and some would be very difficult politically (e.g. halving of relative poverty or a target for declines in income inequality).

However there need to be some around climate, resource use and energy efficiency that are applied to all. When the goals apply to all countries, there should be a compensation mechanism or differentiated target for the poorer countries who are signing up to reduce global "bads" such as pollution, global warming, unfair trade, unregulated financial flows, unregulated arms and drug trade.

3. Targets: was the MDG 'target-based' approach a success? Should it be retained? How should progress be measured?

The 'target-based' approach should be retained within a new framework as without targets the goals are devalued. A new framework should include some indicators and targets on inputs such as spending, policies and charters as well as outcomes. This would strengthen the accountability framework. It is difficult to hold governments solely accountable for outcomes that can have multiple and international causes. It will be easier to hold them to account to commitments on spending, policy reform and signing up to charters and rights (for one example, see the IDS work on the hunger reduction commitment index at www.hrcindex.org)

4. Timescale: what period should the new framework cover? Was the 15-year timescale for the MDGs right?

The timescale for the existing MDGs of 15 years, with measurements on 25 years, was probably too short. It took at least two to three years to build awareness of the MDGs and discussions around what succeeds the MDGs have been underway since 2010. This has left less time to focus attention on accelerating progress towards meeting the existing goals. Bearing this in mind, a new framework should adopt a longer timescale of 20-25 years.

5. The content of future goals: what would be a good set of global goals? What continuity should there be with the MDGs, and how should the unfulfilled MDGs be taken forward?

A balance between continuity, learning and the changing world needs to be struck in a new post 2015 framework. A new framework needs to be underpinned by a theory of change – see figure below for a rough example. It should set out the human well being outcomes the framework is seeking to achieve i.e. freedom from hunger, good health, peace and security. The values of freedom, dignity, equality, solidarity, tolerance and respect for nature described in the original Millennium Declaration could serve as a good starting point. It should outline the enablers necessary to realise these values i.e. secure employment, education; the connectors such as access to energy, water, sanitation, infrastructure, ICTs and the sustainers including resource intensity and pollution targets. The commitments to these elements should be tracked and the gender dimensions monitored. Any model should be have a small number of goals but be more expansive on indicators.

The breadth of policies that drive a focused set of goals should be broadened to go beyond aid. A focused set of goals does not mean a limited set of policies. A more integrated approach and set of policy responses is required that incorporates climate, energy, trade, security, immigration, finance and intellectual property.

For more information regarding this submission or IDS’ work on the post 2015 agenda please contact Hannah Corbett at h.corbett@ids.ac.uk or on 01273 915640.

31 October 2012

Development Horizons: Big in Norway--But Why?

Development Horizons has been going for about 3 years now--about 300 blogs and 185,000 page views.

One of my collaborators in India has been telling me that I am hopeless at getting the blog to have a wider readership, and so I have been looking at the stats that Blogger software generates.

One of the stats is the top ten page reads by country of reader. The table below summarises the countries in order of page reads. I have calculated the page reads by population size and by number of internet users (columns 2 and 3 respectively).
Which country has the top readership per internet user? Not the UK, but Norway. Other interesting points:
  • 185,000 feels low--600 reads per post on average. Other bloggers---what do you think?
  • China and India and 3rd and 5th in terms of absolute readership--emerging powers indeed
  • On a per internet user basis, Ukraine is the fourth largest readership--again, I have no idea why
  • I was surprised that Japan is not in the list--perhaps a language issue? Japanese readers, what do I need to do to get the blog more widely read
  • No country in Africa listed, although lately South Africa has featured more strongly-this is partly to do with internet access and speeds--anything to do here?
  • No country in Latin America listed--I am relatively uninformed about the region, and it shows (and the language does not help)
As for Norway.  Why Norway?  I'm not sure.

Norway has a deep interest in international development, peace, participation and human rights and these surely resonate with IDS interests and values. But are Denmark and Sweden so far behind? (By now I have probably alienated the entire Norwegian readership by comparing them to other Scandinavian countries. It could have been worse, I could have gone for the Nordic comparison.)

Perhaps it is institutional connections? IDS has some connections with Noragric, FAFO and Chr. Michelsen Institute, but we have more with Swedish organisations. We have a Board member, Jon Lomoy, from Norway too. We also have one Norwegian Research Fellow, Lars Otto Naess, head of our climate change team.

Personal connections? I gave a talk last year at the Research Council of Norway's annual meetings. I know some of the Noragric folks (Ruth Haug and colleagues) and some of the food security, nutrition and human rights community (Arne Oshuag, Asbjorn Eide and Wenche Barthe-Eide). But I know as many from other countries in the region.

Does any of this really explain a rate equivalent to over one-in-a-thousand internet users in Norway visiting Development Horizons in the latest 3 years? Maybe it is the name of a prominent tourism website in Norway? In any case, a puzzle.

In the final analysis, perhaps it all boils down to Norway's inherent good taste!

In any case, a warm tusen takk to you all.

28 October 2012

Australia in the Asian Century: UK Please follow suit

A new White Paper from the ruling Coalition Government of Australia, headed by PM Julia Gillard (left), was launched yesterday at the Lowy Institute for International Policy.

Now Government policy, the White Paper is entitled: Australia in the Asian Century.

The report is a wide ranging analysis of how Australia can benefit from and contribute to the emerging countries of Asia.  It does not generate too many specific policy priorities or spending pledges, but that is not it's point.  It is a strategic document about positioning Australia in a rapidly changing context.

Reactions have been largely positive, although there have been criticisms about the lack of specific pledges. Some also say it is motivated primarily by domestic politics: it contrasts the current Government with the Opposition who are perceived by many as more Anglocentric.

Some have praised it for its positive focus on opportunity, but have criticised it for the relative lack of discussion about the risks inherent in closer links with the region.  There are other risks too, for example this document seems to be steering Australia firmly away from the New Zealand model towards Australia not settling for being a small and prosperous country, but one that wants to be prosperous and a much bigger player in the region and the world (note it was recently voted onto the UN Security Council).

Some have said it is too state-centric and does not recognise that Australian businesses need to take more responsibility (see the CPA Australia competitiveness survey of Asian and Australian companies which concludes that regional business leaders see Australian businesses as disengaged from the region).

I obviously have not read the 300 page report in full--I have gone through the useful slide pack and some of the press releases and the exec summary.

While it is clear that the main audience was domestic, it is a little disappointing that there was not more emphasis on international development and on Australia's emerging role.  

This is not surprising I suppose--it is difficult to sell Asia to your population if you focus on its poverty, malnutrition and inequity.  To be fair, the White Paper does spend some time on the inequality issues, but only in the sense of noting how internally focused many of the Asian giants will have to be over the next few years to deal with these growing internal tensions.

I found the strategic international ambition of the paper really refreshing.  The attitude on promoting student exchange between Asia and Australia is particularly impressive because it comes in the context of the concerns many Australians have over immigration levels.  The Paper says:

"Australia’s university system is a powerful link with the Asian region through the number of students who come to Australia for their education, and more can be done to strengthen these links."

This is in stark contrast to what is happening in the UK.

A recent Economist article ("Shutting out foreign brains is a good way to foster mediocrity") paints a gloomy "Little Britain" picture with restrictions on student visas and work permits being used to bring down net numbers on UK immigration.  This is hurting everyone.

Mr. Cameron, follow Ms. Gillard's lead: tear down this wall--you are not helping us compete in the so-called "global race".

26 October 2012

Climate Change: It's the Economy, Stupid

As we near the US Presidential Election, it was noted that for the first time in 24 years climate change did not feature once in any of the 3 Presidential Debates.

This prompted David Attenborough, one of the UK's leading naturalists, to speculate on what, if anything would change the US public's minds. He said it would have to be a disaster, clearly linked to global warming.
A related report in Foreign Policy by Kate Sheppard, says, its not that people don't think the climate is changing, but that they do not have the capacity to worry about it that much due to the global economic downturn.

She cites a 2012 paper by Scruggs and Benegal in Global Environmental Change which poses the question: "Declining public concern about climate change: Can we blame the great recession?" Their answer is yes.

They use 3 data sources for their regression work (fitting lines to data and then figuring out if there is causation): (1) pooled averages from US opinion surveys from 3 different organisations from the past 15 years, (2) individual level responses from PEW surveys and (3) data on country level opinions about climate change from EuroBarometer data between 2008-2009.

They find plenty of evidence correlating unemployment with belief and concern that global warming is happening and is a serious threat: stronger than media effects and stronger than local weather anomalies--although church going and a very conservative ideology had stronger associations than unemployment. But they can't really test for causation with their small sample sizes.
They suggest that when the economy picks up, more capacity can be directed to addressing climate change. Ironically, the pick up in the economy will accelerate emissions--there is no counter cyclical force on attention to climate change.

It is clear that when economic growth picks up, we need to be ready to strike while the iron--and the earth--is hot.

25 October 2012

Plan International and Nutrition: Helping to Scale?

Founded 75 years ago, Plan is one of the oldest and largest children's development organisations in the world.

Plan International's website says: "We work in 50 developing countries across Africa, Asia and the Americas to promote child rights and lift millions of children out of poverty. In 2011, Plan reached 56,500,000 children in 58,053 communities. Plan is independent, with no religious, political or governmental affiliations." 

This is a phenomenal outreach and, in a context where the world is working hard to scale up nutrition action, having a such a well regarded and far reaching organisation such as Plan up its ambition on nutrition is a major plus. 

Yesterday I was invited to be on a panel that was addressing Plan's country and national Directors.

My argument was (a) if you don't focus on nutrition status, you are ignoring the very foundation of child development--all your other programmes and effort will not realise their potential, (b) a focus on the first 1000 days after conception helps break the intergenerational cycle of hunger and child deprivation and helps make sure that any resilience work is pro-poor, (c) focus more on the most marginal areas, the fragile contexts and the urban contexts--this is increasingly where the malnourished children are, and (d) become fragile development experts--the wider development community needs organisations who can bridge the disasters-development space.

I also saw presentations from Peter Walker of Tufts University and Steve Collins of Valid International. 

Peter presented some interesting stuff on (a) how we are hardwired for diet diversity and how supermarkets and food processing plays on this to generate the consumption of the wrong types of foods, even in poorer environments.  (As a side note, it was fascinating to hear that the first supermarket, Piggly Wiggly in the US, secured a patent in 1917 for its new supermarkets in terms of store plan layouts that forced customers to pass through all the aisles to get to the checkouts.) and (b) how concentrated the decision making is in food systems--for one country (I did not catch which) Peter said that for 3.2 million farmers, there were 160,000 suppliers, 8600 manufacturers, 600 chains, 110 buying desks, 170,000 outlets and 170 million customers--that is a lot of concentration of power in those 110 buyer desks.

Steve gave a nice summary of community managed acute malnutrition and was urging the nutrition community to find ways of working with the private sector to incentivise them to be more concerned with healthy and affordable foods.  One idea he proposed was to have SUN social branding if companies meet certain standards.  Nice idea, but who has the capacity to enforce? 

I was impressed with the quality of reflections and questions from the leaders in the audience and I wish Plan well as it (hopefully) enters the nutrition field in a strong way.

22 October 2012

George McGovern: Leader against hunger and malnutrition

In a world where too few political leaders put themselves on the line for things that don't help them get elected, George McGovern will be surely missed.  The former Senator and Presidential candidate, 90, died over the weekend.

He was described as "disastrously attached to principle" by the Economist (he was trounced by a Watergate fuelled Nixon in the 72 Election) and as a "Prairie Liberal, Trounced but Never Silenced" by the New York Times.

I ran across his work early on in my career, when I was writing my Master's Thesis on Women Infant and Children Nutrition Centres in the poorer neighbourhoods of Massachusetts.  This is because Senator McGovern was a champion of nutrition--at home and abroad--when it was terribly unfashionable to be one, and he was a great advocate of political solutions to hunger.

 A World War Two bomber pilot, he was no pacifist, but was staunchly against the Vietnam war and proud of his stance.

It is hard to think of politicians of this age in the US or the UK who were as willing to stand up for things they believe in but which have little immediate political payoff.

19 October 2012

How Effective are Cash Transfer Programmes at Improving Nutritional Status?

A new paper by James Manley, Seth Gitter (both Towson University  in Maryland US) and Vanya Slavchevska (American University in Washington DC) asks "How effective are cash transfer programmes at improving nutrition status?". 

The paper is a rapid evidence assessment of all the studies that evaluate the impact of conditional and unconditional cash transfers on various measures of nutrition status. 

(It is not a systematic review--the restricted time period available to find the studies probably led to some of the difficult to obtain and foreign language material being excluded and this might bias the findings because easier to find studies--i.e. published--tend to be more likely to find statistically significant results.  It still seemed to me like a very careful study.).

Their search uncovers 24 papers on 18 programmes in 11 countries.  

The authors focus most of their energy on the analysis of the impacts of cash transfer programmes on height for age as this is the outcome for which they have most data (18 studies looking at 15 programmes in 10 countries which generate 117 estimates).  

The multiple estimates are averaged out per study per outcome indicator and then used in statistical meta analyses (simple regressions or analysis of variance) to see if the impacts varied by study features (e.g. quality, RCT, sample size), programme features (e.g. conditionality, size of transfer), child characteristics (e.g. sex, age) and country level features (e.g. infant mortality rates and health service provision).  For the height for age outcome this generated 18 observations which (I think--they don't actually say) from the basis for the regressions (n=18) in the meta-analyses.
The paper is thorough and, for the most part, well done and generates some interesting results.  

I liked the fact that the authors attempted to do meta analyses on the estimates of the impact of the programmes on height for age.  

My main problem is that the authors used all 117 estimates stating that "Each estimate contains useful information so we want to include all of them, but at the same time we must control for the correlation between estimated impacts for different estimators or treatment groups in the same programme." 

Well, they may all contain interesting information, but but that does not mean they should all be included.  Some are reported in the original studies to test whether more sophisticated estimates are needed and if they are needed then the more basic estimates should be discarded by the meta analysis. In other words only the preferred estimates from the original papers should be included.  From the review, I could not tell if the non preferred estimates had been discarded from the meta-analysis.   

If known biased estimates are included in the meta analysis, then this is obviously a big problem for all the conclusions of the paper. 

If--and it is a big if--the conclusions are drawn from meta analyses that exclude the known biased estimates, then they are really interesting.  

* "The average effects of the programmes on height for age are positive but statistically indistinguishable from zero, and the conditions, including the country characteristics, recipient population characteristics, and the programme characteristics all matter." (In other words, these interventions are not proven in all contexts, and design matters--this echoes the 2008 Lancet conclusions from Bhutta (Table 1).)

* Which programme characteristics matter?  The only one found to be significant was conditionality that is not tied to health and education (not terribly clear what these non health and education conditionalities were--I presume they had something to do with employment)--this had a negative effect.  There was no statistical difference between programmes that did not condition and those that conditioned on health and education (I was surprised at this--my priors were that the latter would have a bigger effect). Wisely the authors state that there are probably many other programme features that are more important than conditionality and we should avoid over-focusing on this design feature, although they recognise the political as well as technical rationale for this feature (but, then again, many of the others are political too--think of transfer size!).

* Which study characteristics mattered? The analysis could not find any significant correlations between outcome and study characteristics. So use of RCTs, quality, peer reviewed--no significant differences. 

* Which child characteristics matter?  The impacts tend to be higher for girls. No significant age effects found. The girl effect is not discussed in the report.  Nor is the programme characteristic of who the transfer is given to within the household explored--perhaps there is some link there? 

* Which country characteristics matter?  When infant mortality is high and hospital infrastructure is poorest, impacts are most positive.  As the authors note, this provides some supportive evidence for the UNICEF Narrowing the Gaps to Meet the Goals approach of reaching the most marginalised. 

But the meta analysis is a tease.  Only 18 observations.  And these might include known biased estimates of impact.  We need clarification of the last point and multiplication of the data points.   

Nevertheless this is a fascinating paper and perhaps its greatest contribution is the creation of a universe of studies with a coding of some of their key features.  

As the paper says, we need more studies.  I agree, and they should be informed by this report.