Don't kill off the data revolution before it has even started: review of new paper

There are many people, me included, who are calling for a data revolution nutrition. 

This means considering data as an essential rather than a luxury item, not tolerating data gaps when they are a block to effective action and not making do with old data. 

But the assumptions behind this are that (a) data are not prohibitively expensive to collect and (b) they actually have returns that are several multiples of the cost.

So I really welcome a new paper by Morten Jerven, under the Copenhagen Consensus banner, which asks the general question: is a data revolution for the SDGs a good investment? His answer is a resounding no.  However I think there are some fundamental problems with his analysis.

How does he come to this conclusion?

He proposes, for every country, a population census every 10 years, a demographic and health survey every 5 years, a living standards measurement survey every 5 years and an annual Core welfare Indicator Questionnaire.  He applies this to the 25 year MDG period, 1990-2015.

He gets cost estimates for each, based on published data.

He applies this to 138 countries for the 8 MDGs (18 targets and 60 indicators).

The grand total is $27 billion, which works out to just over $1 billion per year over the 1990-2015 period.  He suggests this is an underestimate because data collection capacity costs are not included.

He then notes that the 17 SDGs have 169 targets (we don’t know how many indicators) and therefore the SDGs will cost (169/18) x $27 billion or $256 billion.

He then compares this to official development assistance (ODA) and says it is nearly twice a large.

He also says it is really hard to get a sense of the returns to improved data and says the benefit-cost ratio is likely to be less than one.

There are a few problems with this. 

First, it is highly unlikely that the SDGs will have nearly 10 times as many targets and indicators as the MDGs.  That would be nearly 600 SDG indicators.  Simply not going to happen.  My guess would be fewer than 100 targets and less than 200 indicators, but it may well be a lot less than that.  So let’s increase the $27 billion by a factor of 5.  To about $100 bn.

Second, $100 bn is still a lot of money.  But this has to be divided by 15 years.  Per year this is about $7 billion.  Increase this to $10 billion, because the number of countries is increasing from 138 MDG countries to 193 SDG countries (i.e. all of them). 

Third, so $10 billion per year.  This is a lot of money but less than 10% of ODA. But the comparison with ODA is spurious, because most of the costs will be covered from domestic resources for all but the poorest countries.  I would imagine only about one fifth of the cost of indicator collection will come from ODA.  If so, that is $2bn a year, which is just under 2% of ODA.  Still a lot, but that is the lower bound ratio most organisations factor into their project costs for M&E.

The better arguments for caution, I think, lie on the benefits side.  Jerven rightly points out that data that guides action is most valuable.  He makes the argument that the most valuable of this type of data is collected frequently at a rather granular level and that most of the SDG indicators will not be like this, therefore they will be less valuable.   I’m less pessimistic.  First, we can try to get more of these intermediate indicators into the SDGs—things like spending, coverage and staffing.  Second, the impact indicators on nutrition status, say, are useful for civil society to put pressure on governments to act, and for governments to calibrate their actions.  These data are also essential for research and analysis to guide action.

So my conclusion is that this is a very partial paper.  Data collection costs are certainly not prohibitive.  And the benefits can be maximized by asking when is the absence of data a barrier to action that we know is effective.

The data revolution is just that—a call for better data, but also a different way of thinking about data in terms of costs and benefits and also in terms of effective actions supported and bad actions suspended due to better data.  

Don’t kill off the data revolution before it has even had a chance to turn things around.