20 September 2012

When Worlds Collide: Trying to work across the health-development divide

Today we held the concluding workshop of a project between IDS, the London International Development Centre (LIDC) and the London School of Hygiene and Tropical Medicine (working title: When Worlds Collide).

Four papers—each with some variant of systematic review--were presented on the following questions:

(1) To reduce malarial infections should we be investing more in malaria-preventing development (e.g. building prevention into the design of shelter) or in malaria control (e.g. drugs, insecticides). In an era of pressure on aid flows and when resistance to malarial insecticides and drugs are increasing, the paper concludes that we should be spending more on malaria-preventing development. This paper used systematic review processes and non-systematic review approaches to find its evidence.

(2) Why do some institutional arrangements seem to offer better quality of health (and other) services to low income populations? The paper found that directing resources (whether private or public) to institutions as opposed to individuals improves quality because institutions can signal quality more clearly, are less fly by night, and have stronger incentives for repeat transactions.

(3) Does agriculture-driven food price policy affect undernutrition and diet related chronic disease? Out of hundreds of studies found through online and bibliographic searches, we only found 4 studies that linked agriculture-driven food price policy with nutrition status. The direction of impact were as predicted—higher food prices increased undernutrition and lower food prices led to increased overweight and obesity. The surprising result was the lack of evidence (and we included ex-ante modelling studies and ex post evaluation studies) on the direct link between agricultural driven food price policy and nutrition status.

(4) Do systematic reviews miss multiple effects of interventions? Here the paper was a re-review of an existing systematic review looking at the impact of water, sanitation and hygiene interventions on diarrhoea rates. It found that a high proportion of the studies in the original systematic review contained impacts on other outcomes, sometimes of direct relevance to the interpretation of the diarrhoea outcomes.

The original plan was for the 4 papers—supported by DFID—to come out simultaneously in the Lancet and World Development. In retrospect, this was a ridiculously unrealistic goal. Richard Horton the Editor of the Lancet, and one of the panellists at the workshop, dubbed the project a “glorious failure”. That’s a bit unfair, but not much.

Why failure?

It was a failure in the sense that we could not successfully apply systematic reviews to the really big cross-sectoral questions (the questions just could not be framed specifically enough). We also failed in our quixotic attempt to get the 4 papers published as a set simultaneously in the two journals (although one is with the Lancet, one with World Development, and the other 2 are about to be submitted to 2 different journals).

Why glorious?

Perhaps because those involved in the project learnt a lot and can share a lot. What did we experience?

• We have very different languages (and we were folks generally used to working across disciplinary boundaries). “You say consumption (diet) and I say consumption (expenditure)--let’s call the whole thing off.”

• The development and health journals are not set up to evaluate transboundary research (one journal editor said “there aren’t enough interdisciplinary reviewers”)

• UK universities are facing incentives such as the Research Excellence Framework which tend to place a premium on single disciplines

• The nature of credible evidence to each community is very different—health colleagues tend to be more enamoured with randomised controlled trials (RCTs) and classical systematic reviews, while the development colleagues tend to not worry so much about comprehensiveness and systematic inclusion but more about understanding external validity (context, nuance, formative research)

• Research funders know that they provide important incentives to researchers, and they do not seem as bound by disciplines and sectors as other research actors

• Those in civil society are probably scratching their head about all this as they tend to be eclectic users of evidence in any case

We also felt that there are not many champions for this kind of work. This is a pity and is perhaps an ethical challenge. Many if not most of the policy questions of the next decade will respect no boundaries—how will we get anything but fragments of answers to them? How to create champions? New multidisciplinary MA programmes are surely part of the answer.

Unfortunately the 4 papers are not available for wide sharing just yet, but if you want to find out more contact Lucy Tusting (LSHTM) for the malaria paper, David Leonard (IDS) on institutions, Alan Dangour (LSHTM) on agriculture and Michael Loevinsohn IDS) on water, sanitation and hygiene.


Stephen Jones said...

Thanks for a good summary of the discussions. I was at the workshop (but not affiliated to any of the organisations involved) and shared some of the 'civil society head-scratching' that you mention. In particular, some conclusions like 'we need to understand context' and 'we need to appreciate people's own agency' would seem like starting points rather than conclusions to most social scientists (and fall close to the pitfalls Duncan Green recently mentioned that researchers sometimes fall into when writing conclusions for policy-makers - see http://www.oxfamblogs.org/fp2p/?p=11659).

Clearly there is huge expertise and experience amongst the people and organisations involved, so I wonder how many of the challenges were related to the initial stage of developing the project's questions and the restrictions that emerged then? You mention some of the issues - different disciplinary languages and understandings of evidence - and there was mention of a fifth paper setting out some of these conceptual issues and discussions which sounded like it would be useful. Is this paper going to be published too?

I like your suggestion for multidiscipinary MAs - but I wonder if the demand-side (research funding, how peer reiew works etc) is more important than the supply-side. Regarding journals and peer-review, is there no way 2 (or more) reviewers from different disciplines could be assigned to a paper and asked to discuss it together? Some of the blind nature of the review might be lost, but if I understand this right a similar thing happends with many research funding panels.

Lawrence Haddad said...

Hi Stephen, thanks for this..I agree that the demand side is more important..

I like the idea of a discussion between peer reviewers..not sure if that ever happens on a paper submitted to a journal, but you are right, it often happens on funding panels (and IDS Fellow review panels) -- and why not a journal?

I think the systematic review format was more restrictive than we realised (or perhaps we were not creative enough in bending it to our will).


Amelia Parry said...

Good explanation on a complex subject matter. I'm hoping for a discussion on respiratory health. I would like that e cigarettes be more visible to the public. The public awareness for these should be increased for the smokers to have an idea of its health benefits.

Cameron Salmond said...

Water sanitation should always be a priority. If the government or concerned institutions or organizations will continue to ignore this issue, people's health are at risk. There are always solutions for problems like this.

Lance Harcourt said...

Despite the fact that a concrete consensus in applying the combined approaches to better understand a serious problem, I have to say that this conference was a great move. A channel of communication was opened between parties that, though unrelated, are both affected by the issue at hand (that is, the health-development divide).

Therese Ligert said...

Finally it seems that the gates are opening for an inter-party cooperation with regards to the health sector. That's a good step forward; we need more interoperability to bring health programs to more people.