tag:blogger.com,1999:blog-6335146197342151188.post9115082334674384004..comments2024-02-29T13:07:00.519+00:00Comments on Development Horizons by Lawrence Haddad: Evidence on the impact of research on international development? New DFID literature review. Lawrence Haddadhttp://www.blogger.com/profile/17265061444076801962noreply@blogger.comBlogger2125tag:blogger.com,1999:blog-6335146197342151188.post-24750586352628491952014-10-22T12:07:18.244+01:002014-10-22T12:07:18.244+01:00Hi Kirsty, thanks for the thoughtful blog--you mak...Hi Kirsty, thanks for the thoughtful blog--you make many sensible points. On 1, I still think the review of reviews could have been more systematic. On point 2, I think that this is the most important question, so its a pity it was not focused on more (perhaps for a follow up?). On 3, capacity is of course key--for me the big issue is where to know to invest and how to know if it has worked? On 4, glad to know there are some trustworthy researchers out there. Best, Lawrence Lawrence Haddadhttps://www.blogger.com/profile/17265061444076801962noreply@blogger.comtag:blogger.com,1999:blog-6335146197342151188.post-78921101947285747732014-10-22T11:09:05.938+01:002014-10-22T11:09:05.938+01:00Many thanks for blogging about this Lawrence and I...Many thanks for blogging about this Lawrence and I am glad to hear you found it interesting. I was the lead author on the review and certainly found it interesting to write. We chose to do it because we were asked so often what is the evidence that investing in research is a good thing - and we felt that the only answers we could offer were pretty flimsy. I was particularly interested to find that the review suggested that a couple of 'mantras' on research impact were actually not backed up by the evidence. I have written much more about the review on my own blog starting here: http://kirstyevidence.wordpress.com/2014/10/07/science-to-the-rescue-part-1/<br /><br />I hope its ok if I offer a few responses to your comments. <br /><br />1. On why this was not a systematic review - the quick answer is that the topic was just too big! In fact it would need to have been multiple systematic reviews but we decided instead to do essentially a review of existing reviews - a kind of meta-review. Therefore, wherever possible, we limited our search to existing synthesis products. That is the reason that the primary evidence you mention is not included but it is covered by some of the reviews we included. Now that we have done this paper, we may choose to commission some systematic reviews for the areas where none exist - but possibly more importantly was the fact that in some areas there was almost no good quality primary evidence. So that is another priority area. <br />2. Regarding what features of research make it more likely to have impact, this is covered in brief in the paper but was not a primary focus. In fact there has been a large amount written on this topic in the past - see for example Fred Carden's book or work from RAPID. I think such reserach is interesting BUT... it needs to be interpreted cautiously. There is a danger that people try to imitate the features of research which has impact without asking the more important question of which research ought to have impact. So for example, there is a lot of guidance suggesting that people should get better at writing their research outputs up as policy briefs and publicising via social media. I have a worry that such approaches, in the absense of decision makers who can differentiate good from bad, can lead to poor research with good comms having disproportionate impact.<br />3. It is true that capacity is not the only factor hindering research uptake - in the paper we talk about both capacity and motivation/incentives (which we use broadly to include a range of political cultural and personal factors). I agree that the latter is very important but I also think that the former tends to get glossed over. There are hundreds of studies looking at the political economy of reserach use - some of the best come again from the RAPID team. However, from my experience working with developing country policy making institutions, I have seen time and time again that reserach is not used even where there is a political will simply because there are not people there to find it, appraise it and summarise it for decision makers. I find this to be a big difference between policy making organisations in the UK (e.g. the UK parliament or UK ministries) which have armies of technical advisors and analysts to support senior decision makers. This layer is simply missing in many developing country policy making organisations. I think the reason capacity is often overlooked is that it is quite politically sensitive to mention. People find it much more palatable to say that reserach isn't used because politicians are swayed by politics that to suggest that there might be a lack of staff with capacity to understand the research. <br />4. On the final question... I don't think we should distrust all researchers ;-) But I do think we should look at research into rates of return critically and be sure that the methods used are actually giving an acurate reflection of reality and not being driven by other motivations.Kirsty Newmanhttp://kirstyevidence.wordpress.comnoreply@blogger.com