Tracking the Impacts of the GFC in Real Time: How Accurate Are Qualitative Methods?

One of the fragilities laid bare by the global financial crisis was our slowness in actually tracking the impacts of crises on households. In February 2009 IDS undertook case studies across 5 countries to get a rapid take on who was being affected by the crisis, how badly, and how well households and governments were responding.

Now Neil McCulloch, a Fellow at IDS, and Amit Grover have published a paper that compares the qualitative results from Indonesia with 3 waves of a national labour force survey, the crisis hitting late 2008 between waves 2 and 3.

How well do the qualitative and quantitative results match up? Pretty well actually. First, some caveats. As McCulloch and Grover note, the rapid qualitative survey was only undertaken in a few villages which are likely to be affected negatively by the shock and is by no means representative. And the quantitative survey is not large enough to give meaningful results at the village level. Nevertheless, where did the methods converge and where not?

Very similar conclusions:

• school enrollment and attendance (no change in enrollment or continued improvements for 13/14 year olds)
• female labour force participation (no change)
• informality of employment (some suggestions of increases)
• unemployment rising for young workers

Differences

• qualitative survey did not show any income gainers during the crisis, but the labour force survey showed large increases in wages for employees involved in industries that were supplying the large domestic market in the wake of the large drop in exports and imports. The survey did conclude that workers in the informal sector did not benefit.
• the qualitative work also showed the weaknesses in the way the formal survey treated migrant workers

The differences are important: the qualitative surveys, selected to find for negative impacts if they existed, found negative impacts. But the similarities are striking. As the authors say, the methods are complementary.

This adds more grist to the mill about how we need to devise better methods for tracking the impacts of crises. We need to strengthen methods, and this study gives us some hope.