The report is built on 10 years of DFID-supported research. It is unique in that it draws on a nonrandom sample of 100 case studies of citizen engagement from 20 countries, and maps over 800 observable effects of citizenship participation in 4 categories of outcomes: (a) construction of citizenship, (b) strengthening the practices of participation, (c) strengthening responsive and accountable states and (d) development of inclusive and cohesive societies. Effects are classified into positive and negative outcomes in these four categories.
The study finds that 75% of the citizen engagement outcomes are positive and 25% negative.
The things I liked about the study:
- Its ambition and vision. As far as I know, no other DFID Research Centre has attempted this type of systematic case study analysis
- It covers the more instrumental development outcomes (Did people get more access to state services and resources? Was the state more responsive and accountable? Were there new forms of collective action? Were new issues identified? For example the review points out how a campaign in Mexico led to budget reforms around maternal mortality, how South African citizens helped expand access to antiretroviral vaccines, how the urban reform movement in Brazil increased public housing and increase state capacity for urban planning)
- But it also covers the more intrinsic outcomes (increased civic and political knowledge—which can allow more informed citizen choices, and a greater sense of agency and empowerment)
- The authors are really careful about not simply following an established systematic review protocol—they adapt one (EPPI) to their own data, innovating as they go along
- They create their final universe of studies in a transparent way: their initial universe is the “hundreds” of outputs produced by the Centre between 2003 and 2010. These are then put through a 4 criterion filter (does citizen engagement occur? Presence of empirical work? Original research product? English language?). The authors do not use an explicit quality criterion for weighting purposes, arguing that the “shared origins” of the cases means that this is less important (although we are never told why this should be the case—perhaps because one quality screen has already been applied by the Centre)
- The authors take on the tensions between particularization and generalization—something usually either ignored or deemed too difficult
- They resist the temptation to accentuate the positive outcomes of citizen engagement but also describe the negatives
But the most interesting thing…
- They look at the distribution of outcomes by (a) country democracy type and (b) type of citizen engagement. They find that more democratic contexts do not necessarily generate more positive outcomes of citizen engagement.
- They also find that successful outcomes in the least democratic contexts are associated with local associations and in the most democratic contexts successful outcomes are associated with social movements and campaigns.
A step too far?
It would be interesting to take these 800 or so outcomes and begin to run some regressions on them, exploring the statistical associations between the contexts, types of citizen engagement, and different types of outcomes. For that, however, we would probably have to weight the observations in some way, and that might be pushing the data too far!