by Keetie Roelen
In his recent article in World Development, Paul Shaffer states that
“[...] the recent decade has been marked by a significant increase in the quantity of mixed methods materials produced on poverty [...]”.
This week the University of East Anglia (UEA) and the Institute of Development Studies (IDS) co-hosted a two-day workshop on Mixed Methods Research in Poverty and Vulnerability at UEA in London. The workshop brought together researchers, consultants, practitioners and students from multiple countries and disciplines with the aim to share ideas and learn lessons about current practice in mixed methods research in poverty and vulnerability in the context of both developing and developed countries. The workshop focused on three different themes – poverty measurement, poverty dynamics and impact evaluation. In doing so it gave rise to discussions about individual methods, the challenges in combining methods, and ways in which to meaningfully communicate findings following the combination of methods. Presentations encompassed technologies such as video, computer gaming, and GIS illustrating the range of methods now being mixed to enrich understandings of particular contexts and dynamics.
A number of recurrent and key issues can be drawn out from the many rich discussions during the workshop:
- Firstly, the notion that despite great advances in mixed methods research in the past decade, it still appears to be hard to sell. Although there is wide acceptance that disciplinary dichotomies and strong distinctions between quantitative and qualitative methods are not conducive to gaining insightful understandings of poverty and vulnerability, workshop participants felt that such dichotomies and distinctions are still very much alive. Researchers experience resistance from colleagues working within strictly defined disciplinary parameters in accepting a mixing of methods as equally rigorous. By the same token, discussions within the theme of impact evaluation revealed that the push for evidence-based policy making and need to demonstrate value for money have led many policy makers and practitioners to favour quantitative and ‘confirmatory’ approaches over qualitative and ‘exploratory’ approaches, as highlighted by James Copestake from the University of Bath.
- But even amongst the ‘converted’ of mixed method researchers, many new and recurrent challenges around individual methods and their integration exist. One such challenge is the multiple meanings of terms used in poverty measurement surveys such as ‘necessity’ and the potential of group-based qualitative methods to reach consensus on such meanings. Eldin Fahmy from the University of Bristol argued that qualitative research might help us take account of individual as well as patterned differences, which means that claims to present a consensus on anything, rather than simply the majority view, should perhaps be challenged. Many other presentations, including that of Janet Seeley from UEA, reflected on the role of qualitative research more widely. Often its role is seen as that of quality control whereby it is used to ‘verify’ or ‘triangulate’ findings from quantitative methods. Both the timing of the qualitative research and the imperative to integrate qualitative and quantitative research might reduce its subversive potential. Janet Seeley, for example, pointed towards the power of stories in their own right, whilst James Copestake argued that the current emphasis in development on evaluating against a theory of change reduced the potential of qualitative research to identify genuine impact or absence thereof.
- The increasing availability of secondary quantitative data sets also calls for reflection on the appropriate role of qualitative research in analysis and more innovative ways of integrating both types of data.
The link between research and policy making was discussed throughout the course of the workshop. Questions were raised about how to communicate findings from mixed methods research avoiding oversimplification or ‘cherry-picking’ of messages in policy briefs. This concern was not exclusive to the discussion around policy, though. It was also pointed out that the strict word limits in academic journals force researchers to segregate analysis and discussion of findings, which means that claims often need to be taken on trust.
The push for researchers to formulate policy recommendations was also expressed as a real concern as much of the mixed methods research in poverty and vulnerability is geared towards gaining understanding rather than seeking to provide answers or solutions to the problems.
The wealth of experiences and range of ongoing initiatives that were presented during the workshop indicate that there is still great momentum around mixed methods research in poverty and vulnerability. The large range of studies that came about in the last decade have ensured that the ways in which mixing methods can add value to research in poverty and vulnerability are now well understood. In the years to come, it is time to be more critically reflective about individual methods, the ways in which they can be meaningfully combined and the engagement with those outside of the mixed methods ‘space’.
This blog post was written by Keetie Roelen (IDS) and Laura Camfield (UEA). A version of this post can also be found on the UEA website:
http://www.uea.ac.uk/international-development/dev-blog. Proceedings from the workshop can soon be found here:
http://www.uea.ac.uk/international-development/research/poverty-and-vulnerability
