 |
Evaluating mixed methods research
|
|
|
|
EVALUATING MIXED METHODS RESEARCH
Once the results from each component have been successfully combined to create a set of meta-inferences, it is important to evaluate the quality of the findings pro- duced. However, this is not necessarily straightforward in mixed methods research, primarily because of the different epistemological and ontological positions associ- ated with quantitative and qualitative research; Onwuegbuzie and Johnson (2006) call this ‘the problem of integration’.
Creswell and Plano Clark (2011: 239) suggest that mixed methods researchers should aim for ‘validity’. This is a term widely used by both quantitative and qualita- tive researchers, and as such represents common ground between the two. They characterize validity as the result of a process which involves ‘employing strategies that address potential issues in data collection, data analysis and the interpretations that might compromise the merging or connecting of the quantitative and qualitative strands of the study and the conclusions drawn from the combination’. However, validity is a somewhat contentious term in mixed methods research because it is argu- ably overused, often rather meaninglessly, and is disliked by many qualitative researchers due to its strong associations with quantitative strategies (Creswell and Plano Clark, 2011). As a result, novel evaluative terminology has been created spe- cifically for use in mixed methods research. Teddlie and Tashakkori (2003, 2010) use the concept of inference quality instead of validity to judge the value of the conclu- sions and interpretations that stem from the combination of inferences from each of the components. Greene (2007: 167) suggests that the process of assessing the qual- ity of inferences should reflect a multiplistic stance that:
· focuses on the available data support for the inferences, using data of multiple and diverse kinds
· could include criteria or stances from different methodological traditions
· attends to the nature and extent of the better understanding that is reached with this mixed methods design, as this is the overall aim of mixed methods inquiry.
Furthermore, Teddlie and Tashakkori (2003) also use the term ‘inference transfera- bility’, which reflects quantitative notions of generalizability and qualitative concerns with transferability.
All of this leads to a rather practical question: Which evaluative term(s) should you use in your mixed methods research project? There is clearly no single correct answer to this question. The terms you use will reflect a number of things, such as your own identity as a researcher, your own preferences and the nature of the project you are working on. They will be, in short, the product of your own academic judgement.
In practice, a sensible approach to evaluating mixed methods research is to:
1. Evaluate the research methodology and findings from each data collection com- ponent using an appropriate evaluative term for the strategy involved (e. g. relia- bility and validity for quantitative, and trustworthiness and authenticity (Guba and Lincoln, 1994) for qualitative).
|
|
|
2. Select which terminology you are going to adopt to evaluate the meta-inferences generated through the integration of your findings (validity OR inference quality/ inference transferability). This will depend on:
a. Your personal feelings towards the terms in question based on your perspec- tives of epistemology and ontology
b. Your understanding of, and confidence in using, terminology that is specific to mixed methods
c. The nature of your research project, which may grant priority to either the quantitative or qualitative components.
3. Justify your selection and be consistent in your application of terminology throughout your project and the associated written work.
CREATING A MIXED METHODS RESEARCH PROJECT
Developing a research project based on just a single research strategy can be a chal- lenging enough endeavour, so planning a mixed methods research project with multiple components requires an additional level of preparatory thinking and plan- ning. It makes sense to develop your project only once you have a sound grasp of mixed methods research philosophy, mixed methods designs, and how to combine and evaluate mixed methods research. This section will discuss the creation of a coherent proposal for a mixed methods research project, putting everything explored
in the previous sections into practice. It will consider issues such as the composition of the central research question, how the central research question drives the research design, methodology and ethics, and the task of presenting your work.
|
|
|
