Mixed methods research expanding the evidence base

	Journal of Practical Studies in Education
	ISSN: 2634-4629
	www.jpse.gta.org.uk

Mixed-Methods Research: A Discussion on its Types, Challenges, and Criticisms

Volume: 2 Issue: 2

How to cite this paper: Dawadi, S., Shrestha, S., & Giri, R. A. (2021). Mixed-Methods Research: A Discussion on its Types, Challenges, and Criticisms. Journal of Practical Studies in Education, 2(2), 25-36
DOI: https://doi.org/10.46809/jpse.v2i2.20

Keywords: Mixed Methods Research, Research Paradigm, Challenges, Criticism

1. Introduction

An in-depth understanding of the research paradigms is essential for a researcher. When novice researchers encounter a social problem, they must know how best to approach it. For instance, they must understand the paradigms that guide their methodological decisions in collecting information (data), analysing and interpreting them, and reporting findings. In other words, new researchers must understand what research designs are there that can best address their research problems and guide them throughout the research process. With novice researchers in view, this article introduces the most prevalent research paradigms and the resultant research methods. It particularly focuses on the mixed-methods research (MMR) - its characteristics, reasons for using it, and its major types. The language and organisation of the article are deliberately simple to assist researchers to understand what different types of MMR approaches there are, how to decide which type of MMR is appropriate for their research study, and what the key considerations are when choosing a mixed-method design. Additionally, the chapter provides an understanding of practical considerations and the potential challenges a researcher is likely to experience when adopting a particular MMR design. What follows, then, is a brief discussion of major research paradigms followed by an introduction to mixed-methods research, its types, key considerations, and challenges.

2. Major Research Paradigms

JPSE 2 (2): 25-36 Dawadi, S., Shrestha, S., & Giri, R. A.

study what interests them and are of value to them. They study research problems in different ways that they deem appropriate. Therefore, the main reason for adopting a pragmatist position in a study is to allow a researcher to have a pluralistic stance of gathering all sorts of data in order to best answer the research questions. In essence, a pragmatist employs a mixed-methods design to follow one or multiple combinations of some of the prevalent research paradigms mentioned above. In a mixed-methods research design, qualitative research approaches help understand the situation through indicative results by exploring through the tools like participant observation and interviews whereas quantitative approaches help derive objective findings by using the tools like a survey. A description of mixed methods as a research design is presented below.

4. Why Mixed Methods?

Mixing two methods might be superior to a single method as it is likely to provide rich insights into the research phenomena that cannot be fully understood by using only qualitative or quantitative methods. A mixed-methods design can integrate and synergize multiple data sources which can assist to study complex problems (Poth & Munce, 2020). The application of MMR, as mentioned in the previous section, means purposeful data consolidation which allows researchers to seek a wide view of their study by enabling them to view a phenomenon from different perspectives and research lenses (Shorten & Smith, 2017).

conclusion (Maxwell, 2016; Morgan, 2014). In other words, mixing two methods helps to produce a more complete picture and provides an opportunity for a greater assortment of divergent or complementary views; which are valuable as they not only lead to extra reflection and enrich our understanding of a phenomenon, but also open new avenues for future inquiries (Teddlie & Tashakkori, 2009). Additionally, findings from mixed-methods research offer a holistic view of a phenomenon and provide additional insights into different components of a phenomenon which might help for generating substantive theories (Ventakesh et al., 2013).

Third, an MMR approach helps “to overcome the epistemological differences between quantitative and qualitative paradigms and to provide a royal road to true knowledge” (Bergman, 2008, p. 4). Indeed, a principled combination of the two methods supports researchers in developing an in-depth and comprehensive understanding of a research phenomenon (Lund, 2012). For example, while using a quantitative method, concepts can be operationalised in terms of well-defined indicators, tracing trends and relationships, making comparisons, and using large and perhaps representative samples, a qualitative method has the strengths of sensitivity to multiple meanings, logical ground, great methodological flexibility and in-depth study of smaller samples which helps to study the process and change.

In a mixed-methods study, the selection of a proper design is not an easy task for most researchers. Careful consideration should be given to three major aspects while selecting an MMR design. The first decision is about the relative priority of the approaches. Priority refers to the relative importance of the qualitative and quantitative data for answering research questions (Plano Clark & Ivankova, 2016). The priority usually depends on the research questions or the goals of the research and its participants. A study can have three priority options: quantitative priority (i.e., more emphasis on the quantitative data collection and analysis), qualitative priority (i.e., more emphasis on the qualitative data collection and analysis), or equal priority (i.e., considering both data sets to be equally important to answer the research questions) (Plano Clark & Ivankova, 2016). A researcher, then, must weigh carefully the purpose of their research and the data they need to address it before prioritising research approaches.

The second decision accentuates the level of interaction between the data sets. It refers to the extent to which qualitative and quantitative approaches “are kept independent or interact with each other” (Creswell & Plano Clark, 2011, p. 64). When they are independent, the researcher mixes the two approaches only at the final stage, i.e., after the analysis of the data. As one of the purposes of using mixed methods methodology in a study is to obtain different but complementary data on the same issue to best understand the research problems, the data can be collected separately, and the findings can be mixed before interpreting the results. Creswell and Plano Clark (2011) discuss four possible stages for mixing two data sets: at the level of design, during data collection, during data analysis, and during data interpretation.

Regarding sequential combination, Achterberg (1988) suggests that a qualitative method should precede quantitative methods so that detailed information can be collected and more directed, specific quantitative procedures can be developed. However, the type of combination should be driven by research goals and context. In general, if the research goal is to understand the phenomenon as it happens, it seems that a concurrent approach will be better, but if the researcher expects that findings from a method (either qualitative or quantitative) will support the later (quantitative or qualitative) study, then a sequential approach should be used (Creswell, 2003).

In addition to the above key considerations, the sample size in a mixed methods research design can be different for qualitative and quantitative strands. The sample participating in a qualitative strand can be a subset of the participants who participate in the quantitative study. The researcher should also be aware of the issue that it will bring complexity in the merging process while analysing and interpreting the data. And since one of the purposes is also to synthesize different results into a complementary picture of the issue being explored (Creswell & Plano Clark, 2018), the size differential should not be a big issue. Creswell and Plano Clark (2018) state that having a small size in qualitative component and larger size in quantitative component supports researchers to get in-depth qualitative exploration and rigorous quantitative examination of the issue.

In the analysis phase, a researcher can always look for the common concepts across both sets of findings. Integration in convergent design can be done in two ways: a) by presenting findings of the qualitative study followed by the quantitative study or vice versa or b) by transforming the qualitative data into counts and integrating the transformed qualitative dataset into quantitative data (Creswell & Plano Clark, 2018). In the latter case, a researcher can count the identified codes or themes.

29

In this design, firstly, a quantitative strand is designed and implemented, and later the specific quantitative findings which will be explained is decided. Secondly, a qualitative strand is designed and developed to explain the quantitative findings. And finally, the quantitative results are summarized and interpreted. Creswell and Plano Clark (2018) argue that integration in this design takes place in two ways: a) by connecting the quantitative findings to the qualitative data collection and b) by drawing integrated findings after combining two sets of results after the qualitative phase is completed.

This design is useful when a researcher and research issue is more quantitatively oriented; when s/he has already identified a variable to measure; when s/he has an ability to access the participants to collect the qualitative data; when s/he has time to collect data in two phases; and when s/he is the sole investigator, collecting and analysing the data one at a time. Wilkinson and Staley (2019) in their study found that sampling was one of the problems associated with this design. They pointed out that the reviewers of the research papers, which they analysed, were concerned with "how well the sample for the qualitative component represented the phenomenon identified in data from a larger sample of participants and analysed in the quantitative portion of the study" (p. 70). Creswell and Plano Clark (2018) enumerate the extended time needed for completion, the complexity in specifying the qualitative phase in advance, the compulsion for the identification of quantitative results to be followed up, and the need to specify the participants who can provide the explanation as challenges of this design. In this design, a researcher needs to spend too much time to implement two phases and a researcher might face difficulty to get approval from the institutional board since it will be challenging for a researcher to specify the qualitative phase beforehand. A researcher also needs to decide the quantitative results to be followed on, and they also need to decide who to study and what will be the criteria for sampling (Creswell & Plano Clark, 2018). The following is an example of the study that explains explanatory sequential design.

The first connection of the quantitative and qualitative phase was the use of the quantitative results to create the focus group questions. The second connection was the mixing that happened after the qualitative data were collected and analyzed. The results were connected to gain a better understanding of the findings from both phases. (pp. 204-205)

The above discussions indicate that the quantitative phase informed the qualitative phase and the qualitative phase provided further explanations to the quantitative results. Thus, the two data sets helped the authors to reflect on the research issues with sufficient breadth and depth.

JPSE 2 (2): 25-36 Dawadi, S., Shrestha, S., & Giri, R. A.

Third, quantitative and qualitative methods are guided by different epistemological and philosophical frameworks. Therefore, the concerns in integrating them "include whether the assumptions in each paradigm get the same value or attention in the study and whether the data derived from the two methodologies are viewed as incommensurable" (Salehi & Golafshani, 2010, p.189). Similarly, Yu (2012) points out, ''the difficulty associated with this design is the quantitative measures must be compatible with the qualitative findings, which requires distinct and accurate themes to be found in the qualitative data’’ (p. 375). Although researchers give equal priority/value and weight to both methods by considering that they complement each other, a big challenge may arise when the findings drawn from one method contradict the those from the other method, questioning the reliability and validity of one method (Salehi & Golafshani, 2010). As pointed out by Plano Clark and Creswell (2018), most mixed-methods researchers obtain conflicting results from the qualitative and quantitative strands. There might also be frequent problems in relating two different kinds of information and drawing a conclusion from them (Hammersley, 2014). A researcher, therefore, needs to ensure that different methods are suitably combined so that there is no compromise on the robustness and reliability of the research. Therefore, a mixed-methods researcher needs to have a wider set of skills to conduct research rigorously. To reiterate, mixing methods is not a solution in itself, it might also create some problems.

The fourth challenge associated with the mixed-methods approach is that of choosing a proper design and maintaining quality in data integration. Sometimes, there might be a case that one method may influence data collection and interpretation of another method. For instance, in a sequential design, the findings drawn from the first method (e.g., survey data) may influence the second method (e.g., interview). A concurrent design might have similar issues. Leal et al. (2018) express their concern that ‘‘the concurrent collection of both quantitative and qualitative in a single written survey from the sample participants could result in each data method unintentionally influencing the other’’ (p. 51).

8. Criticism of Mixed Methods Research

Researchers hold different views with regard to the use of both quantitative and qualitative methods within a single study. Hammersley (1996), for example, states that qualitative and quantitative research paradigms are “founded on incommensurable philosophical and/or political presuppositions'' (p. 2). Indeed, a quantitative approach, which is guided by positivism, envisions the world mostly as static but for a qualitative approach, which is guided by interpretivism, the reality is multiple and dynamic. Therefore, incompatibilists argue that the two approaches are incompatible as they have different conceptions of reality, truth, the relationship between the researcher and object of investigation, and so forth. Guba (1987) claims, "The one [paradigm] precludes the other just as surely as belief in a round world precludes belief in a flat one" (p.

various paradigms are beginning to ‘interbreed’ such that two theories previously thought to be in irreconcilable conflict may now appear, under a different theoretical rubric [eclecticism in this case], to be informing one another's arguments. (p. 254)

Thus, the driving motive for combining the two approaches is the belief that both kinds of research have value, that in some respects they are complementary to each other, and that there are benefits of combining them together. As such, there are several rationales for using a mixed-methods approach.

In answering the question of ‘why mixed-method research design?’, the article provided a number of justifications. First and foremost, the article pointed out that the need for a mixed-method research design is to address the need of researching

33

References

Achterberg, C. (1988). Qualitative methods in nutrition education evaluation research. Journal of Nutrition Education, 20(5), 244-250.

Creswell, J. W. (2007). Qualitative Inquiry and Research Design: Choosing Among Five Approaches. Sage Publications. Creswell, J. W. (2015). A concise Introduction to Mixed Methods Research. Sage Publications Ltd.

Creswell, J. W., & Plano Clark, V. L. (2011). Designing and Conducting Mixed Methods Research. Sage Publications. Creswell, J. W., & Plano Clark, V. L. (2018). Designing and Conducting Mixed Methods Research (3rd ed.). Sage Publications.

Guba, E. G. (1987). What have we learned about naturalistic evaluation? Evaluation Practice, 8(1), 23-43.

Hammersley, M. (1996). The relationship between qualitative and quantitative research: Paradigm loyalty versus methodological eclecticism. In J. Richardson (Ed.) Handbook of qualitative research methods for psychology and the social sciences. British Psychological Society Books.

Maxwell, J. A. (2016). Expanding the history and range of mixed methods research. Journal of Mixed Methods Research, 10(1), 12–27. https://doi.org/10.1177/1558689815571132
Morgan, D. L. (2014). Integrating qualitative and quantitative methods: A pragmatic approach. Sage Publications. Munce, S. E. P., Guetterman, T. C. & Jaglal, S. B. (2021). Using the exploratory sequential design for complex intervention development: Example of the development of a self-management program for spinal cord injury. Journal of Mixed Methods Research.15(1) 37–60. https://doi.org/10.1177/1558689820901936
Plano Clark, V. L. & Ivankova, N. V. (2016). Mixed methods research. A guide to the field. Sage Publications.

Poth, C., & Munce, S. E. P. (2020). Commentary—Preparing today’s researchers for a yet unknown tomorrow: Promising practices for a synergistic and sustainable mentoring approach to mixed methods research learning. International Journal of Multiple Research Approaches, 12(1), 56-64. doi:10.29034/ijmra.v12n1commentary
Salehi, K. & Golafshani, N. (2010). Using mixed methods in research studies: An opportunity with its challenges.

Taylor, P. C., & Medina, M. (2011). Educational research paradigms: from positivism to pluralism. College Research Journal, 1(1), 1-16.

Teddlie, C., & Tashakkori, A. (Eds.) (2009). Foundations of Mixed Methods Research: Integrating Quantitative and Qualitative Approaches In the Social and Behavioral Sciences. Sage Publications.

35

JPSE 2 (2): 25-36 Dawadi, S., Shrestha, S., & Giri, R. A.