The term mixed methods refers to an emerging research methodology that promotes the systematic integration, or blending, of quantitative and qualitative data within a single sustained research or program of inquiry. The basic premise of this methodology is that such integration allows a more complete and synergistic use of the data than the collection and analysis of quantitative and qualitative data separately.
Mixed methods research originated in the social sciences and has recently expanded into the health and medical sciences. It includes fields such as nursing, family medicine, social work, mental health, pharmacy, and allied health, among others. In the last decade, its procedures have been developed and refined to suit a wide variety of research questions (Creswell & Plano Clark, 2011).
These procedures include advancing rigor, offering alternative mixed-method designs, specifying a shorthand notation system to describe designs in order to increase communication between fields. It includes visualizing procedures through diagrams, looking at research questions that may particularly benefit from integration, and developing rationales for conducting various forms of mixed methods studies.
Characteristics of Mixed Methods
The main features of a well-designed mixed methods study in research are as follows:
- Collect and analyze quantitative (closed) and qualitative (open) data.
- Use rigorous procedures in data collection and analysis appropriate to the tradition of each method, such as ensuring adequate sample size for quantitative and qualitative analysis.
- Integrate data during data collection, analysis or discussion.
- Use procedures that apply qualitative and quantitative components simultaneously or sequentially, with the same sample or with different samples.
- Framing procedures within philosophical/theoretical models of research, such as within a social constructionist model that seeks to understand multiple perspectives on the same topic, for example, what patients, carers, clinicians, and office staff would characterize as “high quality treatment”.
Uses of Mixed Methods Research Designs
Mixed methods may be an ideal technique for evaluating complex interventions (Homer, Klatka, Romm, et al., 2008; Nutting, Miller, Crabtree, et al., 2009). Evaluators can choose from five primary mixed methods designs depending on the research questions they want to answer and the resources available for the evaluation.
Validate results using quantitative and qualitative data sources
Evaluators can use a convergent design to compare results from qualitative and quantitative data sources. This implies:
- The collection of both types of data at approximately the same time.
- The evaluation of information using parallel constructs for both types of data
- Separate analysis of both types of data; and the comparison of the results by procedures such as side-by-side comparison in a debate, transformation of the qualitative data set into quantitative scores.
- The joint presentation of both forms of data.
For example, the researcher may collect qualitative data to assess the personal experiences of patients. At the same time, collect data from survey instruments that measure quality of care. The two types of data can validate each other and also create a strong basis for drawing conclusions about the intervention.
Use qualitative data to explore quantitative results
This explanatory sequential design usually includes two phases:
(1) an initial quantitative instrument phase, followed by
(2) a qualitative data collection phase, in which the qualitative phase is based directly on the results of the quantitative phase.
In this way, the quantitative results are explained in more detail through the qualitative data. For example, the findings of the instrument data on costs can be further explored with qualitative focus groups to better understand how individuals’ personal experiences match the instrument results. This type of study illustrates the use of mixed methods to explain qualitatively how quantitative mechanisms can work.
Develop survey instruments
Another mixed methods study design could support the development of appropriate quantitative instruments that provide precise measurements. This exploratory sequential design implies, first, the collection of qualitative exploratory data, the analysis of the information and the use of the results to develop a psychometric instrument well adapted to the sample under study.
Next, this instrument is administered to a sample of the population. For example, a study on PCMH might begin with a qualitative exploration through interviews with primary care providers to assess what constructs should be measured to better understand improving quality of care. From this exploration, an instrument could be developed using rigorous scale development procedures that is then tested on a sample. In this way, researchers can use a mixed methods approach to develop and test a psychometric instrument that improves on existing measures.
Using qualitative data to augment a quantitative outcome study
An outcome study, such as a randomized controlled trial, to which qualitative data collection and analysis are added, is called an integrated design. In this type of outcome study, the researcher collects and analyzes quantitative and qualitative data. Qualitative data may be incorporated into the study early on (for example, to help design the intervention); during the intervention and after the intervention (for example, to help explain the results). Thus, qualitative data augments the study of outcomes, which is a popular approach within implementation and dissemination research (Palinkas, Aarons, Horwitz, et al., 2011).
Engage Community Stakeholders
A participatory community-based approach is an example of a multi-stage design. This advanced mixed-methods approach engages community participants in many quantitative and qualitative phases of research to bring about change (Mertens, 2009). All of the multiple phases address a common goal of evaluating and refining the models. Key stakeholders participate as co-investigators in a project, providing input on their needs, ways to address them, and ways to implement changes.
Advantage
Using a mixed methods study has several advantages, which we discuss below.
Compare quantitative and qualitative data
Mixed methods are especially useful for understanding contradictions between quantitative and qualitative results.
Reflect the point of view of the participants
Mixed methods give study participants a voice and ensure that study conclusions are based on participants’ experiences.
Encourage academic interaction
These types of studies expand the research of multidisciplinary teams by encouraging the interaction of specialists in quantitative, qualitative and mixed methods.
Provides methodological flexibility
Mixed methods have great flexibility and adapt to many study designs, such as observational studies and randomized trials, to elucidate more information than can be obtained in purely quantitative research.
Collect rich and complete data
Mixed methods also reflect the way individuals naturally collect information, integrating quantitative and qualitative data. For example, sports stories often integrate quantitative data (scores or number of errors) with qualitative data (descriptions and images of highlights) to provide a more complete story than either method would on its own.
Limitations
Mixed methods studies are difficult to implement, especially when used to evaluate complex interventions. Several challenges are discussed below.
Increases the complexity of evaluations
Mixed methods studies are complex to plan and carry out and require careful planning to describe all aspects of the research, including the study sample for the qualitative and quantitative parts (identical, integrated or parallel), the schedule (the sequence of the qualitative and quantitative parts) and the plan to integrate the data. Integrating qualitative and quantitative data during analysis is often a difficult phase for many researchers.
Depends on a multidisciplinary team of researchers
Conducting high-quality mixed methods studies requires a multidisciplinary team of researchers who, in the service of the larger study, must be open to methods that may not be their area of expertise. Finding qualitative experts who are also comfortable discussing quantitative analyzes and vice versa can be challenging in many settings. Since each method must adhere to its own standards of rigor, ensuring adequate quality for each component of a mixed methods study can be difficult.
For example, quantitative analyzes require much larger sample sizes to obtain statistical significance than qualitative analyses, which require meeting the goals of saturation (not discovering new information by conducting more interviews) and relevance. Embedded samples, in which a qualitative subsample is embedded within a larger quantitative sample, can be useful in cases of inadequate statistical power.
requires more resources
Lastly, mixed methods studies are labor intensive and require more resources and time than is required to conduct a single method study.
Conclusions
The integration of quantitative and qualitative data in the form of a mixed methods study has great potential to reinforce the rigor and enrich the analysis and conclusions of any evaluation. By carefully selecting the mixed method design that best fits the evaluation questions and meets their resource constraints, evaluators can facilitate deeper and more meaningful learning regarding the effectiveness and implementation of the models.
