Chapter 10: Mixed Methods Procedures 

Chapter 10: Mixed Methods Procedures 

1. Discuss the reasons for using mixed methods presented in the chapter. For each reason, provide an example of a study that would benefit.
2. Discuss at least one example of integration in mixed methods research. Use the appropriate term (merging, explaining,building, embedding) and give an example.
3. Chapter 10 gives six factors considered important when choosing a particular mixed methods design. Consider a study you are interested in conducting. First, identify a mixed methods design. Then,discuss each of these factors in providing a rationale for your design choice.
4. Select one of the complex mixed methods designs(experimental, case study, participatory-social justice, evaluation). What is an example of a study you might conduct using that particular design?
5. Review the mixed methods intervention design presented in Figure 10.2. What are the validity threats that may be present in this design?

How would you write a mixed methods procedure section for your proposal or study? Up until this point, we have considered collected quantitative data and qualitative data. We have not discussed “mixing” or combining the two forms of data in a study. We can start with the assumption that both forms of data provide different types of information (open-ended data in the case of qualitative and closed-ended data in the case of quantitative). If we further assume that each type of data collection has both limitations and strengths, we can consider how the strengths can be combined to develop a stronger understanding of the research problem or questions (and, as well, overcome the limitations of each). In a sense, more insight into a problem is to be gained from mixing or integration of the quantitative and qualitative data. This “mixing” or integrating of data, it can be argued, provides a stronger understanding of the problem or question than either by itself. Mixed methods research, therefore, is simply “mining” the databases more by integrating them. This idea is at the core of a new methodology called “mixed methods research.”

Conveying the nature of mixed methods research and its essential characteristics needs to begin a good mixed methods procedure. Start with the assumption that mixed methods is a methodology in research and that the readers need to be educated as to the basic intent and definition of the design, the reasons for choosing the procedure, and the value it will lend to a study. Then, decide on a mixed methods design to use. There are several from which to choose; consider the different possibilities and decide which one is best for your proposed study. With this choice in hand, discuss the data collection, the data analysis, and the data interpretation, discussion, and validation procedures within the context of the design. Finally, end with a discussion of potential ethical issues that need to be anticipated in the study, and suggest an outline for writing the final study. These are all standard methods procedures, and they are framed in this chapter as they apply to mixed methods research. Table 10.1 shows a checklist of the mixed methods procedures addressed in this chapter.

COMPONENTS OF MIXED METHODS PROCEDURES

Mixed methods research has evolved into a set of procedures that proposal developers and study designers can use in planning a mixed methods study. In 2003, the Handbook of Mixed Methods in the Social and Behavior Sciences (Tashakkori & Teddlie, 2003) was published (and later added to in a second edition, see Tashakkori & Teddlie, 2010), providing a comprehensive overview of this approach. Now several journals emphasize mixed methods research, such as the Journal of Mixed Methods Research, Quality and Quantity, Field Methods, and the International Journal of Multiple Research Approaches. Additional journals actively encourage this form of inquiry (e.g., International Journal of Social Research Methodology, Qualitative Health Research, Annals of Family Medicine). Numerous published research studies have incorporated mixed methods research in the social and human sciences in diverse fields such as occupational therapy (Lysack & Krefting, 1994), interpersonal communication (Boneva, Kraut, & Frohlich, 2001), AIDS prevention (Janz et al., 1996), dementia caregiving (Weitzman & Levkoff, 2000), occupational health (Ames, Duke, Moore, & Cunradi, 2009), mental health (Rogers, Day, Randall, & Bentall, 2003), and in middle school science (Houtz, 1995). New books arrive each year solely devoted to mixed methods research (Bryman, 2006; Creswell, 2015; Creswell & Plano Clark, 2018; Greene, 2007; Morse & Niehaus, 2009; Plano Clark & Creswell, 2008; Tashakkori & Teddlie, 1998, 2010; Teddlie & Tashakkori, 2009).

Table 10.1 A Checklist of Questions for Designing a Mixed Methods Procedure

___________

Is a basic definition of mixed methods research provided?

___________

Are the reasons (or justification) given for using both quantitative and qualitative data in your study?

___________

Does the reader have a sense for the potential use of mixed methods research?

___________

Are the criteria identified for choosing a mixed methods design?

___________

Is the mixed methods design identified?

___________

Is a visual model (a diagram) presented that illustrates the research strategy?

___________

Are procedures of data collection and analysis mentioned as they relate to the chosen design?

___________

Are the sampling strategies for both quantitative and qualitative data collection mentioned for the design?

__________

Are specific data analysis procedures indicated for the design?

__________

 

Are the procedures for validation mentioned for the design and for the quantitative and qualitative research?

___________

Is the narrative structure of the final study or dissertation or thesis mentioned, and does it relate to the type of mixed methods design being used?

Describe Mixed Methods Research

Because mixed methods research is still somewhat unknown in the social and human sciences as a distinct research approach, it is useful to convey a basic definition and description of the approach in a method section of a proposal. This might include the following:

A definition. Begin by defining mixed methods. Recall the definition provided in Chapter 1. Elements in this definition can now be enumerated so that a reader has a complete set of core characteristics that describe mixed methods (see a more expanded view of defining mixed methods research in Johnson, Onwuegbuzie, & Turner, 2007):

It involves the collection of both qualitative (open-ended) and quantitative (closed-ended) data in response to research questions or hypotheses.

It includes the rigorous methods (i.e., data collection, data analysis, and interpretation) of both quantitative and qualitative data.

The two forms of data are integrated in the design analysis through merging the data, explaining the data, building from one database to another, or embedding the data within a larger framework.

These procedures are incorporated into a distinct mixed methods design that indicates the procedures to be used in a study.

These procedures are often informed by a philosophy (or worldview) and a theory (see Chapter 3).

Terminology. Explain that many different terms are used for this approach, such as integrating, synthesis, quantitative and qualitative methods, multimethod, mixed research, or mixed methodology but that recent writings, such as the SAGE Handbook of Mixed Methods in the Social & Behavioral Sciences and SAGE’s Journal of Mixed Methods Research, tend to use the term mixed methods (Bryman, 2006; Creswell, 2015; Tashakkori & Teddlie, 2010).

Background of methodology. Educate the reader about the background of mixed methods by reviewing briefly the history of this approach to research. It can be seen as a methodology originating around the late 1980s and early 1990s in its current form based on work from individuals in diverse fields such as evaluation, education, management, sociology, and health sciences. It has gone through several periods of development and growth, and it continues to evolve, especially in procedures. Several texts outline these developmental phases (e.g., Creswell & Plano Clark, 2011, 2018; Teddlie & Tashakkori, 2009). This section could also include a brief discussion about the importance or rise of mixed methods today through indicators such as federal funding initiatives, dissertations, and the discipline-specific discussions about mixed methods found in journals across the social and health sciences (see Creswell, 2010, 2011, 2015).

Reasons for choosing mixed methods research. Follow this section with statements about the value and rationale for the choice of mixed methods as an approach for your project. At a general level, mixed methods is chosen because of its strength of drawing on both qualitative and quantitative research and minimizing the limitations of both approaches. At a practical level, mixed methods provides a sophisticated, complex approach to research that appeals to those on the forefront of new research procedures. It also can be an ideal approach if the researcher has access to both quantitative and qualitative data. At a procedural level, it is a useful strategy to have a more complete understanding of research problems and questions, such as the following:

Comparing different perspectives drawn from quantitative and qualitative data

Explaining quantitative results with a qualitative follow-up data collection and analysis

Developing better contextualized measurement instruments by first collecting and analyzing qualitative data and then administrating the instruments to a sample

Augmenting experiments or trials by incorporating the perspectives of individuals

Developing cases (i.e., organizations, units, or programs) or documenting diverse cases for comparisons

Developing a more complete understanding of changes needed for a marginalized group through the combination of qualitative and quantitative data

Evaluating both the processes and the outcomes of a program, an experimental intervention, or a policy decision

Indicate the type of mixed methods design that will be used in the study and the rationale for choosing it. A detailed discussion of the primary strategies available will be discussed shortly. Include a figure or diagram of these procedures.

Challenges to design. Note the challenges this form of research poses for the inquirer. These include the need for extensive data collection, the time-intensive nature of analyzing both qualitative and quantitative data, and the requirement for the researcher to be familiar with both quantitative and qualitative forms of research. The complexity of the design also calls for clear, visual models to understand the details and the flow of research activities in this design.

TYPES OF MIXED METHODS DESIGNS

There have been several typologies for classifying and identifying types of mixed methods strategies that proposal developers might use in their proposed mixed methods study. Creswell and Plano Clark (2018) identified several classification systems drawn from the fields of evaluation, nursing, public health, education policy and research, and social and behavioral research. In these classifications, authors used diverse terms for their types of designs, and a substantial amount of overlap of types existed in the typologies. For purposes of clarifying the design discussion in the mixed methods field, we will identify three core mixed methods designs (as shown in Figures 10.1 and 10.2)—the convergent design, the explanatory sequential design, and the exploratory sequential design—and then briefly mention more complex designs (i.e., the mixed methods experimental design, the mixed methods case study design, the mixed methods participatory–social justice design, and the mixed methods evaluation design) in which the core designs can be embedded. Each approach will be discussed in terms of a description of the design, the forms of data collection and data analysis and integration, interpretation, and validity challenges.

Convergent Mixed Methods Design

Description of the design. The convergent mixed methods design is probably the most familiar of the core and complex mixed methods approaches. Researchers new to mixed methods typically first think of this approach because they feel that mixed methods only consists of combining the quantitative and qualitative data. In this single-phase approach, a researcher collects both quantitative and qualitative data, analyzes them separately, and then compares the results to see if the findings confirm or disconfirm each other (see Figure 10.1). The key assumption of this approach is that both qualitative and quantitative data provide different types of information—often detailed views of participants qualitatively and scores on instruments quantitatively—and together they yield results that should be the same. It builds off the historic concept of the multimethod, multitrait idea from Campbell and Fiske (1959), who felt that a psychological trait could best be understood by gathering different forms of data. Although the Campbell and Fiske conceptualization included only quantitative data, the mixed methods researchers extended the idea to include the collection of both quantitative and qualitative data.

Figure 10.1 Three Core Mixed Methods Designs

Data collection. The qualitative data can assume any of the forms discussed in Chapter 9, such as interviews, observations, documents, and records. The quantitative data can be instrument data, observational checklists, or numeric records, such as census data, as discussed in Chapter 8. Ideally, the key idea with this design is to collect both forms of data using the same or parallel variables, constructs, or concepts. In other words, if the concept of self-esteem is being measured during quantitative data collection, the same concept is asked during the qualitative data collection process, such as in an open-ended interview. Some researchers will use this design to associate certain themes with statistical data using different forms of data for the quantitative and qualitative data collection. For instance, Shaw et al. (2013) compared quality improvement practices in family medicine clinics with colorectal cancer screening rates. Another data collection issue is the sample size for both the qualitative and quantitative data collection process. Unquestionably, the data for the qualitative data collection will be smaller than that for the quantitative data collection. This is because the intent of data collection for qualitative data is to locate and obtain information from a small sample but to gather extensive information from this sample; whereas, in quantitative research, a large N is needed in order to infer meaningful statistical results from samples to a population.

How is this inequality resolved in a convergent mixed methods design? Sometimes mixed methods researchers will collect information from the same number of individuals on both the qualitative and quantitative databases. This means that the qualitative sample will be increased, and it will limit the amount of data collected from any one individual. Another approach would be to weight the qualitative cases so that they equal the N in the quantitative database. One other approach taken by some mixed methods researchers is not to consider the unequal sample sizes a problem. They would argue that the intent of qualitative and quantitative research differs (one to gain an in-depth perspective and the other, to generalize to a population) and that each provides an adequate count. Another issue in sampling is whether the individuals for the sample of qualitative participants should also be individuals in the quantitative sample. Typically, mixed methods researchers would include the sample of qualitative participants in the larger quantitative sample, because ultimately researchers make a comparison between the two databases and the more they are similar, the better the comparison.

Data analysis and integration. Data analysis in a convergent design consists of three phases. First, analyze the qualitative database by coding the data and collapsing the codes into broad themes. Second, analyze the quantitative database in terms of statistical results. Third comes the mixed methods data analysis. This is the analysis that consists of integrating the two databases.

This integration consists of merging the results from both the qualitative and the quantitative findings. One challenge in this design is how to actually merge the two databases since bringing together a numeric quantitative database with a text qualitative database is not intuitive. There are several ways to merge the two databases:

 

 

The first approach is called a side-by-side comparison. These comparisons can be seen in the discussion sections of mixed methods studies. The researcher will first report the quantitative statistical results and then discuss the qualitative findings (e.g., themes) that either confirm or disconfirm the statistical results. Alternatively, the researcher might start with the qualitative findings and then compare them to the quantitative results. Mixed methods writers call this a side-by-side approach because the researcher makes the comparison within a discussion, presenting first one set of findings and then the other. A good example of this can be seen in the Classen and colleagues’ (2007) study.

Researchers can also merge the two databases by changing or transforming qualitative codes or themes into quantitative variables and then combining the two quantitative databases—a procedure in mixed methods research called data transformation. The researcher takes the qualitative themes or codes and counts them (and possibly groups them) to form quantitative measures. Some useful procedures that mixed methods researchers have used can be found in Onwuegbuzie and Leech (2006). This approach is popular among researchers trained in quantitative research who may not value or see the worth of an independent qualitative interpretive database.

A final procedure involves merging the two forms of data in a table or a graph. This is called a joint display of data, and it can take many different forms. It might be a table that arrays the themes on the horizontal axis and a categorical variable (e.g., different types of providers such as nurses, physician assistants, and doctors) on the vertical axis. It might be a table with key questions or concepts on the vertical axis and then two columns on the horizontal axis indicating qualitative responses and quantitative responses to the key questions or concepts (Li, Marquart, & Zercher, 2000). The basic idea is for the researcher to jointly display both forms of data—effectively merging them—in a single visual and then make an interpretation of the display (see Guetterman, Fetters, & Creswell, 2015).

Interpretation. The interpretation in the convergent approach is typically written into a discussion section of the study. Whereas the results section reports on the findings from the analysis of both the quantitative and qualitative databases, the discussion section includes a discussion comparing the results from the two databases and notes whether there is convergence or divergence between the two sources of information. Typically the comparison does not yield a clean convergent or divergent situation, and differences exist on a few concepts, themes, or scales. When divergence occurs, steps for follow-up need to be taken. The researcher can state divergence as a limitation in the study without further follow-up. This approach represents a weak solution. Alternatively, mixed methods researchers can return to the analyses and further explore the databases, collect additional information to resolve the differences, or discuss the results from one of the databases as possibly limited (e.g., the constructs were not valid quantitatively or the qualitative themes did not match the open-ended questions). Whatever approach the researcher takes, the key point in a convergent design is to further discuss and probe results when divergent findings exist.

Validity. Validity using the convergent approach should be based on establishing both quantitative validity (e.g., construct) and qualitative validity (e.g., triangulation) for each database. Is there a special form of mixed methods validity that needs to be addressed? There are certainly some potential threats to validity in using the convergent approach, and several of these have already been mentioned. Unequal sample sizes may provide less of a picture on the qualitative side than the larger N on the quantitative side. Generally we find the use of unequal sample sizes in a convergent design study, with the researcher acknowledging the different perspectives on size taken by quantitative and qualitative researchers. The use of different concepts or variables on both sides, quantitative and qualitative, may yield incomparable and difficult-to-merge findings. Our recommended approach is to use the same concepts for both the quantitative and qualitative arms of the research study, but we acknowledge that some researchers use the convergent design to associate different qualitative and quantitative concepts. A lack of follow-up on conclusions when the scores and themes diverge also represents an invalid strategy of inquiry. In this discussion we have recommended several ways to probe divergence in more detail and would recommend the use of one or more of these strategies in a convergent design project.

Explanatory Sequential Mixed Methods Design

Description of the design. The explanatory sequential mixed methods approach is a design in mixed methods that appeals to individuals with a strong quantitative background or from fields relatively new to qualitative approaches. It involves a two-phase data collection project in which the researcher collects quantitative data in the first phase, analyzes the results, and then uses the results to plan (or build on to) the second, qualitative phase. The quantitative results typically inform the types of participants to be purposefully selected for the qualitative phase and the types of questions that will be asked of the participants. The overall intent of this design is to have the qualitative data help explain in more detail the initial quantitative results, thus it is important to tie together or to connect the quantitative results to the qualitative data collection. A typical procedure might involve collecting survey data in the first phase, analyzing the data, and then following up with qualitative interviews to help explain confusing, contradictory, or unusual survey responses.

 

Data collection. The data collection proceeds in two distinct phases with rigorous quantitative sampling in the first phase and with purposeful sampling in the second, qualitative phase. One challenge in this strategy is to plan adequately what quantitative results to follow up on and what participants to gather qualitative data from in the second phase. The key idea is that the qualitative data collection builds directly on the quantitative results. The quantitative results that then are built on may be extreme or outlier cases, significant predictors, significant results relating variables, insignificant results, or even demographics. For example, when using demographics, the researcher could find in the initial quantitative phase that individuals in different socioeconomic levels respond differently to the dependent variables. Thus, the follow-up qualitatively may group respondents to the quantitative phase into different categories and conduct qualitative data collection with individuals representing each of the categories. Another challenge is whether the qualitative sample should be individuals that are in the initial quantitative sample. The answer to this question should be that they are the same individuals, because the intent of the design is to follow up the quantitative results and explore the results in more depth. The idea of explaining the mechanism—how the variables interact—in more depth through the qualitative follow-up is a key strength of this design.

Data analysis and integration. The quantitative and the qualitative databases are analyzed separately in this approach. Then the researcher combines the two databases by the form of integration called connecting the quantitative results to the qualitative data collection. This is the point of integration in an explanatory sequential design. Thus, the quantitative results are then used to plan the qualitative follow-up. One important area is that the quantitative results cannot only inform the sampling procedure but it can also point toward the types of qualitative questions to ask participants in the second phase. These questions, like all good qualitative research questions, are general and open-ended. Because analysis proceeds independently for each phase, this design is useful for student research and perhaps easier to accomplish (than the convergent design) because one database explains the other and the data collection can be spaced out over time.

Interpretation. The mixed methods researcher interprets the follow up results in a discussion section of the study. This interpretation follows the form of first reporting the quantitative, first-phase results and then the qualitative, second phase results. However, this design then employs a third form of interpretation: how the qualitative findings help to explain the quantitative results. A common misstep at this point by beginning researchers is to merge the two databases. While this approach may be helpful, the intent of the design is to have the qualitative data help to provide more depth, more insight into the quantitative results. Accordingly, in the interpretation section, after the researcher presents the general quantitative and then qualitative results, a discussion should follow that specifies how the qualitative results help to expand or explain the quantitative results. Because the qualitative database questions narrows the scope of the quantitative questions, a direct comparison of the overall results of the two databases is not recommended.

Validity. As with all mixed methods studies, the researcher needs to establish the validity of the scores from the quantitative measures and to discuss the validity of the qualitative findings. In the explanatory sequential mixed methods approach, additional validity concerns arise. The accuracy of the overall findings may be compromised because the researcher does not consider and weigh all of the options for following up on the quantitative results. We recommend that researchers consider all options for identifying results to follow up on before settling on one approach. Attention may focus only on personal demographics and overlook important explanations that need further understanding. The researcher may also contribute to invalidated results by drawing on different samples for each phase of the study. If explaining the quantitative results in more depth, then it makes sense to select the qualitative sample from individuals who participated in the quantitative sample. This maximizes the importance of one phase explaining the other. These are a few of the challenges that need to be built into the planning process for a good explanatory sequential mixed methods study.

Exploratory Sequential Mixed Methods Design

Description of the design. If we reverse the explanatory sequential approach and start with a qualitative phase first followed by a quantitative phase, we have an exploratory sequential approach. A three-phase exploratory sequential mixed methods is a design in which the researcher first begins by exploring with qualitative data and analysis, then builds a feature to be tested (e.g., a new survey instrument, experimental procedures, a website, or new variables) and tests this feature in a quantitative third phase. Like the explanatory sequential approach, the second feature builds on the results of the initial database. The intent of this design is to explore with a sample first so that a later quantitative phase can be tailored to meet the needs of the individuals being studied. Sometimes this quantitative feature will include developing a contextually sensitive measurement instrument and then testing it with a sample. Other times it may involve developing new variables not available in the literature or attuned to a specific population being studied, or designing a website or an Internet application shaped to the needs of the individuals being studied. This design is popular in global health research when, for example, investigators need to understand a community or population before administering English-language instruments.

In this design, the researcher would first collect focus group data, analyze the results, develop an instrument (or other quantitative feature such as a website for testing), and then administer it to a sample of a population. In this case, there may not be adequate instruments to measure the concepts with the sample the investigator wishes to study. In effect, the researcher employs a three-phase procedure with the first phase as exploratory, the second as instrument (or quantitative feature) development, and the third as administering and testing the instrument feature to a sample of a population.

 

Data collection. In this strategy, the data collection would occur at two points in the design: the initial qualitative data collection and the test of the quantitative feature in the third phase of the project. The challenge is how to use the information from the initial qualitative phase to build or identify the quantitative feature in the second phase. This is the integration point in an exploratory sequential design.

Several options exist, and we will use the approach of developing a culturally sensitive instrument as an illustration. The qualitative data analysis can be used to develop an instrument with good psychometric properties (i.e., validity, reliability). The qualitative data analysis will yield quotes, codes, and themes (see Chapter 9). The development of an instrument can proceed by using the quotes to write items for an instrument, the codes to develop variables that group the items, and themes that group the codes into scales. This is a useful procedure for moving from qualitative data analysis to scale development (the quantitative feature developed in the second phase). Scale development also needs to follow good procedures for instrument design, and the steps for this include ideas such as item discrimination, construct validity, and reliability estimates (see DeVellis, 2012).

 

Developing a good psychometric instrument that fits the sample and population under study is not the only use of this design. A researcher can analyze the qualitative data to develop new variables that may not be present in the literature, to identify the types of scales that might exist in current instruments or to form categories of information that will be explored further in a quantitative phase. The question arises if the sample for the qualitative phase is the same for the quantitative phase. This cannot be, because the qualitative sample is typically much smaller than a quantitative sample needed to generalize from a sample to a population. Sometimes mixed methods researchers will use entirely different samples for the qualitative (first phase) and quantitative components (third phase) of the study. However, a good procedure is to draw both samples from the same population but make sure that the individuals for both samples are not the same. To have individuals help develop an instrument and then to survey them in the quantitative phase would introduce confounding factors into the study.

 

Data analysis and integration. In this strategy the researcher analyzes the two databases separately and uses the findings from the initial exploratory database to build into a feature that can be analyzed quantitatively. So integration in this design involves using the qualitative findings (or results) to inform the design of a quantitative phase of the research such as the development of a measurement instrument or new variables. This means that the researcher needs to pay careful attention to the qualitative data analysis steps and determine what findings to build on. If, for example, the researcher uses grounded theory (see Chapter 9), the theoretical model generated may provide a model to be tested in the second, quantitative phase. A qualitative case study can yield different cases that become the focus of important variables in the second quantitative phase.

Interpretation. Researchers interpret the mixed methods results in a discussion section of a study. The order of interpretation is to first report the qualitative findings, the development or design of the feature to be tested (e.g., the development of an instrument, the development of new quantitative measures), and then the quantitative test in the final phase of the study. It does not make sense to compare the two databases, because they are typically drawn from different samples (as noted above in the data collection discussion) and the intent of the strategy is to determine if the qualitative themes in the first phase can be generalized to a larger sample.

Validity. Researchers using this strategy need to check for the validity of the qualitative data as well as the validity of the quantitative scores. Special validity concerns arise, however, in using this design that need to be anticipated by the proposal or mixed methods report developer. One concern is that the researcher may not use appropriate steps to develop a good psychometric instrument. Developing a good instrument is not easy, and adequate steps need to be put in place. Another concern is that a researcher may develop an instrument or measures that do not take advantage of the richness of the qualitative findings. This occurs when the qualitative data lacks rigor or occurs simply at the theme level without the further data analysis steps associated with using one of the qualitative design-types, such as ethnography, grounded theory, or case study procedures. Finally, as previously mentioned, the sample in the qualitative phase should not be included in the quantitative phase as this will introduce undue duplication of responses. It is best to have the sample of qualitative participants provide information for scale, instrument, or variable (or website) design, but the same individuals should not complete the follow-up instruments. This sample strategy, therefore, differs from the sampling strategy needed for an explanatory sequential design.

 

Several Complex Mixed Methods Designs

After working with these three core designs—convergent, explanatory sequential, and exploratory sequential—that are the foundation of good mixed methods research, we have now branched out to incorporate more designs that typically fit complex projects. By complex we mean that the designs involve more steps and procedures than are embodied in the three core designs. These mixed methods designs are not more “advanced.” They simply involve more steps and incorporate the core designs into “processes” of research. We have come to this position based on key readings in the mixed methods literature that have surfaced in the past few years. The first step involved isolating and thinking about the types of more complex features that the core designs could be embedded within.

 

A useful typology emerged in the work of Plano Clark and Ivankova (2016). Their book was helpful in conceptualizing the many types of applications of complex designs. In an entire chapter they discussed the intersection of mixed methods with other approaches to form “advanced applications” (p. 136). They recommended a framework for considering the possibilities of these complex applications:

 

Intersecting a secondary method (mixed methods) within a primary quantitative or qualitative research design. A research design is a set of formal procedures for collecting, analyzing, and interpreting data such as those found in a quantitative experiment or qualitative case study. In this framework, a mixed methods core design could be embedded as a secondary (or supportive) method within a primary quantitative or qualitative design. The typical form of this application is to embed qualitative data collection and analysis within a quantitative experimental or intervention design.

Intersecting mixed methods within another methodology. A methodology is a set of procedures that guide the use of design. These procedures exist in the research at a more practical level than the design. In this framework, a mixed methods core design could be added to another methodological approach. For example, a core design could be added to a case study, an evaluation approach, action research, social network analysis, longitudinal research, Q methodology, phenomenology, or grounded theory.

Intersecting mixed methods within a theoretical framework. A theoretical framework advances an abstract and formalized set of assumptions to guide the design and conduct of the research. In this framework, a mixed methods core design could be intersected with an established theory. This theoretical lens could be drawn from perspectives such as social justice, feminism, critical theory, participatory involvement, or other conceptual frameworks that advance the needs and involvement of special populations and often call for action or change.

These three types of complex designs deserve additional attention because many researchers are conducting evaluations, using theoretical orientations such as gender or social inequality theories, and conducting experiments or interventions using mixed methods. In our discussion of mixed methods, we simply need to account for these complex applications and assess how core designs might be embedded within them.

 

Another step forward in designs appeared in Nastasi and Hitchcock (2016). Their book brought forth several ideas that we now incorporate into our complex designs. They suggested that distinct “processes” occur in research in which both quantitative and qualitative data might be used in distinct steps in the overall process. Their book focused on two ideas: the use of mixed methods in program evaluation and its use in experimental, intervention trials. It also relied heavily on the authors’ mixed methods study in Sri Lanka that addressed the mental health of youth, and they advanced the steps in their evaluation process and embedded into these steps the use of qualitative and quantitative data in multiple core designs. From their work we then have some practical examples of incorporating core designs into the complex procedures of an evaluation and an experimental, intervention trial.

 

Specifically, we see embedding the core designs into larger processes. As in Creswell and Plano Clark (2018), here we briefly discuss four examples of complex designs and then discuss a general model for embedding the core designs in these processes:

 

Mixed methods experimental (intervention) design. The mixed methods experimental (or intervention) design involves the researcher collecting and analyzing both quantitative and qualitative data and integrating the information within an experiment or intervention trial (see Figure 10.2). This design adds qualitative data collection into an experiment or intervention so that the personal experiences of participants can be included in the research. Thus the qualitative data become a secondary source of data embedded in the experimental pre- and posttest data collection. It requires the researcher to understand experiments and to be able to design them in a rigorous way (e.g., a randomized controlled trial). As shown in Figure 10.2, researchers add the qualitative data to the experiment in different ways: before the experiment begins, during the experiment, or after the experiment (Sandelowsi, 1996). The basic ideas are to embed the core exploratory sequential design into the experiment to carry out exploration before conducting the experiment; to embed a convergent core design during the experiment to assess participants’ experiences with the intervention; or to add an explanatory sequential design into the experiment after the study to follow up on the experimental outcomes. The points at which the qualitative data collection and findings connect to the experiment represent the integration in the mixed methods study. In this design it is important to be explicit about the reasons for adding the qualitative data. We enumerated several important reasons in Figure 10.2. These lists are representative of the examples of mixed methods research we have found in the literature. The qualitative data collection can occur at a single point in time or at multiple points in time depending on the resources available to the researcher. This type of mixed methods use has become popular in the health sciences.

 

Figure 10.2 Mixed Methods Intervention Design

 

Source: Adapted from Sandelowski (1996).

 

Case study design. The mixed methods case study design involves the use of one or more core designs (i.e., convergent, explanatory sequential, exploratory sequential) within the framework of a single or multiple case study design. The intent of this design is to develop or generate cases based on both quantitative and qualitative results and their integration. We have found two basic variants of this design. One is a deductive approach where researchers establish the cases at the outset of the study and document the differences in the cases through the qualitative and quantitative data. A second is more of an inductive approach where the researcher collects and analyzes both quantitative and qualitative data and then forms cases—often multiple cases—and then makes comparisons among the cases. Regardless of the approach, the challenge is to identify the cases before the study begins or to generate cases based on the evidence collected. Another challenge is to understand case study research (Stake, 1995; Yin, 2014) and effectively intersect case study design with mixed methods. The type of core design embedded within this approach can vary, but we can find good illustrations of the design using a convergent design (Shaw, Ohman-Strickland, & Piasecki, 2013). Within this framework, the typical mixed methods case study design is one where both types of data are gathered concurrently in a convergent core design and the results are merged together to examine a case and/or compare multiple cases. This type of mixed methods case study design is shown in Figure 10.3. In this hypothetical example, the researcher gathers both survey quantitative data and qualitative interview data at roughly the same time. Analysis of both databases produces results that can be merged to identify specific cases. These cases portray different profiles found in the databases, and they can be compared in a cross-case comparison.

Participatory-social justice design. The mixed methods participatory-social justice design is a mixed methods design in which the researcher adds a core design within a larger participatory and/or social justice theoretical or conceptual framework (see Figure 10.4). The intent of this design is to give voice to participants and collaborate with them in shaping the research and to build evidence from both quantitative and qualitative data. As a complex design, these frameworks span the entire mixed methods study. The framework can be, for example, a feminist theory or a racial theory. It might also be a participatory theory of the involvement of stakeholders in many aspects of the mixed methods study (Ivankova, 2015), although it could be debated as to whether participatory action research exists in a study as a conceptual framework or as methodological procedures. This aside, in addition to seeing the strong placement of this theory in the study, we can also identify one or more of the core designs operating. Within a feminist mixed methods study, for example, we can see both the flow of the theory into many aspects of the project (e.g., informing the problem, shaping the research questions, highlighting the outcomes) as well as an embedded core design such as an explanatory sequential design where an initial survey is followed by one-on-one interviews. In Figure 10.4, we see this type of core design embedded within a participatory-social justice framework. This is a study discussing the transition of homeless individuals from a hospital to a shelter (Greysen, 2012). The element that makes this study participatory research is the substantial involvement of community personnel in many aspects of the study. What makes the project mixed methods is the collection and analysis of both quantitative and qualitative data. As shown in Figure 10.4, we see that multiple core designs were embedded within the study. An exploratory sequential core design connected identifying the research priorities and developing a survey. Then, the data collection and analysis portrayed a convergent design with the combination of themes and statistical results.

 

Figure 10.3 Mixed Methods Case Study Design

 

 

Figure 10.4 Mixed Methods Participatory-Social Justice Design

 

Source: Adapted from Greysen et al. (2012); reported in Creswell & Plano Clark (2018).

 

Evaluation design. The mixed methods evaluation design consists of one or more core designs added to the steps in an evaluation procedure typically focused on evaluating the success of an intervention, a program, or a policy (see Figure 10.5). The intent of this design is to engage in a process of research in which both quantitative and qualitative data and their integration shapes one or more of the steps in the process. This complex design illustrates a core design within another methodology. This approach is typically used in program evaluation where quantitative and qualitative approaches are used over time to support the development, adaptation, and evaluation of programs, experiments, or policies. We often find multiple core designs occurring throughout these projects. For example, researchers might start by conducting a qualitative needs assessment study to understand the meaning of smoking and health from the perspective of adolescents in this community. Using these results, the researchers might develop an instrument and quantitatively assess the prevalence of different attitudes across the community. In a third phase, the researchers might develop a program based on what they have learned and then examine both the process and the outcomes of this intervention program. Across these phases the researchers would make use of exploratory (phase 1 to phase 2), explanatory (phase 2 to phase 3), and convergent (phase 3) core designs.

Examine Figure 10.5. This mixed methods evaluation design was used in a study of the mental health of youth in Sri Lanka (Nastasi & Hitchcock, 2016). In the outer circle we see the general steps in the evaluation process. Within boxes in the circle we also find the combination of quantitative and qualitative research. In short, looking across these boxes within the circle, we see that the authors have incorporated multiple core designs at different stages in the evaluation process. The figure also shows within the boxes the dates on which the data were collected.

 

A PROCEDURE FOR EMBEDDING CORE DESIGNS INTO COMPLEX DESIGNS

In the mixed methods evaluation design example in Figure 10.5, we see that core designs can be embedded within a process of evaluation. This provides important clues as to how to embed the core designs within complex procedures such as other designs, theories, or methodologies. It also speaks to how to draw a diagram of the mixed methods procedures. In our thinking, we embed the core designs into more complex procedures using these steps:

 

 

Figure 10.5 Mixed Methods Evaluation Design

 

Source: Nastasi & Hitchcock (2016). Used with permission from SAGE Publishing.

 

Identify the quantitative and qualitative data collection in your study. Refer to whether the data source is closed-ended (quantitative) or open-ended (qualitative).

Draw a diagram of the steps in the procedure. These steps (represented by boxes) may be the phases in an experimental design, the generation of cases, or the phases of an evaluation.

Look into the steps (boxes) and ask yourself at what steps in the procedure you have an opportunity to collect both quantitative and qualitative data. Such data collection, you will recall from Chapter 1, represents a core defining characteristic of mixed methods research.

In those boxes where you are collecting both forms of data, ask yourself further how the databases are being connected. Are they being merged (as in a convergent mixed methods design) or connected (as in an explanatory sequential mixed methods design or an exploratory sequential mixed methods design).

 

Discuss the procedures of using the core mixed methods designs, paying attention to how the data are being integrated in each step.

As is evident in our discussion, we believe in drawing diagrams of procedures, whether of core designs or more complex designs. Besides thinking about how to draw these diagrams, you might consider some of the notations that have emerged in the field of mixed methods research. Mixed methods notation provides shorthand labels and symbols that convey important aspects of mixed methods research, and they provide a way that mixed methods researchers can easily communicate their procedures (see Table 10.2). Morse (1991) first developed the notation, and it has been added to by writers such as Tashakkori and Teddlie (1998) and Plano Clark (2005) who suggest the following:

 

QUAL and QUAN capitalization indicates an emphasis or priority on the quantitative or qualitative data, analysis, and interpretation in the study. In a mixed methods study, the qualitative and quantitative data may be equally emphasized, or one may be more emphasized than the other. Capitalization indicates that an approach or method is emphasized. Lowercase indicates lesser priority or emphasis on the method.

Quan and Qual stand for quantitative and qualitative, respectively, and they use the same number of letters to indicate equality between the forms of data.

A plus sign—+—indicates a convergent or merging integration of data collection—with both quantitative and qualitative data collected at same time.

An arrow—→—indicates a sequential form of data collection; one form (e.g., qualitative data) builds or connects with the other (e.g., quantitative data).

Parentheses—( )—indicate that one form of data collection is embedded within another or embedded within a larger design.

Double arrows—→←—mean that the flow of activities can go both ways.

Also in the figures we see boxes that highlight important major components of the design—such as data collection or data analysis.

Table 10.2 Notation Used in Mixed Methods Research

 

 

Notation

 

What It Indicates

 

Example

 

Citation Establishing Notation

 

Uppercase letters

 

Greater emphasis given to a method

 

QUAN, QUAL

 

Morse (1991)

 

Lowercase letters

 

Lesser emphasis given to a method

 

quan, qual

 

Morse (1991)

 

+

 

Convergent methods

 

QUAN + QUAL

 

Morse (1991)

 

→

 

Sequential methods

 

QUAL→quan

 

Morse (1991)

 

( )

 

Embed within a design or framework

 

QUAN(qual)

 

Plano Clark (2005)

 

→#xF0AC;

 

Recursive

 

QUAL→←QUAN

 

Nastasi et al. (2007)

 

[ ]

 

Study within a series

 

QUAL → [QUAN + qual]

 

Morse & Niehaus (2009)

 

FACTORS IMPORTANT IN CHOOSING A MIXED METHODS DESIGN

The choice of a particular mixed methods design is based on several factors that relate to the intent of the procedures as well as practical considerations. We will begin with the procedural reasons for choosing a particular mixed methods strategy. It should be recognized that many variations exist in mixed methods designs, and the particular approach an investigator has in mind may not conform exactly to the approaches specified here. However, these designs represent the common underlying features of many designs, and, with modification, researchers can find their own strategy. To choose a design for your project, consider the following factors:

 

Choice based on outcomes expected or intent. Earlier in this chapter, we reviewed the reasons for choosing mixed methods research. In Table 10.3, we repeat the reasons but this time link them to expected outcomes of a mixed methods project and the type of mixed methods strategy. This thinking calls for the researcher to determine the outcome anticipated at the end of the mixed methods study and then to link it to the types. These outcomes are, in turn, shaped by the intention behind including and integrating both quantitative and qualitative data.

Choice based on integrating the data together. To choose a mixed methods strategy beyond considering the outcome anticipated, the researcher needs to consider whether mixed methods integration of the two databases will be merged (convergent mixed methods design), explaining (explanatory sequential design), building (exploratory sequential design), or embedded (the complex designs). Merging the data involves combining the quantitative and qualitative data through the procedures of a side-by-side comparison, data transformation, or a joint display. Connecting the data means that the analysis of one data set is used to lead into or build into the second data set. In short, the data analysis of one data set informs the data collection of the other data set. In embedding, one data set—involving quantitative, qualitative, or combined data—is embedded within a larger design, theory, or methodology.

Table 10.3 Choosing a Mixed Methods Project, Expected Outcomes, Type of Design

 

 

Reasons for Choosing Mixed Methods

 

Expected Outcomes

 

Recommended Mixed Methods Design

 

Comparing different perspectives drawn from quantitative and qualitative data

 

Merging the two databases to show how the data converge or diverge

 

Convergent mixed methods design

 

Explaining quantitative results with qualitative data

 

A more in-depth understanding of the quantitative results (often cultural relevance)

 

Explanatory sequential mixed methods design

 

Developing better measurement instruments

 

A test of better measures for a sample of a population

 

Exploratory sequential mixed methods design

 

Understanding experimental results by incorporating perspectives of individuals

 

An understanding of participant views within the context of an experimental intervention

 

Mixed methods experimental (intervention design)

 

Comparing one or more case studies

 

An understanding of the differences and similarities among several cases

 

Mixed methods case study design

 

Developing an understanding of needed changes for a marginalized group

 

A call for action

 

Mixed methods participatory-social justice design

 

Understanding the need for an impact of a program, intervention, or policy

 

A formative and summative evaluation

 

Mixed methods evaluation design

 

For example, in a convergent design the two are considered to be independent and the data collection and analysis for each database proceed separately. In an embedded experimental design, the qualitative data may be collected independently of the experiment and used to support or augment the larger design, the experiment. Alternatively, the two databases may be connected, with one building on the other. This is a sequential type of design (explanatory sequential design or an exploratory sequential design), and one database does not stand in isolation of the other database. In these sequential designs, the data collection in the second phase cannot be conducted until the first phase results are in. In short, the follow-up data collection builds directly on the results of the initial data collection.

 

Choice based on the timing of the data collection. A related factor is timing in mixed methods data collection—whether the two databases are collected concurrently, at roughly the same time, or with one following the other, sequentially. A convergent strategy typically involves collecting data concurrently while the explanatory sequential and the exploratory sequential strategies means that the data will be collected in sequence. Sometimes this criterion is difficult to identify in published mixed methods studies, but it should go into the thinking about choosing a mixed methods strategy. In complex designs, the timing may vary and be included at multiple time points in the design.

 

Choice based on the emphasis placed on each database. Like timing, the emphasis placed on each database in mixed methods research is also somewhat difficult to determine and to apply to the question of choice. A mixed methods study can illustrate an equal emphasis (or priority or weight) on both databases, or an unequal emphasis. For example, a mixed methods project can stress the qualitative phase of the research and give minimal attention to the quantitative phase. How can we tell? We can look at the number of pages in a study to determine emphasis, how the study begins (e.g., with a strong quantitative theory orientation or personal qualitative stories), the amount of depth and sophistication given to the qualitative and quantitative data collection and analysis, or even the background training of the investigator. As mentioned earlier in the section on notation, capital letters may be used in the notation for greater emphasis (e.g., QUAN) and lowercase letters for less emphasis (e.g., quan). The emphasis can help determine the choice of a mixed methods strategy. Typically if the researcher seeks to emphasize both databases, a convergent approach is best. Alternatively, if a stronger emphasis is sought for the quantitative approach, then an explanatory sequential strategy is used because it began with the quantitative component of the study. If a qualitative approach is to be emphasized, then an exploratory sequential strategy is chosen. These are not firm guidelines, but they may play into the overall decision about a choice of strategy.

Choice based on type of design most suited for a field. On a practical level, the choice of a strategy depends on the inclination of fields toward certain mixed methods designs. For quantitatively oriented fields, the explanatory sequential approach seems to work well because the study begins (and perhaps is driven) by the quantitative phase of the research. In qualitatively oriented fields, the exploratory sequential approach may be more appealing because it begins with an exploration using qualitative research. However, in this approach, an outcome may be a measurement instrument that is tested so that the outcome, a quantitative outcome, outweighs in importance how the study began. In some fields, the choice of approach may be dependent on collecting data efficiently, and this would argue for a convergent mixed methods study in which both quantitative and qualitative data are typically collected at roughly the same time rather than at different times that require more visits to the research site.

Choice based on a single researcher or team. A final practical reason for a choice of a strategy depends on whether a single researcher (e.g., graduate student) conducts the study or a team of researchers (e.g., funded long-term investigation). If the investigator is a single researcher, the sequential strategies of an explanatory sequential or exploratory sequential approach are best because the investigation can be divided into two manageable tasks rather than multiple data collection and analysis procedures. The study can be projected out over a period of time rather than collecting multiple forms of data at the same time as in a convergent approach. When time is a problem, we encourage students to think about a convergent design. In this design, both forms of data are gathered at roughly the same time, and it does not require repeated visits to the field to gather data. Complex designs are well-suited for a team of researchers who assist in the multiple phases of the research and for well-funded projects that unfold over several years.

We recommend that students find a published mixed methods journal article that uses their design and introduce it to advisers and faculty committees so that they have a working model to understand the design. Since we are at the early stage of adopting mixed methods research in many fields, a published example of research in a field will help to create both legitimacy for mixed methods research and the idea that it is a feasible approach to research for graduate committees or other audiences. If a research team is conducting the study, multiple forms of data collection at the same time or over a long period of time are possible, such as in an embedded or a multiphase design. Although a single researcher can conduct a participatory–social justice study, the labor-intensive nature of collecting data in the field involving participants as collaborators typically suggests more of a team approach than the inquiry by a single investigator.

EXAMPLES OF MIXED METHODS PROCEDURES

Examples 10.1–10.4 illustrate mixed methods studies that use both the sequential and convergent strategies and procedures.

Example 10.1A Convergent Parallel Mixed Methods Design

Classen et al. (2007) studied older driver safety in order to develop a health promotion intervention based on modifiable factors influencing motor vehicle crashes with older drivers (age 65 and older). It was a good example of a convergent mixed methods study. The central purpose of the study was identified in the abstract:

This study provided an explicit socio-ecological view explaining the interrelation of possible causative factors, an integrated summary of these causative factors, and empirical guidelines for developing public health interventions to promote older driver safety. Using a mixed methods approach, we were able to compare and integrate main findings from a national crash dataset with perspectives of stakeholders. (p. 677)

This purpose statement identified the use of both quantitative (i.e., a national crash data set) and qualitative (i.e., stakeholders’ perspectives) data. From one of the research questions in the study, we learned that the authors compared the qualitative stakeholder perspectives, needs, and goals for safe and unsafe driving with the quantitative results of the factors that influenced driving injuries. So the expected outcome was to compare the findings. The method section commented on the quantitative national data set, the statistical analysis of this data set, and then the qualitative data set and its analysis. Although not stated explicitly, the data were used together to form results, not used for one database to build on another, and the timing was to look at both databases concurrently. A diagram illustrated the procedures involved in both collecting and analyzing the information. A results section first reported the quantitative results and then the qualitative results. More emphasis was given to the quantitative results, leading to the conclusion that this study favored the quantitative research. However, these reports on the results from the two databases were followed by an analysis of key findings in which the quantitative and qualitative results were compared for supportive and non-supportive findings. In this discussion section, the researchers merged the two databases in a side-by-side comparison. Looking more broadly at the topic and the authors, we saw that the quantitative emphasis would probably be better accepted in the field of occupational therapy than qualitative research. Also, a scan of the authors’ biographical sketches showed that this mixed methods study was completed by a team of researchers drawn from individuals with quantitative and qualitative expertise.

As suggested by this statement, the expected outcome of the study was projected to be a detailed picture of resilience and the personal perspectives of the survivors as learned through qualitative data. Also, the authors intended to probe the quantitative findings, to explain them in more detail through the qualitative data. With this intent, the study set up as a sequential approach, with the two databases connected and one building on the other. Also, with this approach, the timing illustrated the qualitative data collection followed the quantitative results. It was difficult to discern whether this study placed greater emphasis on the quantitative or qualitative component of the project. The project began with a quantitative longitudinal phase with extensive discussions of the measures used to gather data. The authors detailed the quantitative results. However, the qualitative findings illustrated many themes that emerged from the interviews with the women. These themes pointed toward new issues that helped to develop the concept of resilience, such as the turning points in the women’s lives, the ongoing nature of recovery, and the role of spirituality in recovery. The study was conducted by a team of researchers from psychology and criminal justice and supported by the National Institutes of Health (NIH).

Example 10.2An Explanatory Sequential Mixed Methods Design

In 2007, Banyard and Williams conducted an explanatory sequential mixed methods study examining how women recover from childhood sexual abuse. The quantitative component of the study consisted of structured interviews (with 136 girls in 1990 and a subset of 61 girls in 1997) looking at resilience, correlates of resilience, over time across 7 years of early adulthood. The qualitative aspect consisted of interviews with a subset of 21 girls about their life events, coping, recovery, and resilience. The intent of the mixed methods study was to use the qualitative interviews to “explore and make sense” of the quantitative findings (p. 277). Here was the purpose statement:

Multiple methods are used to examine aspects of resilience and recovery in the lives of female survivors of child sexual abuse (CSA) across 7 years of early adulthood. First quantitative changes in measures of resilience over time were examined. To what extent did women stay the same, increase, or decrease in functioning in a variety of sphere across 7 years during early adulthood? Next, the role of re-traumatization as an impediment to ongoing resilience and correlates of growth or increased well-being over time were examined. Finally, because resilient processes in adulthood have not been the focus of much research and require further description, qualitative data from a subset of participants was used to examine survivors’ own narratives about recovery and healing to learn about key aspects of resilience in women’s own words. (p. 278)

In this mixed methods study, the expected outcome was clearly to develop good psychometric measures and then to use the measures as outcomes in an experimental project. It was also to use the qualitative data to develop hypotheses that might be tested using the intervention in the experiment. The initial phase qualitative data collection was connected to the subsequent quantitative measures and their rigorous testing for scores on validity and reliability. The entire project was timed for the quantitative phase to follow the qualitative phase, and the quantitative phase could be stated as the development of the measures (and survey) and the experimental intervention study. If we were to diagram this project, it would be qual → QUAN → QUAN. As this notation shows, the emphasis in the project favored quantitative research, and the project could be seen as pointing toward the program intervention test at the end of the article. Recognizing that the researchers came from public health, an organization called Partners in Health, and a children’s hospital, the strong quantitative orientation of the project makes sense. Overall, this mixed methods study illustrated both the core exploratory sequential design and the more advanced embedded experimental design with a sequential focus. To conduct such a complex project, the study involved a team of researchers both in the United States and in Rwanda.

Example 10.3An Exploratory Sequential Mixed Methods Design

A good example of an exploratory sequential study with an experimental test outcome is found in Betancourt et al. (2011). This study used mixed methods research to adapt and evaluate a family strengthening intervention in Rwanda. The investigators sought to examine the mental health problems facing HIV-affected children in Rwanda. They first began with an exploratory, qualitative first phase of interviews with children and their caregivers. From a qualitative thematic analysis of the data, they then performed an extensive review of the literature to locate standardized measures that matched their qualitative findings. They found some measures and added some new ones to develop a survey instrument. This instrument went through several refinements following rigorous procedures of instrument-scale development (e.g., backward and forward translations, a discussion of items, reliability and validity) to develop good construct validity for the measures. These measures (e.g., family communication, good parenting, and others) then became the pretest and posttest assessments in an experimental (intervention) study. For the intervention in the study, the researchers were led to a strengths-based, family-based prevention program that was hypothesized to be related to the measures. The final step in the mixed methods process was to use the validated measures within a program that featured the prevention program. At various points in this study, the researchers also collaborated with stakeholders to help to develop good measures. Thus, this study illustrated a good, complex mixed methods project with an initial qualitative phase, an instrument development phase, and an experimental phase. It shows how an initial exploration qualitatively can be used to support a later quantitative testing phase. They stated the purpose of the study as follows:

In the multi-step process used in this mental health services research, we aimed to (1) carefully unpack locally-relevant indicators of mental health problems and protective resources using qualitative methods; (2) apply qualitative findings to the adaptation of mental health measures and the development of a locally-informed intervention; (3) validate the selected mental health measures; and (4) apply the measures to rigorous evaluation research on the effectiveness of the intervention chosen through the mixed methods process. (p. 34)

Example 10.4A Social Justice Design

The final example is a feminist study using a mixed methods social-justice explanatory sequential study by Hodgkin (2008). This study investigated the concept of social capital for men and women in households in a regional city in Australia. Social capital described norms and networks that enabled people to work collectively together to address and resolve common problems (e.g., through social activities, the community, and civic participation). The basic mixed methods approach was an explanatory sequential design with an initial survey, a quantitative phase, followed by an interview, qualitative phase. As stated by the author, “the qualitative study elaborated on and enhanced some of the results from the quantitative study” (p. 301). In addition, the author declared that this was a feminist mixed methods project. This means that Hodgkin used a feminist framework (see Chapter 3) to encase the entire mixed methods project. She also referred to the Merten’s transformative research paradigm (Mertens, 2007) that gave voice to women, used a range of data collection methods, and bridged the subjective and objective ways of knowing (see the epistemology discussion in Chapter 3). The purpose of the study was this:

The author will provide examples of quantitative data to demonstrate the existence of different social capital profiles for men and women. Stories will also be presented to provide a picture of gender inequality and expectation. The author will conclude by arguing that despite reluctance on the part of feminists to embrace quantitative methods, the big picture accompanied by the personal story can bring both depth and texture to a study. (p. 297)

Thus, in this mixed methods study, the expected outcome for the study was to help explain the initial survey results in more depth with qualitative interview data. Added to this would be the transformative perspective of seeking to provide a picture of gender inequality and expectations. The databases were used sequentially with the qualitative interviews following and expanding on the quantitative surveys. While the surveys were sent to both men and women in households (N = 1431), the interviews included only women in the survey sample (N = 12). The women interviewed were of different ages, they varied in terms of their work activities (inside and outside the home), they were mothers, and they varied in their educational level of attainment. The timing of the data collection was in two phases with the second-phase qualitative interviews building on the results from the first-phase quantitative surveys. In fact, the survey data indicated that men and women differed in terms of their level of social participation in groups, and in community group participation. The emphasis in this study seemed to be equal between the quantitative and qualitative components, and clearly the sole author of the study sought to provide a good example of mixed methods research that used a feminist framework.

How was this framework used? The author announced at the beginning of the study that “the aim of this article is to demonstrate the use of mixed methods in feminist research” (p. 296). Then the author discussed the lack of qualitative research in the empirical studies of social capital and noted the White, middle-class notion of community that dominated the discussions of social capital. Further, the author talked about lifting up the voices of those disenfranchised by gender and engaged in a study that first pointed out gender differences in social, community, and civic participation within a large sample of men and women, and then focused a qualitative follow-up on only women to understand the women’s role in more depth. The qualitative findings then addressed themes that influence women’s participation, such as wanting to be a “good mother,” wanting to avoid isolation, and wanting to be a good citizen. A summary of the qualitative findings indicates specifically how the qualitative data helped to enhance the findings of the initial survey results. Unlike many feminist mixed methods studies, the conclusion did not indicate a strong call for action to change the inequality. It only mentioned in passing that the mixed methods study provided a powerful voice to gender inequality.

Summary

In designing the procedures for a mixed methods discussion, begin by defining mixed methods research and its core characteristics, briefly mentioning its historical evolution; discuss your chosen mixed methods design; and note the challenges in using the design. Convey a diagram of your procedures that includes good notation to help the reader understand the flow of activities. As you discuss your design, convey the elements that go into it, such as the procedures used in a convergent parallel, an explanatory sequential, or an exploratory sequential mixed methods study. Also consider whether you will overlay your project with a more complex procedure that embeds the data within a larger design, theoretical framework, or methodology. Finally, consider factors that play into your choice of a mixed methods design. These involve considering the intent of the design, what outcomes you expect from the study, the integration of the databases, the timing of them, the emphasis placed on each database, the choice of design that matches your field, and the conduct of the project either by yourself or a team of researchers.

 

WRITING EXERCISES

Design a combined qualitative and quantitative study that employs two phases sequentially. Discuss and provide rationales for why the phases are ordered in the sequence you propose.

Design a combined qualitative and quantitative study that gives emphasis to qualitative data collection and less emphasis to quantitative data collection. Discuss the approach to be taken in writing the introduction, the purpose statement, the research questions, and the specific forms of data collection.

Develop a figure and specific procedures that illustrate the use of a theoretical lens, such as a feminist perspective. Use the procedures of either an explanatory or exploratory design for conducting the study. Use appropriate notation in the figure.

Additional Readings

Creswell, J. W., & Plano Clark, V. L. (2018). Designing and conducting mixed methods research (3rd ed.) Thousand Oaks, CA: Sage.

John Creswell and Vicki Plano Clark provide two chapters on mixed methods research designs. Chapter 3 discusses the three core mixed methods designs: convergent mixed methods designs, explanatory sequential mixed methods design, and exploratory sequential mixed methods designs. Chapter 4 advances the four examples of complex designs: mixed methods intervention designs, mixed methods case study designs, mixed methods participatory–social justice designs, and mixed methods evaluation designs. The authors provide examples and diagrams of each type of design and detail important characteristics such as their integrative features.

Greene, J. C., Caracelli, V. J., & Graham, W. F. (1989). Toward a conceptual framework for mixed-method evaluation designs. Educational Evaluation and Policy Analysis, 11(3), 255–274.

Jennifer Greene and associates undertook a study of 57 mixed methods evaluation studies reported from 1980 to 1988. From this analysis, they developed five different mixed methods purposes and seven design characteristics. They found the purposes of mixed methods studies to be based on seeking convergence (triangulation), examining different facets of a phenomenon (complementarity), using the methods sequentially (development), discovering paradox and fresh perspectives (initiation), and adding breadth and scope to a project (expansion). They also found that the studies varied in terms of the assumptions, strengths, and limitations of the method and whether they addressed different phenomena or the same phenomena; were implemented within the same or different paradigms; were given equal or different weight in the study; and were implemented independently, concurrently, or sequentially. Using the purposes and the design characteristics, the authors have recommended several mixed methods designs.

Morse, J. M. (1991). Approaches to qualitative-quantitative methodological triangulation. Nursing Research, 40(1), 120–123.

Janice Morse suggests that using qualitative and quantitative methods to address the same research problem leads to issues of weighing each method and their sequence in a study. Based on these ideas, she then advances two forms of methodological triangulation: (a) simultaneous, using both methods at the same time, and (b) sequential, using the results of one method for planning the next method. These two forms are described using a notation of capital and lowercase letters that signify relative weight as well as sequence. The different approaches to triangulation are then discussed in the light of their purpose, limitations, and approaches.

Plano Clark, V. L. & Creswell, J. W. (2008). The mixed methods reader. Thousand Oaks, CA: Sage.

Creswell and Plano Clark have developed a comprehensive guide to designing the steps in conducting mixed methods research. This design theme is carried forward in specific examples of published mixed methods studies in this reader. Examples are provided of the convergent design, the explanatory sequential design, and the exploratory sequential design. Also, the book contains key articles throughout the years that have informed the development of the field of mixed methods.

Tashakkori, A., & Teddlie, C. (Eds.). (2010). SAGE handbook of mixed methods in social & behavioral research (2nd ed.). Thousand Oaks, CA: Sage.

This handbook, edited by Abbas Tashakkori and Charles Teddlie, represents a major effort to map the field of mixed methods research. The chapters provide an introduction to mixed methods, illustrates methodological and analytic issues in its use, identifies applications in the social and human sciences, and plots future directions. For example, separate chapters illustrate the use of mixed methods research in evaluation, management and organization, health sciences, nursing, psychology, sociology, and education.

https://edge.sagepub.com/creswellrd5e

Students and instructors, please visit the companion website for videos featuring John W. Creswell, full-text SAGE journal articles, quizzes and activities, plus additional tools for research design.