Qualitative Data Analysis

19 The Qualitative Approach

The Qualitative Approach

Mikaila Mariel Lemonik Arthur

At the most basic level, qualitative research is research that emphasizes data that is not numerical in nature, data like words, pictures, and ideas. In contrast, quantitative data emphasizes numbers, or at least variables that can relatively easily be translated into numerical terms. In other words, quantitative data is about quantities, while qualitative data is about qualities. Beyond this basic distinction, qualitative research can look very similar to quantitative research or it can take a very different approach. Later in this chapter, you will learn more about different ways of thinking about data and how they might apply to qualitative data analysis. When people talk about qualitative approaches to research, however, they are often focused on those approaches that are distinct from what quantitative researchers do.

So what are some of the unique features of qualitative research? First of all, qualitative research tends to rely on the use of rich, thick description. In other words, qualitative research does not just provide summaries of data and findings, it really takes the reader or consumer of research there and lets them explore the situation and make conclusions for themselves by drawing on extended descriptions and excerpts from the data. Qualitative research also leaves room for focus on feelings and emotions, elements of the social world that can be harder to get at with quantitative data. For the qualitative researcher, data should not just depict specific actions or occurrences, but rather the contexts and backgrounds that lead up to what happened. More broadly, qualitative research tends to focus on a deep understanding of a specific place or organization or of a particular issue, rather than providing a wider but shallower understanding of an area of study.

Among the strengths of qualitative research are that it provides for the development of new theories and the exploration of issues that people do not know much about. It is very high in validity since it is so connected to real life. And it permits the collection and analysis of more detailed, contextual, and complex kinds of data and information. Of course, with strengths come limitations. The higher validity of qualitative data is matched with lower reliability due to the unique circumstances of data collection and the impact of interviewer effect. The greater ability to develop new theories is matched with a greater difficulty testing existing theories, especially causal ones, given the impossibility of eliminating alternative explanations for the phenomena under investigation. The ability to collect more detailed and complex information comes in large part due to the focus on a much smaller number of participants or cases, which in turn limits generalizability and in some cases can limit representativeness. And while there is no reason to conclude that any of these factors make qualitative research more prone to bias than quantitative research, which after all can be profoundly impacted by slight variations in survey question wording or sample design, those who are not well informed about research methodology may discount the strengths of qualitative research by suggesting that the lack of numbers or the close interaction between participants and researchers bias the results.

In their classic text on qualitative data analysis, Miles and Huberman (1994) present the following as among the key elements of qualitative data analysis:

  • It involves more prolonged contact with more ordinary aspects of human life;
  • It has a holistic rather than a particularistic focus, aiming to keep data and findings in context;
  • Multiple interpretations and understandings of data are possible, and researchers should preserve respondents’ own understandings of their worlds and lives;
  • There is a lack of standardization and measurement, with the researcher themselves becoming the primary measurement instrument; and
  • Analysis is done primarily with words.

For the purposes of this text on data analysis, which focuses on what we do after we collect data rather than on how we go about obtaining the data in the first place, the last of these elements is most important. However, other scholars would argue that qualitative analysis is not limited to words—it may also involve visuals ways of engaging with and presenting data.

Types of Qualitative Data

The data that we analyze in qualitative research consist primarily of words and images drawn from observation, interaction, interviewing, or existing documents. In particular, the types of data collection that tend to result in qualitative data include interviews and focus groups, ethnography and participant observation, and the analysis of existing documents. These different data collection strategies imply a variety of analytical strategies as well, and indeed qualitative data analysis relies on a breadth of techniques. Thus, part of the process of formulating and selecting qualitative data is selecting the right kinds of strategies to apply to the particular data being utilized.

One of the most common ways in which qualitative data is collected is through talking to people. We often refer to these people as respondents or participants. Sometimes, they may be called subjects, though many qualitative researchers find that term to be inappropriate. In contrast to respondents or participants, subjects implies a more passive kind of relationship to the research process, a relationship in which research is done to a person rather than one in which a person is a party to the research process.

Research involving talking to people usually involves interviews of various kinds, whether they be in-person or via video chat, short and structured or long oral histories, of an individual or of a larger focus group. The data collected from interviews may include interview notes and audio or video recordings. Alternatively, researchers may conduct observational research or participant-observation (often called ethnography). In this method, researchers observe real social life in all its detail, either with or without participating in it. Typically, the data collected from observation and ethnography entails detailed fieldnotes recording what has been encountered in the setting.

It is beyond the scope of this text to discuss the process of data collection. However, the next chapter will detail some of the strategies that researchers may want to consider in designing their studies and collecting their data in order to ensure that data is obtained in a form that is useful for analysis.

There are other kinds of qualitative data that do not involve talking to people. These include trace analysis, or observing the traces of life that people have left behind (this is what archeologists do), as well as the use of existing documents or images as a data source. For example, researchers might collect social media posts, photographs of social events, newspaper articles, or archival materials like letters, journals, and meeting minutes.

Paradigms of Research

Researchers approach their research from different perspectives or paradigms. A paradigm is a set of assumptions, values, and practices that shapes the way that people see, understand, and engage with the world, and thus the particular paradigm that a researcher inhabits shapes the fashion in which they carry out their research. Philosophers use the term epistemology to refer to the study of the nature of knowledge, and thus we can take an epistemological perspective to understanding how paradigms of research might vary.

Two paradigms that commentators often juxtapose are the positivist and interpretivist approaches. Positivism assumes that there is a real, verifiable reality and that the purpose of research is to come as close to it as possible. Thus, a positivist would argue that we can understand the world, subject it to prediction and control, and—through the processes of research and data analysis—empirically verify our claims. Positivist research projects can utilize a variety of methods, but experimental and quantitative survey data are especially likely. Among qualitative approaches, positivism is often associated with the type of observational study once common in anthropology, which aimed at uncovering the “real” social practices of a group. These methods tend to involve keeping some degree of distance between the researcher and the participants and positioning the researcher as the expert on both research methods and the participants’ own lives. From a positivist perspective, standards of rigor like reliability, validity, and generalizability are important and attainable markers of good research as they contribute to the likelihood that the research arrives at the right answer. As this suggests, objectivity is an essential goal of positive research. Good research, to a positivist, is that which is valid, reliable, generalizable, and has strong, significant results.

In contrast, interpretivism suggests that our knowledge of the world is created by our own individual experiences and interactions, and thus that reality cannot be understood as existing on its own in a form separate from our distinct existences. Thus, an interpretivist would argue that understandings are always based in a particular time and on a particular interpreter and are always open to reinterpretation. Interpretivist research projects utilize naturalistic research methods that are rooted in real social contexts, especially in-depth interviewing and participant-observation. These methods tend to involve a closer and more reciprocal relationship between the researcher and the participants, with a greater concern for ethical treatment and in some cases an emphasis on possibilities for social change. Interpretivist researchers also value participants’ expertise and their understandings of their own lives rather than assuming the researcher’s perspective is necessarily more accurate. From an interpretivist perspective, validity may not be attainable due to the fact that truth is not certain, and in any case standards of rigor are far less important than considerations like ethics, morality, the degree to which biases are made clear, and what the world can learn from the research. As this might suggest, interpretivists would tend to believe that objectivity is probably not attainable, and that even it is, the pursuit of it may not be worthwhile. To an interpretivist, good research is that which is done in a careful, respectful manner, contributes to knowledge, is reflective, and takes appropriate political and ethical considerations into account.

Lisa Pearce (2012) has outlined a paradigm she calls pragmatist. This approach is sometimes understood as a kind of middle position between positivist and interpretivist ways of thinking. Thus, its proponents neither believe that strict objectivity is possible nor abandon efforts to seek objectivity at all, instead engaging in reflexivity as they consider how researchers influence both research participants and research findings. While pragmatist approaches can be used with various methods of data collection, they tend to be employed by those using mixed-methods approaches, especially those combining quantitative and quantitative strategies.

Another paradigm of research is feminist in nature. While there are of course many ways to do research from a feminist perspective, one of the most important elements of feminist epistemology is the idea that everyone comes to research—whether they are a researcher, a research participant, or a consumer of research—from their own standpoint. In other words, each person’s individual life experiences and social positions shape their point of view on the world, and this point of view will in turn impact how the individual understands and interprets phenomena they encounter, including those that are part of research. Dorothy Smith (1987), one of the figures associated with feminist standpoint approaches, notes that this approach to methods requires that we be able to describe the social “in ways that can be checked back to how it actually is” (1987:122). Such approaches are powerful not only for understanding the experiences of women, but also for understanding the experiences of other minoritized, marginalized, and/or oppressed groups, including people who are Black, Indigenous, or of color, and those living with disabilities. Feminist research has much in common with the broader paradigm of interpretivist research, but it pays greater attention to the importance of standpoints and of inequality and oppression in shaping the dynamics of research.

While the discussion of paradigms here is not exhaustive—there are many other approaches to research, many other epistemologies—it does provide an overview of some of the possible ways to think about research and data analysis. One important thing to remember is that while there are criteria for good research, criteria that will be further outlined in subsequent chapters of this text, there are no objective or empirical standards for which paradigm is “correct.” In other words, individual researchers or research teams approach their research from the perspective or philosophy that makes sense to them, and while others may have reasons for disapproving, they cannot say that such a choice is right or wrong. Researchers must make these sorts of decisions for themselves.

Inductive and Deductive Approaches

Another question we might ask about the epistemology of research processes is whether our data emerges from our analysis or whether our data generates our analysis. If you argue that data emerges from analysis, you are suggesting that you begin the research process with a theory and then look to the data you have collected to see whether or not you can find support for your theory. This approach enables the testing of theories. It is typically understood as a deductive approach to research. In deductive approaches, researchers develop a theory, collect data, analyze the data, and use their analysis to test their theory. Positivist research is often deductive in its approach.

Instead, if you argue that data generates analysis, you are suggesting that you begin the research process by collecting data and you then look to see what you can find within it. This approach enables the building of theories. It is typically understood as an inductive approach. In inductive approaches, researchers begin by collecting data. Then they analyze that data and use that analysis to build new understandings. Interpretivist and feminist research are often inductive in their approach.

While qualitative research can be conducted using both deductive and inductive approaches, it is a bit more common for qualitative researchers to use inductive approaches. Such approaches are far less possible in quantitative analysis because of the need for more precisely-designed data collection techniques. Thus, one advantage of qualitative research is that it permits for an inductive approach and is thus especially useful in contexts in which very little is already know or where new explanations need to be uncovered. It is also possible to conduct research using what some call abduction, or an interplay between deductive and inductive approaches (Pearce 2012). Such an approach may also be found in mixed-methods research. This text will focus primarily on inductive approaches to qualitative data analysis, given that they are far more common. But deductive approaches do exist. For example, consider a researcher who is interested in what sorts of circumstances give rise to nonprofit organization boards deciding to replace the organization’s director. More typically, a qualitative researcher with this question would interview a wide variety of non-profit board members and, based on the responses, would build a theory—an inductive approach. In contrast, the researcher could choose to conduct her study deductively. Then, she would read the prior literature on management and organizational decision-making and develop one or more hypotheses about the circumstances that give rise to leadership changes. She would then interview board members looking specifically for the constellation of circumstances she hypothesized to test whether these circumstances were associated with the decision to replace the director.

Research Standards

As researchers design and carry out their data collection and data analysis strategies, there are a variety of issues they must consider in terms of ensuring their research meets appropriate disciplinary and professional standards for quality and rigor. These include considerations of generalizability or representativeness, reliability and validity, and ethics and social responsibility. It is also important to note that researchers must be attentive to ensuring that they are not overstating the degree to which their research can demonstrate evidence of causation. It is only possible for research to demonstrate causation if it meets three essential criteria:

  • Association, which means that there must be clear empirical evidence of a relationship between the factor understood as the cause and the factor understood as the effect,
  • Temporal order, which means that it must be known that the causal factor happened earlier in time than the effect, and
  • Elimination of alternatives, which means that the research must have eliminated all possible alternative explanations for the effect.

It is not generally possible to eliminate all possible alternative explanations—even if a research project is able to eliminate all the ones the researcher thought of, there are still other possibilities. Thus, research can only make true causal claims if its finding come from a properly-controlled laboratory experiment in which the only element that could possibly have change the outcome was the one under examination. If research does not involve a properly-controlled laboratory experiment, researchers must be cautious about the way they describe their findings. They cannot say that their study has proven anything or that it shows that A causes B. Instead, they can say something like “these findings are consistent with the hypothesis that A causes B.” Qualitative research cannot conclusively show causal relationships, even though it can be suggestive of them.

Generalizability refers to whether the research findings from a particular study can be assumed to hold true for the larger population. Research can only be generalized if it is the result of a properly-conducted random sample, also called a probability sample, and then only to the population that was sampled from. In other words, if I conduct a random sample of students at a particular college, I can only assume my findings will hold true for students at that college—I cannot assume they accurately reflect dynamics at other colleges or among people who are not college students. Furthermore, because probability sampling can involve what is called sampling error, even it can not guarantee generalizability to the population from which the sample has been drawn. It simply optimizes the chance that such generalizability exists.

And if my sample was not random, I cannot assume that my findings reflect the broader dynamics of that college. This is because the randomization that is part of developing a random sample is designed to eliminate the potential for sample bias that might shape the results. For example, if I conduct a non-random sample of college students by posting an ad on a social media site, then my participants will only be those who saw or heard about the ad, and they may be different in some way from students who did not see or hear about the ad.

While it is possible to conduct qualitative research using a random sample, a considerable portion of qualitative research projects do not use random sampling. This is because it is only possible to develop a random sample if you have a list of all possible people in the population (or can use sampling methods like cluster sampling that allow you to randomize without such a list). Clearly, if I want to study students at a particular college, I can get a list of all possible students at that college. But what if I wanted to study people who play the video game Fortnite? Or individuals who enjoy using contouring makeup? Or parents who have a child with autism as well as a child who is neurotypical? There are no lists of people in these categories, and thus a random sample is not possible. In addition, it can be hard to use random sampling for studies in which the researcher will ask participants for more lengthy time commitments, such as in-depth interviewing and ethnographic observation.

Where generalizability is not possible, researchers can instead strive for representativeness. Having a representative sample means having a sample that includes a sufficient number of people from various subgroups within the population such that the research can understand whether the dynamics it uncovers are applicable broadly across groups or whether they only apply to specific subgroups. Which characteristics must be reflected to ensure representativeness will vary depending on the study in question. A study of students’ participation in extracurricular activities probably should consider both residential students and those who commute to campus. A study of retail employees might need to include both full-time and part-time workers as well as those who do and do not hold managerial positions. Race, gender, and class, as well as other axes of inequality, are very common subgroups used to ensure representativeness. Note that it is entirely ok to exclude various subgroups from a study, as long as the study makes clear who and what it is studying. In other words, it would be reasonable to conduct a study of mothers of children with autism. It would not be acceptable to conduct a study of parents with autism but only include mothers in the sample.

Reliability refers to the extent to which repeated measures produce consistent results. Usually, discussions of reliability refer to the consistency of specific measures. For instance, if I ask you what you ate for breakfast on Wednesday in a conversation on Wednesday evening and then on Friday morning, will you give me the same answer? Or if I administer two different self-esteem scales, do you come out with similar results? Changes in the way questions are asked, the context in which they are asked, or who is doing the asking can have remarkable impacts on the responses, and these impacts mean reliability is reduced. Some concerns about reliability, such as that illustrated with the self-esteem scale, refer to consistency between different approaches for measuring the same underlying idea. Others have to do with repeatability, replicability, or reproducibility (Plesser 2017). An example of the issue of repeatability is the question about what you ate for breakfast—if the same researcher repeats the same measurement, do they get the same results? Replicability refers to situations in which a different researcher uses the same measurement approaches on the same type of population, though the research may take place in a different location. While a researcher can never ensure that their research will be replicable, researchers who strive to ensure replicability do endeavor to make their research process as clear as possible in any publications so that others will be able to take the same exact steps in trying to replicate it. However, this can be difficult in qualitative studies as the impact the researcher has on the context through phenomena such as interviewer effect may mean that a different researcher or research team cannot exactly replicate the original conditions of data collection. Finally, reproducibility refers to whether a different research team can develop its own methodological approach to answering the research question but still find results consistent with those in the original study. It is always possible that a study fails to reproduce not because the findings are inherently irreproducible but rather because some variation in the population or setting is responsible for the different results. Another element of reliability is inter-rater reliability. To understand inter-rater reliability, consider a study in which a researcher is trying to determine whether the degree of sexism displayed in advertisements differs depending on the type of product being advertised. In order to collect this data, a team of research assistant has to examine each advertisement and rate, on a scale of 1 to 5, how sexist the advertisement is. It’s not surprising that different research assistants might judge the same advertisement differently–and this can impact the results of the study. Measuring inter-rater reliability helps determine how different these multiple raters’ ratings are from one another, and if the differences are large, the researcher can go back and retrain the research assistants so they can more consistently apply the intended rating scale.

Validity refers to the extent to which research measurements accurately reflect the underlying reality. Well-designed qualitative approaches, especially in-depth interviewing and participant-observation, tend to be high in validity. This is because such methods come the closest of all social science methods to reflecting real life in all of its complexity. Validity can be increased by careful attention to research design, the use of method triangulation (multiple research methods or approaches), and deep reflection on process and findings.

While a full treatment of research ethics is beyond the scope of this book, it is essential to remember that good research always attends to the highest ethical standards. People who talk about research ethics often focus their primary attention to the treatment of human subjects, or the people who actually participate in the research project. Ethical treatment of participants includes ensuring that any risks they face are limited, that they have given fully-informed consent to their participation in research, that their identity will be protected,[1] and that they do not experience coercion to participate. An interesting example of the kinds of issues that a commitment to research ethics raises has to do with the legal risks inherent in research. Shamus Khan, a researcher studying sexual assault, has written about an instance in which he became embroiled in the court process after his research materials were subpoenaed in a lawsuit. The subpoena would have entitled the litigants to materials that would have disclosed confidential personal information, information research participants were assured would remain confidential. Khan details the lengths that he had to go to in order to protect participants’ information and the complex ethical questions his case raises, ultimately concluding that a real commitment to research ethics requires some changes in how the institutions that sponsor research think about and manage their responsibilities (Khan 2019).

Many commentators who discuss research ethics suggest that researchers’ ethical responsibility goes much further. For example, feminist researchers often suggest that research participants be given the opportunity to review interview transcripts for errors, omissions, or statements they would have preferred not to make and issue corrections, even if their words and experiences will be used anonymously. Attention should also be paid to ensuring that people and communities who participate in research are able to share in the benefits of that research. For example, if a program is developed through research on a particular community of homeless people, those people should be among the first to be able to access the new program. If researchers profit financially from the research they have done, they might consider sharing the profits with those they have studied.

While traditional treatments of research ethics consider only the researcher’s responsibility to research participants, a broader treatment of ethics—in keeping with interpretivist and feminist paradigms—would also include social responsibility as an ethical touchstone. Researchers concerned with social responsibility might consider whether their approach to publication or the content of their publications might have harmful impacts on the populations they have studied, stigmatizing them or exposing them to disadvantageous policy consequences. For example, Robert Putnam, a political scientist, conducted a study that examined the impact of neighborhood diversity on social cohesion and trust. When he found that diversity can reduce trust, he worried that his findings would be used as a political weapon by those opposed to diversity, racial equity, and immigration. Thus, while he made some data available to other researchers, he withheld publication for several years while he developed policy proposals designed to mitigate the potential harm of his findings. Some commentators felt that withholding publication was itself unethical, while Putnam felt that publishing without due consideration of the impact of his findings was the unethical thing. A commitment to social responsibility might also include attention to ensuring equity in citation practices, an issue that has been brought to the fore by the social media campaign #CiteBlackWomen, which urges scholars and teachers to ensure that they read publications by Black women, acknowledge Black women’s scholarship through citation as well as inclusion in course reading lists, and ensure that Black women are represented as speakers at conferences, among other things (Cite Black Women Collective n.d.).

As noted above, research paradigms influence the particular qualities that researchers value in their research. In addition, it is not always realistic or even possible to maximize all of these qualities in a given project. Thus, most research, including most excellent research, will emphasize some of these standards and not others. This does not mean the research is lacking in rigor. Good research, however, is always explicit about its own limitations. Thus, researchers should indicate whether or not their results can be generalized, and if so, to whom. They should be clear on which subgroups they included in their efforts to ensure representativeness.

The Process of Qualitative Research

So, how does one go about conducting an inductive qualitative research project? Well, there are a series of steps researchers follow. However, it is important to note that qualitative research and data analysis involve a high degree of fluidity and are typically iterative, meaning that they involve repeatedly returning to prior steps in the process.

First, researchers design their data collection process, which includes developing any data collection instruments such as interview guides and locating participants. Then, they collect their data. To collect data researchers might conduct interviews, observations, or ethnography, or they might locate documents or other sources of textual or visual data. While deductive quantitative approaches require researchers collect all their data and only then analyze it, inductive qualitative approaches provide the opportunity for more of a cyclical process in which researchers collect data, begin to analyze it, and then use what they have found so far to reshape their further data collection.

Once data is collected, researchers need to ensure that their data is usable. This may require the transcription of audio or video recordings, the scanning or photocopying of documents, typing up handwritten fieldnotes, or other processes designed to move raw data into a more manipulable form.

Next, researchers engage in data reduction. Research projects typically entail the collection of really large quantities of data, more data that can possibly be managed or utilized in the context of one paper. This is especially likely in the case of qualitative research because of the richness and complexity of the data that is collected. Therefore, once data collection is completed, researchers use strategies and techniques to reduce the hundreds or thousands of pages of fieldnotes or interview transcripts or documents into a manageable form. Activities involved in data reduction, which will be taken up in a later chapter, include coding, summarization, the development of data displays, and categorization.

Once data reduction has made data more usable, researchers can develop conclusions based on their data. Remember, however, that this process is iterative, which means that it is a continuing cycle. So, when researchers make conclusions, they also go back to earlier stages to refine their approaches. In addition, the process of developing conclusions also requires careful consideration to limitations of the data and analytical approaches, such as those discussed earlier in this chapter.

Finally, researchers present their findings. During each project, researchers must determine how best to disseminate results. Factors influencing this determination include the research topic, the audience, and the intended use of the results—for instance, are these the results of basic research, designed to increase knowledge about the phenomena under study, or are they the results of applied research, conducted for a specific audience to inform the administration of a policy or program? Findings might be disseminated in a graphical form like an infographic or a series of charts, a visual form like a video or animation, an oral form like a lecture, or a written form like a scholarly article or a report. Of course, many projects incorporate multiple forms of dissemination.

While this chapter is titled “The Qualitative Approach,” it is actually inaccurate to suggest that there is just one overall approach to qualitative research. As this chapter has shown, there are some core characteristics that qualitative approaches to research have in common, such as data that relies on words or images rather than numbers and a richer, more contextual understanding of the phenomena under study. But there are also many ways in which qualitative approaches to research vary. They use different methods of data collection. They take place within different paradigms and epistemologies. They focus their attention on emphasizing different standards for research quality. And, as the following chapters will show, they utilize different methods for preparing and managing data, analyzing that data, and disseminating their findings.

Exercises

    1. Find a few grocery store circulars from your area. The ones that get delivered with your mail are fine, or you can locate them online on the website of your local grocery stores. Spend some time examining the circulars. Look at the words and images, the types of items represented, the fonts and layouts, anything that catches your eye, and then answer two questions: first, what do the circulars tell you about the lives of people today who live in your area, and second, what did you do, cognitively, to figure that out?
    2. Locate a recent scholarly journal article in your field of study and read it. Do you think this article used a more positivist or more interpretivist paradigm of knowledge? Explain how you know, drawing on the key elements of these paradigms.
    3. What do you think it means to do good research? Which of the various standards for good research do you think are most important to the topics or issues you are interested in? And what are some of the strategies you might employ to be sure your research lives up to these standards?

  1. However, there are research participants who wish to disclose their real identity, and some qualitative researchers argue that truly ethical research gives participants the option to make informed decisions about such disclosure.
definition

License

Icon for the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License

Social Data Analysis Copyright © 2021 by Mikaila Mariel Lemonik Arthur is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License, except where otherwise noted.