Can you generalize qualitative data

For a result to be considered generalizable outside of the test group, it must produce the same results with different forms of measurement. A third type of generalizability concerns the subjects of the test situation.

Although the results of an experiment may be internally valid, that is, applicable to the group tested, in many situations the results cannot be generalized beyond that particular group. Researchers who hope to generalize their results to a larger population should ensure that their test group is relatively large and randomly chosen. However, researchers should consider the fact that test populations of over 10, subjects do not significantly increase generalizability Firestone, No matter how carefully these three forms of generalizability are applied, there is no absolute guarantee that the results obtained in a study will occur in every situation outside the study.

In order to determine causal relationships in a test environment, precision is of utmost importance. Yet if researchers wish to generalize their findings, scope and variance must be emphasized over precision. Therefore, it becomes difficult to test for precision and generalizability simultaneously, since a focus on one reduces the reliability of the other. One solution to this problem is to perform a greater number of observations, which has a dual effect: first, it increases the sample population, which heightens generalizability; second, precision can be reasonably maintained because the random errors between observations will average out Runkel and McGrath, Transferability describes the process of applying the results of research in one situation to other similar situations.

In this section, we establish a practical working definition of transferability as it's applied within and outside of academic research. We also outline important considerations researchers must be aware of in order to make their results potentially transferable, as well as the critical role the reader plays in this process.

Finally, we discuss possible shortcomings and limitations of transferability that researchers must be aware of when planning and conducting a study that will yield potentially transferable results. Transferability is a process performed by readers of research. Readers note the specifics of the research situation and compare them to the specifics of an environment or situation with which they are familiar.

If there are enough similarities between the two situations, readers may be able to infer that the results of the research would be the same or similar in their own situation. In other words, they "transfer" the results of a study to another context.

To do this effectively, readers need to know as much as possible about the original research situation in order to determine whether it is similar to their own. Therefore, researchers must supply a highly detailed description of their research situation and methods.

Results of any type of research method can be applied to other situations, but transferability is most relevant to qualitative research methods such as ethnography and case studies. Reports based on these research methods are detailed and specific. However, because they often consider only one subject or one group, researchers who conduct such studies seldom generalize the results to other populations.

The detailed nature of the results, however, makes them ideal for transferability. Transferability is easy to understand when you consider that we are constantly applying this concept to aspects of our daily lives. If, for example, you are an inexperienced composition instructor and you read a study in which a veteran writing instructor discovered that extensive prewriting exercises helped students in her classes come up with much more narrowly defined paper topics, you could ask yourself how much the instructor's classroom resembled your own.

If there were many similarities, you might try to draw conclusions about how increasing the amount of prewriting your students do would impact their ability to arrive at sufficiently narrow paper topics.

In doing so, you would be attempting to transfer the composition researcher's techniques to your own classroom. An example of transferable research in the field of English studies is Berkenkotter, Huckin, and Ackerman's study of a graduate student in a rhetoric Ph.

In this case study, the researchers describe in detail a graduate student's entrance into the language community of his academic program, and particularly his struggle learning the writing conventions of this community. They make conclusions as to why certain things might have affected the graduate student, "Nate," in certain ways, but they are unable to generalize their findings to all graduate students in rhetoric Ph. It is simply one study of one person in one program. However, from the level of detail the researchers provide, readers can take certain aspects of Nate's experience and apply them to other contexts and situations.

This is transferability. First-year graduate students who read the Berkenhotter, Huckin, and Ackerman study may recognize similarities in their own situation while professors may recognize difficulties their students are having and understand these difficulties a bit better. The researchers do not claim that their results apply to other situations. Instead, they report their findings and make suggestions about possible causes for Nate's difficulties and eventual success.

Readers then look at their own situation and decide if these causes may or may not be relevant. When designing a study researchers have to consider their goals: Do they want to provide limited information about a broad group in order to indicate trends or patterns? Or do they want to provide detailed information about one person or small group that might suggest reasons for a particular behavior?

The method they choose will determine the extent to which their results can be transferred since transferability is more applicable to certain kinds of research methods than others. Thick Description: When writing up the results of a study, it is important that the researcher provide specific information about and a detailed description of her subject s , location, methods, role in the study, etc.

This is commonly referred to as "thick description" of methods and findings; it is important because it allows readers to make an informed judgment about whether they can transfer the findings to their own situation.

For example, if an educator conducts an ethnography of her writing classroom, and finds that her students' writing improved dramatically after a series of student-teacher writing conferences, she must describe in detail the classroom setting, the students she observed, and her own participation.

If the researcher does not provide enough detail, it will be difficult for readers to try the same strategy in their own classrooms. If the researcher fails to mention that she conducted this research in a small, upper-class private school, readers may transfer the results to a large, inner-city public school expecting a similar outcome. The Reader's Role: The role of readers in transferability is to apply the methods or results of a study to their own situation.

In doing so, readers must take into account differences between the situation outlined by the researcher and their own. If readers of the Berkenhotter, Huckin, and Ackerman study are aware that the research was conducted in a small, upper-class private school, but decide to test the method in a large inner-city public school, they must make adjustments for the different setting and be prepared for different results. Likewise, readers may decide that the results of a study are not transferable to their own situation.

For example, if a study found that watching more than 30 hours of television a week resulted in a worse GPA for graduate students in physics, graduate students in broadcast journalism may conclude that these results do not apply to them. Readers may also transfer only certain aspects of the study and not the entire conclusion.

For example, in the Berkenhotter, Huckin, and Ackerman study, the researchers suggest a variety of reasons for why the graduate student studied experienced difficulties adjusting to his Ph. Although composition instructors cannot compare "Nate" to first-year college students in their composition class, they could ask some of the same questions about their own class, offering them insight into some of the writing difficulties the first-year undergraduates are experiencing.

It is up to readers to decide what findings are important and which may apply to their own situation; if researchers fulfill their responsibility to provide "thick description," this decision is much easier to make. Understanding research results can help us understand why and how something happens.

However, many researchers believe that such understanding is difficult to achieve in relation to human behaviors which they contend are too difficult to understand and often impossible to predict. Cziko's point is important because transferability allows for "temporary understanding. Transferability takes into account the fact that there are no absolute answers to given situations; rather, every individual must determine their own best practices.

Transferring the results of research performed by others can help us develop and modify these practices. However, it is important for readers of research to be aware that results cannot always be transferred; a result that occurs in one situation will not necessarily occur in a similar situation.

Therefore, it is critical to take into account differences between situations and modify the research process accordingly. Although transferability seems to be an obvious, natural, and important method for applying research results and conclusions, it is not perceived as a valid research approach in some academic circles. Perhaps partly in response to critics, in many modern research articles, researchers refer to their results as generalizable or externally valid.

Therefore, it seems as though they are not talking about transferability. However, in many cases those same researchers provide direction about what points readers may want to consider, but hesitate to make any broad conclusions or statements. These are characteristics of transferable results.

Generalizability is actually, as we have seen, quite different from transferability. Unfortunately, confusion surrounding these two terms can lead to misinterpretation of research results. Emphasis on the value of transferable results -- as well as a clear understanding among researchers in the field of English of critical differences between the conditions under which research can be generalized, transferred, or, in some cases, both generalized and transferred -- could help qualitative researchers avoid some of the criticisms launched by skeptics who question the value of qualitative research methods.

Generalizability allows us to form coherent interpretations in any situation, and to act purposefully and effectively in daily life. Transferability gives us the opportunity to sort through given methods and conclusions to decide what to apply to our own circumstances. In essence, then, both generalizability and transferability allow us to make comparisons between situations.

For example, we can generalize that most people in the United States will drive on the right side of the road, but we cannot transfer this conclusion to England or Australia without finding ourselves in a treacherous situation. It is important, therefore, to always consider context when generalizing or transferring results. Whether a study emphasizes transferability or generalizability is closely related to the goals of the researcher and the needs of the audience.

Studies done for a magazine such as Time or a daily newspaper tend towards generalizability, since the publishers want to provide information relevant to a large portion of the population. A research project pointed toward a small group of specialists studying a similar problem may emphasize transferability, since specialists in the field have the ability to transfer aspects of the study results to their own situations without overt generalizations provided by the researcher.

Ultimately, the researcher's subject, audience, and goals will determine the method the researcher uses to perform a study, which will then determine the transferability or generalizability of the results.

Although generalizability has been a preferred method of research for quite some time, transferability is relatively a new idea. In theory, however, it has always accompanied research issues. It is important to note that generalizability and transferability are not necessarily mutually exclusive; they can overlap.

From an experimental study to a case study, readers transfer the methods, results, and ideas from the research to their own context. Therefore, a generalizable study can also be transferable. For example, a researcher may generalize the results of a survey of people in a university to the university population as a whole; readers of the results may apply, or transfer, the results to their own situation. They will ask themselves, basically, if they fall into the majority or not.

However, a transferable study is not always generalizable. For example, in case studies , transferability allows readers the option of applying results to outside contexts, whereas generalizability is basically impossible because one person or a small group of people is not necessarily representative of the larger population.

Research in the natural sciences has a long tradition of valuing empirical studies; experimental investigation has been considered "the" way to perform research. As social scientists adapted the methods of natural science research to their own needs, they adopted this preference for empirical research. Therefore, studies that are generalizable have long been thought to be more worthwhile; the value of research was often determined by whether a study was generalizable to a population as a whole.

However, more and more social scientists are realizing the value of using a variety of methods of inquiry, and the value of transferability is being recognized. It is important to recognize that generalizability and transferability do not alone determine a study's worth.

They perform different functions in research, depending on the topic and goals of the researcher. Where generalizable studies often indicate phenomena that apply to broad categories such as gender or age, transferability can provide some of the how and why behind these results.

However, there are weaknesses that must be considered. Researchers can study a small group that is representative of a larger group and claim that it is likely that their results are applicable to the larger group, but it is impossible for them to test every single person in the larger group.

Their conclusions, therefore, are only valid in relation to their own studies. Another problem is that a non-representative group can lead to a faulty generalization.

For example, a study of composition students'; revision capabilities which compared students' progress made during a semester in a computer classroom with progress exhibited by students in a traditional classroom might show that computers do aid students in the overall composing process.

However, if it were discovered later that an unusually high number of students in the traditional classrooms suffered from substance abuse problems outside of the classroom, the population studied would not be considered representative of the student population as a whole. Therefore, it would be problematic to generalize the results of the study to a larger student population. In the case of transferability, readers need to know as much detail as possible about a research situation in order to accurately transfer the results to their own.

However, it is impossible to provide an absolutely complete description of a situation, and missing details may lead a reader to transfer results to a situation that is not entirely similar to the original one.

The degree to which generalizability and transferability are applicable differs from methodology to methodology as well as from study to study. Researchers need to be aware of these degrees so that results are not undermined by over-generalizations, and readers need to ensure that they do not read researched results in such a way that the results are misapplied or misinterpreted.

Research Design Case studies examine individuals or small groups within a specific context. Research is typically gathered through qualitative means: interviews, observations, etc.

Data is usually analyzed either holistically or by coding methods. Assumptions In research involving case studies, a researcher typically assumes that the results will be transferable. Generalizing is difficult or impossible because one person or small group cannot represent all similar groups or situations.

For example, one group of beginning writing students in a particular classroom cannot represent all beginning student writers. Also, conclusions drawn in case studies are only about the participants being observed. Results of a Study In presenting the results of the previous example, a researcher should define the criteria that were established in order to determine what the researcher meant by "writing skills," provide noteworthy quotes from student interviews, provide other information depending on the kinds of research methods used e.

Readers are then able to assess for themselves how the researcher's observations might be transferable to other writing classrooms. Research is completed through various methods, which are similar to those of case studies, but since the researcher is immersed within the group for an extended period of time, more detailed information is usually collected during the research. Assumptions As with case studies, findings of ethnographies are also considered to be transferable.

Unlike a case study, the researcher here discovers many more details. Also, since analysts completing this type of research tend to rely on multiple methods to collect information a practice also referred to as triangulation , their results typically help create a detailed description of human behavior within a particular environment.

Example The Iowa Writing Program has a widespread reputation for producing excellent writers. In order to begin to understand their training, an ethnographer might observe students throughout their degree program. During this time, the ethnographer could examine the curriculum, follow the writing processes of individual writers, and become acquainted with the writers and their work.

By the end of a two year study, the researcher would have a much deeper understanding of the unique and effective features of the program. Results of a Study Obviously, the Iowa Writing Program is unique, so generalizing any results to another writing program would be problematic. However, an ethnography would provide readers with insights into the program. Readers could ask questions such as: what qualities make it strong and what is unique about the writers who are trained within the program?

At this point, readers could attempt to "transfer" applicable knowledge and observations to other writing environments. Research Design A researcher working within this methodology creates an environment in which to observe and interpret the results of a research question.

A key element in experimental research is that participants in a study are randomly assigned to groups. In an attempt to create a causal model i. Assumptions Experimental research is usually thought to be generalizable. Since participants are randomly assigned to groups, and since most experiments involve enough individuals to reasonably approximate the populations from which individual participants are drawn, generalization is justified because "over a large number of allocations, all the groups of subjects will be expected to be identical on all variables" Example A simplified example: Six composition classrooms are randomly chosen as are the students and instructors in which three instructors incorporate the use of electronic mail as a class activity and three do not.

When students in the first three classes begin discussing their papers through e-mail and, as a result, make better revisions to their papers than students in the other three classes, a researcher is likely to conclude that incorporating e-mail within a writing classroom improves the quality of students' writing.

Depending on how the researcher has presented the results, they are generalizable in that the students were selected randomly. Since the quality of writing improved with the use of e-mail within all three classrooms, it is probable that e-mail is the cause of the improvement. Readers of this study would transfer the results when they sorted out the details: Are these students representative of a group of students with which the reader is familiar?

What types of previous writing experiences have these students had? What kind of writing was expected from these students? The researcher must have provided these details in order for the results to be transferable. Research Design The goal of a survey is to gain specific information about either a specific group or a representative sample of a particular group.

Survey respondents are asked to respond to one or more of the following kinds of items: open-ended questions, true-false questions, agree-disagree or Likert questions, rankings, ratings, and so on. Results are typically used to understand the attitudes, beliefs, or knowledge of a particular group. Assumptions Assuming that care has been taken in the development of the survey items and selection of the survey sample and that adequate response rates have been achieved, surveys results are generalizable.

Note, however, that results from surveys should be generalized only to the population from which the survey results were drawn. Results of a Study The generalizability of surveys depends on several factors.

Whether distributed to a mass of people or a select few, surveys are of a "personal nature and subject to distortion. Depending on whether or not the survey designer is nearby, respondents may or may not have the opportunity to clarify their misunderstandings. It is also important to keep in mind that errors can occur at the development and processing levels.

A researcher may inadequately pose questions that is, not ask the right questions for the information being sought , disrupt the data collection surveying certain people and not others , and distort the results during the processing misreading responses and not being able to question the participant, etc. One way to avoid these kinds of errors is for researchers to examine other studies of a similar nature and compare their results with results that have been obtained in previous studies.

This way, any large discrepancies will be exposed. Depending on how large those discrepancies are and what the context of the survey is, the results may or may not be generalizable. For example, if an improved understanding of Derrida is apparent after students complete E, it can be theorized that E effectively teaches students the concepts of Derrida.

Issues of transferability might be visible in the actual survey questions themselves; that is, they could provide critical background information readers might need to know in order to transfer the results to another context. In Miles and Huberman's book Qualitative Data Analysis , quantitative researcher Fred Kerlinger is quoted as saying, "There's no such thing as qualitative data.

Everything is either 1 or 0" p. To this another researcher, D. Campbell, asserts "all research ultimately has a qualitative grounding" p. This back and forth banter among qualitative and quantitative researchers is "essentially unproductive" according to Miles and Huberman. They and many other researchers agree that these two research methods need each other more often than not. However, because typically qualitative data involves words and quantitative data involves numbers, there are some researchers who feel that one is better or more scientific than the other.

Another major difference between the two is that qualitative research is inductive and quantitative research is deductive. In qualitative research, a hypothesis is not needed to begin research. However, all quantitative research requires a hypothesis before research can begin. Another major difference between qualitative and quantitative research is the underlying assumptions about the role of the researcher.

In quantitative research, the researcher is ideally an objective observer that neither participates in nor influences what is being studied. These basic underlying assumptions of both methodologies guide and sequence the types of data collection methods employed. Although there are clear differences between qualitative and quantitative approaches, some researchers maintain that the choice between using qualitative or quantitative approaches actually has less to do with methodologies than it does with positioning oneself within a particular discipline or research tradition.

The difficulty of choosing a method is compounded by the fact that research is often affiliated with universities and other institutions. The findings of research projects often guide important decisions about specific practices and policies.

The choice of which approach to use may reflect the interests of those conducting or benefitting from the research and the purposes for which the findings will be applied. Decisions about which kind of research method to use may also be based on the researcher's own experience and preference, the population being researched, the proposed audience for findings, time, money, and other resources available Hathaway, Some researchers believe that qualitative and quantitative methodologies cannot be combined because the assumptions underlying each tradition are so vastly different.

Other researchers think they can be used in combination only by alternating between methods: qualitative research is appropriate to answer certain kinds of questions in certain conditions and quantitative is right for others.

And some researchers think that both qualitative and quantitative methods can be used simultaneously to answer a research question. To a certain extent, researchers on all sides of the debate are correct: each approach has its drawbacks. Quantitative research often "forces" responses or people into categories that might not "fit" in order to make meaning. Qualitative research, on the other hand, sometimes focuses too closely on individual results and fails to make connections to larger situations or possible causes of the results.

Rather than discounting either approach for its drawbacks, though, researchers should find the most effective ways to incorporate elements of both to ensure that their studies are as accurate and thorough as possible. It is important for researchers to realize that qualitative and quantitative methods can be used in conjunction with each other. In a study of computer-assisted writing classrooms, Snyder employed both qualitative and quantitative approaches.

The study was constructed according to guidelines for quantitative studies: the computer classroom was the "treatment" group and the traditional pen and paper classroom was the "control" group.

Both classes contained subjects with the same characteristics from the population sampled. Both classes followed the same lesson plan and were taught by the same teacher in the same semester.

The only variable used was the computers. Although Snyder set this study up as an "experiment," she used many qualitative approaches to supplement her findings. She observed both classrooms on a regular basis as a participant-observer and conducted several interviews with the teacher both during and after the semester. However, there were several problems in using this approach: the strict adherence to the same syllabus and lesson plans for both classes and the restricted access of the control group to the computers may have put some students at a disadvantage.

Snyder also notes that in retrospect she should have used case studies of the students to further develop her findings. Although her study had certain flaws, Snyder insists that researchers can simultaneously employ qualitative and quantitative methods if studies are planned carefully and carried out conscientiously. Babbie, Earl R. The practice of social research. Belmont: Wadsworth Publishing Company, Inc.

Berkenkotter, C. Conventions, conversations, and the writer: Case study of a student in a rhetoric Ph. Research in the Teaching of English 22 1 , Black, Susan. Redefining the teacher's role. Executive Educator,18 8 , Blank, Steven C. Set off in different pathways, qualitative research regarding the individual's wellbeing will be concluded with varying validity. For sampling, procedures and methods must be appropriate for the research paradigm and be distinctive between systematic,[ 17 ] purposeful[ 18 ] or theoretical adaptive sampling[ 19 , 20 ] where the systematic sampling has no a priori theory, purposeful sampling often has a certain aim or framework and theoretical sampling is molded by the ongoing process of data collection and theory in evolution.

For data extraction and analysis, several methods were adopted to enhance validity, including 1 st tier triangulation of researchers and 2 nd tier triangulation of resources and theories ,[ 17 , 21 ] well-documented audit trail of materials and processes,[ 22 , 23 , 24 ] multidimensional analysis as concept- or case-orientated[ 25 , 26 ] and respondent verification.

In quantitative research, reliability refers to exact replicability of the processes and the results. In qualitative research with diverse paradigms, such definition of reliability is challenging and epistemologically counter-intuitive.

Hence, the essence of reliability for qualitative research lies with consistency. Silverman[ 29 ] proposed five approaches in enhancing the reliability of process and results: Refutational analysis, constant data comparison, comprehensive data use, inclusive of the deviant case and use of tables.

As data were extracted from the original sources, researchers must verify their accuracy in terms of form and context with constant comparison,[ 27 ] either alone or with peers a form of triangulation. Most qualitative research studies, if not all, are meant to study a specific issue or phenomenon in a certain population or ethnic group, of a focused locality in a particular context, hence generalizability of qualitative research findings is usually not an expected attribute.

However, with rising trend of knowledge synthesis from qualitative research via meta-synthesis, meta-narrative or meta-ethnography, evaluation of generalizability becomes pertinent.

A pragmatic approach to assessing generalizability for qualitative studies is to adopt same criteria for validity: That is, use of systematic sampling, triangulation and constant comparison, proper audit and documentation, and multi-dimensional theory.

Despite various measures to enhance or ensure quality of qualitative studies, some researchers opined from a purist ontological and epistemological angle that qualitative research is not a unified, but ipso facto diverse field,[ 8 ] hence any attempt to synthesize or appraise different studies under one system is impossible and conceptually wrong.

From a realism standpoint, Porter then proposes multiple and open approaches for validity in qualitative research that incorporate parallel perspectives[ 43 , 44 ] and diversification of meanings. In summary, the three gold criteria of validity, reliability and generalizability apply in principle to assess quality for both quantitative and qualitative research, what differs will be the nature and type of processes that ontologically and epistemologically distinguish between the two.

Source of Support: Nil. Conflict of Interest: None declared. National Center for Biotechnology Information , U. J Family Med Prim Care. Lawrence Leung 1, 2. Author information Copyright and License information Disclaimer. Address for correspondence: Prof. E-mail: ac. This is an open-access article distributed under the terms of the Creative Commons Attribution-Noncommercial-Share Alike 3. This article has been cited by other articles in PMC. Abstract In general practice, qualitative research contributes as significantly as quantitative research, in particular regarding psycho-social aspects of patient-care, health services provision, policy setting, and health administrations.

Keywords: Controversies, generalizability, primary care research, qualitative research, reliability, validity. Nature of Qualitative Research versus Quantitative Research The essence of qualitative research is to make sense of and recognize patterns among words in order to build up a meaningful picture without compromising its richness and dimensionality. Impact of Qualitative Research upon Primary Care In many ways, qualitative research contributes significantly, if not more so than quantitative research, to the field of primary care at various levels.

Overall Criteria for Quality in Qualitative Research Given the diverse genera and forms of qualitative research, there is no consensus for assessing any piece of qualitative research work. Reliability In quantitative research, reliability refers to exact replicability of the processes and the results. Generalizability Most qualitative research studies, if not all, are meant to study a specific issue or phenomenon in a certain population or ethnic group, of a focused locality in a particular context, hence generalizability of qualitative research findings is usually not an expected attribute.

Food for Thought Despite various measures to enhance or ensure quality of qualitative studies, some researchers opined from a purist ontological and epistemological angle that qualitative research is not a unified, but ipso facto diverse field,[ 8 ] hence any attempt to synthesize or appraise different studies under one system is impossible and conceptually wrong.

Footnotes Source of Support: Nil. References 1. Br J Gen Pract. Physician colorectal cancer screening recommendations: An examination based on informed decision making. Patient Educ Couns.

Streamline triage and manage user expectations: Lessons from a qualitative study of GP out-of-hours services. Evaluating care pathways for community psychiatry in England: A qualitative study.

J Eval Clin Pract. Identifying children's health care quality measures for Medicaid and CHIP: An evidence-informed, publicly transparent expert process. Acad Pediatr. Patient difficulty using tablet computers to screen in primary care. J Gen Intern Med. Exploring barriers to participation and adoption of telehealth and telecare within the Whole System Demonstrator trial: A qualitative study.

The problem of appraising qualitative research. Qual Saf Health Care. Sage Publications; Paradigmatic controversies, contradictions, and emerging confluences, revisited.

The Sage Handbook of Qualitative Research; pp. Barbour RS. Checklists for improving rigour in qualitative research: A case of the tail wagging the dog?

Rationale and standards for the systematic review of qualitative literature in health services research. Qual Health Res. Sale JE. How to assess rigour…or not in qualitative papers. Meyrick J. What is good qualitative research? J Health Psychol. Quality in qualitative research. Med J Aust. Waterman AS. The humanistic psychology-positive psychology divide: Contrasts in philosophical foundations.

Am Psychol. Finfgeld-Connett D. Generalizability and transferability of meta-synthesis research findings. J Adv Nurs. Adm Policy Ment Health. Coyne IT. Sampling in qualitative research. Purposeful and theoretical sampling; merging or clear boundaries? Becker PH. Common pitfalls in published grounded theory research. Newbury Park, London: Sage Publications; Naturalistic Inquiry; p.

The qualitative research audit trail: A complex collection of documentation. Res Nurs Health.