Generalization and Transferability - The WAC Clearing House (2023)

In this chapter we discuss generalizability, transferability, and the interrelationship between the two. We also explain how these two aspects of research work in different methodologies and show how researchers can apply these concepts throughout the research process.

Generalizability Overview

Generalization is used by researchers in an academic setting. It can be defined as the extension of the research results and conclusions of a study carried out in a population sample to the general population. The reliability of this extension, although not absolute, is statistically probable. Because strong generalizability requires data from large populations, quantitative research, eg, experimental, provides the best foundation for broad generalizability. The larger the sample population, the more generalized the results may be. For example, a comprehensive study of the role computers play in the writing process might reveal that students who do most of their composition on a computer are statistically likely to move more blocks of text than students who do not compose on a computer. computer.

Portability overview

Transferability is requested by theLectorthe investigation. Although generalization generally only applies to certain types of quantitative methods, transferability can apply, to varying degrees, to most types of research. Unlike generalization, transferability does not imply broad statements, but instead invites readers of research results to make connections between elements of a study and their own experiences. For example, high school teachers could selectively apply the results of a study showing that heuristic writing exercises help college-level students to their own classrooms.

connections

Generalizability and transferability are important elements of any research methodology, but they are not mutually exclusive: generalizability depends to varying degrees on the transferability of the research results. It is important that researchers understand the implications of these two aspects of research before designing a study. Researchers intending to make a generalizable claim should carefully examine the variables involved in the study. These include the sample population used and the mechanisms behind the formulation of a causal model. In addition, if researchers wish to transfer the results of their study to another setting, they must maintain a detailed description of their research setting and include a full description of that setting in their final report. Armed with the knowledge that the sample population was large and diverse, as well as detailed information about the study itself, readers of the research can more confidently generalize the results and apply them to other situations.

generalizability

Generalization is not only common in research, but also in everyday life. In this section we provide a working definition of generalizability as used within and outside academic research. We also define and consider three different types of generalizability and some of their possible applications. Finally, we discuss some of the potential shortcomings and limitations of generalization that researchers need to consider when constructing a study that they hope will produce potentially generalizable results.

Definition

In many ways, generalization boils down to nothing more than making predictions based on recurring experience. If something happens frequently, we assume that it will continue to do so in the future. Researchers use the same type of reasoning when they generalize the results of their studies. Once researchers have collected enough data to support a hypothesis, a premise can be formulated regarding the behavior of that data, allowing it to be generalized to similar circumstances. However, due to its probabilistic basis, such a generalization cannot be considered definitive or exhaustive.

While generalization can occur in informal, non-academic contexts, it generally only applies to specific research methods in academic studies. Quantitative methods allow some generalization. Experimental research, for example, often leads to generalizable results. However, such experiments must be rigorous to find generalizable results.

example

An example of generalization in everyday life is driving a car. Operating a motor vehicle on the highway requires drivers to make assumptions about the likely outcome of certain actions. When a driver approaches an intersection where a driver is preparing to make a left turn, the driver crossing the intersection in a straight line assumes that the left-turning driver has the right-of-way before turning. The driver passing through the intersection carefully applies this assumption and recognizes the possibility of the other driver making a premature turn.

American motorists also generalize that everyone will drive on the right side of the road. However, if we try to generalize this assumption to other settings like England, we are making a potentially catastrophic mistake. So it's obvious that generalization is necessary to form consistent interpretations in many different situations, but we don't expect our generalizations to work the same way in all circumstances. Given enough evidence, we can make predictions about human behavior, but at the same time we must recognize that our assumptions are based on statistical probability.

Consider this example of generalizable research in English studies. A study of undergraduate composition teacher evaluations might show that there is a strong correlation between how well students grade in a course and whether they give their teacher good grades. The study may reveal that 95% of students who expect to earn a "C" or worse in their class give their teacher a grade of "average" or below. Therefore, there is a high probability that prospective students expecting a "C" or lower will not give their teacher good grades. However, the results would not necessarily be conclusive. Some students might buck the trend. Additionally, a number of different variables can affect how students rate a teacher, including the teacher's experience, class size, and relative interest in a particular subject. These variables, and others, would need to be addressed for the study to potentially produce valid results. But even if virtually all variables were isolated, the study results would not be 100% conclusive. At best, researchers can make well-founded predictions about future events or behavior without guaranteeing the prediction in all cases. Therefore, before generalizing, results must be tested through rigorous experimentation that allows researchers to confirm or reject the premises governing their data set.

considerations

There are three types of generalizability that interact to create probabilistic models. All involve generalizing a treatment or measurement to a population outside of the original study. Researchers wishing to generalize their claims must attempt to apply all three forms to their research or the strength of their claims will be weakened (Runkel & McGrath, 1972).

One type of generalization involves researchers determining whether a given treatment will produce the same results under different circumstances. To do this, they must decide whether an aspect within the original setting, a factor outside of the treatment, produced the particular result. The flexibility with which the treatment adapts to new situations is determined. Greater adaptability means that the treatment is generalizable to a greater variety of situations. For example, imagine that a new set of heuristic pre-writing questions designed to encourage college students to consider audiences more fully works so well that students write carefully crafted rhetorical analyzes of their target audience. To responsibly generalize that this heuristic is effective, a researcher would need to test the same pre-writing exercise in a variety of college-level educational settings with different teachers, students, and settings. If the same positive results are obtained, then the treatment is generalizable.

A second form of generalization focuses on measurements rather than treatments. For a result to be considered generalizable outside of the test group, it must return the same results with different ways of measuring. With respect to the heuristic example above, the results become generalizable if the same results are used in the assessment "with slightly different worded questions or if we use a six-point scale instead of a nine-point scale" (Runkel & McGrath, 1972). ). , p. 46).

A third type of generalizability concerns the subjects of the test situation. Although the results of an experiment may be internally valid, that is, applicable to the group being tested, in many situations the results cannot be generalized beyond that particular group. Investigators hoping to generalize their results to a larger population should ensure that their test group is relatively large and randomly selected. However, researchers must consider the fact that test populations of more than 10,000 subjects do not significantly increase generalizability (Firestone, 1993).

possible limitations

No matter how carefully these three forms of generalization are used, there is no absolute guarantee that the results obtained in a study will be true in all situations outside of the study. To determine causal relationships in a test environment, accuracy is of paramount importance. But if researchers want to generalize their results, range and variability must be emphasized over precision. Thus, it becomes difficult to test for precision and generalizability at the same time, since focusing on one reduces the reliability of the other. One solution to this problem is to take a larger number of observations, which has a double effect: first, it increases the sample population, which increases generalizability; Second, precision can be reasonably maintained because random errors cancel out between observations (Runkel and McGrath, 1972).

portability

Transferability describes the process of applying research results in one situation to other similar situations. In this section we provide a practical working definition of transferability as it is used within and outside of academic research. We also describe the key considerations that researchers need to take into account in order for their findings to be potentially transferable, and the critical role the reader plays in this process. Finally, we discuss potential deficiencies and limitations of transferability that investigators need to consider when designing and conducting a study that produces potentially transferable results.

Definition

Portability is a process performed byLectorthe investigation. Readers perceive the details of the research situation and compare them with the details of a setting or situation with which they are familiar. If there are enough similarities between the two situations, readers may conclude that the research results would be the same or similar in their own situation. In other words, they "transfer" the results of a study to another context. To do this effectively, readers need to know as much as possible about the original research situation to determine if it is similar to their own. Therefore, investigators must provide a very detailed description of their situation and research methods.

The results of any type of research method can be applied to other situations, but transferability is most relevant to qualitative research methods such as ethnography and case studies. Reports based on these investigative methods are detailed and specific. However, because they often consider only one subject or group, researchers conducting such studies rarely generalize the results to other population groups. However, the detailed nature of the results makes them ideal for portability.

example

Transferability is easy to understand considering that we constantly apply this concept to aspects of our daily lives. For example, if you are an inexperienced composition teacher and read a study in which an experienced writing teacher found that extensive preparatory practice helped students in her classes develop much more defined paper themes, you might be wondering. ask how similar the teacher's classroom is to yours. own self. If there were many similarities, you might try to draw conclusions about how increasing the number of your students' recipes would affect their ability to come up with specific enough paper topics. In this would you trytransferBring the techniques of the compositional researcher to your own class.

An example of transferable research in the field of English studies is Berkenkotter, Huckin, and Ackerman's (1988) study of a doctoral student in rhetoric. Program. In this case study, the researchers describe in detail a graduate student's entry into the language community of her academic program and, in particular, her struggle to learn the writing conventions of that community. They draw conclusions about why certain things might have affected graduate student "Nate" in a certain way, but are unable to generalize their findings to all graduate students in the Rhetoric program. programs It is simply a study of one person in a program. However, due to the level of detail provided by the researchers, readers are able to take certain aspects of Nate's experience and apply them to other contexts and situations. This is portability. First-year graduate students reading the Berkenhotter, Huckin, and Ackerman study may see similarities in their own situations, while professors may see their students' struggles and understand them a little better. The researchers do not claim that their results are transferable to other situations. Instead, they report their findings to him and offer suggestions about the possible causes of Nate's difficulties and his eventual success. Readers then consider their own situation and decide whether or not these causes may be relevant.

considerations

When designing a study, researchers need to consider their objectives: do they want to provide limited information on a large group to show trends or patterns? Or would you like to provide detailed information about an individual or a small group that could reveal the reasons for specific behavior? The method they choose will determine the extent to which their results are transferable, as transferability is more likely to apply to certain types of research methods than others.

Wide description:When writing the results of a study, it is important that the researcher provide specific information and a detailed description of their subjects, location, methods, role in the study, etc. This is commonly known as a "thick description" of methods and viewpoints; it is important because it allows readers to make an informed judgment about whether they can apply the views to their own situation. For example, if an educator is conducting an ethnography of her writing class and notices that her students' writing has improved dramatically after a series of student-teacher writing conferences, she needs to detail the classroom, the students you observed and your own participation. When the investigator does not provide enough details. , it will be difficult for readers to try the same strategy in their own classrooms. Unless the researcher mentions that she conducted this research in a small, upper-class private school, readers can apply the results to a large, inner-city public school that has a similar expected result.

The role of the reader:The reader's role in transferability is to apply the methods or results of a study to their own situation. In doing so, readers should consider the differences between the situation described by the researcher and their own. If readers of the Berkenhotter, Huckin, and Ackerman study know that the research was conducted in a small, upper-class private school but decide to try the method in a large, inner-city public school, they will need to adjust for the differences. in attitude and prepare for different results.

Also, readers may decide that the results of a study are not applicable to their own situation. For example, if a study found that watching more than 30 hours of television per week resulted in a lower GPA for physics majors, broadcast journalism majors might decide that these results do not apply to them.

Readers may also transfer only certain aspects of the study and not the full conclusion. For example, in the Berkenhotter, Huckin, and Ackerman study, the researchers suggest a variety of reasons why the graduate student had trouble adjusting to his Ph.D. Program. Although composition teachers can't compare "Nate" to the college freshmen in their composition class, they might ask some of the same questions about their own class and give them insight into some of the writing difficulties that composition teachers face. freshmen. It is up to the readers to decide which ideas are important and which may apply to their own situation; When researchers accept their responsibility to provide a "broad description," this decision becomes much easier to make.

possible limitations

Understanding research results can help us understand why and how something happens. However, many researchers believe that such an understanding is difficult to achieve in relation to human behaviors, which they believe are too difficult to understand and often impossible to predict. "Because of the many and varied ways in which individuals differ from one another, and because those differences change over time, exhaustive and definitive experiments are not possible in the social sciences...the most we can achieve independently realist in educational research “prediction and control, but only temporal understanding” (Cziko, 1993, p. 10).

Cziko's point is important because portability allows for "temporal understanding". Instead of applying the research results to every situation that may arise in the future, we can apply a similar method to another similar situation, look at the new results, apply a modified version to another situation, etc. Transferability takes into account the fact that there are no absolute answers for given situations; Rather, each individual must determine their own best practices. Transferring the research of others can help us develop and modify these practices. However, it is important for readers of research to know that results are not always transferable; an outcome that occurs in one situation does not necessarily occur in a similar situation. Therefore, it is crucial to consider the differences between situations and modify the investigative process accordingly.

Although transferability seems to be an obvious, natural and important way to apply research results and conclusions, it is not considered a valid research approach in some academic circles. Perhaps partly in response to criticism, researchers in many modern research articles refer to their findings as generalizable or externally valid. So it seems that they are not talking about portability. In many cases, however, the researchers themselves provide guidance on what points readers should consider, but are reluctant to draw general conclusions or statements. These are properties oftransferable results.

As we have seen, generalizability is actually quite different from transferability. Unfortunately, the confusion surrounding these two terms can lead to misinterpretation of research results. Emphasis on the value of transferable results, as well as a clear understanding among researchers in the English field of the critical differences between the conditions under which research can be generalized, transferred, or, in some cases, generalized and transferred. qualitative researchers to address some of the criticisms to avoid, voiced by skeptics who question the value of qualitative research methods.

Generalizability and Transferability: Synthesis

The ability to generalize allows us to form coherent interpretations in any situation and to act decisively and effectively in daily life. Transferability gives us the ability to classify given methods and conclusions to decide what to apply to our own circumstances. So both generalizability and transferability essentially allow us to make comparisons between situations. For example, we can generalize that most people in the United States drive on the right side of the road, but we cannot extrapolate that conclusion to England or Australia without finding ourselves in a treacherous situation. Therefore, it is important to always consider the context when generalizing or extrapolating results.

Whether a study emphasizes transferability or generalizability is closely related to the goals of the researcher and the needs of the audience. Studies for a magazine likeweatheror a daily newspaper tends to generalize because editors want to provide information that is relevant to a large segment of the population. A research project directed at a small group of specialists studying a similar problem can emphasize transferability, as specialists in the field have the ability to extrapolate aspects of the study results to their own situation without the researcher making obvious generalizations. . Ultimately, the researcher's topic, audience, and objectives determine the method the researcher uses to conduct a study, which then determines the transferability or generalizability of the results.

A comparison of generalizability and transferability

Although generalization has been a favored research method for quite some time, transferability is a relatively new idea. Theoretically, however, it has always accompanied research questions. It is important to note that generalizability and transferability are not necessarily mutually exclusive; they can overlap.

From an experimental study to a case study, the reader translates the research methods, results and ideas into their own context. Therefore, a generalizable study may also be transferable. For example, a researcher can generalize the results of a survey of 350 people at a university to the entire university population; Readers of the results can apply or transfer the results to their own situation. Basically, they will wonder if they belong to the majority or not. However, a transferable title cannot always be generalized. for example incase studies, transferability allows readers the ability to extrapolate results to external contexts, while generalization is inherently impossible because an individual or a small group of individuals is not necessarily representative of the larger population.

Controversy, value, and function

Scientific research has a long tradition of valuing empirical studies; Experimental studies are considered "the" type of research. As social scientists adapted scientific inquiry methods to their own needs, they embraced this penchant for empirical inquiry. Therefore, generalizable studies have long been considered more rewarding; The value of research was often determined by whether a study could be generalized to a population as a whole. However, more and more social scientists are recognizing the value of using a variety of research methods, and the value of transferability is being recognized.

It is important to realize that generalizability and transferability alone do not determine the value of a study. Depending on the topic and the objective of the researcher, they fulfill different functions in the investigation. Where generalizable studies often point to phenomena that apply to broad categories like gender or age, transferability can provide insight into the how and why behind these findings.

However, there are weaknesses that must be considered. Researchers can study a small group that is representative of a larger group and claim that their results are likely to apply to the larger group, but they cannot test every individual in the larger group. Therefore, their conclusions are only valid in relation to their own studies. Another problem is that an unrepresentative group can lead to a wrong generalization. For example, a study of composition students; Skills review comparing student progress over a semester in a computer classroom to student progress in a traditional classroom might show that computers help students through the writing process. However, if unusually high numbers of students in traditional classrooms were later found to have substance abuse problems outside of the classroom, the population studied would not be considered representative of the entire student population. Therefore, it would be problematic to generalize the study results to a larger student population.

When it comes to transferability, readers need to know as much detail as possible about a research situation in order to accurately transfer the results to their own. However, it is impossible to describe a situation with absolute completeness and the lack of details may lead the reader to extrapolate the results to a situation that is not entirely similar to the original.

Applications of research methods.

The extent to which generalizability and transferability are applicable differs both from one methodology to another and from one study to another. Researchers should be aware of these gradations so that results are not undermined by overgeneralization, and readers should ensure that they do not read research results in a way that misapplies or interprets them.

Transferability and generalizability applications: case study

Design of the investigation
Case studies examine individuals or small groups in a specific context. Research is usually collected through qualitative means: interviews, observations, etc. The data is usually analyzed comprehensively or through coding methods.

assumptions
When conducting case study research, a researcher generally assumes that the results will be transferable. Generalization is difficult or impossible because one person or a small group cannot represent all similar groups or situations. For example, a group of beginning writers in a particular classroom may not represent all beginning writers. In addition, the conclusions drawn in the case studies refer only to the observed participants. With rare exceptions, case studies do not attempt to establish cause-and-effect relationships between variables. The results of a case study are transferable in the sense that researchers “suggest further questions, hypotheses, and future implications” and present the results as “directions and questions” (Lauer & Asher 32).

example
To illustrate the writing skills of would-be college writers, a case study researcher might select one or more students in a composition class and talk to them about how they assess their own writing, reading actual essays, and establishing evaluation criteria. review grades on paper/teacher interpretation.

Results of a study
In presenting the results of the example above, a researcher should define established criteria for determining what he meant by "writing skills," provide notable citations from student interviews, and provide different information depending on the type of research methods used (for example, surveys, classroom observation, collected writing samples) and include opportunities to encourage this type of inquiry. Readers can then judge for themselves how the researcher's observations can be transferred to other kinds of writing.

Transferability and Generalizability Applications: Ethnography

Design of the investigation
Ethnographies study groups and/or cultures over a period of time. The goal of this type of research is to understand the particular group/culture by immersing the observer in the culture or group. The research is carried out through various methods similar to case studies, but since the researcher has been immersed in the group for a longer period of time, more detailed information is usually collected during the research. (There Are No Children Here by Jonathan Kozol is a good example.)

assumptions
As with case studies, the findings of ethnographies are also considered transferable. The primary goals of an ethnography are to "identify, operationally define, and relate to each other" within a given context, "variables" that ultimately produce detailed reports or"thick descriptions"(Lauer and Asher 39). Unlike a case study, the researcher discovers much more detail here. Findings from ethnographies should "suggest variables for further investigation" and not generalize beyond the participants in a study (Lauer & Asher 43). Because analysts conducting this type of research tend to rely on multiple methods to gather information (a practice also known as triangulation), their findings typically help provide a detailed description of human behavior to create in a specific setting.

example
The Iowa Writing Program has a widespread reputation for producing excellent writers. To understand their education, an ethnographer might observe students throughout their course. During this time, the ethnographer was able to review the syllabus, follow the writing process of individual authors, and get to know the authors and their work. By the end of a two-year study, the researcher would have a much deeper understanding of the unique and effective features of the program.

Results of a study
Obviously, the Iowa writer is unique, so it would be problematic to transfer the results to another writer. However, an ethnography would give readers an idea of ​​the show. Readers can ask questions like: What qualities make it strong and what is unique about the writers trained in the program? At this point, readers might try to "transfer" applicable knowledge and observations to other writing environments.

Transferability and Generalizability Applications: Experimental Research

Design of the investigation
A researcher working within this methodology creates an environment in which to observe and interpret the results of a research question. A key element in experimental research is being a participant in a study.randomly assigned to groups.In an attempt to create a causal model (ie, to discover the causal origin of a particular phenomenon), groups are treated differently, and steps are taken to determine whether different treatments appear to have different effects.

assumptions
Experimental research is generally considered generalizable. This methodology explores cause and effect relationships through comparisons between groups (Lauer & Asher 152). Because participants are randomly assigned to groups, and because most experiments include enough individuals to reasonably approximate the populations from which each participant is drawn, generalization is warranted because "with a large number of assignments, will be that all groups of subjects are identical in all respects". variables" (155).

example
A simplified example: Six composition rooms are randomly selected (as well as students and teachers) in which three teachers include the use of email as a class activity and three do not. If students in the first three grades begin to discuss their homework over email and, as a result, review their homework better than students in the other three grades, a researcher is likely to conclude that email is embedded in a class of writing and writing needs to be improved. student quality.

Results of a study
Although experimental research is based on cause-and-effect relationships, "certainty" can never be achieved, but the results are "probabilities" (Lauer and Asher 161). Depending on how the researcher presented the results, they can be generalized by randomly selecting students. Since the quality of writing improved in all three classes with the use of email, it is likely that email is the cause of the improvement. Readers of this study would transpose the results to clarify the details: Are these students representative of a group of students known to the reader? What kinds of previous writing experiences did these students have? What type of writing was expected of these students? This information must have been provided by the investigator for the results to be transferable.

Portability and Generalizability Applications: Overview

Design of the investigation
The objective of a survey is to obtain specific information about a specific group or a representative sample of a specific group. Respondents are asked to respond to one or more of the following item types: open-ended questions, true/false questions, agree/disagree (or Likert) questions, ratings, ratings, etc. The results are generally used to understand the attitudes, beliefs, or knowledge of a particular group.

assumptions
The survey results can be generalized, assuming that care was taken in the development of the survey items and the selection of the survey sample, and that adequate response rates were achieved. Note, however, that survey results should only be generalized to the population from which the survey results come.

example
For example, a survey of graduate students of English at Colorado State University to determine how well the French philosopher/critic Jacques Derrida is understood before and after a critical literary theory course might inform professors that Derrida's concepts are generally understood and those of the CSU. The E615 theory course helped students understand Derrida's ideas.

Results of a study
The generalizability of surveys depends on several factors. Whether distributed to a large number of people or a select few, surveys are "personal in nature and subject to bias." Respondents may or may not understand the questions asked. Depending on whether the survey designer is present or not, respondents may have an opportunity to clarify their misunderstandings.

It is also important to remember that errors can occur at the development and processing level. A researcher may ask inappropriate questions (i.e., not ask the right questions for the information sought), interfere with data collection (interview certain people and not others), and distort results during processing (misinterpret responses and be unable to question the participant). etc.). One way to avoid such errors is for researchers to examine other studies of a similar nature and compare their results with the results of previous studies. This will reveal any major discrepancies. Depending on the size of these discrepancies and the context of the survey, the results may or may not be generalizable. For example, when a better understanding of Derrida is evidentafterStudents completing E615 can theorize that E615 effectively teaches them Derrida's concepts. Portability issues may be visible in the survey questions themselves; That is, they could provide important background information that readers may need to know in order to put the results in another context.

The qualitative versus quantitative debate

In Miles and Huberman's 1994 bookQualitative data analysis, quantitative researcher Fred Kerlinger is quoted as saying, “There are no qualitative data. Everything is 1 or 0” (p. 40). In this regard, another researcher, D. T. Campbell, affirms that "all research ultimately has a qualitative basis" (p. 40). This back and forth between qualitative and quantitative researchers is, according to Miles and Huberman, "essentially unproductive." She and many other researchers agree that these two research methods often need each other. However, because qualitative data typically involves words and quantitative data typically involves numbers, there are some researchers who believe that one is better (or more scientific) than the other. Another key difference between the two is that qualitative research is inductive and quantitative research is deductive. In qualitative research, a hypothesis is not required to start the research. However, any quantitative research requires a hypothesis before the research can begin.

Another key difference between qualitative and quantitative research is the underlying assumptions about the role of the researcher. In quantitative research, the researcher is ideally an objective observer who does not participate in or influence the subject being studied. In qualitative research, however, it is believed that the researcher can learn more about a situation by participating and/or immersing himself in it. These basic assumptions underlying both methodologies guide and mandate the types of data collection methods used.

Although there are clear differences between qualitative and quantitative approaches, some researchers argue that the choice between using qualitative or quantitative approaches actually has less to do with methods than with positioning within a particular research discipline or tradition. The choice of method is made more difficult by the fact that the research is often affiliated with universities and other institutions. The results of research projects often guide important decisions about specific practices and policies. The choice of which approach to use may reflect the interests of those conducting or benefiting from the research and the purposes for which the results will be applied. Decisions about which type of research method to use may also be based on the researcher's own experience and preference, the population being studied, the intended audience for the results, time, money, and other available resources (Hathaway, 1995).

Some researchers believe that qualitative and quantitative methods cannot be combined because the assumptions underlying each tradition are very different. Other researchers believe that they can only be used in combination, switching between methods: qualitative research is appropriate for answering certain types of questions under certain conditions, and quantitative is appropriate for others. And some researchers believe that both qualitative and quantitative methods can be used simultaneously to answer a research question.

To some extent, researchers on all sides of the debate are right: Each approach has its drawbacks. Quantitative research often "forces" responses or people into categories that may not "fit" to give meaning. Qualitative research, on the other hand, sometimes focuses too much on individual results and fails to make connections to broader situations or possible causes of the results. However, rather than dismiss either approach due to its shortcomings, researchers must find the most effective ways to integrate elements of both to ensure that their studies are as accurate and comprehensive as possible.

It is important for researchers to recognize that qualitative and quantitative methods can be used together. Snyder (1995) used qualitative and quantitative approaches in a study of computer-assisted writing classes. The study was designed according to the guidelines for quantitative studies: the computer classroom was the 'treatment' group and the traditional pencil-and-paper classroom was the 'control' group. Both classes contained subjects with the same characteristics as the population sample. Both classes followed the same curriculum and were taught by the same teacher in the same semester. The only variable used was the computers. Although Snyder designed this study as an "experiment," he used many qualitative approaches to supplement his findings. As a participant observer, she regularly observed both classrooms and conducted several interviews with the teacher during and after the semester. However, there were several problems with this approach: strict adherence to the same curriculum and lesson plans for both classes, and the control group's limited access to computers may have put some students at a disadvantage. Snyder also notes that, in hindsight, she should have used student case studies to further develop her ideas. Although her study had some flaws, Ella Snyder insists that researchers can use qualitative and quantitative methods simultaneously if the studies are carefully designed and conscientiously conducted.

Annotated bibliography

Babbie, Graf R. (1979).The practice of social research.. Belmont: Wadsworth Publishing Company, Inc.

A comprehensive overview of social science research, including research techniques. The logic behind social science research is discussed.

Berkenkotter, C., Huckin, TN, & Ackerman, J. (1988). Conventions, Conversations, and the Writer: The Case Study of a Ph.D. Student in Rhetoric. Program.research in teaching english22 (1), 9-44.

Describes a case study of a beginning student in a Ph.D. Program. Consider the process of their entry into an academic discourse community.

Black, Susan. (nineteen ninety six). Redefine the role of the teacher.Executive Educator, 18(8), 23-26.

It explains the value of well-trained teachers doing research in their classrooms. points out that teaching research focuses on the particular; it does not look for broad and generalizable principles.

Blank, Steven C. (1984).Methods of practical economic research.. Westport: AVI Publishing Company, Inc.

A comprehensive book on setting up a research project, collecting data, and drawing and reporting conclusions.

Blank, Steven C. (1984).Methods of practical economic research.. Westport: AVI Publishing Company, Inc.

A comprehensive book on setting up a research project, collecting data, and drawing and reporting conclusions.

Bridges, David. (1993). Transferable Skills: A Philosophical Perspective.study higher education18 (1), 43-51.

The transferability of learning competences is discussed, focusing on the notions of interdisciplinary, general, basic and transferable competences and their role in the university curriculum.

Brookhart, Susan M. & Rusnak, Timothy G. (1993). A Pedagogy of Enrichment, Not Poverty: Successful Teaching by Exemplary Urban Teachers.Teacher Training Magazine, 44(1), 17-27.

Reports the results of a study examining the characteristics of effective teachers in the City of Pittsburgh. It suggests that the results could be transferable to urban educators in other contexts.

Bryan, Alan. (1988).Quantity and quality in social research. Boston: Unwin Hyman Ltd.

Metzger, Jude. (1994, julio).Case and cohort study components in research on teacher education.Paper presented at the Australian Teacher Education Association Annual Conference, Brisbane, Queensland, Australia.

He believes that studies of teacher development will be generalizable if a wide range of data collection methods are used, if the data collected is extensive and intensive, and if the methods used take into account differences between the people and situations studied.

Carter, Duncan. (1993). Critical Thinking for Authors: Transferable Skills or Subject-Specific Strategies?Composition Studies/Freshman English News, 21(1), 86-93.

It questions the context dependency of critical thinking and whether critical thinking skills are transferable to writing tasks.

Carter, Kathy. (1993). The place of history in teaching and teacher training.educational scientist, 22(1), 5-12.

It explains the benefits of storytelling in the classroom and in teacher education, but cautions teachers who are currently unfamiliar with storytelling in current pedagogical structures to be cautious when implementing this method in their classrooms.

Clonts, Jean G. (1992, January).The concept of reliability in relation to data from qualitative studies.Paper presented at the Southwest Educational Research Association Annual Meeting, Houston, TX.

Presents a review of the literature on reliability in qualitative studies and defines reliability as the extent to which studies can be replicated using the same methods and yielding the same results. Strategies to improve reliability through study design, data collection, and analysis are suggested. Generalizability as an estimate of reliability is also explored.

Connelly, Michael F. and Clandinin D. Jean. (1990). Experience stories and narrative research.educator, 19(5), 2-14.

He describes the narrative as a site of inquiry and a qualitative research methodology in which the experiences of the observer and the observed interact. This form of research requires the development of new criteria, which may include appearance, credibility, and transferability (7).

Crocker, Linda y Algina, James. (1986).Introduction to classical and modern proof theory.Nueva York: Holt, Rinehart y Winston.

Explains test theory and its application to psychometrics. Chapters range from an overview of important topics to statistical methods and applications.

Cronbach, Lee J. et al. (1967).The reliability of behavioral measures: multifaceted studies of generalizability.Stanford: Stanford ARRIBA.

A technical research paper that includes statistical methods to test multifaceted generalization with classical reliability.

Cziko, Gary A. (1992). Appropriate behavior as control of perception: Implications for educational research.educational scientist, 21(9), 10-18. El Hasan, Karma. (nineteen ninety five). Student evaluation of the lesson: generalization of the results.Studies in educational research21 (4), 411-29.

Issues of dimensionality, validity, reliability, and generalizability of student instructional ratings are discussed in connection with a study in which 610 college students rated their teachers on the Teacher Effectiveness Scale.

Feingold, Alan. (1994). Gender differences in the variability of intellectual ability: a cross-cultural perspective.Gender Roles: A Research Journal20 (1-2), 81-93.

Feingold is conducting a cross-cultural quantitative review of contemporary findings on gender differences in variability of verbal, mathematical, and spatial ability to assess the generalizability of the US findings that men are more variable in mathematics. and spatial ability than women and that the genders are equally variable in verbal ability.

Firestone, William A. (1993). Alternative arguments to generalize the data applied in qualitative research.educational scientist, 22(4), 16-22.

It focuses on generalization in three areas of qualitative research: extrapolation from the sample to the population, analytical generalization, and case-to-case transfer (16). It explains the underlying principles, related theories, and criteria for each approach.

Fyans, Leslie J. (ed.). (1983). Generalizability Theory: Conclusions and Practical Applications. in theNew instructions for testing and measuring: Vol. 18.San Francisco: Jossey-Bass.

A collection of articles on the theory of generalization. The aim of the book is to present various aspects and applications of generalizability theory so that the reader can apply the theory.

Hammersley, Martin. (Hrsg.). (1993).Social research: philosophy, policy and practice.Newbury Park, CA: Sage Publications.

A collection of articles offering an overview of positivism; contains an article on increasing the generalizability of qualitative research by Janet Ward Schofield.

Hathaway, R. (1995). Assumptions underlying quantitative and qualitative research: implications for institutional research.university research, 36(5), 535-562.

Hathaway says that the choice between using qualitative or quantitative approaches has less to do with methodology and more to do with alignment with particular theoretical and academic traditions. She concluded that the two approaches approach problems in very different ways, each with its own advantages and disadvantages.

Devils, Ronald H., Marcoulides, George A. (1996). .research in teaching english22 (1), 9-44.

Describes a case study of a beginning student in a Ph.D. Program. Consider the process of their entry into an academic discourse community.

Hipps, Jerome A. (1993).Reliability and authenticity: alternative ways of evaluating authentic appraisals.Paper presented at the American Educational Research Association Annual Meeting, Atlanta, GA.

It contrasts the basic assumptions of the constructivist approach to traditional research and the positivist approach to authentic assessment in relation to generalization and other research questions.

Howe, Kenneth and Eisenhart, Margaret. (1990). Standards for Qualitative (and Quantitative) Research: A Preliminary.educator, 19(4), 2-9.

Huang, Chi-yu, ua (abril de 1995).A theoretical-generalizing approach to the study of teacher evaluation instruments carried out by students.Paper presented at the American Educational Research Association Annual Meeting, San Francisco, CA.

It presents the results of a study that used generalization theory to examine the reasons for variability in a teacher and course evaluation mechanism.

Hungerford, Harold R. et al. (1992).Investigation and evaluation of environmental problems and measures: modules of development of competences.

A guide designed to teach students to study and evaluate environmental issues and policies. The guide is presented in six modules that include information gathering and surveys, questionnaires and opinion sheets.

Jackson, Philip W (1990). The tasks of educational research.educator 19(7), 3-9. Johnson, Randell G. (1993, abril).A validity generalization study of multiple assessment test and program services.Paper presented at the American Educational Research Association Annual Meeting, Atlanta, GA.

Presents the results of the examination of the validity reports of the Multiple Assessment Test and Program Services, using quantitative analysis to determine the generalizability of the results.

Jones, Elizabeth A y Ratcliff, Gary. (1993).Critical thinking skills for college students.(National Center for Post-Secondary Teaching, Learning and Assessment). University Park, Pennsylvania.

Reviews the research literature that examines the nature of critical thinking; analyzes the extent to which critical thinking can be generalized across disciplines.

Karpinski, Jakob. (1990).Causality in sociological research🇧🇷 Boston: Kluwer Academic Publishers.

Discusses causality and causal analysis in relation to sociological research. Provides equations and explanations.

Kirsch, Irwin S. and Jungeblut, Ann. (1995).Use results of large-scale assessments to identify and assess generalizable indicators of literacy.(National Center for Adult Literacy, Publication No. TR94-19). Philadelphia, Pennsylvania.

Reports on the analysis of data collected during a comprehensive literacy survey to help understand the different variables involved in reading literacy. points out that literacy can be predicted in large and heterogeneous populations, but not as effectively in homogeneous populations.

Lauer, Janice M. y Asher, J. William. (1988).Compositional Research: Empirical Designs.New York: Oxford Press.

It explains the selection of topics, the formulation of hypotheses or questions, the collection of data, the analysis of data, and the identification of variables through the discussion of each design.

LeCompte, Margaret and Goetz, Judith Prissle. (1982). Reliability and validity problems in ethnographic research.Educational Research Journal, 52.(1), 31-60.

It focuses on educational research and ethnography and shows how reliability and validity can best be considered in ethnographic research.

Marcoulides, George; Simkin, Mark G. (1991). Assessment of student work: the case of peer review.Business education magazine67 (2), 80-83.

A preprinted assessment form and generalization theory are used to assess the reliability of students' grading of papers.

Maxwell, Joseph A (1992). Understanding and validity in qualitative research.Harvard Education Magazine, 62(3), 279-300.

It examines the five types of validity used in qualitative research, including generalizable validity, and explores potential threats to research validity.

McCarthy, Christine L. (1996, spring). What is critical thinking"? Is it generalizable?Pedagogical Theory, 46217-239.

Review, compare and contrast a selection of essays from Stephen P. Norris's book The Generalizability of Critical Thinking: Multiple Perspectives on an Education Ideal for exploring the diversity of the topic of critical thinking.

Miles, Matthew B. and Huberman, A. Michael. (1994).Qualitative data analysis.Thousand Oaks: Sage Publications.

A comprehensive overview of data analysis. Topics range from data collection to producing an actual report.

Minium, Edward W. y King, M. Bruce y Bear, Gordon. (1993).Statistical thinking in psychology and pedagogy. . . . . . . . . . . . . . . . New York: John Wiley & Sons, Inc.

A textbook designed to teach students statistical data and theory.

Moss, Pamela A. (1992). Changing notions of validity in educational measurement: implications for performance evaluation.Educational Research Journal, 62.(3), 229-258. Nachmias, David & Nachmias, Chava. (1981).research methods in the social sciences.Nueva York: St. Martins Presse.

It discusses the foundations of empirical research, data collection, data processing and analysis, inferential methods, and the ethics of social science research.

Nagy, Felipe; Jarchow, Elaine McNally. (1981). Estimation of the variance components of test scores in a complex design.Conference / conference contribution.

This article discusses the variables that affect the quality of written compositions and how they can best be controlled to improve the reliability score of writing ability.

Nagy , William E. , Herman , Patricia A. and Anderson , Richard C. (1985).Learning word meanings from context: how generalizable?(University of Illinois at Urbana-Champaign. Center for the Study of Reading, Technischer Bericht Nr. 347). Cambridge, MA: Bolt, Beranek, and Newman.

Reports the results of a study examining how students learn the meanings of words while reading from context. He states that the study should be generalized.

Naiser, Gilbert. (1992, January).Fundamental concepts of generalization theory: A more powerful approach to assessing reliability.Presented at the Southwest Educational Research Association Annual Meeting, Houston, TX.

He explains how a measurement approach called Generalizability Theory (G-Theory) is an important alternative to the more classical measurement theory that provides less useful coefficients. G-theory is about the reliability of behavioral measurements that allow multiple sources of error variance to be estimated simultaneously.

Newman, Isadore and Macdonald, Suzanne. (1993, May).Interpretation of qualitative data: a methodological investigation.Paper presented at the annual meeting of the Ohio Academy of Sciences, Youngstown, OH.

Issues of consistency, triangulation, and generalization are discussed in connection with a qualitative study with graduate student participants. The authors refute Polkinghorne's views on the generalization of qualitative research and argue that quantitative research is better suited to generalization.

Norris, Stephen P. (Hrsg.). (1992).The generalizability of critical thinking: multiple perspectives on an educational ideal.Nueva York: Teachers College Press.A series of essays from different disciplines presenting different perspectives on the topic of the generalization of critical thinking. Authors refer to and respond to each other.Peshkin, Alan. (1993). The quality of qualitative research.educational scientist, 22(2), 23-29.

He explains how effective qualitative research can be in achieving desired results, and concludes that it is an important tool that scientists can use in their explorations. The four categories of qualitative research are examined: description, interpretation, verification, and evaluation.

Raffilson, Fred. (1991, July).The case of generalization of validity.

Describes generalization as a quantitative process. It briefly discusses the theory, method, examples, and applications of validity generalization, emphasizing invisible local methodological problems.

Rhodebeck, Laurie A. The Structure of Men's and Women's Feminist Orientations: Feminist Identity and Feminist Opinion.Gender and Society10(4), 386-404.

This study addresses two questions: the extent to which feminist views differ from feminist identity, and the generalization of these separate constructions across gender and time.

Runkel, Philip J. y McGrath, E. Joseph. (1972).Behavior research: a systematic method guide.Nueva York: Holt, Rinehart und Winston, Inc.

It explains how researchers can take their experiences with human behavior and apply them to research in a systematic and explicit way.

Solomon, Gabriel. (1991). Overcoming the qualitative-quantitative debate: Analytical and systemic approaches to educational research.educational scientist 20(6), 10-18.

Examines complex issues/variables involved in studies. Two types of approaches are explored: an analytical approach that assumes internal and external problems, and a systematic approach in which each component affects the whole. It also explains how a study can never fully measure how much x affects y because there are so many interrelationships. Knowledge is applied differently in each approach.

Schrag, Franz (1992). In defense of positivist research paradigms.educational scientist, 21(5), 5-8.

Positivist critics Elliot Eisner, Fredrick Erikson, Henry Giroux, and Thomas Popkewitz are logically committed to theses that can only be tested through positivist research paradigms. A definition of positivism is collected through examples. In general, the conclusion is reached that educational research need not pretend to be practical.

Sekaran, Uma. (1984).Research Methods for Managers: A Skills Development Approach.New York: John Wiley and Sons.

Explains management approaches to conducting research in organizations. It provides clear definitions and explanations of methods such as sampling and data analysis and interpretation.

Shadish, William R (1995). The logic of generalization: five principles common to experimentation and ethnography.American Journal of Community Psychology23 (3), 419-29.

Both experiments and ethnographies are highly traceable, which is why they are often criticized for their lack of generalization. This article describes generalization logic that can help solve such problems.

Shavelson, Richard J. y Webb, Noreen M. (1991).Generalizability Theory: An Introduction.Newbury Park, CA: Sage Publications.

Snyder, I. (1995). Multiple Perspectives in Literacy Research: Integrating Quantitative and Qualitative.language and education, 9(1), 45-59.

This article details a study in which the author used quantitative and qualitative methods simultaneously to compare computer composition classrooms and traditional classrooms. Although there were some problems integrating the two approaches, Snyder says they can be used together if researchers plan carefully and use their methods judiciously.

Stallings, William M. (1995). Confessions of a quantitative education researcher trying to teach qualitative research.educational scientist, 24(3), 31-32.

It explains the trials and tribulations of teaching a qualitative research course to graduate students. The author describes the successes and failures he encounters and asks his colleagues for suggestions for reading his curriculum.

Wagner, Ellen D. (1993, January).Evaluation of distance learning projects: an approach for comparisons between projects.Paper presented at the Annual Meeting of the Association for Educational Communication and Technology, New Orleans, LA.

It describes a methodology developed to evaluate distance education projects in a way that takes into account specific institutional problems while producing generalizable, valid, and reliable results that allow for discussion among different institutions.

Yin, Robert K. (1989).Case study research: design and methods.London: Wise Publications.

A small section on the application of generalizability in relation to case studies.

appointment information

Jeffrey Barnes, Kerri Conrad, Christof Demont-Heinrich, Mary Graziano, Dawn Kowalski, Jamie Neufeld, Jen Zamora, and Mike Palmquist. (1994-2022). generalizability and transferability. The WAC clearinghouse. Colorado State University. Available at https://wac.colostate.edu/repository/resources/writing/guides/.

registered information

Copyright ©1994-2022Colorado State Universitymethe authors, developers and contributors of this website. Some materials displayed on this site are used with permission.