Recall from Chapter 5 that my philosophy in writing a research project is to not write it from Chapter 1 to Chapter 2 and so on. Instead, I think it is important that the Methods section of the research paper should be written first to ensure that the topic can be studied using empirical methods. Therefore, once the Purposes, Research Questions, and Research Hypotheses are written, then I suggest writing the research Methods.
The purpose of the Methods section of a research paper (Chapter 3 at UniJos) is to describe the study methods and procedures so well that anybody could exactly repeat the research study using only what was written in the Methods section. Recall from Chapter 2 that one principle of scientific research is that it must be verifiable, meaning that the observations must be able to be confirmed by any other individual. Thus, replication of scientific observations is a foundational principle in science: data is not valid unless it can be duplicated by other researchers. This means that researchers need to clearly articulate the exact research design, sample and sampling procedures, instruments, data collection procedures, and data analyses that were used so any other researcher can exactly replicate the results to verify their authenticity.
Therefore, the researcher needs to take great care in being explicit and accurate in reporting the research methods. Inaccurate reporting of research methods reflects either ignorance on the part of the researcher at best or unethical dishonesty at worst. It is the researcher's responsibility to ask questions or find resources to help them fully understand the research terminology and procedures that are reported to ensure that what is written in the Methods section accurately reflects the research methods that were indeed used to collect the data.
When writing the Methods section, keep in mind that each sub-section should report only the relevant information for that section. For example, the Sample section should only report on the participants, not on the research design, instruments, or procedures. The Instruments section should only report on the instruments, not on the sample, data analysis, or procedures. This helps the reader because they will know exactly what to expect in each section. Furthermore, this helps the reader know where to go to find relevant information if they are just skimming the work. Perhaps when reading through the data analysis, the reader might decide that she needs to refresh her memory about the research Instruments. If the writer has presented all of the information about the Instruments in the Instrument section, then she knows exactly where to look in the report to find the relevant information.
Most educational research studies will reflect one of five designs listed below. By this point, the researcher should have already identified the research design that will be used. In this step, the researcher writes the Research Design sub-section of the research Methods. When writing the Research Design sub-section, the writer has three basic goals:
Before getting into the specifics of writing the research design, it is important to understand the difference between independent and dependent variables. There are a few ways that you can think about the differences between the independent and dependent variables. The independent variable is thought to be the cause whereas the dependent variable is thought to be the effect. For example, in a study examining the effect of socioeconomic status on literacy skills, the independent variable is socioeconomic skills because that is the presumed cause whereas literacy skills is the dependent variable because it is thought to be the effect.
The dependent variable can also be thought of as the variable that the researcher is trying to explain or predict. In this case, the researcher wants to explain literacy skills, so that is the dependent variable, and they think socioeconomic status can influence literacy skills, so socioeconomic status is the cause, the independent variable. When doing experiments, the treatment is always the independent variable - the cause - and whatever is thought to be influenced by the treatment is the effect - the dependent variable. For a study examining the effect of computerized instruction on math skills, the independent variable is computerized instruction and the dependent variable is maths skills.
Only causal-comparative, experimental, and quasi-experimental research designs have independent and dependent variables. Descriptive studies only describe the current state of a variable, so there are no presumed cause or effects, therefore no independent and dependent variables. Likewise, correlational research studies only examine relationships between variables. Since neither variable in a correlational design is manipulated, it is impossible to determine which is the cause and which is the effect. Thus, correlational research designs do NOT have independent and dependent variables.
Below are the research study purposes from the previous step. For each purpose, identify the independent and dependent variables. Not every purpose will have independent and dependent variables, so write "None" if there are no independent and dependent variables.
Descriptive Design
A descriptive research design describes some phenomenon at a specific point in time. A descriptive study needs a clearly defined phenomenon of interest (e.g., teachers' knowledge of literacy development) that is systematically and precisely measured. After explaining what a descriptive study is and why the descriptive study was chosen, then the Research Design sub-section should also include the specific phenomenon of interest (the variable) in the particular research study.
Causal Comparative Design
Causal comparative designs are also called ex post facto. Causal comparative designs examine the effect of an independent variable that cannot be manipulated by the researcher (i.e., gender, socioeconomic status, age, etc.) on a dependent variable. After explaining the causal-comparative design and why the causal-comparative design was chosen (typically because the independent variable cannot be manipulated), then the independent variables should be clearly identified, as well as any dependent variables. How participant were classified into the groups should also be clearly explained. For example, how was it determined whether participants were high or low socioeconomic status?
Correlational Design
The primary characteristic of a correlational design is that it examines the relationship between two or more variables within the same group of people. Recall that correlational designs do not have independent and dependent variables. After explaining the correlational design, the variables of interest should also be clearly described.
Experimental and Quasi-Experimental Designs
Both experimental and quasi-experimental research designs examine the effect of an independent variable that is manipulated by the researcher on a dependent variable. To manipulate the independent variable, participants are placed in groups: a treatment group that receives the treatment and a control group that is identical to the treatment group except that they do not receive the treatment. Then these two groups are compared on the dependent variable.
Let's return to the study examining the effect of computerized instruction on maths skills, which will be measured by a maths examination. There are many factors that could influence a student's score on a math test: age, quality of instruction in maths, interest in maths, valuing of education, sickness, distraction, etc. All of these variables are extraneous variables, any variable besides the independent variable that might unduly contribute to differences between the treatment and control group. It is the researcher's responsibility to control for extraneous variables, meaning that they carefully plan the research study so that extraneous variables are limited. For example, imagine that the treatment group has a different teacher than the control group. The treatment group's teacher might be better than the control group, so any difference between the treatment and control group on the dependent variable might be caused by the teacher, not by the computerized instruction. This means that the researcher needs to carefully consider any differences between the treatment and control group and make an effort to minimize these differences in the planning stage. The research design should report the steps that were taken to minimize the influences of extraneous variables.
The only difference between an experimental and a quasi-experimental design is how the participants were placed in the groups. Experimental designs must have random assignment to the two groups. Quasi-experimental designs use intact groups, such as a specific classroom. Random assignment (sometimes also called randomization) is different from random selection, and it is important to be clear in these two concepts. Selection refers to how participants were selected from the population to participate in the study as part of the sample. On the other hand, assignment occurs after participants have already been selected. The researcher already has a list of names of those people who will participate in the study, so assignment refers to how those participants are assigned to the treatment and control groups. Random assignment means that participants are randomly assigned to the treatment and control groups: not assigned based on previous test scores, not assigned based on their class teacher, not assigned based on any characteristic of the person, but using random procedures.
The reason that researchers use random assignment is to control for extraneous variables. As previously described, there are a million different factors that can influence scores on the dependent variable. However, when participants are assigned to the treatment and control groups using random procedures, then it is assumed that these extraneous factors balance out. When randomly assigned, one group might be slightly smarter, but the other group might have higher motivation. One group might be slightly sicker on the day of the test, but the other group might have more personal problems that caused anxiety. Random assignment assumes that all of these extraneous variables balance each other out.
Note that the term random, whether in random assignment or random selection, does NOT mean haphazard. Instead, random is a technical term that means that every case has an equal and independent chance of being selected (or assigned). Thus, to accurately use the word random, a study must use specific procedures. Researchers have developed a number of procedures for achieving "random" - whether random assignment or random selection. However, the most simple and most common is the Hat and Draw Method. In the hat and draw method, the names of individuals are placed in a hat and drawn out at random. For random assignment, the first name drawn out of the hat is assigned to the treatment group, the second name drawn is assigned to the control group, the third name is assigned to the treatment group, and so on.
If a study uses random assignment, a pre-test is not necessary. In some cases, such as achievement tests, a pre-test can actually be harmful. In an achievement test, when participants see the questions on the pre-test, they might learn the answers on the questions outside of the treatment. In this case, the pre-test is what influenced the post-test scores and not the treatment itself. Therefore, in some cases, pre-testing is harmful and should be avoided. The researcher should carefully consider the advantages and disadvantages of giving a pre-test as they are planning the study. Again, if random assignment to groups is used, a pre-test is not needed because random assignment assumes that both groups are equal at the start of the treatment.
To summarize, both experimental and quasi-experimental research designs have a control group and at least one treatment group. The only difference between an experimental and quasi-experimental design is that the experimental design uses random assignment whereas the quasi-experimental design does not use random assignment.
Experimental Design
Every experimental design has three key elements: a control group, at least one treatment group (multiple different treatments are possible), and random assignment to the treatment and control groups. After explaining the theory of experimental design and the rationale why experimental design was chosen, you need to explain each of these three elements in your particular study. Exactly how random assignment will be done in the research study also needs to be explained in this section. Furthermore, the researcher needs to explain the independent and dependent variables, as well as all steps that will be taken to control for extraneous variables.
For example: The treatment group will receive six weeks of computerized maths instruction using the ActionFraction program. The control group will receive six weeks of ordinary mathematics instruction on fractions. Participants will be randomly assigned to the treatment and control groups by the hat and draw method. The names of every participant will be dropped in a hat and thoroughly shaken. The first name drawn will be placed in the treatment group, the next name will be placed in the control group, and so on. In this study, the independent variable is the computerized instruction and the dependent variables are math skills and interest in instruction. Both the treatment and control groups will receive one hour of instruction per day for six weeks to control for instruction time, and both groups will also be taught by the same teacher to control for teacher ability.
Quasi-Experimental Design
Every quasi-experimental design also has three key elements: a treatment group, a control group, and groups are created without random assignment. Because random assignment is not used, there is a much greater potential for having extraneous variables influence the dependent variable. Therefore, quasi-experimental designs must give both a pre-test and a post-test. The purpose of the pre-test is to identify any differences between the two groups at the start of the experiment. Then, the ANCOVA statistic (Analysis of Covariance) is used to statistically control for pre-test scores
Like the experimental design, the quasi-experimental design should explain the treatment and control group as well as the independent and dependent variables and steps taken to control for extraneous variables (read more in the Experimental Design section for more information). Additionally, for quasi-experimental designs, the researcher should also explain why random assignment was not used and how participants were assigned to the treatment and control groups.
For example: The principal would not allow the students to be randomly assigned to different classes. Instead, intact classrooms had to be used. Therefore, Monday's class was randomly assigned to be the treatment group whereas Tuesday's class was randomly assigned to the control group. To control for extraneous factors, both pre and post tests were used for the dependent variables.
To summarize, the Design section should include:
Two example research designs are given below. The first is from a causal comparative study (Korb, 2009). The second is an experimental study (Korb, n.d.).
The design of this study was causal-comparative because it compared students who cheated to students who did not cheat, a variable that could not be manipulated. A causal-comparative research design examines the effect of an independent variable that the researcher does not manipulate on a dependent variable (Gall, Gall, & Borg, 2003). The current study sought to determine whether students who cheated demonstrated lower academic achievement. The independent variable of cheating behavior was operationally defined as whether students blatantly cheated on the assignment or final exam in their 100-level educational psychology course. The dependent variable of academic achievement was operationally defined as the final exam score in the same course. |
|
An experimental design was used. An experimental design has three key defining features: a treatment group, a control group, and random assignment to those groups. That design was chosen because students were randomly assigned to either a treatment group where participants had teachers who attempted to build a relationship with the student or a control group where participants had teachers who did not attempt to build a relationship. The independent variable was relatedness between the teacher and student. Relatedness was manipulated in two experimental conditions. In the involvement condition, the teacher made conversation with the participant before and after the task. The teacher also made eye contact, smiled, and laughed because these behaviors facilitate relatedness. In the No Involvement condition, the teacher only communicated to the participant about the teaching task. The dependent variable was intrinsic motivation in the teaching task. Intrinsic motivation was assessed by two measures: the Intrinsic Motivation Inventory designed and validated by Ryan (1982) and the time the participant spent engaged in the teaching task after the teacher left the room for 10 minutes. Participants were randomly assigned to the Involvement and No Involvement conditions by putting the names of all of the participants in a hat. The first name drawn was placed in the Involvement condition the second name drawn was placed in the No Involvement condition, and so on. The following procedures were used to control for extraneous variables. The three teachers were research assistants in the study. The teachers spent two hours being trained about behaviors to the Involvement and No Involvement conditions. The teachers also conducted four trial-runs where the experimenter gave the teachers feedback on their performance in the Involvement and No Involvement conditions. The teachers were not informed of the purpose of the study to avoid teacher bias affecting the results. The teacher was the same sex as the participant. For each participant, the experimenter was blind to the condition that the participant was assigned to so the experimenter would not bias the time spent engaged in the teaching task after the study was completed. |
Return to Educational Research Steps
Copyright 2012, Katrina A. Korb, All Rights Reserved