Adapted from Singleton & Straits (2010)
As can be seen from the figure above, once the research design has been determined, then the researcher has two separate responsibilities: identifying the sample and determining how the key variables will be measured. The key variables should have been identified in Step 2: Identify Key Variables and Research Design. Now that the sample has been identified, the researcher must now transition to thinking about what the variables mean both theoretically and practically to determine how to measure the key variables in the study. Measurement consists of assigning numbers or labels to the units of analysis that accurately represent their position on the variables under study. For example, if a research study is examining Maths achievement of JS1 students, each unit of analysis (JS1 student) must be assigned a number that represents their position in Maths achievement. This can simply be done by assigning them a score on a Maths exam.
Except for qualitative research designs in which the key sources of data include words, pictures, or objects, the key variables need to be translated into numbers so the researcher can analyze the data using statistics. The process of measurement consists of moving from the theoretical definition of a variable (typically called the construct definition) to the concrete mode of measuring that variable in the research study. Measurement is typically done by developing an instrument, which can be a questionnaire, an examination, an interview, an observation schedule, etc.
When designing an instrument, keep in mind the following:
Developing an instrument takes a lot of work and advanced preparation. It is very important that the instrument has been thoroughly critiqued, evaluated, and pilot tested by the student, supervisor, and others before it is administered for the actual study. Once the instrument has been administered, it cannot be changed. Any problems with the instrument after it has been administered will require the student to completely redo the data collection process, wasting considerable time and money. Therefore, researchers must take great thought and care when developing their instrument.
There are many steps that must be done in developing an instrument, which will be described below.
As just mentioned in the previous Step 8: Search Literature, the first step in measurement is to find other research studies that have examined the same variables. This step is helpful in developing the instrument for two reasons.
The second step in measurement is to develop a construct definition of each of the key variables. The construct definition is the theoretical definition, and this definition is best obtained by referencing other research studies that have measured the same variable. For example, a well-established definition for socio-economic status is a person's economic standing based on lifestyle, prestige, power, and control of resources (Liu, Ali, Soleck, Hopps, Dunston, Pickett, 2004).
A researcher must develop a construct definition for each key variable identified from Step 2: Identify Key Variables and Research Design. The variables that are associated with a treatment such as a new teaching method or counseling procedure, are not measured but manipulated. These variables will not pass through the measurement process, but instead will be manipulated by treatment and control groups. For more information on manipulating variables, go to Step 11: Manipulate the Treatment.
Once the construct definition has been developed, the researcher must translate that theoretical definition into a concrete way of measuring the variable so numbers can be assigned. The abstract construct definition of socioeconomic status provides no information about how to measure socioeconomic status. Therefore, the researcher needs to think of how to make that definition more concrete, or how it will be practically measured, a process called operationalize. The operational definition of a variable consists of a statement of specifically how the construct will be measured or implemented in the study. In most studies, socio-economic status is typically measured by a composite of an individual's level of income, occupation, and level of education.
The operational definition must directly relate to the construct definition. For example, the operational definition of socio-economic status must be closely related to the definition "a person's economic standing based on lifestyle, prestige, power, and control of resources." It would be inaccurate for a person to give an operational definition of socioeconomic status as "amount of time spent at work" because this operational definition does not closely relate to the construct definition. The best operational definition is one that has the clearest relationship to the construct definition.
In causal-comparative studies, it is also important that the operational definition states how participants will be assigned to the groups. If gender is the key independent variable in a causal-comparative study, it is clear that a participant's self-report of their gender will assign them into the appropriate group. However, other studies might not be quite as clear. For example, a causal-comparative study might classify participants into groups based on socio-economic status. What criteria will be used to determine whether a person is low or medium status? This criteria must be clearly specified.
When developing the operational definition, a researcher must consider practical limitations. Imagine you want to measure a primary 1 child's level of socio-economic status. Can a small child report their family's income, occupation, and level of education? Even measuring income by adults might be difficult as non-civil servants oftentimes do not have a steady income, or may be hesitant to report how much they earn. How can these limitations be overcome? This requires perseverance and creativity on behalf of the researcher: perseverance by reviewing other research studies to find out how other researchers overcame this difficulty and creativity by thinking up new methods when necessary. Perhaps socio-economic status can be measured in small children by the type of school they attend (public for low socio-economic status and private for middle socio-economic status) or by mode of transportation (taxi for low, own vehicle for middle, and multiple vehicles for high) for adults.
Other research studies and types of variables have other practical considerations. Can small children complete a questionnaire? How then will a researcher examine the emotional development of a small child? Perhaps through an observational checklist by a parent or teacher. How will a researcher measure engagement in examination malpractice? Can people be honest in reporting their dishonesty? Maybe a questionnaire will work, but participants need to be continually assured of their anonymity in responding to the questionnaire so they feel comfortable being honest. Developing an operational definition of a variable requires considerable thought.
Another practical consideration is the knowledge of the participant. Sometimes, I have seen questionnaires that ask participants to indicate socio-economic status by circling either low, middle, or high. How would a participant know whether they are low, middle, or high status? How is the participant supposed to know what "low" means? Participants need objective criteria so they can give an unbiased account of their socioeconomic status. The process of measurement requires finding an objective, indisputable way of assigning numbers to low, middle, and high. A participant must have enough knowledge to be able to objectively state their response.
Types of Measurement
There are multiple types of measurement to consider when thinking about how to operationally define a variable.
Each variable in a research study needs to be measured separately. A research study examining the effect of intrinsic motivation on academic achievement may be able to use the Intrinsic Motivation Inventory to measure intrinsic motivation. Another instrument will be necessary to measure academic achievement. Because each variable must be measured separately, a researcher may use five or six different "instruments" may be used for one research study. For information on how to combine the instruments, go to Developing the Questionnaire Format.
"Development of new tests is a complex and difficult process that requires considerable training in educational and psychological measurement. Therefore, we recommend that you make certain no suitable test is available before developing your own" (Gall, Gall, & Borg, 2003, p. 216).
Using a pre-developed instrument to measure the key variables in your study has a number of advantages.
Finding an already-existing instrument is best done through a literature review search. When reading over the instruments section of a research study, look to see if a table or an appendix provides the actual items on the questionnaire. Other times, the researcher will cite the original source of the instrument, so check the citation. For example, a line might read, "Subtests from the Intrinsic Motivation Inventory (Ryan, 1982) were used to assess relatedness, intrinsic motivation, autonomy, and competence." The source of the instrument is Ryan (1982), so find this reference in the References section. If you can get a copy of the Ryan article, then you likely will get a copy of that instrument in either the appendix or a table.
A number of resources are also available on the internet to assist in finding an instrument that will measure the key variables in your research study.
As Gall, Gall, and Borg (2003) stated, a researcher should make a thorough search of the literature and internet to try to find an instrument to use that has already been developed. A researcher should make their own questionnaire only if an appropriate instrument is not found.
There is currently debate in the Faculty of Education at UniJos about whether there should be an Operational Definitions section to Chapter 1. However, a researcher absolutely must develop an operational definition for each variable as part of the planning process regardless of whether this section is required or not. It is best if a researcher presents each operational definition to the supervisor when getting approval for the instrument even if it is not required as part of the thesis itself.
The operational definition should include the following components:
For example, a good operational definition of socio-economic status might read:
Socio-economic status is typically defined as a person's economic standing based on lifestyle, prestige, power, and control of resources (Liu, Ali, Soleck, Hopps, Dunston, Pickett, 2004). In this study, socio-economic status was defined as participants' self-report of their average monthly income.
Another operational definition might read:
Intrinsic motivation is defined as the motivation for a behavior is based on the inherent enjoyment in the behavior itself (Ryan and Deci, 2002). In this study, intrinsic motivation was measured by responses to seven Likert-scale items on a self-report questionnaire.
For a causal-comparative study, an operational definition might read:
Cheating is typically defined as dishonest practices by students that misrepresent their true understanding of course content. Cheating behavior was operationally defined as whether students were caught blatantly cheating on the assignment or final exam in their 100-level educational psychology course.
Return to Educational Research Steps
Copyright 2012, Katrina A. Korb, All Rights Reserved
Copyright 2012, Katrina A. Korb, All Rights Reserved