Educational Psychology

Conducting Educational Research
Step 12: Write Method of Data Collection

Introduction

Again, recall that the purpose of the Methods section is to describe the research study in enough detail so that any other researcher can repeat the study exactly. The purpose of the Methods of Data Collection portion is to describe the specific procedures and logistics used to collect data in the study, including when, how, where, and for how long the research study took place. The specifics of this section will depend on the type of data collected.

Administering a Questionnaire

When administering a questionnaire, report the following:

Who administered the questionnaire: the teacher, the principal investigator, research assistants, etc.?
Where was the questionnaire administered: at school, at home, at church, etc.?
When was the questionnaire administered: in the morning, just before school holiday, one week after the treatment was completed, etc.?
How long did participants have to complete the questionnaire: 45 minutes in class, take home and return within two weeks, etc.?
How was the questionnaire administered? Specifically include the directions that were given to the participants.

Interviews

When conducting interviews, report the following:

Who conducted the interview: the principal investigator, research assistants, etc.?
Where did the interview take place: at school, at home, at church, etc.?
When did the interview take place: in the morning, just before school holiday, etc.?
How long did the interview last: 45 minutes, two sessions of 30 minutes apiece, etc.?
What explanations and directions were participants given prior to the interview?
How were interview responses recorded: audio, written, etc.?

Archival Data

Recall that archival data consists of materials such as achievement test scores from student records and attendance records. If archival data was collected, report:

Who collected the data: the principal investigator, research assistants, teachers, etc.?
Where was the data collected: at the school office, LGA headquarters, etc.?
How was the data collected: who gave permission, how was data recorded, etc?

Observational Data

If the researchers will be observing behavior for data collection, report the following:

Who was observed: students, teachers, parents, etc.?
Who did the observations: the principal investigator, research assistants, etc.?
Where were participants observed: in the classroom, on the playground, at home, etc.?
When did the observations take place: in the morning, during English class, at free time, etc.?
How long did the observations last: twice per week, 45 minutes, etc.?

Multiple Administrations of an Instrument

In some studies, particularly in pre-post test experimental designs, the instrument must be administered multiple times. If this is the case, for each time the instrument is administered, report on each instrument administration as detailed above. Also report on the length of time between administrations of the instrument. How long before the treatment was the pre-test administered? How long after the treatment was the post-test administered? Also, if there were any differences in logistics between the administration of the pre- and post-test, these must also be reported.

Treatments

In experimental and quasi-experimental designs, the treatment must be clearly explained in great detail. A reader should be able to conduct your exact treatment based on the description given in the Methods of Data Collection and Appendix. Report the following in great detail in the Methods of Data Collection.

What was the overall goal of the treatment?
How long the treatment lasted: one week, six weeks, the entire term, etc.
How often the participants met for the treatment: once per day, once per week, etc.
How long each treatment meeting lasted: 30 minutes, two hours, etc.
Where the treatment meetings held: at school, at the church, in the hostels, etc.
What was the typical format of each meeting?

Recall that the control group needs to be identical to the treatment group in every way except that they do not receive the treatment. Therefore, the same information should be reported for the control group. What were they doing during the times when the treatment group was meeting?

For student projects, the specific objectives and methodology of each meeting in the treatment must be detailed. However, this information is oftentimes best placed in an Appendix. I recommend including the treatment scheme in an Appendix. Be sure to add a sentence in the Methods of Data Collection that this treatment scheme is presented in the Appendix.

Research Assistants

If it is necessary to recruit research assistants, it is important that these are well trained and professional individuals. It would be a shame for a research study to fail because incompetent, inexperienced, unmotivated, or poorly trained research assistants messed up the data collection procedures. This is a potentially fatal source of error in a study that can easily be avoided by collecting the data yourself. I much prefer to be directly involved in all aspects of data collection to ensure that all procedures are performed correctly and with professionalism. I have had a number of situations where I allowed somebody else to collect data, and then had to discard the entire research study because I realized that poor data collection procedures had been used. Although it is more work to collect the data myself, I believe that using accurate research procedures is worth the extra time and effort.

If research assistants are necessary, it is very important to select competent individuals and train them thoroughly and effectively. In addition to effective training, a protocol should be developed that gives a word-for-word script that the research assistants should follow, as well as what the researchers should do. This helps to ensure that all research assistants are behaving similarly. The protocol should include:

What materials should be brought to the experiment: pencils, questionnaires, informed consent documents, etc.?
What preliminary directions should be given to the participants?
How should the instruments be distributed?
How should the instruments be collected?
Should there be any closing remarks?

The training should include:

General information about the purpose of the study. This is important so the research assistants can appropriately handle any problems that might come up in the study. If something unexpected happens, the research assistant can use their knowledge of the purpose of the study to act appropriately.
The importance of following the directions exactly. If the research assistants do not follow the direction exactly, then it will introduce unnecessary error in the research study, which will invalidate the research findings.
An overview of what the instrument is measuring. Like understanding the purpose of the study, understanding the purpose of the instrument will help the research assistants answer unexpected questions during the study.
A review of the instrument item-by-item and a description of the purpose of each item.
A review of the protocol. Provide explanation where necessary and allow the research assistants to ask any questions they might have about the procedures.
A demonstration of how the research procedures should be carried out. Pretend that a few of the research assistants are the participants. Then you should demonstrate the protocol.
Each research assistant should have an opportunity to practice the procedures, perhaps by pairing up research assistants: one as the researcher and the other as a participant. Provide feedback to each research assistant about procedures they can improve.
Allow the research assistants ample opportunities to ask questions. Answer all questions thoroughly.
If any research assistant demonstrates poor research skills or a lack of motivation to follow the procedures, then it is vital to find a different research assistant to replace them.

In the Methods of Data Collection section, report the following for the research assistants:

How many research assistants were used?
How were the research assistants selected?
What were the minimum qualifications for the research assistant?
Briefly describe the training for the research assistants.
What tasks in the study did research assistants perform?

Examples

At the end of a class session, the instructor gave directions for the questionnaires, including that the students should complete the questionnaire in good faith and return it to the course instructor within three weeks. Class representatives distributed the questionnaires to all of the students. Because students were receiving course credit for completing the questionnaires, they had to list their name and matriculation number on the completed questionnaire. To encourage students to complete the questionnaire honestly, the following procedures were used to assure students that their responses would never be linked to them personally. A solid line was drawn immediately below the area where they were to write their name and matriculation number with the words "Do not write your name below. The top section will be removed upon submission" written in capital letters. The instructor informed students that as soon as they submitted the questionnaire, their name and matriculation number would be cut off along the solid line so they would receive credit for completing the questionnaire, but they could not subsequently be linked to their responses.

All mathematics exams were individually administered by the researcher to the child. The researcher removed the children one-by-one from the classroom and took them into a quiet room where they could complete the exam undisturbed. The seven exams were administered on two separate days, typically within the same week. The more, less, addition, and subtraction exams were all administered on the first day while equivalence, comparison, and missing number exams were administered on the second day. Children completed three practice items for each exam. Students were given one point for each correct answer.

Participants in the treatment group attended a one-week workshop that covered the development of literacy skills and effective literacy instructional practices. After introductions on the first day, the pre-test was administered. Participants were instructed to complete the questionnaire based on their own understanding of how children learn to read. They were given 30 minutes to complete the pre-test. All participants finished within this time period. During the workshop, three literacy experts used discussion, lecture, video demonstrations, small group discussions, and practice teaching demonstrations to educate participants about new developments in the field of literacy. (See Appendix B for a detailed list of objectives, activities, and discussion points.) Participants met for eight hours for all five days. After the last educational session on the final day, participants were asked to complete the post-test questionnaire. Again, they were given 30 minutes to complete the questionnaire. Participants in the control group attended a workshop with a similar format, but the content was on teaching mathematics skills. The control group also completed the same pre- and post test, just as the treatment group.

Return to Educational Research Steps