1
Chapter 10 Experimental research
Suggested Activities
The content in Chapter 10 tends to be a bit overwhelming for novice students. There is a lot of jargon and
a lot to remember! Several examples of most text content are really necessary for them to ‘get it’. I have
been successful using the four-part approach below when teaching the content.
1. Introduce threats to validity with a variety of examples. Next, present each of the main research
designs briefly with an example. Pull together how each design addresses the threats to validity.
This is a bit overwhelming for the class so also use another example and step through it again.
2. One example that works exceptionally well for this exercise is SAT preparation training
programs. As a class consider the ways one could conduct SAT training programs. The example
3. As you work through both Activities 1 and 2, diagram the designs with the class. At first they
think it is really weird but as they catch on it is really helpful to their understanding to use the
notations.
4. Next, allow students to work in groups to work through another example. Have students generate
how a study could be conducted for all types of designs. Some topics to consider include
a. Type of language learning programs is another example that works well. Any form of text
supplements, such as animation, pictures, or added questions.
5. An ies publication produced by the coalition of evidence based policy and the what works
clearinghouse entitled Key items to get right when conducting a randomized control trial in
education written by J. Baron can serve as outside text support regarding how to conduct
experimental work in education. The short 2005 publication is essentially a checklist with
discussion that breaks the research process down into 4 sections and provides key items to
2
Handout 10.5
Given the Baron, J. (2005). Key items to get right when conducting a randomized controlled trial in
education. Answer the following questions.
1. The first section of the publication notes key items to get right in planning the study.
a. What does the author mean by surrogate outcomes? In a study of mathematical problem solving
what might be an example of a surrogate outcome? Why are surrogate outcomes a concern?
b. The author presents two reasons when it may be more advisable to randomly assign groups
instead of individuals. In the first reason, ‘spillover’ effects are presented. How do such
spillover effects relate to threats to validity as discussed in Chapter 10 of your text? Select two
examples and explain.
c. As discussed elsewhere, the author promotes the use of a power analysis. What key features of
the design are necessary for including in an a priori power analysis? Pick one and provide an
example.
2. The second section of the publication notes key items to consider during the random
assignment process.
a. Why would it be advisable to have someone not invested in the study conduct the random
assignment?
b. Random assignment is sometimes challenging. What does the author mean when he says make
sure the random assignment is truly random? What is one example of a non truly random
assignment someone might conduct? Why does example present a concern?
c. Why would a participant need a unique personal identifier?
d. The author suggests that researchers monitor what services the control group might be receiving.
Suggest one reason why this can be a concern. Provide an example.
3. The third section of the publication notes key items to get right in measuring outcomes for
the study sample.
a. What are some of the key characteristics you would seek in your instrumentation? Explain in
your own words why each is important.
b. What is one way you might be able to limit subjectivity in administration?
c. Why might you want to keep track of those who fail to complete the study? How does this relate
to the threats discussed in your text?
3
d. What are some of the benefits and drawbacks of collecting long term outcomes?
4. The fourth section notes key items to get right in the analysis of study results
a. How should you analyze those individuals who may have benefited from cross-over effects?
Why?
b. In addition to statistical significance of the intervention’s effect what else should be reported?
4
Handout 10.5 (Points to lead discussion)
Given the Baron, J. (2005). Key items to get right when conducting a randomized controlled trial in
education. Answer the following questions.
1. The first section of the publication notes key items to get right in planning the study.
a. What does the author mean by surrogate outcomes? In a study of mathematical problem solving
what might be an example of a surrogate outcome? Why are surrogate outcomes a concern?
b. The author presents two reasons when it may be more advisable to randomly assign groups
instead of individuals. In the first reason, ‘spillover’ effects are presented. How do such
spillover effects relate to threats to validity as discussed in Chapter 10 of your text? Select two
examples and explain.
c. As discussed elsewhere, the author promotes the use of a power analysis. What key features of
the design are necessary for including in an a priori power analysis? Pick one and provide an
example.
5
2. The second section of the publication notes key items to consider during the random assignment
process.
a. Why would it be advisable to have someone not invested in the study conduct the random
assignment?
b. Random assignment is sometimes challenging. What does the author mean when he says make
sure the random assignment is truly random? What is one example of a non truly random
assignment someone might conduct? Why does example present a concern?
c. Why would a participant need a unique personal identifier?
d. The author suggests that researchers monitor what services the control group might be receiving.
3. The third section of the publication notes key items to get right in measuring outcomes for the
study sample.
a. What are some of the key characteristics you would seek in your instrumentation? Explain in
your own words why each is important.
b. What is one way you might be able to limit subjectivity in administration?
c. Why might you want to keep track of those who fail to complete the study? How does this relate
to the threats discussed in your text?
d. What are some of the benefits and drawbacks of collecting long term outcomes?
6
4. The fourth section notes key items to get right in the analysis of study results
a. How should you analyze those individuals who may have benefited from cross-over effects?
Why?
b. In addition to statistical significance of the intervention’s effect what else should be reported?
7
6. The text notes some of the difficulties educational researchers face when employing experimental
designs. These difficulties center around several key issues. Some of these issues are listed
Chapter 20.
Who wants to be a control group member? Does there have to be a control group? How do we
conduct an experiment and still teach all learners if the outcome is achievement?
Sometimes it isn’t ethical NOT to provide a treatment (especially one with some efficacy data) to
some students and not to others. How can we resolve this ethical concern?
How can we effectively randomly assign students to treatments? In school settings this
How do we balance the need for sound research and the amount of instructional time it
takes to conduct research?
In classroom-based research there are numerous threats to external validity. Does the
teacher matter? Does the group dynamic of the class influence our findings? How do we
conduct research that can generalize?
8
Chapter 10 Test Items
1. Compared to all other types of research, experimental research
a. requires greater number of participants.
b. provides evidence concerning cause-effect relationships.
c. measures dependent and independent variables.
d. collects samples to generalize to populations.
2. Treatment variable is another name for
a. independent variable.
b. dependent variable.
c. extraneous variable.
d. mediating variable.
3. Replications in experimental research are generally conducted to facilitate
a. internal validity.
b. external validity.
c. internal reliability.
d. external reliability.
• 4.-5. Gavin’s study compared reading error rates and reading comprehension scores
between two groups of 50 third-grade students that received different reading programs.
His t-test data analysis did not indicate any differences between the two types of
instruction.
• 4. The most likely reason for his findings is that
a. his measures did not measure reading.
b. his sample is too small.
c. he conducted the wrong analysis.
d. his groups were too similar.
• 5. The treatment variable in Gavin’s study is
a. error rates.
b. third graders.
c. reading instruction.
d. reading comprehension.
• 6.-7. Patsy conducted an experiment that included two groups of jr. high girls. One
group received no training while the other received a 3-hour intervention about the
benefits of careers in math and science domains. Patsy compared the willingness of girls
to enroll in subsequent math courses as her outcome measure. Her t test analysis
indicated no differences between her treatment group and the no-exposure control.
• 6. This finding is likely due to
a. preexisting group differences.
b. limited treatment exposure.
c. sampling error.
d. too small a sample size.
9
• 7. In Patsy’s study, willingness to enroll in future math and science courses is the
a. manipulated variable.
b. control variable.
c. criterion variable.
d. independent variable.
8. The final step in an experimental study is the
a. selection and definition of a problem.
b. execution of procedures.
c. formulation of conclusions.
d. analysis of data.
9. In experimental studies, but NOT in other studies,
a. the researcher manipulates the dependent variables.
b. the researcher manipulates the independent variables.
c. the researcher tries to eliminate the effects of extraneous variables.
d. the researcher examines a priori differences between groups.
10. In experimental studies, the sample may be derived from
a. one population.
b. two populations.
c. three populations.
d. more than three populations.
11. In order for a study to be experimental it must include
a. one assigned variable.
b. one assigned variable and one active variable.
c. one active variable.
d. two or more active variables.
12. A researcher’s efforts to remove influence of extraneous variables is referred to as
a. experimental control.
b. sample assignment.
c. sorting independent variables.
d. determining dependent variables.
13. The type of validity concerned with whether findings can be generalized to other groups
or other settings is referred to as
a. internal validity.
b. external validity.
c. content validity.
d. construct validity.
14. The type of validity concerned with whether differences between groups are due to the
manipulated variable is referred to as
a. internal validity.
b. external validity.
c. content validity.
10
d. construct validity.
• 15. Marcy is concerned that her findings may be due to an extraneous uncontrolled
variable and not her treatment. Marcy is most concerned with
a. internal consistency reliability.
b. inter-rater reliability.
c. external validity.
d. internal validity.
• 16.-17. Consider the example of Cyr, a researcher who studies the role of worked
examples in engineering education. He measures students test performance and ratings
of their instructors. He randomly assigns classes to condition. In some classes worked
examples are shown, some where no worked examples are used, and a third set of classes
in which students have access to worked examples if they elect to look at them.
• 16. The independent variable in Cyr’s research is
a. Engineering education
b. Worked example exposure
c. Test performance
d. Teacher ratings
• 17. Cyr argues that his research is experimental. However, you know better given
the following reason.
a. He randomly assigned in-tact groups not students
b. Ratings of instructors are not measured in experiments.
c. He uses a correlational approach to the study.
d. There is no dependent variable in his research study.
• 18.-19. Given a laboratory study that compared learning from text in two conditions, one with
underlining of important terms and one without underlining, found that the underlining
group did better on factual items; Marina conducted a study with tenth-grade history
students and compared one group that read a chapter with the important text underlined
and one group that read a chapter without any underlining. In her study she found no
effects for the underlining.
• 18. Marina’s study illustrates an example of a(n)
a. replication study.
b. internal validity study.
c. test for reliability.
d. regression study.
• 19. Marina’s findings may indicate that the underlining treatment has
a. low internal reliability.
b. low external reliability.
c. low internal validity.
d. low external validity.
11
20. Which of the following is a threat to external validity?
a. Statistical regression.
b. Testing effects
c. Specification of variables.
d. Instrumentation.
21. Which of the following is a threat to internal validity?
a. Differential selection of participants
b. Pretest treatment interference
c. Posttest sensitization
d. Multiple-treatment interactions
22. The threat to validity defined as the effects of the occurrence of events that are not part of
the treatment but affect the dependent variable is
a. history.
b. testing.
c. statistical regression.
d. instrumentation.
• 23. Laura was conducting a study on two treatment interventions to increase
patriotism. During her study, terrorist attacks on a federal building occurred. Laura
should be concerned with which of the following validity threats?
a. Statistical regression
b. Instrumentation.
c. History
d. Testing
• 24. Anne’s study compared pre-post differences in prekindergarten children’s
strategies for alternatives to violent displays when frustrated by others. She found that
there were differences between pre and post testing. With which validity threat should
Anne be most concerned?
a. Differential selection
b. Statistical regression
c. Maturity
d. Instrumentation
• 25. In Katia’s remedial mathematics study, she collected pretest data from a group of
participants. She divided the participants into three groups. One group received no
treatment, one group received instruction by a teacher-delivered intervention, one group
received peer tutoring on the same problems. After the intervention she tested their math
skills. With which validity threat should Katia be most concerned?
a. Maturation
b. Testing
c. Instrumentation
d. Mortality
12
• 26. Mo, an early childhood researcher, is particularly interested in the effects of
reading aloud on students’ reading attitudes. She wants to do an experimental study to
test the question of whether reading attitudes are more positive in children who are read
aloud to during morning free time in classrooms. She randomly assigns students to either
be read to or not to be read to daily for the course of one month. After she is two days
into her study, children ask the researchers if they can go to other activities instead of
going to read aloud time. Ethically she is bound to let them go to free time. By the end of
her study she has only half as many participants as were in her initial experimental
condition. Given Mo’s study, the loss of participants in the experimental condition can be
referred to as
a. Placebo effect.
b. attenuation.
c. attrition.
d. novelty effect.
• 27.-29. PJ conducts an experimental study on the effects of soft music during high stakes
science testing. He randomly assigns students at the school. In one condition he does not
provide music for testing while in the other group he does provide the music. He
administers a pretest at the beginning of the year and a posttest at the end of the year.
One concern in his study is that during the year the music teacher at the school he
administered the study shared with students the physiological benefits of soft music.
• 27. PJ’s study can best be described as
a. descriptive.
b. correlational.
c. causal-comparative.
d. experimental.
• 28. PJ’s design is best represented by which of the following?
a. Static-group comparison
b. Pretest-posttest control group design
c. One-shot case study
d. One-group pretest-posttest design
• 29. The introduction of the benefits of music during the duration of his study can be
considered which of the following threats?
a. History
b. Maturation
c. Mortality
d. Testing
13
• 30-33. As a secondary mathematics teacher, Hernandez conducted a study that explored
whether giving children recess prior to testing helped their test performance. For one of
the semesters, he sends half of his classes out for 10 minutes of recess prior to testing for
the other half, he provides 10 minutes of free time after the test.
• 30. Which of the following best represents the design of Hernandez’s study?
a. Static-group comparison
b. Post-test only control group
c. Solomon four group
d. One-shot case study
• 31. That Hernandez is the experimenter and the teacher administering the conditions,
introduces which potential threat to validity?
a. Experimenter
b. Placebo
c. John Henry
d. Instrumentation
• 32. Given the study was conducted in a school, students in the recess condition
classes told students in the other classes that they got to do recess before the tests. This is
an example of which of the following?
a. History effect
b. Instrumentation bias
c. Treatment diffusion
d. Placebo effect
• 33. The students in the non recess condition decided they would try harder on the
tests because they didn’t get recess. This is an example of a(n)
a. Experimenter threat.
b. John Henry effect.
c. Placebo effect.
d. Hawthorne effect.
• 34. Lisa’s study assessed the effects of an intervention to facilitate the math skills of
low-achieving second grade learners. She administered a treatment to the students in the
form of structured computer lesson and found that students’ skills increased. With which
validity threat should Lisa be most concerned?
a. History
b. Statistical regression
c. Instrumentation
d. Mortality
• 35. Max’s study compared the differences in teachers’ attitudes about inclusion
before and after a Saturday in-service training. He assessed their post-intervention
attitudes with a self-report questionnaire. With which of the following threats to validity
should Max be particularly concerned?
a. Instrumentation
b. Mortality
c. Selection
d. History
14
• 36. Russell’s study compared GPA of those students who volunteered for academic
study skills training and those who did not elect to take the training. He found that those
who had the training also had higher GPA. With which validity threat should Russell be
most concerned?
a. Maturation
b. Instrumentation
c. History
d. Selection
• 37. Shelly’s study compared the effects of two different types of phonemic
instruction on young children who volunteered to participate in the study. She gave each
participant who completed the study two pencils and three stickers. These incentives are
likely a strategy to decrease the threat of
a. history.
b. instrumentation.
c. differential selection.
d. mortality.
38. One way to decrease the threat of pretest-treatment interaction is
a. lengthen the time of the treatment.
b. decrease the sample size.
c. increase the significance level.
d. increase the number of conditions.
39. According to your text the research should report any difficulty obtaining samples
because of the increased threat of
a. pretest-treatment interaction.
b. multiple-treatment interference.
c. selection by treatment interaction.
d. experimenter effects.
• 40. Molly’s study examines the benefits of exercise on weight loss. In one group she
advocates exercise as well as a new diet program. In the comparison group she only
presents the diet program. Those in the comparison group find out that some of Molly’s
clients are in an exercise program too, so they form an exercise group. This is an
example of
a. compensatory rivalry.
b. placebo effect.
c. treatment diffusion.
d. experimenter effects.
15
• 41. Nadine conducted a pretest-posttest control group design. The comparison group
received traditional lecture of course content while the treatment group received the
lecture and instructor-provided summary notes. She found, however, no differences
between conditions, due in part to students sharing the instructor summaries across
conditions. This is an example of
a. compensatory rivalry.
b. treatment diffusion.
c. placebo effect.
d. experimenter effects.
• 42. In a recent study of young children’s reading skills, Gregg and Steve both
collected data. Gregg did not push the children and his scores are systematically lower
than are Steve’s. This example illustrates
a. compensatory rivalry.
b. placebo effect.
c. treatment diffusion.
d. experimenter effects.
43. Given a choice of the following research designs, which design controls the most threats
to internal validity?
a. One-shot case study
b. One group pretest-posttest
c. Static group comparison
d. Pretest-posttest control group
44. The most significant threat to internal validity of the time series design is
a. history.
b. maturation.
c. regression.
d. mortality.
• 45. Linda is conducting a federally-funded research project that examines literacy
skills in middle school learners. Bonnie recently started a new that specifically focuses
on reading strategies. Some of the students are participating in both studies.
a. This is of no concern because there are two different studies.
b. This is a mostly a threat to internal validity.
c. This is mostly a threat to external validity.
d. This is an example of the placebo effect.
46. Of the following, the design that best controls for internal validity threats is the
a. Posttest only control group design.
b. Pretest-posttest group design.
c. Time series design.
d. Soloman four-group design.
47. Which of the following is the primary drawback of the posttest-only design?
a. History.
b. Selection.
c. Mortality.
d. Regression.
16
48. For which of the following designs is multiple treatment interference a substantial threat?
a. Time series design.
b. Counterbalanced design.
c. Solomon Four group design.
d. Posttest-only control group design.
49. Which of the following is a drawback of the Soloman Four group design?
a. Lack of control for threats to internal validity.
b. The large number of participants needed.
c. Systematic desensitization.
d. Increased experimenter effects.
50. Of the following, which is an example of a pre-experimental design?
a. Time series design
b. Static group comparison
c. Posttest only control group design
d. Nonequivalent control group design
51. Which of the following is a concern with the nonequivalent control group design?
a. Selection interactions.
b. History.
c. Mortality.
d. Maturation.
52. A factorial design includes:
a. One manipulated independent variable.
b. More than one manipulated independent variable.
c. Two or more independent variables, one of which is manipulated.
d. Multiple dependent variables.
• 53.-54. Jack wants to test if runners on his cross-country team improve with his new
training regimen. He takes their times at the beginning of the season and again at the end
of the season.
• 53. Which of the following designs is Jack best illustrating?
a. One-shot case study
b. One-group pretest-posttest design
c. Pretest-posttest control group design
d. Time series design
• 54. Runners in Jack’s study improving their times due to running generally and not
necessarily due to the new regimen is an example of which of the following threats?
a. Selection
b. Maturation
c. History
d. Instrumentation
17
• 55. The third-grade teachers in one school are interested in the effects of seating
assignments on self-esteem of students. All students take a self-esteem measure at the
beginning of the school year. At the end of the school year, students take a self-esteem
measure. Each teacher uses a different seating strategy for the year. This study best
illustrates which of the following designs?
a. Nonequivalent groups design
b. Soloman four-group design
c. Pretest-posttest control group design
d. One group pretest-posttest design
• 56. Jorge studies the effects of mnemonic training on learning vocabulary. He
randomly assigns one group to a no treatment control and he gives mnemonic training to
the other group. He assesses them at the end of the study and compares how many words
they know. Which of the following designs is Jorge best illustrating?
a. Pretest-posttest control group design
b. Nonequivalent group control group design
c. Posttest-only control group design
d. Time series design
• 57. Nora is interested in differences in amount of content retained from a lecture
based on time of class. She compares her two sections—one that meets at 8 am and one
that meets at 2 pm— by testing them, unannounced, on content that she taught two weeks
ago. Which of the following designs is Nora best illustrating?
a. One shot case study
b. One group pretest-posttest design
c. Posttest only control group design
d. Static group comparison
58. Given the following notation, which design is illustrated?
X O
a. One shot case study
b. One group pretest-posttest design
c. Posttest only control group design
d. Static group comparison
59. Given the following notation, which design is illustrated?
X1 O
X2 O
a. One shot case study
a. One group pretest-posttest design
b. Posttest only control group design
c. Static group comparison
18
60. Given the following notation, which design is illustrated?
O X1 O
O X2 O
a. Pretest-posttest control group design
b. Posttest only control group design
c. Nonequivalent groups control group design
d. Static group comparison design
61. Given the following notation, which design is illustrated?
R X1 O
R X2 O
a. Pretest-posttest control group design
b. Posttest only control group design
c. Nonequivalent groups control group design
d. Static group comparison design
• 62. Kari wants to make sure that all students in her study get both of the treatments
she is testing. To assure all subjects get both interventions, which of the following
designs should Kari consider?
a. Within-subjects design
b. Soloman Four-Group design
c. Counterbalanced design
d. Nonequivalent groups control group design
63. How many cells does a 2 x 4 factorial design have?
a. 2
b. 4
c. 6
d. 8
• 64. A study that investigates the effects of three different types of instruction (text
only, text with pictures, text with animation), and spatial ability (high, low) would be
symbolized as a
a. 2×2 factorial design
b. 3×2 factorial design
c. 2×3 factorial design
d. Soloman four group design
• 65. A study that addressed the effects of gender and type of practice (massed,
distributed, massed and distributed) would be symbolized as
a. 2×2 factorial design
b. 3×2 factorial design
c. 2×3 factorial design
d. Soloman four group design
19
66. One caution with factorial designs is that
a. they are less valid than most other designs.
b. they require more participants.
c. interactions impede the significant findings.
d. they are not used in educational studies.
• 67. Chin-Hsing conducted a study that examined motivation for sport and academics
in a Taiwanese school for gifted athletes. One concern that Chin-Hsing has is that the
findings from the study may not relate to findings that would be found in another school
or country. The validity threat most evident in his concern is
a. Internal validity
b. External validity
c. Construct validity
d. Predictive validity
• 68.-69. Use the follow scenario to answer these questions.
Archimedes read about a study that found that soft music during stressful events was
soothing for many workers. He thinks that test taking is stressful for many learners and
wants to conduct an experiment to determine if music playing during testing assists in
student test performance. Therefore, he randomly assigns all of the introductory biology
classes into one of three groups. One group that tests as usual without music, one group
that listens to soft music during the test, and the final group that listens to a CD with the
latest pop music tracks playing.
• 68. Given Archimedes’ study, the dependent variable is __________________.
a. biology class.
b. test performance.
c. type of music.
d. test anxiety.
• 69. Which of the following designs best represents Archimedes’ study?
a. One shot case study
b. Static group comparison
c. Post-test only control group design
d. Soloman four group design
• 70. Ingrid works as a teacher in a small private school that does not have a food service
program. She sees what kids bring to school and is very worried that families are not
supporting their children’s nutritional needs enough. This year she decides to conduct a
study to see if sending home literature and homework that parents must do with their
children makes a difference in their application of appropriate selections in children’s
lunches at the end of the year. Which of the following designs best represents Ingrid’s
research design plan?
a. One shot case study
b. Static group comparison
c. Pretest-post-test control group
d. Soloman Four group
20
• 71. Ewa is an athletic trainer at the university. She conducts research on violence in sport.
She has designed a study to examine the effects of a new sportsmanship class. She
randomly assigns teams by sport to be in either her new class or not. That is, some teams
are in the class and others are not. She then compares incidents of violence between
groups. What is one threat that Ewa must be particularly concerned about?
a. Instrumentation
b. Mortality
c. Selection
d. History
• 72.-74. Grace conducts research on race and achievement in mathematics. She designs a study to
see if a new intervention she designed increases African-American students’ math
achievement scores. She goes to a local school with a representative Aftrican-American
student population and randomly assigns college-bound African-American students to the
intervention or to a control condition. Grace gives a mathematics achievement test in
September to both groups. Through the academic year the treatment condition receives the
intervention. She uses end of year achievement tests to see if there are differences between
conditions.
• 72. Which of the following external validity threats should most concern Grace regarding her
study?
a. Selection
b. History
c. Instrumentation
d. Maturation
• 73. Which of the following best represents the design of Grace’s study?
a. One shot case study
b. One-group pretest-posttest
c. Pretest-posttest control group
d. Two by two factorial
• 74. Grace found that students studied together for the end of year achievement tests. She
believes that some of the students in the treatment condition shared intervention
information with the control group. Which of the following threats does Grace now have
to be most concerned about?
a. Maturation by treatment
b. Treatment diffusion
c. Selection by treatment
d. Statistical regression
• 75. Terence, one of the participants in the control group told Grace that his group found out
that ‘the other kids were getting a program to help their math scores’ and therefore they
decided to get together in a study group once a week to help each other. Grace was very
concerned. What threat does this student action represent?
a. John Henry Effect
b. Multiple treatment interference
c. Experimenter effects
d. Treatment diffusion
21
• 76.-77. Kathy trains nurses. She designed a new assertive communication unit to include in her
training. She administers a pretest and then randomly assigns nurses to her treatment or
a control condition. She tests their communication skills after the treatment. She sees
significant differences at immediate testing but after eight weeks she retests the nurses
and finds no differences between her treatment and control students.
• 76. Which of the following best represents the design of Kathy’s study?
a. Non-equivalent groups
b. Solomon four group
c. Pretest-posttest control group
d. Post-test only control group
• 77. It appears Kathy should be most concerned with which of the following threats to her
study?
a. Participant effect
b. Researcher effect
c. Novelty effect
d. Placebo effect
• 78.-79. Audrey examines students’ learning of science principles from interactive games. She
randomly assigns the fifth grade students in her study to a control condition with no play
time, a timed play condition, or an unlimited time play condition. She also groups her
students into either high or low ability groups. She measures her students with a
knowledge pretest, a post-test, and a delayed post-test.
• 78. Which of the following best represents Audrey’s study design?
a. Pre-test post test design
b. A 3 x 3 factorial design
c. A 3 x 2 factorial design
d. A 3x2x3 factorial design
• 79. Which of the following is an independent variable in Audrey’s study?
a. Ability
b. Fifth grade
c. Post-test exam
d. Science principles
• 80.-81. Khalad is conducting a study to determine if motivational posters inspire middle school
students to be more pro-social and to complete more homework. He goes to a large
school district to conduct his study. In one of the district’s middle schools he puts the
posters up and in the second school he does not. After the posters are up for two months
he compares the GPA and number of disciplinary referrals between the two schools.
• 80. Which of the following designs is most representative of Khalad’s study?
a. Two by two factorial
b. Static group comparison
c. Post-test only control
d. Solomon four group
• 81. Which of the following represents the dependent measure in Khalad’s study?
22
a. Pro-social behavior
b. Homework completion
c. Disciplinary referral
d. Motivational poster
• 82.-83. Laura has developed a new treatment to increase spatial ability. She randomly assigns
students to either her intervention or a control condition. In the treatment condition
students engage in working puzzles for twenty minutes a day for 10 days. Unfortunately,
students don’t like her intervention and refuse to do puzzles for all of the days. When
she examines post-test means on the spatial ability measure, there are no differences
between her treatment and the control condition.
• 82. Which of the following designs best describes Laura’s study?
a. Factorial
b. Post-test only control group
c. One group pretest-posttest
d. Static group comparison
• 83. Which of the following demonstrates a threat to validity demonstrated in this study?
a. Maturation
b. Testing
c. Instrumentation
d. Mortality
• 84.-85. Mario is a STEM researcher who wants to test if field-based experiences in high school
lead to increased long term engagement in STEM fields. He first measures students’
attitudes toward STEM careers in the beginning of their junior year of high school.
Then, students are randomly assigned to one of three conditions. Students in the first
condition are assigned to no field-based experiences, the second condition provides one
three-week experience, and the third condition provides two different three-week
experiences. He not only measures attitudes at the end of the academic year but tracks
students for three years to test attitudes again one year and three years after the
treatment to examine if any effects.
• 84. Mario’s study can best be described as a(n)
a. 3 X 2 factorial design.
b. Pretest-posttest control group design
c. Post-test only control group design
d. Non-equivalent groups control group design
• 85. How many levels of the independent variable are represented in Mario’s study?
a. 1
b. 2
c. 3
d. 4
23
Chapter 10 Answers
24