1
CHAPTER 7
REPEATED MEASURES DESIGNS
CHAPTER OUTLINE AND OBJECTIVES
I. Overview
II. Why Researchers Use Repeated Measures Designs
Researchers choose to use a repeated measures design in order to (1) conduct an experiment when
few participants are available, (2) conduct the experiment more efficiently, (3) increase the sensitivity of
the experiment, and (4) study changes in participants= behavior over time.
III. The Role of Practice Effects in Repeated Measures Designs
Repeated measures designs cannot be confounded by individual differences variables because the
same individuals participate in each condition (level) of the independent variable.
Participants’ performance in repeated measures designs may change across conditions simply
because of repeated testing (not because of the independent variable); these changes are called
practice effects.
Practice effects may threaten the internal validity of a repeated measures experiment when the
different conditions of the independent variable are presented in the same order to all participants.
The two types of repeated measures designs, complete and incomplete, differ in the specific ways they
control for practice effects.
A. Defining Practice Effects
B. Balancing Practice Effects in the Complete Design
Practice effects are balanced in complete designs within each participant using block
randomization or ABBA counterbalancing.
In block randomization, all of the conditions of the experiment (a block) are randomly ordered each
time they are presented.
In ABBA counterbalancing, a random sequence of all conditions is presented, followed by the
opposite of the sequence.
Block randomization is preferred over ABBA counterbalancing when practice effects are not linear,
or when participants’ performance can be affected by anticipation effects.
C. Balancing Practice Effects in the Incomplete Design
Practice effects are balanced across subjects in the incomplete design rather than for each subject,
as in the complete design.
2
The rule for balancing practice effects in the incomplete design is that each condition of the
experiment must be presented in each ordinal position (first, second, etc.) equally often.
The best method for balancing practice effects in the incomplete design with four or fewer
conditions is to use all possible orders of the conditions.
Two methods for selecting specific orders to use in an incomplete design are the Latin Square and
random starting order with rotation.
Whether using all possible orders or selected orders, participants should be randomly assigned to
the different sequences.
IV. Data Analysis of Repeated Measures Designs
A. Describing the Results
Data analysis for a complete design begins with computing a summary score (e.g., mean, median)
for each participant.
Descriptive statistics are used to summarize performance across all participants for each condition
of the independent variable.
B. Confirming What the Results Reveal
The general procedures and logic for null hypothesis testing and for confidence intervals for
repeated measures designs are similar to those used for random groups designs.
V. The Problem of Differential Transfer
Differential transfer occurs when the effects of one condition persist and influence performance in
subsequent conditions.
Variables that may lead to differential transfer should be tested using a random groups design because
differential transfer threatens the internal validity of repeated measures designs.
Differential transfer can be identified by comparing the results for the same independent variable when
tested in a repeated measures design and in a random groups design.
VI. Summary
REVIEW QUESTIONS AND ANSWERS
These review questions appear in the textbook (without answers) at the end of Chapter 7, and can be used
for a homework assignment or exam preparation. Answers to these questions appear in italic.
1. Describe what is balanced in a random groups design and what is balanced in a repeated measures
design.
In a random groups design, random assignment of subjects to conditions is used to balance, or
8. Briefly describe the techniques that researchers can use to balance practice effects in the repeated
measures experiments using an incomplete design. Identify which of these techniques is preferred and
explain why.
9. Explain why an additional initial step is required to summarize the data for an experiment involving a
complete repeated measures design.
10. Describe how researchers can determine if differential transfer has occurred in a repeated measures
experiment.
CHALLENGE QUESTIONS AND ANSWERS
These questions appear in the textbook at the end of Chapter 7, and can be used for a homework
assignment, in-class discussion, or exam preparation. Answers to these questions appear in italic below.
[Answer to Challenge Question 1 also appears in the text.]
1. The following problems represent different situations in the repeated measures designs in which
practice effects need to be balanced.
A. Consider a repeated measures experiment using a complete design involving one independent
variable. The independent variable in the experiment is task difficulty with three levels (Low,
Medium, and High). You are to prepare an order for administering the conditions of this experiment
so that the independent variable is balanced for practice effects. You are first to use block
randomization to balance practice effects and then to use ABBA counterbalancing to balance
practice effects. Each condition should appear twice in the order you prepare. (You can use the first
row of the random number table (Table A.1) in the Appendix to determine your two random orders
for block randomization.)
6
C. Suppose the intern decides to use all possible orders to balance practice effects, and assigns one
participant to each of the 24 possible orders of the conditions. Consider only the first ordinal
position of this experiment (i.e., the first photo each participant saw). Which experimental design is
used when you look only at the first ordinal position across the 24 participants? How many
participants are in each of the four conditions?
D. Considering your answer to part C, how could the intern test whether differential transfer occurred
when all possible order are used to balance practice effects?
random groups design, differential transfer may have taken place.
3. The following table represents the order of administering the conditions to participants in a repeated
measures experiment using an incomplete design in which the independent variable was the difficulty
level of a children’s electronic puzzle game. Four-year old children used a tablet to play a game
requiring them to find hidden figures in three difficulty levels. The levels of difficulty were defined by the
size of the figures embedded in the pictures displayed on the screen: extremely small (ES), very small
(VS), and small (S). The dependent variable was the number of figures found by a child at each of the
three difficulty levels (maximum = 10). Six children were tested and the values in the table indicate the
number of figures found at each level of difficulty. Use this table to answer questions below.
Participant Order of Conditions
1 ES (2) VS (9) S (9)
2 VS (3) S (5) ES (7)
3 S (4) ES (3) VS (5)
4 ES (6) S (10) VS (8)
5 VS (7) ES (8) S (6)
6 S (8) VS (4) ES (4)
A. What method was used to balance practice effects in this experiment?
B. Present the values you would use to describe the overall effect of the difficulty variable. Include a
verbal description of the effect along with the descriptive statistics that you use as a basis of your
description.
7
8
9
10
Reading Research Critically
Read the following description of a research study to answer the questions that follow. This description is
based on an article by: Meier, B. P., Robinson, M. D., & Clore, G. L. (2004). Why good guys wear white:
Automatic inferences about stimulus valence based on brightness. Psychological Science, 15, 82–87.
Brightness and Affect
Numerous examples exist in culture that depict an association between brightness and affect. Light or
brightness is associated with Agood@ and darkness is associated with Aevil.@ In the movie Star Wars, for
example, Luke Skywalker and Princess Leia were dressed in white and Darth Vader was completely in
black. Across many religions, including Buddhism, Christianity, Hinduism, Islam, and Zoroastrianism, an
association exists between light and God or goodness, and between darkness and Satan or evil.
Whether something is considered good or bad is referred to as an affective judgment. In contrast, our
experience of light and dark is a sensory perception. The focus of this research was to determine whether
people automatically judge brighter objects as good and darker objects as bad. To test this hypothesis,
participants in a series of experiments judged whether 100 words presented on a computer screen were
negative or positive. In a separate study, 50 of the words were rated as reflecting positive affect (e.g.,
candy, love, pretty, sleep), and 50 were rated as negative affect words (e.g., bitter, cancer, devil, rude).
Each word was presented one at a time on a gray background. The researchers manipulated whether the
words were presented in a bright font or a dark font. Thus, half of presentations of positive-affect words
were in bright font and half were in dark font. Similarly, half of the presentations of negative-affect words
were in dark font and half were in bright font. One dependent variable was the time it took participants to
respond whether the word on the screen was positive or negative.
The researchers were most interested in comparing participants= reaction times when the affect and font
brightness Amatched@ (e.g., love presented in a bright font and cancer presented in a dark font) and when
the affect and font brightness conflicted (e.g., love presented in a dark font and cancer presented in a bright
font). Their results indicated that when the affect and font of the word conflicted, participants took longer
and made more errors when judging whether the word was positive or negative, compared to when the
affect and font of the word matched.
To explain this finding, Meier et al. (2004) considered theories suggesting that conceptual thinking, such as
making an affective judgment, is automatically tied to physical perception. The researchers suggested that
people cannot judge the affect of a word (or any other object) without first automatically considering its
physical features, such as brightness. In their experiments, when the brightness conflicted with the correct
affective judgment, additional cognitive processing (i.e., time, attention, thought) was required for
participants to override their automatic association between brightness and affect in order to make the
correct judgment about whether the word was negative or positive. Meier et al. noted the applied
implications of their findings for racial stereotyping by stating, AIf there really is an automatic tendency to
relate stimulus color to stimulus valence, then people who are dark skinned may be at a disadvantage in
interpersonal relations… with prejudice perhaps a manifestation of this tendency@ (p. 86).
A. Identify the independent variables in this study and indicate whether an independent groups design or a
repeated measures design was used to manipulate the independent variables.
11
B. The primary dependent variable in this research was participants’ reaction time for indicating whether
the presented words were positive or negative. Identify some features of the stimuli that the researchers
would need to control by holding conditions constant so that reaction time measures would not by
influenced by these potential confounding features.
C. Although procedures differed slightly across several experiments, we will consider procedures for the
experiments in which each of the 100 words was presented twice, once in dark font and once in bright
font, for a total of 200 trials. To simplify here, we will select 10 positive and 10 negative words for a total
of 40 trials (each word presented twice).
Clearly, if all the Amismatched@ stimuli were presented last in the 200 trials, response latencies for
these trials might be longer because of practice effects (i.e., fatigue, boredom). Counterbalancing the
trials is needed to average practice effects across the trials. Use the following stimuli to generate a
counterbalanced order of presentation. Be sure to present each word twice, once in bright (B) and once
in dark (D) font (e.g., love-B, love-D). Provide a rationale for the counterbalancing procedure you
selected.
Positive Affect Words Negative Affect Words
baby bitter
candy cancer
dream crime
gentle cruel
kiss dead
love devil
neat liar
pretty nasty
sleep rude
wise ugly
2. More Challenges
These additional challenge questions (along with the concepts illustrated in each question) can be used for
class discussion or as possible test questions.
A. The pursuit rotor is a test of perceptual-motor coordination. It involves a turntable with a disk about
the size of a dime embedded in it. The participant is given a pointer and is asked to keep the pointer
on the disk while the turntable is rotating. The dependent variable is the percentage of time on each
trial that the participant keeps the pointer on the disk. Learning on this task is linearly related to
trials over many periods of practice, and the task generally takes a long time to master. A
researcher wants to study the influence of time of day on the performance on this task with four
different times (10 A.M., 2 P.M., 6 P.M., and 10 P.M.). The participants will receive a constant
number of trials under each of the four conditions, and participants will be tested in one condition
per day over four consecutive days.
(1) What design is being used for the time–of-day variable in this experiment?
(2) Prepare a Latin Square to balance practice effects across the conditions of the experiment.
(3) The researcher decides to use all possible orders to balance practice effects. The researchers
assigns each participant to one of the 24 possible orders of the conditions. Which experimental
design is included when you look only at the first condition to which each participant was
assigned?
(4) How could the researcher test whether differential transfer occurred when all possible orders
were used to balance practice effects?
13
(1) What method was used to balance practice effects for the drug variable?
(2) Compare the mean for the two values when the 50-unit dose followed the 0-unit dose with the
mean for the two values when the 50-unit dose followed the 100-unit dose to determine
whether there is evidence of differential transfer influencing performance in the 50–unit
condition.
(3) Compare the means for the 0-, 50-, and 100-unit conditions at the first ordinal position (where a
random groups design is present) to the means for the three conditions over the whole
experiment (compute the means for each of the three conditions by averaging the six values for
each condition). Compare these two sets of means and decide whether they indicate that
differential transfer has affected performance in the experiment.
D. Cognitive psychologists make extensive use of the repeated measures design, especially when
reaction time is used as the dependent variable. Because participants can respond within a second
or two it is possible to use the complete repeated measures design to test participants on hundreds
of trials in an experiment of reasonable length. One illustrative area of this type of research in
cognitive psychology involves the relationship between cognition and brain processes. More
specifically, reaction time experiments have been used to explore differences in processing
between the two hemispheres of the cerebral cortex. One question researchers have explored is
whether information is processed more efficiently when it is presented to only one hemisphere at a
time or when it is presented to both hemispheres simultaneously.
A task commonly used in these experiments is a letter-matching task. In this task participants
see a display showing two letters and they are asked if the letters match. For example, the display
Aa would be a match and the display Ab would not be a match. The critical independent variable in
experiments comparing the processing efficiency of one hemisphere as compared to both
hemispheres is the position on the display of the presented letters. In the experiment we will be
describing for this problem, there were three different positions.
In the first condition both letters were presented only in the left visual field (unilateral left
condition) with one above and one below the middle of the screen.
In the second condition both letters were presented only in the right visual field (unilateral right
15
condition), again, one above and one below the middle of the screen.
In the third condition one letter was presented on either side of a central fixation point on the
screen (bilateral condition).
There are many factors that need to be controlled in conducting an experiment of this type. For
example, half the trials need to be match trials and half need to be no-match trials. The length of
time the letters are presented, the size of the letters, and the brightness of the letters also need to
be controlled. For this problem, however, we will ignore these important control considerations. We
will only be concerned with the three levels of the position variable. Twenty-five participants were
tested on 100 trials for each of the three conditions in a complete repeated measures design.
Practice effects were balanced for the position variable using block randomization.
The investigators expected to find that processing would be more efficient for the bilateral
condition than for the unilateral conditions. The name for this phenomenon is the bilateral
superiority effect and evidence for it is provided when participants’ reaction times are faster in the
bilateral condition than in the combined unilateral conditions. In this problem you will be following
the basic steps in testing for a bilateral superiority effect.
(1) The following table presents the reaction times (measured in milliseconds) for a subset of four
participants who had match trials for four blocks of the experiment. You are to use these data to
prepare a matrix showing each participant’s performance in each of the three conditions. The
symbols in the table are B for bilateral, L for unilateral left, and R for unilateral right.
Block
1
2
3
4
P#
B
R
L
L
B
R
L
B
R
R
L
B
1
665
690
710
695
680
675
705
660
685
670
690
675
2
685
665
705
720
700
680
700
695
660
675
715
680
3
665
690
735
750
655
705
745
650
695
710
730
670
4
695
705
720
735
710
715
740
690
720
700
725
705