978-0077825362 Chapter 7 Part 2

16

mean of the two unilateral conditions (i.e., the mean of left and right) was statistically

significant. What would you claim about the bilateral superiority effect?

17

LEARNING BY DOING RESEARCH

A simple experiment involving repeated measures that would also be familiar to students is a Ataste test.@

These procedures could be used as a class demonstration or students could conduct this simple

experiment with friends as participants. Presented here are procedures that could be followed for

comparing two conditions, but it would be easy to expand the number of conditions depending on students’

research question. An advantage of the taste-test experiment is that students can appreciate the

complications associated with differential transfer in the repeated measures design.

Step 1: Students choose their research question. For example, students might ask, ACan people

distinguish between Coke and Diet Coke?@ (a third condition might test Coke Zero), or ACan people tell the

difference between Coke and Pepsi?@ (a third condition might test a generic product). Instead of Atell the

difference@ questions, students may choose a question relating to preference (e.g., ADo people prefer

Coke or Pepsi?@). Students may identify other products of interest for comparison.

Step 2: Define the dependent variable. To illustrate computational procedures using the repeated measures

design, students could use a rating scale for each taste sample. For example, for a research question

regarding whether people can distinguish between products, participants could rate the likelihood that each

product is Coke on a 1-10 scale. They would make the rating for each taste sample. If the research question

concerns preference, the dependent variable question could ask participants to use a rating scale anchored

by Ado not like at all@ to Alike very much@ for each taste sample.

Step 3: Decide what type of repeated measures design to use. Students should decide whether each

participant samples each product once (incomplete design) or more than once (complete design). Once this

decision is made, students should decide the method for balancing practice effects in their design.

Step 4: Design the procedure for recording participants= responses based on the dependent variable and

repeated measures design selected. Other decisions, such as how much of each product should be tasted,

how to keep participants (and experimenters) blind to condition, random assignment to orders of conditions,

and extraneous variables to hold constant, should be addressed.

Step 5: Tabulate the results, making sure to Aunwind@ the order of conditions and product condition (i.e.,

Aproduct A@ will appear in the 1st ordinal position and the 2nd ordinal position, etc.). Use descriptive

statistics to describe the findings, and inferential statistics as appropriate to the classroom situation.

Step 6: Address issues of differential transfer: Is it possible that sampling AProduct A@ first influences the

taste of AProduct B,@ and vice versa? A simple way to test for the possibility of differential transfer is to

compare the overall results for the repeated measures design with the results for just the first ordinal

position. Are ratings of the products different in the first ordinal position, in which no prior tasting could

influence participants= ratings, compared to the overall results?

Step 7: Students may be asked to write a brief report to describe their findings. In their report they should

describe their research question; the research design, including how the independent variable was

manipulated and counterbalancing procedures; the dependent variable; a summary of the results, including

descriptive and inferential statistics (as appropriate); a discussion of their test for differential transfer; and

their conclusions.

18

INSTRUCTOR=S LECTURE/DISCUSSION AIDS

The following pages outline content from Chapter 7 and may be used to facilitate lecture or discussion.

1. Repeated Measures Designs: This page describes the repeated measures design and the reasons

researchers use this design.

2. Practice Effects: This page illustrates the problem of practice effects using an example.

3. Balancing Practice Effects: This page introduces complete and incomplete repeated measures designs

for counterbalancing practice effects.

4-5. Complete Repeated Measures Design: These two pages describe the complete repeated measures

design, block randomization and ABBA counterbalancing.

6. Incomplete Repeated Measures Design: The incomplete design is differentiated from the complete

design on this page.

7-8. Counterbalancing in the Incomplete Design: Methods for counterbalancing, all possible orders and

selected orders (Latin square, random starting order with rotation) are described on these two pages.

9. Data Analysis for Repeated Measures Designs: This page identifies the additional step required when

participants complete the experimental conditions in the complete design.

10. The Problem of Differential Transfer: This page describes differential transfer.

11. Comparison of Two Designs: This page compares repeated measures designs and random groups

designs on two dimensions: independent variable and what is controlled through balancing.

19

Repeated Measures Designs

! Each individual participates in each condition of the experiment.

 Completes DV measure with each condition

 Hence, Arepeated measures@

! Also called Awithin-subject design@

 Entire experiment is completed Awithin@ each subject

! Advantages

 No need to balance individual differences across conditions of

experiment

 Fewer participants needed

 Convenient and efficient

 More sensitive design

! A sensitive experiment can detect the effect of an IV

even if the effect is small

! Error variation is reduced

! Variability due to individual differences is eliminated.

20

Practice Effects

! Main disadvantage of repeated measures designs

 People change as they are tested repeatedly.

! Performance may improve over time.

! People may become tired or bored as number of Atrials@ increases.

! Practice effects become a potential confounding variable if not controlled.

! Example

 Researcher tests 2 study methods, A and B, for participants’

comprehension of text passages

! Condition A: highlight text while studying, then take test.

! Condition B: Read text and create sample test questions and answers,

then take test.

 Suppose researchers tests all participants in condition A first, then B.

 Results show test scores are higher with Method A than Method B.

 Problem: Confounding of study method IV with order of presentation

! Can’t determine effect of IV

! Practice effects (boredom, fatigue) may explain poorer performance

for Method B.

21

Balancing Practice Effects

! Balance practice effects across conditions

! Counterbalance the order of conditions

 Half of the participants do Condition A first, then Condition B.

 The remaining participants do Condition B, then A.

! Distribute practice effects equally across conditions

 Practice effects aren’t eliminated.

 Balance, or average, practice effects across the conditions of

experiment.

! Two types of repeated measures designs

 Complete repeated measures design

 Incomplete repeated measures design

! Complete and incomplete designs differ in how practice effects are

counterbalanced.

22

Complete Repeated Measures Design

! Balance practice effects within each participant.

 Each participant experiences each condition several times.

 Each participant forms a Acomplete@ experiment.

 Use different orders each time.

 Best when each condition is brief

 e.g., simple judgments about stimuli

! Two methods for counterbalancing order of conditions

 Block randomization

 ABBA counterbalancing

! Block randomization

23

Complete Repeated Measures Design, continued

! ABBA counterbalancing

 Present conditions only a few times to each participant

 Use one random sequence of conditions (e.g., DABC)

 Then the opposite of the sequence (CBAD).

 Repeat with new random sequence and its opposite, etc.

When anticipation effects can occur:

 Participants form expectations about which condition will appear next

24

in sequence.

 Responses may be influenced by expectations, not independent

variable.

 If anticipation effects are likely (e.g., conditions are predictable), use

block randomization.

25

Incomplete Repeated Measures Design

! Each participant experiences each condition once.

 Not many times, as in the complete design.

! Balance practice effects across participants (not within)

! General rule for balancing practice effects:

 Each condition must appear in each ordinal position (1st, 2nd, 3rd, etc.)

equally often.

! Two methods for balancing practice effects

 All possible orders

 Selected orders

26

Counterbalancing in the Incomplete Design

! All possible orders

 Use with 4 or fewer independent variable conditions

 2 conditions (A, B)  2 possible orders: AB, BA

Randomly assign half of participants to do condition A then B,

other half do condition B then A

 3 conditions (A, B, C)  6 possible orders:

ABC, ACB, BAC, BCA, CAB, CBA

Randomly assign participants to one of the 6 orders

 4 conditions (A, B, C, D)  24 possible orders

 Need at least 1 participant randomly assigned to each order.

! Selected orders

 Select particular orders of conditions to balance practice effects

 Two methods: Latin Square and Random Starting Order with Rotation

 Each IV condition appears in each ordinal position exactly once.

27

Selected Orders for Counterbalancing

in the Incomplete Design

! Latin square

Ordinal Position

1st 2nd 3rd 4th

A B D C

B C A D

C D B A

D A C B

 Each condition appears in each ordinal position to balance practice

effects.

 Another advantage: Each condition precedes and follows every other

condition once (AB and BA, BC and CB, etc.).

 This helps to control for possible order effects.

! Random starting order with rotation

 Generate random order of conditions (e.g, ABCD)

 Rotation: Move each condition one position

Ordinal Position

1st 2nd 3rd 4th

A B C D

B C D A

C D A B

D A B C

28

 Each condition appears in each ordinal position.

 Possible order effects are not balanced.

Data Analysis for Repeated Measures Designs

! Complete repeated measures design requires one additional step.

 Compute a summary score (e.g., mean) for each participant for each

independent variable condition.

 This score represents each participant=s average performance in

each condition.

29

The Problem of Differential Transfer

! Do not use repeated measures designs when differential transfer is

possible.

 Effects of one condition persist and affect participants’ experience of

subsequent conditions.

 Use independent groups design instead.

 Assess whether differential transfer is a problem by comparing results

for repeated measures design and random groups design.

 Compare performance in 1st ordinal position to overall results.

1st 2nd 3rd 4th

A B D C

B C A D

C D B A

D A C B

 1st ordinal position represents a random groups design.

 Differential transfer may be a problem if performance in 1st ordinal

position differs from results for entire repeated measures design.

30

Comparison of Two Designs

! Differences between repeated measures design and independent

(random) groups design

! Compare how independent variable is manipulated

 Repeated measures: each participant experiences every condition of

the IV

 Independent (random) groups design: each participant experiences

only one condition of the IV

! Compare what is balanced across conditions in order to rule out

alternative explanations for findings (confoundings)

 Repeated measures: practice effects

 Independent (random) groups: individual differences