21
Mindfulness, continued
2. Langer and Piper’s findings were statistically significant. Comment critically on each of the following
statements and decide whether the conclusions are appropriate based on the fact that the findings
were statistically significant.
(a) The findings in Langer and Piper’s study have only a small probability of occurring if the null
hypothesis is true.
(b) Langer and Piper’s found that 65% of the students in the conditional group solved the problem by
identifying the object as an eraser while only 25% of students in the unconditional group did so.
Because this finding is statistically significant, do we know that the finding is meaningful and
important?
(c) If the probability of Langer and Piper’s obtaining their findings assuming the null hypothesis is
true was .02, then we can say that the probability that the null hypothesis is true for their findings
is .02.
22
Reading Research Critically
Read the following description of a research study to answer the questions that follow. This description is
based on an article by: Crusco, A. H., & Wetzel, C. G. (1984). The Midas touch: The effects of
interpersonal touch on restaurant tipping. Personality and Social Psychology Bulletin, 10, 512-517.
Interpersonal Touch
Through the act of touching someone we communicate a variety of intentions, ranging from love or sexual
desire to dominance or aggression. The authors of the study reviewed results of previous research on
interpersonal touch, showing, for example, that an innocuous touch of another person can have positive
effects, such as increasing ratings of liking a person. There also appear to be differences in the way
males and females touch one another and in how they respond to being touched. In a field experiment,
the effects of interpersonal touch were examined in a natural setting: a restaurant. Male and female
diners were administered either no touch, or one of two kinds of touches immediately before the
customers left their tip. The diners were touched briefly by a waitress either on the hand or on the
shoulder during the change-returning transaction. The authors speculated that a touch on the shoulder,
which often signals dominance, may be viewed less favorably, especially by male diners, than a touch on
the hand. The effects of interpersonal touch were assessed both by measuring the size of the gratuity (tip
percentage) and by the results of a brief written survey asking about the customers’ dining experience.
Most of the data were obtained at one restaurant and by one waitress who was blind to (unaware of) the
research hypothesis. The waitress randomly assigned diners to the experimental conditions after she had
collected the money but before she returned the change. The touch was administered at the time the
change was returned to the diner. After returning the change, the waitress also asked the customers to
complete a restaurant survey and leave it in a sealed envelope on the table.
The statistical analysis revealed that the average percentage tip differed significantly as a function of
touch. Specifically, although the mean tip in the Shoulder Touch (M = 14.4%) did not differ from the mean
tip obtained in the Hand Touch condition (M = 16.7%), these two touch conditions did result in larger
average tips than did the No Touch condition (M = 12.2%). In addition, the authors reported that male
diners tipped significantly more on the average than did female diners (means of 15.3% and 12.6%,
respectively). When the results of the survey were analyzed, the only significant finding was that males
rated their restaurant experience more positively than did females; there were no effects of the touch
manipulation on the survey results.
The authors suggested that the failure to find a difference between the two kinds of touches, especially
for male diners, was possibly due to the fact that the diners in this setting felt secure in their role and
viewed the shoulder touch “benevolently.” Several reasons were given to explain why an effect of touch
was not observed on the restaurant survey. These included the possibility that the effect of touch is short–
lived (and had diminished by the time customers filled out the survey). Nevertheless, the results did show
that interpersonal touch had a positive, even if fleeting, effect on behavior. The possibility was raised that
the effect may be “subliminal,” that is, that the diners were not really aware that they had been touched.
1. Despite the observation that diners were apparently unaware of being touched in the field experiment,
why is it possible to conclude that the touch caused the diners to leave a larger tip?
2. Crusco and Wetzel (1984) were interested in examining the factors that influence the amount of
money left for a tip. Explain why it was important to use the percentage of the total bill left for a tip as
the dependent variable rather than the amount of money left for the tip. What potential variable is
23
controlled for by using the percentage tip as the dependent variable?
3. The researchers held conditions constant in their experiment by employing one female wait-staff
person as their confederate. Explain how this might have influenced the results they observed.
LEARNING BY DOING RESEARCH
1. Class Research Project
When we cover the material on experimental research methods in the text we sometimes have
students conduct one research project and prepare one proposal. The experimental research project
represents either a random groups design or a repeated measures design with only one independent
variable. It is important that students also become familiar with the steps for data analysis described
in Chapters 11 and 12. Students report the experiment in an APA-style paper. The research proposal
involves a complex design (Chapter 8). Students present their proposal in a poster as a final
assignment in the course. We will describe here the general outline of the experimental project.
Outline of Steps for Conducting an Experimental Research Project
The instructor designs an experiment or selects one from available software. Our experience has
been that the experiment need not be original research for students to engage in the project. We
typically have students do replications or partial replications of published research. Students in the
methods class can be the participants in the experiment tested by the instructor or a student
assistant. Or, the methods students can be the experimenters who test each other or their friends
(one way to increase sample size). In recent years we have decided to do experiments that are
computer controlled.
After completing the data collection students are introduced in class to APA format (Chapter 13). The
final product of the assignment is an APA-style report written by each student or written by small
teams of students working collaboratively.
In the introduction of their paper students typically are asked to include:
A. A primary reference for the experiment (i.e., the article that is the basis of the replication or partial
replication); often students are given a photocopy of this reference or a link to locate an online
version.
B. A second reference related to the experiment published in the last 3 or 4 years; students identify
this reference using electronic searches.
C. An outline of the experiment.
D. The predictions (hypotheses) being tested in the experiment.
Students are given a handout that outlines the basic subsections of the method and the necessary
descriptive and inferential statistics for the results section. [Students can be involved in the computer
analysis of the data as time and facilities permit.]
Students develop their discussion section based on class discussions of the experiment. They are
required to conclude their discussion with a proposed follow-up experiment. Their proposed
24
experiment can become the basis for the research proposal assignment. As part of their discussion
students are asked to consider:
A. What were the major findings in your experiment?
B. How did your findings compare to your predicted findings?
C. How do your findings relate to the references you cited in your introduction?
D. What factors, if any, limit your ability to interpret your findings or to generalize your findings?
E. What specific experiment would you propose to follow up on this experiment?
2. Research Project Based on Dittmar, Halliwell, and Ive (2006)
Students may enjoy conducting a class research project that builds on the findings reported in
Chapter 6 for the Dittmar, Halliwell, and Ive (2006) study. Although the Dittmar et al. study
investigated body image among very young girls, students will find an abundance of relevant studies
conducted using college student samples in an electronic search of the research literature. As they
develop their ideas for a research project in this area, students can address many topics covered in
Chapter 6:
A. Replication
1. What aspects of the Dittmar, Halliwell, and Ive (2006) study can be replicated?
2. How does the idea of “conceptual replication” impact the choice of materials to be used in a
replication?
3. How can a subsequent study be designed so as to be a partial replication?
4. Describe the findings from a follow-up study that would increase the external validity of the
findings regarding exposure to thin images and body dissatisfaction.
B. Experimental Control
1. Identify an independent variable manipulation of exposure to thin images that could be used
in a college student sample. Describe how this independent variable could be manipulated.
What control conditions would be necessary to allow the causal inference that exposure to
thin images, and not any body images, causes body dissatisfaction (e.g., contrast thin
images, athletic images, and neutral images).
2. Describe the dependent variable for this study and how it should be measured. [NB: Dittmar
et al. used the Child Figure Rating Scale; a similar version is available for adults.]
3. What aspects of the experiment would need to be held constant to allow an unambiguous
interpretation of the results (e.g., amount of time of exposure, size and color of the images)?
4. What procedures would be required to rule out the alternative explanation that individual
differences among participants in the experimental groups were related to findings?
5. What procedures would be necessary to prevent threats to internal validity due to extraneous
variables, demand characteristics, and experimenter effects? Would it be important to
disguise the true nature of the investigation?
C. Sample
25
1. Identify the sample of participants for this study. How would a choice of females or males
influence the external validity of the findings regarding exposure to thin images? [NB:
Students may wish to conduct a study investigating differences between male and female
students’ responses to body images in a complex design.]
2. How should the choice of materials used for implementing the independent variable be
affected by the choice of the sample?
D. Results
1. Describe how null hypothesis significance testing could be used to analyze the data for this
study.
2. Describe how confidence intervals could be used to analyze the data.
3. What are the expected findings for the effect of the independent variable on the dependent
variable? Is the effect size predicted to be small, medium, or large?
INSTRUCTOR’S LECTURE/DISCUSSION AIDS
The following pages reproduce content from Chapter 6 and may be used to facilitate lecture or
discussion.
1. Why Psychologists Conduct Experiments: This page identifies three main uses for experiments.
2. Experimental Research: Independent and Dependent Variables are described on this page.
3. Internal Validity: This page defines internal validity and confoundings.
4. Research Example: The main features of the Dittmar, Halliwell, and Ives (2006) experiment are
outlined on this page.
5. Independent Groups Designs: This page identifies the essential feature of the independent groups
design and the three types of designs.
6. Random Groups Designs: This page describes the logic of the random groups design and block
randomization.
7. Threats to Internal Validity: This page identifies the problems of intact groups, extraneous variables,
selective subject loss, and demand characteristics/experimenter effects.
8. Analysis and Interpretation of Experimental Findings: This page begins the topic of data analysis of
experiments.
9. Analysis of Experiments, Descriptive Statistics: The basics of mean, standard deviation, and effect
size are identified on this page.
10. Analysis of Experiments, Confirm What the Data Reveal: This page describes the goal of statistical
inference and the two main methods for inference (NHST, confidence intervals).
11. Null Hypothesis Significance Testing (NHST): The basic steps for NHST are outlined on this page.
26
12. Confidence Intervals: This page outlines the use of confidence intervals for statistical inference.
13. External Validity: This page describes key points associated with external validity.
14. Matched Groups Design: The main feature of the matched groups design are described on this page.
15. Natural Groups Design: This page differentiates selected and manipulated independent variables and
describes the use of the natural groups design.
Why Psychologists Conduct Experiments
• Test
▪ Hypotheses derived from theories
▪ Effectiveness of treatments and programs
• Third goal of psychological research
▪ Explanation: examine the causes of behavior
• Multimethod approach
▪ Seek converging validity for research findings across methods
27
Experimental Research
• An experiment must include
▪ Independent Variable (IV)
▪ Dependent Variable (DV)
• Independent Variable
▪ Manipulated (controlled) by experimenter
▪ At least two conditions (levels)
o Treatment
o Control
• Dependent Variable
▪ Measured by experimenter
▪ Used to determine effect of IV
o Typically researchers measure several dependent variables to
assess effect of IV.
28
Internal Validity
• Definition
▪ An experiment has internal validity when differences in performance
(DV) can be attributed unambiguously to the effect of the independent
variable (IV).
• Three conditions for causal inference
▪ Covariation
▪ Time-order relationship
▪ Elimination of alternative causal explanations (confoundings)
• Confoundings
▪ When IV covaries with a different, potential independent variable
▪ Alternative explanations for an experiment’s findings
▪ Experiment free of confoundings has internal validity
• Control techniques to eliminate confoundings
▪ Balancing
▪ Holding conditions constant
29
Research Example: Body Image Among Young Girls
• Research prediction
▪ Young girls exposed to a very thin body image will experience greater
body dissatisfaction than young girls who are exposed to realistic or
neutral body images.
• Independent Variable: Version of picture book with 3 levels
▪ Barbie (very thin body image)
▪ Emme (realistic body image)
▪ Neutral (no body images)
• Dependent Variables: Several measures of body image and body
dissatisfaction
▪ Child Figure Rating Scale
o Identify perceived actual body shape
o Identify ideal body shape
o Difference score: Negative scores indicate desire to be thinner
• Control techniques
▪ Manipulation
o Independent Variable: Girls in each condition have a different
experience (Barbie, Emme, Neutral images)
▪ Holding conditions constant
o IV is the only thing that varies systematically across groups
o Dittmar et al. (2006) held constant:
All girls listened to the same instructions and story.
All completed the same questions after the story.
▪ Balancing
o Random assignment to conditions balances subject
characteristics, on average.
o Groups are equivalent prior to IV manipulation.
o All subject variables are balanced (e.g., body weight, number of
Barbies, preexisting levels of body dissatisfaction)
30
Independent Groups Designs
• Different individuals participate in each condition of the experiment
▪ No overlap of participants across conditions
• Three types
▪ Random groups design
▪ Matched groups design
▪ Natural groups design
31
Random Groups Design
• Individuals are randomly assigned to conditions of the IV.
▪ Random assignment balances subject characteristics.
• Logic of causal inference
▪ If groups are equivalent at the beginning of an experiment (through
balancing), and conditions are held constant
▪ then any differences among groups on dependent variable are
caused by the manipulated independent variable.
• Block randomization
▪ Block: a random order of all conditions in an experiment
▪ Randomly assign subjects one block at a time.
▪ Advantages
o Creates groups of equal size
o Controls for time-related events that occur during the course of an
experiment
32
Threats to Internal Validity
• Ability to make causal inferences is threatened when
▪ Intact groups of subjects are used.
o Do not use already-formed groups (i.e., not randomly assigned).
▪ Extraneous variables are not controlled.
o Hold conditions constant
▪ Selective subject loss occurs
o Mechanical subject loss is not a problem.
▪ Demand characteristics and experimenter effects are not controlled.
o Use placebo-control and double-blind procedures.
33
Analysis and Interpretation of Experimental Findings
• Use statistical analysis to claim
▪ IV produced an effect on DV
▪ Rule out the alternative explanation that chance produced any
observed effect
• Replication
▪ Best way to determine whether findings are reliable
▪ Repeat experiment and see if same results are obtained
• Three steps for data analysis
▪ Check the data
o Errors? outliers?
▪ Describe the results
o Descriptive statistics such as mean, standard deviation, effect size
▪ Confirm what the data reveal
o Inferential statistics
o Confidence intervals
34
Analysis of Experiments: Descriptive Statistics
• Mean (central tendency)
▪ Average score on a DV, computed for each condition
▪ Not interested in each individual score, but how people responded on
average in a condition.
• Standard deviation (variability)
▪ Average distance of each score from the mean of a group
▪ Not everyone responds the same way to an experimental condition.
• Effect size
▪ Measure of strength of relationship between IV and DV
▪ Cohen’s d =
difference between treatment and control means
average variability for all participants’ scores
▪ Guidelines for interpreting Cohen’s d
o small effect: d = .20
o medium effect : d = .50
o large effect: d = .80
▪ Meta-analysis
o Summarize effect sizes across many experiments that investigate
same IV or DV
o Select experiments based on their internal validity and other
criteria
o Allows psychologists to gain confidence in general psychological
principles
35
Analysis of Experiments:
Confirm What the Data Reveal
• Use inferential statistics to determine whether the IV produced a reliable
effect on the DV.
• Rule out whether findings are due to chance (error variation).
• Two types of inferential statistics
▪ Null Hypothesis Significance Testing (NHST)
▪ Confidence intervals
36
Null Hypothesis Significance Testing (NHST)
• Statistical procedure to determine whether mean difference among
conditions is greater than what might be expected based on chance
(error variation)
• Effect of IV is statistically significant when probability of the observed
results being due to chance is low.
▪ p < .05
• Steps for NHST
▪ Assume null hypothesis (H0) is true.
o The population means for the groups in the experiment are equal.
▪ Use sample means to estimate population means.
▪ Is the observed difference between means greater than what is
expected if H0 were true (zero difference)?
▪ Compute appropriate inferential statistic.
o 2 means: t-test
o 3 or more means: F-test (ANOVA)
▪ Identify the probability associated with the inferential statistic.
o Check computer output or statistical tables.
▪ Compare observed probability with predetermined level of statistical
significance, alpha (p = .05).
▪ If the observed p value is greater than .05, then do not reject H0.
o Conclude IV did not produce a reliable effect in this experiment.
▪ If the observed p value is less than .05, then reject H0.
o Conclude IV produced a reliable effect.
37
Confidence Intervals
• Logic of using confidence intervals for statistical inference:
▪ Use sample means to estimate population means.
▪ Confidence interval for a mean provides the range of values that
contains the true population mean.
o Researchers typically use probability associated with range of
values, p = .95.
▪ Compute confidence interval around sample mean for each condition
of experiment.
▪ Decision rules
o If confidence intervals do not overlap,
then we gain confidence that the population means for the
conditions are different and conclude the IV produced a reliable
effect.
o If confidence intervals overlap slightly,
we are uncertain about the true mean difference and cannot draw
conclusion about effect of IV.
o If the confidence intervals overlap such that the sample mean for
one condition lies within the confidence interval for another
condition,
we conclude the population means do not differ.
38
External Validity
• Extent to which findings from an experiment can be generalized to
describe individuals, settings, and conditions beyond the scope of a
specific experiment.
▪ Any single experiment has limited external validity.
▪ External validity of findings increases when findings are replicated in
a new experiment.
• Questions of external validity
▪ Would the same findings occur
o in different settings?
o with different conditions?
o with different participants?
▪ Does research with college student samples have low external
validity?
▪ Theory testing: Questions of external validity less important
o Sample often doesn’t matter.
• Increasing external validity
▪ Include characteristics of settings, conditions, and population to which
researchers seek to generalize
▪ Partial replications
▪ Field experiments
▪ Conceptual replications
39
Matched Groups Design
• Random groups design requires large samples to balance subject
characteristics.
▪ Sometimes only small numbers of subjects are available.
▪ Select 1 or 2 variables for matching in matched groups design.
• Procedure
▪ Select matching variable
o Must be related to outcome or DV
▪ Match pairs (or triples, quadruples, etc. depending on number of
conditions) of identical or similar scores on matching variable.
▪ Randomly assign individuals within each match to different IV
conditions.
▪ Groups not equivalent for all individual difference variables.
40
Natural Groups Design
• Psychologists often ask questions about how individuals differ, and how
these individual differences are related to important outcomes.
• Individual differences (subject variables)
▪ Characteristics or traits that vary across individuals (e.g., male,
female)
▪ Select individual variables; cannot manipulate these variables
▪ Can’t randomly assign participants to these groups
• Procedure
▪ Classify individuals into groups based on subject variables and
measure DVs.
▪ Correlational research
▪ Can’t make causal inference for natural groups variable.
▪ Describe and predict using relationships between natural groups
variables and DVs
▪ Method to improve causal inferences
o Study natural groups variable in combination with manipulated IVs