McDaniel & Gates – Marketing Research, 9th Edition Instructor’s Manual
CHAPTER 9
Primary Data Collection: Experimentation and Test Markets
LEARNING OBJECTIVES
1. To understand the nature of experiments.
3. To learn about the experimental setting.
5. To learn the limitations of experimentation in marketing research.
7. To gain insight into test marketing.
KEY TERMS
Experiment
Causal research
Concomitant variation
Randomization
Physical control
Design control
Instructor’s Manual
Multiple time-series design Test market
CHAPTER SCAN
This chapter examines experimentation, a method of primary marketing research. An experiment
occurs when the researcher changes or manipulates an independent variable in order to observe
its effect on a dependent variable. Experiments typically intend to prove causation. Experiments
can be conducted in a laboratory or in a field setting. When conducting an experiment, the
researcher hopes to prove causation of one variable on another. Doing so involves showing that
CHAPTER OUTLINE
1. What Is An Experiment?
2. Demonstrating Causation
I. Causal Research
A. Causal Research Defined
B. Requirements to Demonstrate Causation
3. The Experimental Setting
I. Laboratory Experiments
A. Laboratory Experiments Defined
4. Experimental Validity
I. Validity
5. Experimental Notation
I. Standard System of Notation
6. Extraneous Variables
I. Examples of Extraneous Variables
McDaniel & Gates – Marketing Research, 9th Edition Instructor’s Manual
A. History
7. Experimental Design, Treatment, and Effects
I. Experimental Design
8. Limitations of Experimental Research
I. The High Cost of Experiments
A. Time
9. Selected Experimental Designs
I. Pre-experimental Designs
McDaniel & Gates – Marketing Research, 9th Edition Instructor’s Manual
10. Test Markets
I. Test Market
A. Test Market Defined
B. Designed to Provide information
11. Summary
CHAPTER SUMMARY
1. WHAT IS AN EXPERIMENT
I. Experiment
A. Experiment Defined—the researcher is an active participant in the process, the research
2. DEMONSTRATING CAUSATION
I. Causal Research
A. Causal Research Defined – research that has the potential to demonstrate that a change in
one variable causes some predictable change in another variable
B. Requirements to Demonstrate Causation
1. Concomitant variation (correlation)
3. Elimination of other possible causal factors
C. Scientific Definition–Causation and Causality
1. Popular view–X is only one of a number of determining conditions that caused the observed
change in Y
2. Scientific view–X can be a cause of Y if the presence of X makes the occurrence of Y more
probable
4. Causal relationships are inferred rather than demonstrated conclusively beyond a shadow of a
doubt
II. Concomitant Variation
A. Concomitant Variation (Correlation)–evidence that A and B vary together in some
predictable fashion, either a positive or an inverse relationship might exist.
B. Causation–concomitant variation by itself does not prove causation; there must be true
III. Appropriate Time Order of Occurrence
McDaniel & Gates – Marketing Research, 9th Edition Instructor’s Manual
A. Appropriate Time Order of Occurrence Defined–change in an independent variable
occurred before an observed change in the dependent variable
B. Demonstration that A caused B
1. To demonstrate that A (independent variable) caused B (dependent variable), the researcher
must be able to show that A occurred before B occurred.
IV. Elimination of Other Possible Causal Factors
A. Difficult to Demonstrate
2. Competitive environmental changes or changes in the economy or other factors or
combinations of factors may have caused the change in dependent variable.
3. EXPERIMENTAL SETTING
I. Laboratory Experiments
A. Laboratory Experiments Defined–experiments conducted in a controlled setting
B. Advantages
2. Can focus on the first two elements–concomitant variation and appropriate time order of
occurrence
3. Greater internal validity
C. Disadvantages
1. Results not directly transferrable to the actual marketplace
2. Problems with external validity
II. Field Experiments
A. Field Experiments Defined–tests conducted outside the laboratory, such as a marketplace
B. Advantages
1. Creates realism of the environment
C. Disadvantages
1. Problems with internal validity
McDaniel & Gates – Marketing Research, 9th Edition Instructor’s Manual
2. Cannot control for spurious factors that might influence the dependent variable, such as:
a. Actions of competitors
b. Weather
See Practicing Marketing Research, What’s It Like to Be Old? (pg. 245)
Design products and services for the elderly that promote wellness, autonomy, and social
engagement. One of the biggest takeaways from AgeLab has been simply that elderly people
Questions:
1. The Mirabella is being used as a kind of test market. How do you think this program affects
the way products and services are test-marketed there?
2. Experiments like those discussed here are typically costly. Do you think those costs are worth
it, or do you think researchers might be better off finding a cheaper method entirely?
4. EXPERIMENTAL VALIDITY
I. Validity
A. Validity Defined–degree to which an experiment actually measures what the researcher is
trying to measure.
II. Validity of a Measure
A. Validity of a Measure Defined– depends on the extent to which the measure is free from
both systematic and random error
B. Two Types of Validity
2. External Validity–the extent to which the causal relationships measured in an experiment can
be generalized to outside persons, settings, and times
See Practicing Marketing Research: Know Your Potential Customers, Psychologically
Speaking (pg. 247)
McDaniel & Gates – Marketing Research, 9th Edition Instructor’s Manual
Researchers at Wayne State University in Detroit, Michigan queried 350 applicants for various
bank services on standard demographic variables and 53 psychographic items. The questions
were presented in 10 categories, such as price consciousness, dynamic leadership, credit use, and
innovation, with typical questions such as: “I usually shop a lot for ‘specials.’” Or “I often try
new brands before my friends and neighbors do.” The results showed that “relatively stable”
Discussion Questions
1. In developing an experimental design and identifying key variables to manipulate, how could
you take maximum advantage from the fact that heavy credit card users are not concerned with
price?
2. Identify several factors among heavy credit card users that contradict the psychographic
5. EXPERIMENTAL NOTATION
I. Standard System of Notation
A. X–indicates the exposure of an individual or a group to an experimental treatment.
1. Experimental Treatment–the factor whose effects we want to measure and compare. Factors
such as:
a. Different prices
McDaniel & Gates – Marketing Research, 9th Edition Instructor’s Manual
B. O (for observation)–refers to the process of taking measurements on the test units.
1. Test Units–individuals, groups of individuals, or entities whose response to the experimental
treatments is being tested
1. X’s and O’s can be arranged vertically to show simultaneous exposure and measurements of
different test units–two groups of test units received different experimental treatment at the same
time–two groups were measured simultaneously
X1 O1
X2 O2
6. EXTRANEOUS VARIABLES
I. Examples of Extraneous Variables
A. History–the intervention, between the beginning and end of the experiment, of any variable
or event–other than those manipulated by the researcher (experimental variables)–that might
affect the value of the dependent variable
B. Maturation–changes in subjects throughout the course of the experiment that are a function
of time: it includes getting older, hungrier, more tired, and the like
C. Instrument Variation–changes in measurement instruments that might explain differences in
the measurements taken
D. Selection Bias–encountered in situations where the experimental or test group is
1. Randomization–assigning subjects to test groups and control groups at random
2. Matching–making sure that there is a one-to-one match between people or other units in the
test and control groups in regard to key characteristics.
McDaniel & Gates – Marketing Research, 9th Edition Instructor’s Manual
F. Testing Effects–the fact that the process of experimentation may produce its own effect on
the responses observed
1. Main testing effects–the possible effects of earlier observations on later observations
2. Interactive testing effect–the effect of a prior measurement on a subject’s response to a later
measurement
G. Regression to the Mean– the observed tendency of subjects with extreme behavior to move
toward the average for that behavior during the course of an experiment
II. Controlling Extraneous Variables
A. Confounding Variables–extraneous causal factors confound the treatment conditions,
making it impossible to determine whether changes in the dependent variable are due solely to
the treatments.
B. Four Basic Approaches Used to Control Extraneous Factors
1. Randomization–random assignment of subjects to treatment conditions to ensure equal
representation of subject characteristics
2. Physical Control–holding constant the value or level of extraneous variables throughout the
course of an experiment
3. Design Control–use of the experimental design to control extraneous causal factors
4. Statistical Control–adjusting for the effects of confounded variables by statistically adjusting
the value of the dependent variable for each treatment condition
See Practicing Marketing Research, Data Use: The Insidious Top-box and Its Effects on
Measuring Line Share (pg. 251)
Consumers have limited budgets and consumption capacities, so when companies want to make
changes to a product line, managers want to know how those changes would affect the choices
Questions:
1. Do you think top-box measurement would be more accurate if only one option were presented
rather than multiple options? Why or why not?
2. Do you think any marketing research method can accurately predict action rather than simply
assess desire?
7. EXPERIMENTAL DESIGN, TREATMENT, AND EFFECTS
I. Experimental Design
A. Experimental Design Defined–the researcher has control over and manipulates one or more
independent variables
1. Nonexperimental designs–involves no manipulations, often referred to as ex post facto (after
the fact) research–an effect is observed, and then some attempt is made to attribute this effect to
some causal factor
B. Factors of the Experimental Design
2. Subjects–participants in the experiment
3. Dependent variable–measured
4. Plan or procedure–how to deal with extraneous causal factors
C. Explanation
1. Treatment Variable–independent variable is manipulated in an experiment
a. Manipulation–a process in which the researcher sets the levels of the independent variable to
test a particular causal relationship.
b. Experimental Group
2) Test Group–group that is exposed to manipulation (change) of the independent variable
3) Experimental Effect–effect of the treatment variable on the dependent variable. The goal is
8. LIMITATIONS OF EXPERIMENTAL RESEARCH
I. The High Cost of Experiments
McDaniel & Gates – Marketing Research, 9th Edition Instructor’s Manual
A. Time –need to consider whether the costs of doing an experiment would exceed the value of
B. Money
II. Security Issues
A. Competitors–when conducting a field experiment–gives advance notice to competitors and
allows them an opportunity to decide whether and how to respond–the element of surprise is lost
III. Implementation Problems
A. Difficulty Gaining Cooperation–managers reluctant to permit their area to used as a test
market
B. Contamination–buyers from outside the test area come into the area to purchase the product
being tested, thereby distorting the results of the experiment
C. Differences in Test Markets–test markets may be so different and the behavior of consumers
in those markets so different that a relatively small experimental effect is difficult to detect
D. No Appropriate Group for Control Group
9. SELECTED EXPERIMENTAL DESIGNS
I. Pre-Experimental Designs
A. Pre-E-experimental Designs–designs that offer little or no control over extraneous factors
B. One-Shot Case Study Design–pre-experimental design with no pretest observations, no
control groups, and an after-measurement only. Using standard notation, the design is shown as
follows: X O1
1. Two Basic Weakness
2. Lacks internal validity (probably external validity as well)
C. One-Group Pretest-Posttest Design–pre-experimental design with pre- and post-
1. History–threat to the internal validity of this design because an observed change in the
2. Maturation–threat to internal validity because an observed effect might be caused by the fact
3. Only one pretest observation
II. True Experimental Designs
1. Before and After with Control Group Design–true experimental design that involved
random assignment of subjects or test units to experimental and control groups and pre- and
post-measurements of both groups. The design can be shown symbolically as follows:
Experimental Group: (R) O1 X O2
Control Group: (R) O3 O4
The treatment effect is estimated as (O2 – O1) – (O4 – O3).
a. Mortality–problem if units drop out during the study and these units differ systematically
from the ones that remain
b. History–problem in those situations where factors other than the treatment variable affect the
experimental groups but not the control group, or vice versa
2. After-Only with Control Group–true experimental design that involves random assignment
of subject or test units to experimental and control groups, but no pre-measurement of the
dependent variable. This design is presented symbolically as follows: