CHAPTER 4
RESEARCH DESIGN
LEARNING OBJECTIVES
In this chapter you will learn:
4-1 To understand what research design is and why it is important
4-2 To distinguish three major types of research design: exploratory, descriptive, and
causal
4-3 To learn how exploratory research design helps the researcher gain understanding of
a problem
4-4 To know the fundamental questions addressed by descriptive research and the
different types of descriptive research
4-5 To explain what is meant by causal research and to describe types of experimental
research designs
4-6 To know the different types of test marketing and how to select test-market cities
CHAPTER OUTLINE
Research Design
• A master plan that specifies the methods that will be used to collect and analyze
the information needed for a research project
• Why is Knowledge of Research Design Important?
o Knowledge of the needed research design allows advance planning so that the
project may be conducted in tell time and at less cost
Three Types of Research Designs
• Objectives include gaining background information and to develop hypotheses,
measuring the state of a variable of interest, and to test hypotheses that specify the
relationships between variables
Research Design: A Caution
• Not all research is done the same way or in the same order
Exploratory Research
• Unstructured, informal research to gain background information about the
general nature of the problem
• Uses of exploratory research
o Gain background information
o Define terms
o Clarify problems and hypotheses
o Establish research priorities
o Methods of Conducting Exploratory Research
o Secondary data analysis
o Experience surveys
• Key-informant technique
• Lead-user survey
o Case analysis
o Focus groups
Descriptive Research
• Describes answers to questions of who, what, where, when, and how
• Classification of Descriptive Research Studies
o Cross-sectional studies
o Sample surveys
o Longitudinal studies
• Continuous panels
• Discontinuous panels or omnibus panels
Causal Research
• Causality is a condition in which one or more variables affect one or more other
variables.
• Experiments
o Independent variables
o Dependent variables
• Extraneous variables
Experimental Design
• Pretest
• Posttest
• Control group
• Experimental group
• Before-After with Control Group
o A/B testing
How Valid Are Experiments?
Types of Experiments
• Laboratory experiments
• Field experiments
Test Marketing
• Types of Test Markets
o Standard test markets
o Controlled test markets
o Electronic test markets
o Simulated test markets
• Selecting Test Market Cities
o Representativeness
o Degree of isolation
o Ability to control distribution and promotion
• Pros and Cons of Test Marketing
KEY TERMS
Research design Exploratory research
Experience surveys Degree of isolation
Key-Informant Technique Lead-user survey
Case analysis A/B testing
Descriptive research Ability to control distribution and promotion
Cross-sectional studies Sample surveys
Dependent variables Extraneous variables
Experimental design Pretest
Posttest Control group
Experimental group Before-after with control group
Internal Validity External validity
TEACHING SUGGESTIONS
1. (The following will require giving an assignment to students and having them bring to
class their findings.) Although students have not been fully exposed to exploratory
research methods, they can perform three of them to some extent. Divide the class up
into three groups and assign each group one of the following types of exploratory
research: (1) secondary data analysis, (2) experience surveys, and (3) case analysis.
Here are some possible “assignment” situations. It is necessary to keep the situations
local.
• The university is experiencing a decline (increase) in enrollment and wonders
why. (Chose “decline” or “increase” to fit the situation of your institution.
Alternatively, consider using a particular major or a college as it suits your
situation and students’ abilities.)
• The university bookstore is considering going to an “all rent” policy for
textbooks. (Some universities have this policy in place. Note: the experience
survey will pertain to students’ and instructors’ experiences with buying and
selling back textbooks.)
• The director of your “student life” office is concerned about the high use and
possible abuse of credit cards by college students. Under consideration is a self-
help program for students to learn about credit card abuse and money
management.
Have the groups present their findings and use class discussion to generate
hypotheses.
2. Illustrate the result of a cross-sectional descriptive study by presenting findings from
such a study to the class. If you have used team projects in the past, you have many
from which to pick. Alternatively, you may use a consulting report, or even the
descriptive findings from an academic study may suffice. Use the tables and figures
to illustrate the descriptive nature of the findings. For instance, show how the
demographic profile describes the sample. If there is product use information, use it
to show how it describes the types of product usage.
3. Some students may not readily understand the value of a longitudinal panel. An
analogy is that a cross-sectional survey is like a snapshot; whereas a longitudinal
panel is like a video.
4. A tangential discussion can be generated by asking students the implications of
attrition with an omnibus panel versus a longitudinal panel. In the former case, the
lost panel member can be replaced with someone whose profile is identical or very
similar. That is, if a female elderly sole survivor living on the West Coast drops out,
you would replace her with an elderly sole survivor who lives on the West Coast.
However, with a drop out in a longitudinal panel, the string of observations over time
is broken, so attrition is a much more serious problem.
5. You can have fun with causality because humans have the inborn tendency to make
causal attributions without using good experimental design. Here are some examples.
• It has been documented that human births increase as stork populations grow, so
storks bring babies. (Actually, rain causes crops to grow, so agrarian people
expand their families, while water affords more protection for stork nests so more
storks are sighted.)
• Lizards sometimes jump out of fires, so medieval people believed that lizards
were created by fire. (Actually, lizards jump out of logs when lit on fire, because
they hide in logs and the fire forces them to leave their hiding places.)
• The sun revolves around the earth because one can see it rising and setting. It was
believed for centuries that the sun revolved around the earth because of the
fallacious assumption that the earth stood still. (In truth, the sun’s gravity causes
the earth to revolve around it while the earth spins on its axis.)
6. The descriptions of internal and external validity in the chapter do not utilize terms
used in more comprehensive treatments. For internal validity, the terms are history,
maturation, instrumentation, pretest effect, equivalence, and mortality, while for
external validity, they are sample representativeness, artificiality, and generalizability.
Some instructors may want to introduce these terms to students when reviewing the
two types of validity.
7. The two experimental designs without control groups can be used to illustrate the role
of a control group. The after-only and one-group, before-after designs suffer from
internal validity problems, namely extraneous effects and changes in subjects in both
cases, and measure error and subjects guessing for the before-after design. The way
to overcome these problems is to let them happen to both the experimental and the
control group. That is, you must have a control group to “capture” the internal
validity problem, and then subtract it out. The description of the before-after with
control group and the causal statement formula where E = (O2 – O1) – (O4 – O3) shows
how the control groups O3 and O4 do the subtracting out.
8. There are four types of test markets described: standard, controlled, electronic, and
simulated. Ask students who uses each type and why. One factor is the distribution
system of the company wishing a test market; another is the type of product being
tested; a third factor is the type of information desired.
• Standard test markets fit companies with fully developed distribution systems that
are marketing convenience goods such as grocery products, personal care items,
or over-the-counter drugs. This is a “do–it–yourself” approach with the
information need being primarily to track the test product’s sales.
• Controlled test markets fit companies without fully developed distribution
systems. The test market company establishes the retail structure during the test.
Convenience goods marketers are a good fit. There is more competitive sales
information available because all brands are tracked, so the information needs are
more specific to what brand is gaining or losing during the test.
• Electronic test markets are limited to those companies testing products that are
sold in the stores where the electronic tracking system is in place (grocery and
drug stores primarily). Because electronic test market customers are in a
database, the information would profile the buyers versus nonbuyers and provide
market segmentation implications.
• The simulated test market is suited to a new product where the innovation is tried
and the repurchase rate estimated by mathematical model. The distribution
proximity to the university, students may be able to comment on why it was dropped
or added.
10. The representativeness issue is paramount in selecting a test market city. Although
the cities cited in some textbooks (Tulsa, Oklahoma; Charleston, West Virginia;
Lexington-Fayette, Kentucky; Wichita, Kansas; Rockford, Illinois; Omaha, Nebraska;
and Midland, Texas) are well known for being similar, in many respects, to the
“average U.S. city,” they do not fit the target market definitions of all marketers. Ask
students to indicate products, services, or brands whose target markets are not
“average Americans” and who would be ill-advised to use these cities in their market
tests. Here are some examples of product markets that are not comprised of average
Americans:
ACTIVE LEARNING EXERCISE
Learn About Knowledge Networks’ Panel
Want to learn more about panels? Go to Knowledge Networks at
www.knowledgenetworks.com. Read about its panels under “Resources.” Notice that
Knowledge Networks first recruits panel members using probability sampling methods so
as to ensure it has samples that are representative. For panel members who do not have
online access, Knowledge Networks provides computer and Internet access. The ability to
access probability samples (representative samples) using the speed of the Internet is a
competitive advantage for Knowledge Networks.
The following is taken from the Knowledge Networks Resources materials.
One of the core elements of the Knowledge Networks (KN) advantage is our nationwide
online panel, known as KnowledgePanel®. Bringing unprecedented accuracy to your
research through statistical projectability, KnowledgePanel® is the only online panel that
is representative of the entire U.S. population; it is the valid online resource for such
crucial marketing tasks as:
• Estimating true incidence levels
• Sizing markets and opportunities
• Obtaining publishable findings for peer-reviewed journals
By combining true probability sampling and the Internet’s many advantages as a research
medium, KnowledgePanel® incorporates the views and opinions of all Americans and is
not susceptible to the “professional respondent” problem and other hazards of “opt-in”
online panels. Quite simply, this means that KnowledgePanel® provides the highest level
of accuracy and representation available on the web today—a degree of quality essential
to wise marketing and policy spending.
The unique construction of KnowledgePanel® is suitable to a broad range of commercial,
academic and policy research applications. This versatility is reflected in the impressive
list of universities, government agencies and prominent companies that rely on
KnowledgePanel® for their highest priority research projects.
KnowledgePanel® also offers a variety of other benefits, including:
• Access to our Consumer Profile Database, which contains over 7,000 profile data
points about lifestyle, product use, shopping habits, technological sophistication,
and more
• The ability to target specific respondents – through profile data – with greater
accuracy, reducing costly screen-outs and shortening questionnaires
• High in-panel response rates (65%-75%) and low abandon rates (< 2%), providing
high quality results
Knowledge Networks also allows you to access the benefits of KnowledgePanel® for
data-only projects through our Service Bureau and short public opinion surveys through
our KN/QuickView™ omnibus service. And you can use our profile data to create
custom panels, such as those who shop at specific stores or use certain products on a
regular basis.
ANSWERS TO END-OF-CHAPTER QUESTIONS
1. What is research design?
2. Explain why it is important for marketing researchers to be knowledgeable of
research design.
3. Discuss how research design can lead to ethically sensitive situations.
In the marketing research industry, the following are among the potential ways that
researchers can take advantage of clients in the research design process:
• Recommending a more costly design than is needed
4. Provide an example of exploratory research.
Exploratory research is unstructured, informal research that is undertaken to gain
background information about the general nature of the research problem. It is
Copyright © 2017 Pearson Education, Inc.
better.
Projective techniques Giving students a sheet of paper and asking
them to complete the statement, “This food
service center would be a better place if…”
5. In which type of research design would the key-informant technique be used?
6. What is the difference between longitudinal studies and cross-sectional studies?
7. In what situation would a continuous panel be more suitable than a discontinuous
panel? Under what situation would a discontinuous panel be more suitable than a
continuous panel?
Continuous panels ask panel members the same questions on each panel
8. What type of panel is an omnibus panel?
9. Explain why studies of the “if–then” variety are considered to be causal studies.
Copyright © 2017 Pearson Education, Inc.
Causality may be thought of as understanding a phenomenon in terms of conditional
statements of the form “If X, then Y.” These “if–then” statements, then, become ways
of manipulating independent variables of interest.
10. Define each of the following types of variables and give an example of each in an
experiment designed to determine the effects of an advertising campaign:
independent, dependent, extraneous, control group, and experimental group.
Independent variables are those that the researcher can control and wishes to
11. Explain the two types of validity in experimentation and also explain why different
types of experiments are better suited for addressing one type of validity versus
another.
An experiment is valid if (a) the observed change in the dependent variable is, in fact,
due to the independent variable, and (b) if the results of the experiment apply to the
“real world,” outside the experimental setting. Two forms of validity are used to
12. Distinguish among the various types of test marketing.
Test marketing is the phrase commonly used to indicate an experiment, study, or test
that is conducted in a field setting. The chapter classifies test markets into four types:
Copyright © 2017 Pearson Education, Inc.
standard, controlled, electronic, and simulated. The standard test market is one in
which the firm tests the product and/or marketing mix variables through the
company’s normal distribution channels. Controlled test markets are conducted by
outside research firms who guarantee distribution of the product through pre-specified
types and numbers of distributors. Electronic test markets are those in which a panel
of consumers has agreed to carry an identification card that they present when buying
goods and services. Simulated test markets (STMs) are those in which some limited
amount of data on consumer response to a new product are fed into a model
containing certain assumptions regarding planned marketing programs, which
generates likely product sales volume. There are also consumer versus industrial test
markets. In consumer tests, multiple versions of a more-or-less finished product are
tested by consumers. In industrial test markets, the key technology is presented to
selected industrial users who offer feedback on desired features and product
performance levels.
13. Think of a past job that you have held. List three areas in which you, or some other
person in the organization, could have benefited by having information generated by
research. What would be the most appropriate research design for each of the three
areas of research you have listed?
14. Can you identify research problems that might be addressed through a search of
social media websites? What type of research design would you recommend for these
problems?
15. Design an experiment. Select an independent variable and a dependent variable.
What are some possible extraneous variables that may cause problems? Explain how
you would control for the effects these variables may have on your dependent
variable. Is your experiment a valid one?
16. The Maximum Company has invented an extra-strength instant coffee brand to be
called “Max-Gaff,” and positioned to be stronger tasting than any competing brands.
Design a taste-test experiment that compares Max-Gaff to the two leading instant
coffee brands to determine which brand consumers consider to taste the strongest.
Identify and diagram your experiment. Indicate how the experiment is to be
conducted, and assess the internal and external validity of your experiment.
A taste test is an experiment, so students must demonstrate comprehension of
experimental design concepts.
Internal validity is addressed in the following ways.
External validity is an issue as the taste test is conducted in an artificial setting
(perhaps at a shopping mall or some other facility), and the subjects are told
that they are to rate the taste which means that they will be especially vigilant
Copyright © 2017 Pearson Education, Inc.
to the taste. Also, the long-term taste aspects of the coffees are not tested, so
if Max-Gaff “stays with” the taster, this factor is not captured by the design.
Of course, the sample should be representative of coffee drinkers.
17. Artia Hunt is the CEO of a successful chain of coffee shops in the Midwest.
Ms. Hunt would like to add a small selection of pastries to the current food
offerings at her coffee shops. You have been hired to conduct an exploratory
study using social media sources to develop an initial list of the types of
pastries that should be considered for this new initiative. Based on Marketing
Research Insight, outline the steps you will use to conduct this project and
present the results of your research to Ms. Hunt.
Students have been asked to conduct a search of secondary data from social
18. Coca-Cola® markets PowerAde® as a sports drink that competes with Gatorade®.
Competition for sports drinks is fierce where they are sold in the coolers of
convenience stores. Coca-Cola is thinking about using a special holder that fits in a
standard cooler but moves PowerAde to eye level and makes it more conspicuous
than Gatorade. Design an experiment that determines whether the special holder
increases the sales of PowerAde in convenience stores. Identify and diagram your
experiment. Indicate how the experiment is to be conducted, and assess the internal
and external validity of your experiment.
There is one treatment (the cooler holder for PowerAde) and one control
situation (the current stocking of PowerAde without the cooler holder). A
Copyright © 2017 Pearson Education, Inc.
19. SplitScreen is a marketing research company that tests television advertisements.
SplitScreen has an agreement with a cable television company in a medium-sized city
in Iowa. The cable company can send up to four different television ads
simultaneously to different households. SplitScreen also has agreements with the
three largest grocery store chains, which will provide scanner data to SplitScreen.
About 25 percent of the residents have SplitScreen scan cards that are scanned when
items are bought at the grocery store and wthat allows SplitScreen to identify who
bought which grocery products. For allowing SplitScreen access to their television
hookups and their grocery purchase information, residents receive bonus points that
can be used to buy products in a special points catalog. Identify and diagram an
experimental designs possible using the SplitScreen system. Assess the internal and
external validity of SplitScreen’s system.
Virtually any true experimental design is possible with the Split Screen system, but
the most powerful ones would be before-after with control group or after-only design.
period of unusual shopping because they feel “watched,” the artificiality will be
minimized. The medium-sized city in Iowa is consistent with good market test
selection criteria.
CASE SOLUTIONS
Case 4.1 Memos from a Researcher
Case Objective
This case presents students with three different situations where they must determine the
appropriate research design.
Answers to Case Questions
1. Describe what research design you would recommend for each of client.
2. For each research design you selected for the three clients, discuss why you believe
your choice of design is the correct choice.
The answers are provided in the table below.