CHAPTER 15
Measuring the Effectiveness of Brand
Promotions
PPT 15-1
KEY TERMS
promotion research
reliability
validity
trustworthiness
meaningfulness
account planning
naturalistic inquiry
normative test scores
resonance test
communication test
dummy advertising
vehicles
theater tests
thought listing
attitude change study
physiological measures
eye-tracking systems
psychogalvanometer
voice response analysis
pilot testing
split-cable transmission
split-run distribution
split-list experiments
post-test message
tracking
recall test
recognition tests
log analysis software
inquiry/direct-response
measures
ballot method
single-source tracking
measures
SUMMARY
PPT 15-2
LO1 Discuss issues that shape the evaluation of brand promotion.
First among the issues to consider when exploring the measurement of promotion’s
effectiveness is the scope of promotion research. It is a specialized form of marketing
LO2 Describe how marketers measure the effectiveness of advertising.
Advertising is measured more than other promotional tools because it is conspicuous and
expensive to prepare. It is measured in three main ways: (1) Pretest evaluations use
Chapter 15: Measuring the Effectiveness of Brand Promotions ❖ 2
campaign launches. (2) Marketplace pilot testing uses split-cable transmission for TV
LO3 Identify measures of effectiveness for Internet advertising, direct marketing,
sales promotion, point of purchase, sponsorships, public relations, and corporate
advertising.
Internet advertising is judged by counting visits to websites, hits on banner ads, unique
visitors, and impressions formed. Direct marketing is relatively easy to evaluate by
measuring the responses (sales or inquiries) that are the primary intention for the
LO4 Explain how sales managers evaluate salespeople and the personal-selling
effort.
Measuring the performance of salespeople draws directly on the objectives set for the
personal-selling process and sales objectives. Sales staff can be judged on several
LO5 Describe the process of evaluating the effectiveness of IMC campaigns.
One approach to measuring the effectiveness of the overall IMC program is to measure
each of the promotional tools used in a campaign as if it were independent of the others.
This fragmented approach fails to account for the synergies that are the hallmark of IMC
CHAPTER OUTLINE
INTRODUCTORY SCENARIO: Research and the Campaign for Real Beauty
The story of the Dove’s Campaign for Real Beauty and its initial advertising campaign is
an excellent demonstration of research in the IMC process. Students need to appreciate
Chapter 15: Measuring the Effectiveness of Brand Promotions ❖ 4
• Meaningfulness signifies what (if anything) a piece of research really
means. Here, the difference between the statistical significance of a
finding and the practical significance of a finding must be considered.
A. Research Systems
Account planning is a big story in the industry, so it’s important to compare its role
with traditional promotion research.
• Account planning is a system in which an agency assigns a coequal account
involvement.
o This organizational structure puts research and measurement in a different,
more prominent role.
o “Planning agencies” tend to do more qualitative and naturalistic research
than their more traditional counterparts, which use quantitative techniques
and models to measure effectiveness. Naturalistic inquiry is a broad-
based research approach that relies on data collection methods that are
more qualitative than quantitative and that use video and audio recordings
and photography in an effort to investigate an issue more holistically.
B. Issues in Message Evaluation
At the heart of any successful promotional campaign is the message that will be used
to engage a target audience and give audience members a reason to believe in the
brand. Given the image effects and the meaning consumers take from messages,
much of the research conducted by agencies and their clients involves testing
messages.
1. Motives and Expectations
What is it that marketers and agencies want out of their message tests?
2. Dimensions of Message Assessment
Marketers judge the effectiveness of promotional messages against many
standards. The following are used most often:
Chapter 15: Measuring the Effectiveness of Brand Promotions ❖ 5
• Imparting knowledge about specific claims and dimensions of the brand’s
image—For this, marketers use tests of recall and recognition.
• Shaping attitudes—Message-testing research is frequently structured
around questions related to attitudes.
audience.
II. Effectiveness of Advertisements
PPT 15-5, 15-6, 15-7, 15-8
Advertising has been and continues to be the subject of more measurement effort than
any other promotional tool, even on a global scale.
A. Pretest Evaluations
1. Communication Tests. A communication test seeks to see if a message is
communicating something close to what is desired. These tests are usually
done in a group setting, with data coming from a combination of pencil-and-
paper questionnaires and group discussion.
2. Magazine Dummies. Dummy advertising vehicles are mock-ups of
3. Theater Tests. For theater tests, advertisements are tested in small theaters,
usually set up in or near shopping malls. Members of the theater audience
have an electronic device through which they can express how much they like
or dislike the advertisements shown.
4. Thought Listings. A thought listing (also referred to as cognitive response
analysis) is message research that tries to identify specific thoughts that may
be generated by an ad. The researcher is interested in the thoughts that a
finished or near-finished ad generated in the mind of the consumer.
5. Attitude Change Studies. Typically, an attitude change study uses a before-
and-after ad exposure design. People from the target market are recruited, and
6. Physiological Measures. Several message pretests use physiological
measurement devices. Physiological measures detect how consumers react to
messages, based on physical responses.
Chapter 15: Measuring the Effectiveness of Brand Promotions ❖ 6
• Eye-tracking systems monitor eye movements across print ads.
• A psychogalvanometer measures galvanic skin response (GSR), a
measure of minute changes in perspiration, which suggest arousal related
to some stimulus—in this case, an advertisement.
• Brain wave tracking technology is being applied to measure how effective
banner ads are in causing an emotional response in Web users and how
that response might translate into click-throughs and brand recall.
• Voice response analysis is a high-tech research procedure in which
inflections in the voice when discussing an ad are interpreted as indicating
excitement and other physiological states.
B. Pilot Testing
• Pursuing message evaluation with experimentation in the marketplace is
known as pilot testing. This extended testing can be of the do-it-yourself
variety, or it can be accomplished with a commercial service provider.
• Three types of testing are most common:
o Split-cable transmission allows testing of two different versions of an
advertisement through direct transmission to two separate samples of
o Split-run distribution uses the same technique as split-cable transmission
except the print medium is used. Two different versions of the same
advertisement are placed in every other copy of a magazine.
o Split-list experiments test the effectiveness of various aspects of direct-
• The advantage of all the pilot-testing methods is the natural setting within
which the test takes place. A major disadvantage is that competitive or other
environmental influences in the market cannot be controlled and may affect
the performance of an advertisement without being detected by the researcher.
C. Post-Testing
Post-test message tracking assesses the performance of advertisements during or
after the launch of an advertising campaign. There are several common measures of
1. Recall Testing. The most common method of advertising research is the recall
test, which aims to see how much, if anything, the viewer of an ad remembers of
the message. Recall is used in the testing of print, television, and radio
advertising.
2. Recognition Testing. Recognition tests ask magazine readers and television
viewers whether they remember having seen particular advertisements and
whether they can name the company sponsoring the ad.
Chapter 15: Measuring the Effectiveness of Brand Promotions ❖ 9
campaign against another based on who responded, what region of the country
they came from, and how quickly they responded.
• The data-rich nature of direct marketing and e-commerce also allows for a
campaigns.
C. Sales Promotion and Point of Purchase
Marketers can evaluate sales promotion through pretesting and post-testing using
techniques similar to those used for advertising.
1. Pretesting Sales Promotion. The options for pretesting sales promotion vary
depending on whether the efforts are aimed at consumers or the trade channel.
a. Consumer Sales Promotion
• When a sales promotion is aimed at consumers, marketers pretest it by
measuring the consumer’s perception of the value of a promotion.
to rank their preferences and mail the ballot back to the firm).
b. Sales Promotion in the Trade Channel
• The marketer needs to know if the trade will support a proposed consumer
sales promotion.
• The best approach for pretesting consumer promotions with the trade is
meeting with trade channel managers and working out the details of the
execution with them.
• A survey of key retail partners, asking what they perceive as an energizing
and high-value promotion, will provide important feedback.
2. Post-Testing Sales Promotion
• In the broadest sense, changes in sales can be used with any sales
promotion tool as a measure of effectiveness.
• With contests and sweepstakes, another measure will be the number of
consumer entries.
• With coupons and rebates, the number of coupons redeemed and rebate
forms returned are additional quantitative measures.
3. Point of Purchase
Since the context for point-of-purchase materials offers little opportunity for
“message” execution per se, a legitimate and appropriate measure of effectiveness
is the change in sales.
Chapter 15: Measuring the Effectiveness of Brand Promotions ❖ 10
D. Sponsorship and Supportive Communications
• For sponsorship of an event, the number of people attending the event is easily
obtained.
• If the event is televised, then the number of viewers of the event gives a proxy
measure of exposure to the brand.
interviews with attendees, and offer economic impact studies.
o Others try to identify the event’s impact on the image of the firm.
o Some have tried to calculate the effects on sales.
• Supportive communications should rely on recall of the messages that appear in
the major support media (e.g., billboards, transit, and aerial ads).
E. PR and Corporate Advertising
• Since the primary objectives of public relations and corporate advertising involve
long-term image effects, trying to measure the effect of such programs at any one
time seems pointless.
• However, one easy measure of the effects of public relations is to count the
would be to measure changes in awareness of or attitude toward the firm or the
brand.
IV. Effectiveness of Personal Selling
PPT 15-11, 15-12
• Objective and subjective criteria are defined in Text Exhibit 15.2.
• Objective criteria include the total dollar volume of sales, order size, and expenses
generated.
o A salesperson or an entire sales force might be rated highly on total dollar
volume at the expense of many, repeated small orders at great cost to the firm.
• Subjective criteria can be difficult to define and measure accurately.
o Management must be flexible and allow individual styles to manifest
themselves.
Chapter 15: Measuring the Effectiveness of Brand Promotions ❖ 11
o Criteria related to team relationships are difficult to measure as well.
o The measurement of effectiveness is a primary source of information for an
internal situation analysis.
V. Effectiveness of the IMC Program
PPT 15-13
• Measuring each promotional tool used in a campaign, weighing it independently,
is a fragmented process that makes no attempt to account for the synergies that are
the hallmark of IMC.
• The purpose of creating and executing an integrated campaign is to enhance the
impact of the communication, so marketers should measure the full impact of that
effort. But the limitations of measurement methodologies make that task difficult.
• Despite the methodological challenges, there are some recommendations on how
to proceed. One of these is to use single-source tracking measures, which
provide information from individual households about brand purchases, coupon
use, and television advertising exposure by combining grocery store scanner data
• The main problem with these measures is that it is impossible to determine what
aspects of advertising had positive effects on consumers. And while such a system
is sophisticated, it really focuses on just two promotional tools: television
advertising and couponing.
• Another approach, offered by Helen Katz and Jacques Lendrevie, segments total
exposures of an integrated program. This method assumes that, regardless of the
promotional tools used, communication will occur in up to three ways:
o Media exposure—all exposures to a brand that take place through traditional
media
o Product (brand) impressions, including in-store and in-home contact with the
product, as well as contact with the product on the street or when consumers
are visiting friends and acquaintances or in the workplace
pharmacist, or hair stylist, any of whom can affect brand knowledge and
awareness
• Overall, the challenge of measuring the effectiveness of the total IMC effort is
extremely difficult. Communication is a complex process that varies across times
and places and from individual to individual.
Chapter 15: Measuring the Effectiveness of Brand Promotions ❖ 12
SOLUTIONS TO REVIEW QUESTIONS
1. Identify issues that could become sources of conflict between marketers and agency
personnel in the message-pretesting process. What could go wrong if people in an agency
take the position that whatever the client wants, the client gets?
Message testing can be a function of logic and adaptive decision making, or it may be
driven by custom and history. There is no one right way to test messages that are used
in a campaign, so one can find conflicting advice about how to execute this type of
2. Explain the key distinction between pretesting and pilot testing. What makes direct-
mail promotion so amenable to pilot testing? Does the ease of pilot testing with direct
mail make mass-media advertising less valuable by comparison? Why or why not?
Pretesting a message involves gauging consumer reaction before promotional
3. Identify the key differences between an attitude change study being conducted for
pretesting a promotional message and an attitude change study for post-testing a
message.
When an attitude change study is conducted for pretesting, people from the target
Chapter 15: Measuring the Effectiveness of Brand Promotions ❖ 13
4. How would you explain the finding that ads that achieve high recall scores don’t
always turn out to do a good job in generating sales? Might some features of ads make
them memorable but also turn off consumers and dissuade them from buying the brand?
Give an example from your experience.
High recall scores don’t always turn out signal that ads will do a good job of
5. How do marketers test the effectiveness of sponsorship programs?
6. Identify and explain the different factors used to measure Internet advertising. What
do these factors measure? Is it effectiveness?
The different measurement factors are hits, click-throughs, page views, visits, and
7. If you were a marketing manager, would you insist that your organization measure
the effectiveness of the firm’s public relations and corporate advertising programs? Why
or why not?
8. Discuss the methods that have been proposed for measuring the overall effectiveness
of an integrated marketing communication (IMC) program. Do you think these are
sufficient? Would they produce an accurate measure of the impact of an IMC program?
One approach is to merely take on the measurement of each of the promotional tools
used in a campaign, try to measure the effectiveness of each, and consider their total
Chapter 15: Measuring the Effectiveness of Brand Promotions ❖ 14
proposed by practitioners suggests measuring media exposures, product impressions,
SOLUTIONS TO EXPERIENTIAL EXERCISES
1. Describe a promotion that you believe truly affected your attitude toward a brand and
ultimately your purchase behavior. Find out if anyone else in the class was affected the
same way.
Answers will vary. Look for an understanding of the various promotional tools and
2. This chapter discussed research methods that help managers avoid costly mistakes.
Find an example of a print ad or describe a direct-mail piece you feel is truly awful.
Explain your objections. Now imagine that you have the opportunity to conduct research
before the ad actually runs. What type or types of research would you recommend that
management conduct? Explain your recommendations, and describe how the research
might have changed the ad you selected.
Answers will vary. Students will enjoy the chance to criticize promotional messages
3. In the summer of 2000, a company called Digital Convergence launched the :CueCat,
a mouselike scanner that would let consumers read bar codes in print ads and be whisked
directly to advertisers’ websites. For consumers at a computer, the :CueCat was billed as
a simpler alternative to typing in URLs. The clearer benefit for advertisers was that data
from :CueCat usage could provide information about ad readership and responses. Each
:CueCat device had a unique number, so dossiers could be compiled on their owners.
Go online to research the fate of the :CueCat. Prepare a one-page paper that
summarizes what you learned and discusses whether you think the demise of this device
was a loss to marketers. How else might they gather information similar to what the
:CueCat would have provided?
Today, cell phones offer the ability to scan bar codes, for example, to make purchases.
How might marketers use this technology to measure the effectiveness of their brand
promotion campaigns?
Chapter 15: Measuring the Effectiveness of Brand Promotions ❖ 15
The :CueCat was ridiculed by reviewers after its launch, and the product was
4. A “search voyeur,” such as Dogpile’s SearchSpy (http://www.dogpile.com) and
Google Zeitgeist (http://www.google.com/intl/en/press/zeitgeist/index.html), offers a
snapshot of queries that other visitors to the site are performing. (Some search voyeurs
will censor certain offensive terms.)
Using SearchSpy, Google Zeitgeist, or another search voyeur, discover and list some
of the queries being submitted. Write a brief report that addresses the following
questions: What does this sampling suggest about what other people are using the Web
for? How often do you see typographical errors or poor search terms? How might
marketers use this information? What are its limitations as a measurement of brand
promotion effectiveness? (Consider, for example, the basic criteria for measuring
research effectiveness: reliability, validity, trustworthiness, and meaningfulness.)
A surprising number of Web surfers are looking for smut, and all too often, doing a
poor job of it, with misspellings and poorly chosen keywords. Given that they’re
POWERPOINT PRESENTATIONS
Use the Instructor PowerPoint files to pace your instruction and provide class notes on
key ideas and themes. Each presentation provides a slide-by-slide coordination with the
chapter’s learning outcomes, definitions, and visuals. Encourage students to use the
accompanying Student PowerPoint presentation to align and reinforce classroom
instruction with studying outside of the classroom.
VIDEOS
To view the two videos for this chapter, go to the PROMO book companion website,
www.cengage.com/login.
(*) Indicates the correct answer in the multiple-choice video questions.
Starling
1. The Starling product as seen in the website would allow measurement in which ways?
Chapter 15: Measuring the Effectiveness of Brand Promotions ❖ 16
2. The Starling product as seen in the website is which of the following?
3. The Starling product as seen in the website has which tag line?
4. The Starling product is likely to offer the following measurement capabilities:
5. In the Starling screen shot shown on Creativity online, what device showcases the
product?
Wolfram Alpha
1. Wolfram Alpha was the winner of what award in 2010?
2. What kind of product is Wolfram Alpha?
