Chapter 19
Evaluating IMC Effectiveness
CHAPTER CONTENT
CHAPTER KEY POINTS
1. Why is it important to evaluate brand communication effectiveness?
2. What role do campaign objectives play in the measurement of campaign?
3. What are the key ways in which campaign evaluation is conducted?
4. What are some key challenges faced in evaluating IMC effectiveness?
CHAPTER OVERVIEW
Throughout this text, it has been emphasized that effective campaigns do more than win
awards for creativity. They also work hard to achieve the campaign’s communication and
marketing goals. This chapter discusses the establishment of campaign objectives against
which campaign success can be evaluated and examines the many types of effectiveness
measurement that are.
CHAPTER OUTLINE
BRAND COMMMUNICATION IMPACT: DID IT WORK?
What makes an IMC campaign effective? Is it the fact that people like it or that it moves
people to some kind of action? Or is it something else altogether?
First Things First: The Campaign Objectives
Typically, a brand communication campaign has multiple objectives. It is critical
that the stated objectives be established up front because they provide the
all-important framework for evaluating if a campaign is a success. This is true for
social media, also.
The Campaign Purpose: Brand Building
Many executives feel advertising works only if it produces sales. One researcher,
Professor John Phillip Jones, contends that “advertising must generate an
immediate jolt to sales before it can be expected to produce any further effect.”
Simon Broadbent, another leading figure in effectiveness research and a former
proponent of sales impact, came to realize that ‘long and deep’ effects of
advertising, or its brand building effects, also are important. Many other experts
agree with this view, and it is the perspective supported by this text.
First, determining effectiveness based on sales can be difficult because of the
impact of other environmental factors. Second, sales are not the only reason
1
brands advertise. One of the major objectives of advertising is to create higher
levels of brand awareness among consumers.
Because campaigns often have multiple objectives and because there are so many
ways of measurement available in the IMC tool kit, there are multiple ways to
assess the effectiveness of marketing communication campaigns.
Why Evaluation Matters
Some evaluation of brand communication is informal and based on the judgment
of an experienced manager. There is always a need for the intuitive analysis of
experienced professionals. The important thing, however, is to recognize very
early in the campaign planning stages the need for multiple, formal evaluation
mechanisms.
Structured evaluation not only determines the success of a campaign from an
objective perspective but also provides valuable feedback as brands plan
campaigns for the future. The A Matter of Principle feature in this chapter builds
on the idea of a cycle beginning and ending with research.
From the business perspective, formal evaluation of brand communication is a
must. The first reason is because the stakes in making an advertising misstep are
high. The second reason is because advertising optimization, i.e., reducing risk by
testing, analyzing, tracking performance, and making changes where possible to
increase the effectiveness of the communication, ensures success.
How Evaluation Fits into the Stages of Brand Communication Testing
A complete understanding of the strength of your brand communication is
accomplished through testing, monitoring and measurement. Ideally the result of
preliminary testing should be available before large sums of money are invested
in finished work or in media buys. In this chapter, our focus is upon measurement
– how we evaluate actual effects and results after a campaign effort has been
completed.
EVALUATING THE IMC MESSAGE
Questions about impact are critical and must be addressed. If they are not, there
is no proof that a campaign worked, so companies may be tempted to make the
mistake of cutting communication spending that is driving their business.
Since any attempt to measure the impact of brand messaging by looking only at
sales numbers poses challenges, the impact of brand messaging is typically
measured in terms of its communication effects – the mental responses to a
message – which become surrogate measures for sales impact. Positive
post-campaign changes in measures such brand awareness, knowledge of what the
brand offers, liking of the brand, and intent to purchase all suggest that the
2
advertising message or some other form of brand communication is making a
positive contribution to an eventual purchase decision.
When campaigns don’t work, a very important role of post-campaign message
testing involves understanding what went wrong. Some messages confuse the
audience,while others fail to get consumers’ attention or fail to resonate with
them. In these situations, campaign message evaluation must bring these issues to
light. Solid campaign evaluation measurement tells us precisely what a campaign
has and has not accomplished.
Table 19.1 in the text clusters key message effectiveness measures and then
matches them to the types of research questions that are used to determine
effectiveness.
Experts in Message Evaluation
Many research companies, in addition to some large agency research departments,
specialize in measuring the various dimensions of effectiveness described in
Table. 19.1. The most successful of these companies have conducted so many
tests that they have developed norms for common product and service categories.
Norms allow the brand agency managers to determine whether a particular
campaign message has performed above or below the category average in terms
of “moving the needle” on items such as those listed in Table 19.1.
Most large measurement companies also have developed diagnostic methods that
identify the strong and weak aspects of an IMC campaign. This is important
because understanding what worked and what did not work is what makes us able
to make campaigns better over time.
The most prominent communication evaluation companies are: Ameritest, Ipsos,
Millward Brown, TNS Global, GfK, and Sapient. Also, major online companies
such as Google provide myriad tools for campaign evaluation, from
straightforward analytic tools that are relatively easy to use to more complex tools
that allow marketers to create experiments.
Message Evaluation Techniques
Next we will examine the most common research techniques used. Different types of
measurement are required because brand managers are likely to set different objectives
for different types of campaign messages.
Tracking Studies
Communication tracking studies are conducted from the time a campaign is
launched until after it has concluded and involve the collection of information
from random samples of consumers who live in markets where they were exposed
to a campaign. Research questions that focus on recall, recognition,
comprehension, and relevance are staples of communication tracking surveys.
3
Tracking surveys also include specific questions about how consumers perceive
the personality of the brand, self-identification with the brand, purchase intent,
and purchase rate. If a campaign did not cause consumers to feel increased
affinity with the brand and/or increase their likelihood of buying it, IMC
campaign effectiveness is certain to come under question. At its most basic level,
a tracker includes the components shown in Figure 19.2 of the textbook.
Scanner Analysis
Another common component of posttest evaluation is scanner research. Many
retailers use electronic scanner data to tally up purchase data and collect consumer
buying information. Scanner research is also used to see what type of sales spikes
are created when certain ads and promotions are used in a given market.
Sometimes retailers will also establish a consumer panel so it can track sales
among various consumer groups. In this instance, a consumer would be asked to
join a panel, which might contain hundreds of other customers. The panel
questionnaire will include questions regarding media used by each reporting panel
member, so media performance can also be evaluated.
Single Source Data
Using scanner data and the cooperation of local cable networks, researchers are
closer to showing a causal relationship between IMC programs and sales because
of single source research.Single-source research companies, such as A.C.
Neilson, arrange to have test commercials delivered to a select group of
households within a market to compare changes in behavior to a control group of
households.
The purchasing behavior of each group of householders is collected by scanners
in local stores. Because brand communication is the only manipulated variable,
this method permits a fairly clear reading of cause and effect. Data collected in
this way is called single-source databecause brand communication exposure and
brand purchasing data come from the same household source.
Although fairly expensive, single-source research can produce dependable results.
However, a drawback is that single-source research is better for short-term
immediate sales effects and does not capture very well other brand-building
effects.
Memory Tests
Memory tests are based on the assumption that a brand communication leaves a
mental residue with the person who has been exposed to it or, in other words, the
audience has learned something. One way to measure an advertisement’s
effectiveness, then, is to contact consumers who saw the ad and find out what they
remember. Memory tests fall into two major groups: recognition tests and recall
tests.
4
One way to measure memory is to show a magazine advertisement, for example,
to people and ask them whether they remember having seen it before. This kind of
test is called a recognitiontest. In a recall test, respondents who have read the
magazine are asked to report what they remember from the ad about the brand. In
an aided recall test, the interviewer may go through a deck of cards containing
brand names. Obviously, recall is a more rigorous test than a recognition test.
Similarly, a TV commercial is run on network television within a regular
prime-time program. The next evening, interviewers make thousands of random
phone calls until they have contacted about 200 people who were watching the
program at the exact time the commercial appeared. The interviewer then asks a
series of questions, such as the following:
Do you remember seeing a commercial for any charcoal briquettes?
If no, then, do you remember seeing a commercial for Kingsford Charcoal
briquettes? (Memory prompt)
If yes to either of the previous questions, then, what did the commercial say
about the product? What did the commercial show? What did the commercial
look like? What ideas were brought out?
The first type of question is called unaided recall because the particular brand is
not mentioned. The second question is an example of aided recall, in which the
specific brand name is mentioned. The answers to the third set of questions are
written down verbatim.
If the test requires the respondent to link a specific brand name, or at least a
specific product category, to a specific commercial, a brand linkage test is being
performed. This method was used to measure campaign impact on consumers in
this chapter’s opening case.
Inquiry Tests
A form of action response, inquiry tests measure the number of responses to an
advertisement or other form of brand communication. The response can be a call
to a toll-free number, an e-mail, a website visit, a coupon return, a visit to a dealer,
an entry in a contest, or a call to a salesperson. Inquiry tests are the primary
measurement tool for direct-response communication, and they are also used to
evaluate advertisements and sales promotions when the inquiry is built into the
message design. Inquiry tests are also used to evaluate the effectiveness of
alternative advertisements using a split-run technique in magazines, where there
are two versions of a magazine printed, one with ad A and the other with ad B.
Evaluating the Performance of Various IMC Tools
IMC synergy exists when all campaign components work together to create a
solid and understandable brand meaning. Before overarching campaign synergy
can be measured, evaluation is usually conducted on a tool-by-tool basis. Specific
5
objectives must be set for every IMC tool used in the mix and the effectiveness of
each tool must be measured.
Advertising may be the most visible IMC tool, but other brand communication
tools may be better at achieving the objective of getting people to respond with an
immediate purchase. Most IMC tools have their own metrics through which their
performance is measured at a campaign’s close. Below we explain in more detail
how a few IMC tools are measured in terms of their effectiveness.
Advertising
Table 19.2 in the text shows that advertising has the potential to be particularly
effective in accomplishing a number of objectives. The most common
post-testing or campaign evaluation technique used to evaluate advertising is the
tracking study, which we previously discussed. Although communication tracking
studies represent very large investments on the part of marketers, given the cost of
creating advertising, thorough campaign evaluation represents money well spent.
Public Relations
Public relations typically track the impact of a public relations campaign in terms
of successful outpute.g., how many news release led to stories or mentions in
news stories) and outcome, e.g., attitudinal or behavioral change due to the
impact of materials produced.
The search for methods to tie public relations activities to bottom line business
measures, such as return on investment (ROI) is like the quest for the Holy Grail.
A surrogate ROI measure can be based on shareholder value, which can be seen
as a company or brand’s reputation capital.
Although some still argue that not all the value of public relations programs in
measurable, others claim that it is possible to increase the use of metrics to
determine the impact a public relations campaign is having on consumer
engagement – especially in the digital arena.
Consumer, Trade, and Point-of-Purchase Promotions
Sales promotion managers for packaged goods and other products that use
distribution channels need to evaluate both the impact of consumer (or end user)
directed promotions and promotions targeted at retailers and other channel
members. At the most basic level of evaluation, managers require proof of
execution. One responsibility of the sales force is to do store checks to verify that
stores are doing what they promised. At the most important level of promotion
evaluation, however, the behaviors and types of involvement that promotions are
designed to stimulate must be measured.
Promotions that contain a response device, such as coupons, have a built-in
evaluation measure. Beyond response and redemption rates, however, brand
managers often also measure consumer awareness of promotions, sales force
6
participation in promotions, and appropriate forms of online consumer reaction.
What is measured depends on promotional objectives.
The efficiency of a sales promotion can be evaluated in terms of its financial
returns more easily than can the impact of advertising. We compare the costs of
the promotion, called a payout analysis, to the forecasted sales generated by the
promotion. A break-even analysis seeks to determine the point at which the total
cost of the promotion exceeds the total revenues, identifying the point where the
promotion was not productive. Figure 19.3 depicts this analysis.
Direct Marketing
The primary objective of direct marketing communication is to drive a transaction
or generate some other type of immediate behavioral response, such as a donation
or visit to a dealer. What makes this marketing communication tool so attractive
to marketers is that response is so easily measurable. In many ways,
direct-marketing mechanisms are the easiest IMC tools to evaluate both in terms
of the message efficiency and in terms of return on marketing investment.
Evaluation of Digital IMC Components
Many IMC efforts are conducted online. Some performance indicators are website
traffic volume, such as page views or the simple number of visitors to a site.
Banner advertising and other online ads are evaluated using click-through
rates.One thing the industry has learned is that this form of advertising is
decreasing in effectiveness. Pop-up banners in particular can get more attention,
but they are also seen as more irritating.
Instead of click-through rates, some advertising uses a cost-per-lead metric that
records how well the click-through generates prospects, an attempt to get at ROI.
A more important metric, however, is conversion rate, which is the percentage of
visitors to a site who complete a desired action, such as playing a game, signing
up for a newsletter, or buying something. Of course, online sales are an important
measure of digital effectiveness.
The most important thing to understand in considering evaluation of online efforts
is that while websites used to be the primary digital properties for connecting with
targets and the place where the most digital interaction was happening, that is no
longer true. Today, digital communication is multichannel and multiplatform.
What this means is that in addition to evaluating digital efforts through looking at
conversion rates and sales, experts are now evaluating digital campaign
components in terms of their ability to generate awareness and consideration.
These are measures that used to be applied only to traditional media.
7