Marketing Chapter 17 Evaluating Imc Effectiveness Evaluating Imc Effectiveness Content Key Objectives

Unlock access to all the studying documents.

View Full Document

Chapter 17: Evaluating IMC Effectiveness

Chapter 17

Evaluating IMC Effectiveness

CHAPTER CONTENT

KEY OBJECTIVES

1. Explain why it is important to evaluate brand communication effectiveness.

2. Discuss the role campaign objectives play in the measurement of campaign success.

3. Describe the key ways in which campaign evaluation is conducted.

5. List some of the key challenges faced in evaluating IMC effectiveness.

6. Detail ways to determine if a campaign met its objectives.

CHAPTER OVERVIEW

Throughout this text, it has been emphasized that effective campaigns do more than win

awards for creativity. Their singular and number one purpose is to achieve the campaign’s

communication and marketing goals. You will see in this chapter that there are many ways

to evaluate the effectiveness of an integrated marketing communication program.

Companies evaluate the impact of the program’s message, the performance of its

CHAPTER OUTLINE

BRAND COMMMUNICATION IMPACT: DID IT WORK?

What makes an IMC campaign effective? Is it the fact that people like it or that it moves

people to some kind of action? Or is it something else altogether?

First Things First: The Campaign Objectives

 Typically, a brand communication campaign has multiple objectives: an attitudinal

objective may be to change brand perceptions while a behavioral objective may be to

make people engage in some way with the brand.

 Regardless of the number of objectives a company sets for a campaign, it is critical

that such stated objectives be established up front because they provide the all-

 Principle: Campaign objectives and campaign evaluation work hand in hand. In the

absence of clear campaign objectives, evaluation becomes a much murkier task.

Chapter 17: Evaluating IMC Effectiveness

The Campaign Purpose: Brand Building

 Many executives feel advertising works only if it produces sales. One researcher,

Professor John Phillip Jones, contends that “advertising must generate an immediate

jolt to sales before it can be expected to produce any further effect.”

 First, determining effectiveness based on sales can be difficult because of the impact

of other environmental factors. Second, sales are not the only reason brands advertise.

One of the major objectives of advertising is to create higher levels of brand

awareness among consumers.

 Principle: If you can’t measure it, you can’t manage it.

 It is our view that marketers intend their messages to accomplish a variety of goals.

Because campaigns often have multiple objectives and because there are so many

Why Evaluation Matters

 Some evaluation of brand communication is informal and based on the judgment of

an experienced manager, and there is always a need for that. The important thing,

however, is to recognize very early in the campaign planning stages that there will

also be a need for multiple, formal evaluation mechanisms.

 Evaluation should be planned in to any campaign. This is not an onerous task when

clear campaign objectives have been written, since evaluation can and should flow

directly from those goals. The inclusion of a ‘campaign evaluation phase’ on the

program timeline is mandatory.

Chapter 17: Evaluating IMC Effectiveness

 From the business perspective, formal evaluation of brand communication is a must.

Why? The first reason is because the stakes in making an advertising misstep are

high. The second reason is because advertising optimization, i.e., reducing risk by

testing, analyzing, tracking performance, and making changes where possible to

increase the effectiveness of the communication, helps ensure future success.

 Principle: The sheer cost of brand communication demand that it be evaluated so the

effectiveness can be understood.

How Evaluation Fits into the Stages of Brand Communication Testing

 A complete understanding of the strength of your brand communication is

accomplished through testing, monitoring and measurement. This process is depicted

in Figure 17.1.

 In chapter 6, we discussed message development testing; specifically, how concept

testing, semiotic analysis, and pretesting are used to make a campaign as strong as

possible before it ‘goes live’ in the marketplace. Ideally the result of preliminary

testing should be available before large sums of money are invested in finished work

or in media buys.

EVALUATING THE IMC MESSAGE

 Questions about impact are critical and must be addressed. If they are not, there is no

proof that a campaign worked, so companies may be tempted to make the mistake of

cutting communication spending that is driving their business.

 Any attempt to measure the impact of brand messaging by looking only at sales

numbers poses challenges. Therefore, the impact of brand messaging is typically

measured in terms of its communication effects – the mental responses to a message

that serve as surrogate measures for sales impact. Positive post-campaign changes in

 When campaigns don’t work, a very important role of post-campaign message testing

involves understanding what went wrong. Some messages might confuse the

audience; others might fail to get consumers’ attention; and still others might fail to

resonate with consumers. In these situations, campaign message evaluation must

bring these issues to light. Solid campaign evaluation measurement tells us precisely

what a campaign has and has not accomplished.

Chapter 17: Evaluating IMC Effectiveness

397

 Table 17.1 in the text groups key message effectiveness measures and then matches

them to the types of research questions that are used to determine effectiveness.

Although the word ad is used throughout the table, the questions are relevant to all

IMC tools used in a given campaign.

Experts in Message Evaluation

 Many research companies, in addition to some large agency research departments,

specialize in measuring the various dimensions of effectiveness described in Table.

17.1. The most successful of these companies have conducted so many tests that they

have developed norms for common product and service categories. Norms allow the

brand agency managers to determine whether a particular campaign message has

performed above or below the category average in terms of “moving the needle” on

items such as those listed in Table 17.1.

 Also, major online companies such as Google provide a myriad of tools for online

campaign evaluation that include everything from straightforward analytic tools that

measure site traffic or key word search results to more complex tools that allow

marketers to create experiments. Companies such as MetrixLab use a tagging

technology to determine how many times people are exposed to a campaign and then

measure how differently heavily exposed and lightly exposed consumers differ in

their intent to try the brand.

Message Evaluation Techniques

Next, we will examine the most common research techniques used. Different types of

measurement are required because brand managers are likely to set different objectives

for different types of campaign messages.

Tracking Studies

Chapter 17: Evaluating IMC Effectiveness

 Of the research questions listed in Table 17.1, those focusing on recall, recognition,

comprehension, and relevance are staples of communication tracking surveys.

Scanner Analysis

 Another common component of posttest evaluation is scanner research. Many

retailers use electronic scanner data to tally up purchase data and collect consumer

buying information. Scanner research is also used to see what type of sales spikes are

created when certain ads and promotions are used in a given market. Both retail

chains and manufacturers of brands are interested in such data.

Single-Source Data

 Single-source data are obtained by measuring various media/marketing exposure,

purchase behavior, and loyalty for the same individual or household. Sources may

include data from loyalty card purchases, scanner data, television or cable set-top

boxes, personal people meters, and household demographics. Single-source data

companies, such as Nielsen, arrange to have test commercials delivered to a select

group of households within a market to compare changes in behavior to a control

group of households.

Memory Tests

 Memory tests are based on the assumption that a brand communication leaves a

mental residue with the person who has been exposed to it or, in other words, the

audience has learned something. One way to measure IMC effectiveness, then, is to

contact consumers who were exposed to the campaign and find out what they

remember. Memory tests fall into two major groups: recognition tests and recall tests.

Chapter 17: Evaluating IMC Effectiveness

399

 One way to measure memory is to show a magazine advertisement, for example, to

people and ask them whether they remember having seen it before. This kind of test

is called a recognition test. In a recall test, respondents who have read the magazine

are asked to report what they remember from the ad about the brand. A recall test is

more rigorous than a recognition test.

 Similarly, new television commercials are often run during the Super Bowl.

The next evening, interviewers make thousands of random phone calls until

they have contacted about 200 people who were watching the game at the exact

time the commercial appeared. The interviewer then asks a series of questions,

such as the following:

 Do you remember seeing a commercial for any SUVs?

 If no, then, do you remember seeing a commercial for a Jeep Wrangler?

(memory prompt)

 If yes to either of the previous questions, then, what did the commercial say about

the product? What did the commercial show? What did the commercial look like?

What ideas were brought out?

Inquiry Tests

 Inquiry tests, a form of action response, measure the number of responses to an

advertisement or other form of brand communication. The response can be a call

to a toll-free number, an email, a website visit, a coupon return, a visit to a dealer,

an entry in a contest, or a call to a salesperson. Inquiry tests are the primary

measurement tool for direct-response communication, and they are also used to

Chapter 17: Evaluating IMC Effectiveness

EVALUATING THE PERFORMANCE OF VARIOUS IMC TOOLS

 IMC synergy exists when all campaign components work together to create a solid

and understandable brand meaning. Before overarching synergy of a campaign can be

measured, evaluation is usually conducted on a tool-by-tool basis. They may employ

internal data or they may examine results from an outside organization.

Advertising

 Table 19.2 in the text shows that advertising has the potential to be particularly

effective in accomplishing a number of objectives. The most common post-testing or

campaign evaluation technique used to evaluate advertising is the tracking study,

which we previously discussed. Although communication tracking studies represent

very large investments on the part of marketers, given the cost of creating advertising,

thorough campaign evaluation represents money well spent.

Public Relations

 Public relations practitioners typically track the impact of a public relations campaign

in terms of successful output, such as how many news releases led to stories or

mentions in news stories) and outcome, such as attitudinal or behavioral change due

to the impact of materials produced.

 Output evaluation might be conducted by asking specific questions such as how many

times did our spokesperson appear on talk shows or how much and what kind of buzz

are our tweets generating? The results are presented as counts of minutes, mentions,

or retweets. Outcomes, on the other hand, are usually measured in terms of changes in

public opinion and relationship tracking.

Chapter 17: Evaluating IMC Effectiveness

Consumer, Trade, and Point-of-Purchase Promotions

 Sales promotion managers for packaged goods and other products that use

distribution channels need to evaluate both the impact of consumer (or end user)

directed promotions and promotions targeted at retailers and other channel members.

At the most basic level of evaluation, managers require proof of execution. One

responsibility of the sales force is to do store checks to verify that stores are doing

what they promised. At the most important level of promotion evaluation, however,

the behaviors and types of involvement that promotions are designed to stimulate

must be measured.

 The efficiency of a sales promotion can be evaluated in terms of its financial

returns more easily than can the impact of advertising. We compare the costs

of the promotion, called a payout analysis, to the forecasted sales generated

by the promotion. A break-even analysis seeks to determine the point at which

the total cost of the promotion exceeds the total revenues, identifying the point

where the promotion was not productive. Figure 17.3 depicts this analysis.

Direct Marketing

 The primary objective of direct marketing communication is to drive a transaction

or generate some other type of immediate behavioral response, such as a donation

Evaluation of Digital IMC Components

 Many IMC efforts are conducted online. Some digital performance indicators are

website traffic volume, such as page views or the simple number of visitors to a site.

Banner advertising and other online ads are evaluated using click-through rates.

One thing the industry has learned is that this form of advertising is decreasing in

effectiveness. Pop-up banners in particular can get more attention than banner ads

but they are also seen as more irritating.

Chapter 17: Evaluating IMC Effectiveness

 The most important thing to understand in considering evaluation of online efforts

is that while websites used to be the primary digital properties for connecting with

targets and the place where the most digital interaction was happening, that is no

longer true. Today, digital communication is multichannel and multiplatform.

EVALUATING THE PERFORMANCE OF MEDIA VEHICLES

 Brand communication has little chance to be effective if no one sees it. That is why

analyzing the effectiveness of the media plan is another important part of campaign

evaluation.

Media Optimization

Evaluating Exposure

 For major campaigns, agencies do post-buy analyses, which involve checking the

media plan against the performance of each media vehicle. A critical question is

whether reach and frequency objectives were obtained.

 Verifying audience measurement estimates is a challenge. Media planners are

working sometimes with millions of dollars and they can’t afford to get it wrong.

Research data regarding data impact must be viewed with caution. Just because

Chapter 17: Evaluating IMC Effectiveness

Vehicle-by-Vehicle Evaluation

To better understand the obstacles encountered in media evaluation, we will first look at a

few areas where media performance is hardest to estimate: out-of-home media, digital

media, and alternative media. Then we will review some traditional media evaluation

techniques.

Out-of-Home Media

 Accurately evaluating the mobile audience for outdoor advertising is challenging.

Although traffic counts can be gathered, the problem is that traffic does not equal

exposure. Just because a car drove by a board does not mean that the driver or

passengers actually saw it.

Digital Media

 Until recently, the measures of effectiveness used to evaluate offline campaigns

did not seem to transfer well to the online world. It used to be the case that digital

tracking was tough and offline tracking was stronger in showing the relationship

Alternative Media

 Alternative media programs, such as word-of-mouth, social media, and guerilla

marketing campaigns, are even harder to evaluate, and media planners continue

to search for reliable indicators of exposure numbers from these new sources that

equate to the performance measures for traditional media.

Newspaper Readership Measurement

 For newspapers and other traditional media, assessment is more straightforward.

Newspapers measure their audiences in two ways: circulation, or number of

subscribers, and readership, or number of readers. These same measures are

revisited in post-campaign evaluation.

Magazine Readership Measurement

 Magazine rates are based on the guaranteed circulation that a publisher promises to

provide, as well as figures for their total audience. Post campaign evaluation requires

Chapter 17: Evaluating IMC Effectiveness

Measuring the Broadcast Audience

 A radio station’s coverage is revisited by marketers after the campaign to ensure that

the radio medium delivered. For television, Nielsen data helps advertisers understand

IMC CAMPAIGN EVALUATION CHALLENGES

 The major challenge in overall IMC program evaluation is to pull everything together

and look at the big picture of campaign performance rather than the individual pieces

and parts.

 As companies move from an evaluation mindset that considers measures such as

impressions to one that also looks closely at conversion rates and consumer

Measuring ROI

 Advertisers continue to improve how they measure brand communication ROI, which

compares the cost of creating and running brand communication to the revenue it

generates. However, since the dollar impact of brand communication is difficult to

measure, so is the cost-to-sales ratio. ROI is much easier to calculate for sales

promotions and direct marketing than for the campaign overall.

 One question related to ROI concerns how much spending is too much. That is, how

do you determine whether you are overadvertising or underadvertising? The best way

to answer this question is to use test marketing. Even though test marketing helps

Chapter 17: Evaluating IMC Effectiveness

The Synergy Problem

 Another challenge with evaluating campaigns, particularly IMC campaigns, is

estimating the impact of synergy. Intuitively we know that multichannel

communication with messages that reinforce and build on one another will have more

impact than single messages from single sources; however, that can be difficult to

prove.

 A number of studies have attempted to evaluate IMC impact by comparing campaigns

that use two or three tools to see what is gained when more message sources are

added to the mix. Such studies of both the platforms and components of IMC are

beginning to tease out the effects of synergy, but they are a long way from evaluating

the effects of a total communication program.

 A final complication in evaluating programs is the need to consider other messages

and contact points beyond the brand campaign. Brand experiences such as those

 In the A Principled Practice feature, its author makes three salient points:

o 1) Key management decisions count for a lot. No amount of promotion hype can

compensate for an inferior product offering.

o 2) Social media cannot be ignored or taken lightly. To disregard the impact of

social media is to make a huge management error.

Digital Challenges

 Marketers face several challenges when attempting to capture the success of digital

marketing communication efforts. First and foremost, marketers often want to skip

straight to digital measurement without having a clear sense of their business

objectives.

Chapter 17: Evaluating IMC Effectiveness

 Second, marketers must develop the right digital key performance indicators,

which tell whether or not digital communications are driving the business toward

success. A measure of success can be either direct or indirect, but either way,

success indicators must be succinctly defined.

International Challenges

 International brand communication is difficult to evaluate because of market

differences (e.g., language, laws, and cultural norms) and the acceptability of

various research tools in different countries. There may also be incompatibilities

among various measurement systems and data analysis techniques that make it

difficult to compare the data from one market with similar data from another

market. An international evaluation program for advertising should focus,

at least initially, on pretesting because unfamiliarity with different cultures,

languages, and consumer behaviors can result in major miscalculations.

BACK TO THE BIG PICTURE: DID THE CAMPAIGN WORK?

 The ultimate measure of campaign performance is the answer to one seemingly

simple question: Did the campaign achieve the objectives that were set at the very

beginning of the planning process? Because the planning process is imperfect, this

question is not always so simple.

Chapter 17: Evaluating IMC Effectiveness

 A model of effects can be developed for a specific campaign and used to drive not

only the planning effort, but also evaluation of its effectiveness. An example of this is

the innovative Colorado Pass Club (CPC) introduced by Vail Resorts. Another

Connecting the Dots: Tying Measurement Back to Objectives

 Competent brand communication managers return to the campaign objectives, and

then adequately and realistically measure the campaign’s performance against those

objectives.

 An example of how UPS successfully faced this challenge is discussed in detail in

this chapter. In the example, three campaign objectives are listed, along with how the

campaign performed on each of those objectives. A mix of perception, image, and

 This campaign successfully repositioned UPS within its industry. It also provided the

foundation used by the company to further strengthen its positioning through two

more follow-up campaigns. All of these efforts have been designed to support UPS’s

broader strategy of capturing a greater share of more profitable business services.

Looking Ahead

 Beyond connecting the objectives to the measurements, advertisers continue to search

for methods that will bring all the metrics together and efficiently and effectively

evaluate brand communication effectiveness.

END-OF-CHAPTER SUPPORT

REVIEW QUESTIONS

17-3. Why is setting campaign objectives important?

Typically, a brand communication campaign has multiple objectives. It is critical that the

stated objectives be established up front because they provide the all-important

framework for evaluating the success of a campaign. This is true for social media, also.

408

17-4. Why is campaign measurement becoming more important than it used to be?

17-5. What is single-source research, and how do scanner data relate to it?

Single-source research data is obtained by measuring various media/marketing exposure,

purchase behavior, and loyalty for the same individual or household. Sources may

include data from loyalty card purchases, television or cable set-top boxes, personal

17-6. What is media optimization and how do we evaluate how close we are to

achieving it?

One of the biggest challenges in media planning is media efficiency – that is, getting the

most out of your money invested. The goal of media optimization is to optimize the

409

17-7. Why is an IMC campaign difficult to evaluate?

17-8. What are some unique challenges connected with the evaluation of digital

marketing communication efforts?

There are several challenges marketers face when attempting to capture the success of

digital marketing communication efforts. First, marketers often want to skip straight to

DISCUSSION QUESTIONS

17-9. Most clients want a quick and easy answer to the question of whether an IMC

campaign works. Advertising professionals, however, tend to believe that sales-only

approach to evaluation is not appropriate. Why do they feel that way? If you were helping

an agency prepare for a presentation on its campaign results, what would you suggest the

agency say to explain away the idea that you can evaluate a campaign with a single

measure?

410

17-10. You are hiring a research consulting company to help a client evaluate the

effectiveness of its communication efforts. One of the consultants recommends using

focus groups to evaluate their effectiveness. Another consultant suggests that focus

groups aren’t very effective for campaign evaluation and recommends other measures.

Which viewpoint do you believe is most insightful? For campaign evaluation purposes,

is qualitative or quantitative research used most often?

17-11. Explore the websites of two IMC evaluation companies, such as Ameritest

(www.Ameritest.net), Ipsos ASI (www.ipsos.com), Millward Brown

(www.millwardbrown.com), or Sapient (www.sapient.com), and compare the services

they offer. If you were looking for a company to evaluate a campaign for recall, which

one would you prefer? Why? What about if you were looking for a company to evaluate a

campaign for digital impact? Why?

TAKE-HOME PROJECTS

17-12. Portfolio Project: Put together a portfolio of 10 ads for a set of product categories

targeted to a college audience. Set up a focus group with participants recruited from

among your friends and ask them to look at the ads. In a test of unaided awareness, ask

them to list the ads they remember. Identify the top performing ad and the bottom ad in

this awareness test. Now ask the focus group participants to analyze the headline, the

visual, and the brand identification of each ad. How do the two ads compare in terms of

their ability to get attention and lock the brand in memory?

Chapter 17: Evaluating IMC Effectiveness

ASSIGNMENTS

Individual Assignments

1. In this chapter, a quote from an Advertising Age magazine article is referenced that

states, “the next generation advertising exec will be a data geek with the soul of an

artist, the business acumen of Warren Buffett and the storytelling skills of Don

Draper.” Probe the deeper meaning of this statement by reading the article from

which it came, which can be found at this link:

http://adage.com/article/datadriven-marketing/marketing-quant-101-schools-gear-data-

talent-crunch/240369/

How and why are the skill sets that effective marketers must possess changing? What are

the key issues identified in this article? Conduct additional research to determine what is

being done to address these concerns and summarize your findings in a 350-word report.

2. This chapter discussed the challenges associated with evaluating the effectiveness of

the digital component of IMC campaigns. One of the tools many marketers rely upon

for help in this area is Google Analytics. Follow the link below to review several

success stories on the Google Analytics website that illustrate how this service has

helped companies strengthen, measure, and evaluate their digital campaigns.

Summarize your findings in a 500-word report.

Think-Pair-Share

In small groups, conduct some very informal market research that focuses upon the

overall effectiveness of television advertisements. On youtube.com find five relatively

current 30- or 60-second television ads for five different consumer packaged goods

products. Then construct a survey instrument by selecting one question from each of

Table 17.1’s categories. Next, identify a group of eight to ten survey participants and

OUTSIDE EXAMPLE

Watch “Life Lessons from an Ad Man,” and “Sweat the Small Stuff,” two online videos

featuring Rory Sutherland, who was mentioned at the very end of this chapter. You can

find the videos by going to www.ted.com and typing his name into the search box.

Chapter 17: Evaluating IMC Effectiveness

Make a list of the life lessons that Mr. Sutherland is attempting to convey, and then

compare that list to the IMC principles in your textbook. As you do so, consider the

following questions: (a) Where do the two lists converge and diverge? Pinpoint and

discuss the similarities. (b) What are the fundamental principles regarding human

behavior, economics, and the practice of marketing that Mr. Sutherland cautions us not to

lose sight of? Why do you think it is so easy for marketing and advertising professionals

to stray from them? (c) How could adherence to Mr. Sutherland’s lessons help a young

professional advance their marketing or advertising career?