Part Three
Research Methods for Collecting Primary Data
Chapter 9
Survey Research: An Overview
AT-A-GLANCE
I. Introduction
II. Using Surveys
A. Advantages of surveys
III. Errors in Survey Research
A. Random sampling error
B. Systematic error
C. Respondent error
Nonresponse error
Response bias
oDeliberate falsification
oUnconscious misrepresentation
oTypes of response bias
1. Acquiescence bias
2. Extremity bias
3. Interviewer bias
4. Social desirability bias
D. Administrative error
Data-processing error
Sample selection error
Interviewer error
Interviewer cheating
E. Rule-of-thumb estimates for systematic error
F. What can be done to reduce survey error?
IV. Classifying Survey Research Methods
A. Structured/unstructured and disguised/undisguised questionnaires
B. Temporal classification
Cross-sectional studies
Longitudinal studies
oConsumer panels
V. Total Quality Management and Customer Satisfaction Surveys
A. What is quality?
B. Internal and external customers
C. Implementing total quality management
LEARNING OUTCOMES
1. Define surveys and explain their advantages
2. Describe the type of information that may be gathered in a survey
3. Identify sources of error in survey research
4. Distinguish among the various categories of surveys
5. Discuss the importance of survey research to total quality management programs
CHAPTER VIGNETTE: Media Phones—The Next Wave of
Communication Technology?
What’s next in the world of electronic communication? According to In-Stat, a research firm, it will be
the media phone. The media phone represents a new category of broadband multimedia devices that has
the potential to become the 4th screen in the home, complementing the PC, TV, and mobile handset. It
will combine the power of a PC with the always-on functionality of the home telephone. In-Stat has
conducted extensive survey research to learn consumer perceptions, attitudes, and desires regarding media
phones and estimates the consumer market for this device could reach nearly 50 million units and $8
billion in worldwide revenue by 2013. In-Stat’s research also determined the characteristics of the
product customer desired across consumer age groups and geographic locations.
SURVEY THIS!
Students are asked to review the online survey and address the following:
1. Identify at least three different sources of potential error or bias and offer suggestions on how
this error can be reduced.
2. Classify this survey on the structured, disguised, and temporal dimensions.
3. How could this survey help your academic institution implement a total quality management
program?
RESEARCH SNAPSHOTS
Intuit Gets Answers to Satisfy Customers
Intuit, the maker of Quicken, QuickBooks, and Turbo Tax software for accounting and tax
preparation, has enjoyed years of growth and profits, in part due to its efforts to learn what
customers want. A “net promoter survey” is used to determine which customers are likely to
promote the products to others. To learn more about why customers are satisfied, these
individuals are invited to go online and provide more detailed opinions, and Intuit has made
changes based on these comments. The company also conducts direct observation of customers
using their products, and they have learned that small business owners were not always familiar
with the accounting jargon, so products were changed to include every day terminology.
Overestimating Patient Satisfaction
The challenge facing researchers interested in satisfaction is whether the responses come from a
cross-section of customers. Some researchers studied data from patient satisfaction surveys and
found that the actual data closely matched simulated data in which responses were biased so that
responses were more likely when satisfaction was higher. In other words, more-satisfied patients
were more likely to complete and return the survey, which would overestimate satisfaction.
The “Mere -measurement” Effect
The mere-measurement effect means that simply answering a question about intentions will
increase the likelihood of the underlying behavior – if the behavior is seen as socially desirable.
If it is not, answering the question tends to decrease the likelihood of the behavior. One study
found this to be true for eating fatty food or flossing. However, this effect did not occur if the
surveys indicated that they were sponsored by groups that would be likely to want to persuade the
subjects (i.e., the American Fruit Growers Association and the Association of Dental Products
Manufacturers). Subjects decreased their frequency of flossing if they took the supposedly
manipulative survey that asked about flossing. Thus, if individuals receive information that puts
them on their guard against persuasion, the mere-measurement effect is lessened and sometimes
even generates the opposite behavior.
TIPS OF THE TRADE
Surveys are the most widely used method of collecting primary quantitative data for business
research, but a common error is to begin questionnaire design and the survey process too
soon. Before undertaking primary research activities:
oBe sure to fully exhaust secondary research sources.
oHave a clear understanding of the research issues and objectives.
Error exists in all survey research.
oRandom sampling error is present due to chance variation in the sample elements and can
only be addressed through large sample size.
oSystematic error is due to a flaw in the research design or execution, and it is the job of
the researcher to minimize this error.
OUTLINE
I. INTRODUCTION
Surveys require asking people (respondents) to provide answers to written or spoken
questions.
Questionnaires or interviews collect data through the mail, on the telephone, online, or
face-to-face.
A survey is defined as a method of collecting primary data based on communication with a
representative sample of individuals.
Surveys provide a snapshot at a given point in time.
The more formal term, sample survey, emphasizes that the purpose of contacting
respondents is to obtain a representative sample of the target population
II. USING SURVEYS
The type of information gathered varies considerably depending on its objectives.
Most survey research is descriptive research, which attempts to identify and explain a
particular business activity.
Most surveys typically have multiple objectives; few gather only a single type of factual
information.
Although consumer surveys are a common form of business research, not all survey research
is conducted with the ultimate consumer.
Although surveys are often conducted to quantify certain factual information, certain aspects
of surveys may also be qualitative. For example, in new-product development the qualitative
objective of a survey is often to test and refine new-product concepts.
Advantages of Surveys
Provide a quick, inexpensive, efficient, and accurate means of assessing information
about a population.
Surveys can also be poorly conducted and certain errors can occur to render such surveys
useless.
III. ERRORS IN SURVEY RESEARCH
Exhibit 9.1 outlines the various forms of survey error which can affect the accuracy of a
survey.
Survey error can be broken down into two major sources: random sampling error and
systematic error.
Random Sampling Error
Most surveys try to portray a representative cross-section of a particular target
population, but even with technically proper probability sampling random sampling
errors will occur because of chance variation.
Without increasing sample size, these statistical problems are unavoidable.
However, random sampling errors can be estimated (see Chapters 16 and 17).
Systematic Error
Systematic error results from some imperfect aspect of the research design or from a
mistake in the execution of the research.
Also called nonsampling errors.
A sample bias exists when the results of a sample show a persistent tendency to deviate
in one direction from the true value of the population parameter.
Two general categories are: respondent error and administrative error.
Respondent Error
If the respondents do not cooperate or do not give truthful answers then two major types
of respondent error may cause sample bias.
Nonresponse Error
To use the results of a survey the researcher believes that those who did respond to
the questionnaire are representative of those who did not.
If only those who responded are included in the survey then nonresponse error will
occur.
People who are not contacted or who refuse to cooperate are called nonrespondents.
The number of no contacts has been increasing because of the proliferation
of answering machines and growing usage of Caller ID to screen telephone
calls.
Refusals occur when people are unwilling to participate in the research.
Comparing the demographics of the sample with the demographics of the target
population is one means of inspecting for possible biases in response patterns.
Self-selection may also occur in self-administered questionnaires; in this situation,
only those who feel strongly about the subject matter will respond, causing an over
representation of extreme positions
Response Bias
Response bias occurs when respondents tend to answer with a certain slant.
This bias may be caused by an intentional or inadvertent falsification or by a
misrepresentation of the respondents’ answers.
Deliberate Falsification
Occasionally people deliberately give false answers.
They may become bored with the interview and provide answers just
to get it done.
They may try to appear well informed.
They give answers to please the interviewer.
Average-person hypothesis – individuals prefer to be viewed as
average, so they alter their responses to conform more closely to
their perception of the average person.
Unconscious Misrepresentation
Response bias can arise from the question format, the question content,
or some other stimulus.
In many cases consumers cannot adequately express their feelings in
words.
Language differences also may be a source of misunderstanding.
Time lapse influences people’s ability to precisely remember and
communicate specific factors.
Some consumers may unconsciously avoid facing the realities of a future
buying situation.
Types of Response Bias
1. Acquiescence Bias – a tendency to agree (or disagree) with all or most
questions. Particularly prominent in new-product research.
2. Extremity Bias – some individuals tend to use extremes when
responding to questions; others consistently avoid extreme positions
and tend to respond more neutrally.
3. Interviewer Bias – the interviewer’s presence influences respondents to
give untrue or modified answers.
4. Social Desirability Bias – respondent wishes to create a favorable
impression or save face in the presence of an interviewer.
Administrative Error
The results of improper administration or execution of the research task are
administrative errors.
Such errors are inadvertently caused by carelessness, confusion, neglect, omission or
some other blunder.
There are four types of administrative error:
data-processing error
sample selection error
interviewer error
interviewer cheating.
Data-Processing Error
Processing data by computer is subject to error because data must be edited, coded,
and entered into the computer by people.
Can be minimized by establishing careful procedures for verifying each step in the
data-processing stage.
Sample Selection Error
Sample selection error is systematic error that results in an unrepresentative sample
because of an error in either the sample design or the execution of the sampling
procedures.
For example, selecting a sample from the phone book will have some systematic
error because unlisted numbers are not included; stopping respondents during
daytime hours in shopping centers excludes working people who shop by mail,
Internet, or telephone.
Interviewer Error
Introduced when interviewers record answers but check the wrong response or are
unable to write fast enough to record answers verbatim.
Selective perception may cause interviewers to misrecord data that do not support
their own attitudes and opinions.
Interviewer Cheating
Occurs when an interviewer falsifies entire questionnaires or fills in answers to
questions that have been intentionally skipped.
The term curb-stoning is sometimes used to refer to interviewers filling in responses
for respondents that do not really exist.
Rule-of-Thumb Estimates for Systematic Error
Sampling error may be estimated using certain statistical tools, but ways to estimate
systematic error are less precise.
Many researchers have found it useful to use some standard of comparison in order to
understand how much error can be expected. For example, one cable TV company
knocks down the number of people saying that they intend to purchase the service by a
“ballpark ten percent” because previous experience has indicated a ten percent upward
bias on the intention questions.
What Can Be Done To Reduce Survey Error?
Chapters to follow discuss various techniques for reducing bias in survey research.
Chapter 15 on questionnaire design discusses the reduction of response bias.
Chapters 16 and 17 discuss the reduction of sample selection and random sampling
error.
IV. CLASSIFYING SURVEY RESEARCH METHODS
Surveys can be classified in three ways:
1. method of communication, such as personal interview, telephone interviews, mail
surveys, and Internet surveys (all discussed in Chapter 10)
2. the degrees of structure and disguise in the questionnaire
3. the time frame in which the data are gathered (temporal classification)
Structured/Unstructured and Disguised/Undisguised Questionnaires
A structured question limits the number of allowable responses.
Unstructured questions do not restrict the respondent’s answers (e.g., an open-ended
question).
The researcher can also use undisguised questions or disguised questions.
Using disguised questions is particularly advisable if the subject matter is of a
threatening or sensitive nature.
Other questions do not need to be disguised as it is assumed that the respondent
is willing to reveal the information.
Questions can be categorized according to their degree of structure and disguise.
Unstructured-disguised
Structured-undisguised
Unstructured-undisguised
Structured-disguised
However, it is not always easy to categorize surveys as the categories are not
clear-cut, and most surveys are a hybrid of structured and unstructured questions.
Temporal Classification
Cross-Sectional Studies
Cross-sectional studies collect data at a single point in time.
Various segments of the population are sampled so that relationships among
variables may be investigated by cross-tabulation.
Most business research surveys fall into this category.
Longitudinal Studies
In a longitudinal study respondents are questioned at multiple points in time.
Purpose is to examine continuity of response and to observe changes that occur
over time.
Longitudinal studies which involve two or more samples at different times are
called cohort studies because similar groups of people who share a certain
experience during the same time interval (cohorts) are expected to be included in
each sample.
In applied business research, a longitudinal study that uses successive samples is
called a tracking study because successive waves are designed to compare
trends and identify changes in variables.
Conducting surveys in waves with two or more sample groups avoids the
problem of response bias resulting from a prior interview, but the researcher can
never be sure that the changes in the variable being measured are not actually due
to having different people in the sample.
Consumer Panel
A longitudinal study that gathers data from the same sample of individuals or
households over time.
Diary data that are recorded regularly over an extended period enable the
researcher to track repeat-purchase behavior and changes in purchasing habits
that occur in response to changes in price, special promotions, or other aspects of
marketing strategy.
Expensive to maintain.
Often managed by contractors who offer their services to many organizations
(i.e., National Family Opinion (NFO), Inc., Markettools, Inc.).
Internet panels have grown in popularity.
The first questionnaire typically includes questions about product ownership and
usage, pets, family members, and demographic data.
Economical means for marketers to reach respondents who own their products.
V. TOTAL QUALITY MANAGEMENT AND CUSTOMER SATISFACTION SURVEYS
Total quality management is a business strategy that emphasizes market-driven quality as a
top priority.
Involves implementing and adjusting the firm’s business activities to assure customers’
satisfaction with the quality of goods and services.
In an organization driven by the quality concept, business research plays an important role in
the management of total product quality.
What Is Quality?
Organizations used to define quality by engineering standards, but most companies no
longer see quality that way.
Now, managers believe the level of quality is the degree to which a good or service
corresponds to buyers’ expectations.
Internal and External Customers
A focus on customers must include more than external customers.
Every employee must know exactly who his or her customers are and what output
internal and external customers expect.
Also important to know how customers perceive their needs are being met.
Implementing Total Quality Management
Requires considerable survey research.
A firm must routinely ask customers to rate it against its competitors.
Firm must periodically measure employee knowledge, attitudes, and expectations.
Must monitor company performance against benchmark standards.
Exhibit 9.3 illustrates the total quality management process.
Overall tracking of quality improvement requires longitudinal research.
Stages include:
Commitment and exploration stage.
Management makes a commitment to total quality assurance.
Researchers explore external and internal customers’ needs and
beliefs.
Benchmarking stage
Research must establish quantitative measures that can serve as
benchmarks or points of comparison against which to evaluate
future efforts.
Surveys must establish initial measures of overall satisfaction, of
the frequency of customer problems, and of quality ratings for
specific attributes.
Researchers must identify the company’s or brand’s position
relative to competitors’ quality positions.
Initial quality improvement stage
Establishes a quality improvement process within the
organization.
Establish performance standards and expectations for
improvement.
Continue to track satisfaction and quality ratings in successive
waves.
Continuous quality improvement
Consists of many consecutive waves with the same purpose – to
improve over the previous period.
Allow employees to initiate problem solving without a lot of red
tape.
Reward performance.
Measure performance against customers’ standards – not against standards
determined by quality engineers within the company.
Continuous quality improvement is an ongoing process.
Process applies to service quality as well.