IV.SELF-ADMINISTERED QUESTIONNAIRES
No matter how self-administered questionnaires are distributed (i.e., mail, in high-traffic
locations in stores), they are different from interviews because the respondent takes
responsibility for reading and answering the questions.
Presents a challenge to researchers because they rely on the clarity of the written word rather
than on the skills of the interviewer.
Mail Questionnaires
A mail survey is a self-administered questionnaire sent to respondents through the mail.
Several advantages and disadvantages are listed below.
Geographic Flexibility
Can reach a geographically dispersed sample simultaneously because
interviewers are not required.
Isolated respondents (e.g., farmers) or those who are difficult to reach (e.g., busy
executives) can easily be contacted by mail.
Cost
Relatively inexpensive compared with personal interviews, but they are not
cheap.
Most include follow-up mailings, which require additional postage and printing
costs.
Low response rates contribute to higher cost.
Respondent Convenience
Can be filled out when the respondents have time, so they are more likely to take
time to think about their replies.
Allow respondents to collect facts that they may not be able to recall without
checking.
Anonymity of Respondent
Respondents are more likely to provide sensitive or embarrassing information
when they can remain anonymous.
Can reduce social desirability bias.
Absence of Interviewer
Once the respondent receives the questionnaire, the questioning process is
beyond the researcher’s control.
Respondent does not have the opportunity to question the interviewer.
Respondents can read the entire questionnaire before they answer individual
questions, which could affect responses.
Standardized Questions
Questionnaires typically are highly standardized, and the questions are quite
structured.
Questions and instructions must be clear-cut and straightforward.
Time Is Money
A minimum of 2 or 3 weeks is necessary for receiving the majority of the
responses.
Follow-up mailings require an additional 2 or 3 weeks.
The time between the first mailing and the cut-off date normally is 6 to 8 weeks.
Length of Mail Questionnaire
General rule of thumb is not to exceed six pages in length.
When a questionnaire requires a great deal of effort, and incentive should be
given.
Response Rates
The basic calculation for obtaining a response rate is to count the number of
questionnaires returned or completed, then divide the total by the number of eligible
people who were contacted or requested to participate in the survey.
Typically, the number in the denominator is adjusted for faulty addresses and similar
problems that reduce the number of eligible participants.
The major limitations of mail questionnaires relate to response problems.
Respondents who complete it may not be typical of all people in the sample.
Researcher has no assurance that the intended subject is the person who fills out
the questionnaire (electronic surveying suffers same problem).
Cooperation and response rates rise as home value increases.
Rarely will have response rates of 50 percent or more, but follow-up mailings
and other techniques may increase response rate to an acceptable level.
Increasing Response Rates for Mail Surveys
Individuals who are interested in the general subject of the survey are more likely to
respond than those who are not.
Ways to increase response rates include using a stamped return envelope, using attractive
questionnaires, and wording questions so that they are easy to understand.
Cover Letter
A cover letter that accompanies a questionnaire or is printed on the first page is
an important means of inducing the reader to complete and return the
questionnaire.
First paragraph explains why the study is important.
The basic appeal alludes to the social usefulness of responding.
Two other appeals are asking for help (“Will you do us a favor?”) and the
egotistical appeal (“Your opinions are important!”).
Most cover letters:
promise confidentiality
encourage recipient to use the postage-paid reply envelope
describe any incentive for participation
explain that answering the questionnaire will not be difficult or time
consuming
describe how the person was scientifically selected for participation.
Personalized letter shows the respondent that he or she is important.
Using letterhead rather than a printed form increases response rates.
Money Helps
Monetary incentives appear to be the most effective and least biasing incentive.
Attracts attention and creates a sense of obligation.
Works for all income categories.
Sending the monetary incentive to a charity of the respondent’s choice increases
response rates dramatically.
Interesting Questions
Certain interesting questions can be added to the questionnaire to stimulate
respondents’ interest and to induce cooperation.
Questions may be of little concern to the researcher, but respondents who are
indifferent may be given a reason for responding.
Follow-up
Response rates are relatively high for the first two weeks, then the rates gradually
taper off.
Most studies use a follow-up letter or postcard reminder that may include a
duplicate questionnaire or may merely be a reminder to return the original
questionnaire.
Multiple contacts almost always increase response rates.
Advance Notification
Advance notification (i.e., by letter or telephone) that a questionnaire will be
arriving has been successful in increasing response rates.
Notices that go out closer to the questionnaire mailing time produce better results
than those sent out too far in advance.
Optimum lead time is 3 days before the mail survey is to arrive.
Survey Sponsorship
Sponsorship by well-known and prestigious organizations (i.e., universities or
government agencies) may also significantly influence response rates.
Surveys mailed to a consumer panel receive exceptionally high response rates
because members have already agreed to cooperate.
Other Techniques
Numerous other devices, such as type of postage (i.e., commemorative vs.
regular stamp), envelope size, color of paper, etc, have been varied to increase
response rates.
Limited success.
The researcher should consider his or her particular situation.
Keying Mail Questionnaires with Codes
One device for eliminating those who have already responded from the follow-up
mailing list is to mark the questionnaire so that they may be keyed to identify
members of the sampling frame who are nonrespondents.
Blind keying of questionnaires can entail varying the job number or room
number on the research department on a return envelope.
Visible keying is a visible code number on the questionnaire, and a statement
should indicate to respondents that the purpose is to avoid sending them a
duplicate questionnaire.
Global Considerations
Postal services and cultural circumstances differ around the world.
Issues to consider include the reliability of mail delivery, literacy rates, and trust that
researchers can and will provide confidentiality.
Hand delivery or door-to-door interviewing may be necessary.
Some consumers may be discouraged from talking to an interviewer (e.g., women and
children), so mailed questionnaires would be superior to interviews.
V. SELF-ADMINISTERED QUESTIONNAIRES USING OTHER FORMS OF
DISTRIBUTION
Many forms are similar to mail questionnaires.
Warranty or owner registration cards often are used to collect demographic information and
data about where and why products were purchased.
People who fill these out may differ form those who do not.
The drop-off method, in which an interviewer drops off a questionnaire (often an extremely
long one) and then picks it up later, sacrifices some cost savings because it requires traveling
to each respondent’s location.
Fax Surveys
Potential respondents receive and/or return questionnaires via fax machine.
Disadvantage is that only respondents with fax machines who are willing to exert the
extra effort will return questionnaires.
People with extreme opinions will be more likely to respond.
Researchers may use faxing as one of several options for replying to a survey.
Faxing questionnaires reduces the sender’s printing and postage costs and can be
delivered and returned faster than traditional mail surveys.
Can deal with timely issues.
E-Mail Surveys
Some individuals cannot be reached this way.
Some projects lend themselves to e-mail surveys (i.e., internal surveys of employees or
satisfaction surveys of retail buyers who deal with the organization via e-mail).
Benefits include:
speed of distribution
lower distribution and processing costs
faster turnaround time
more flexibility
less handling of paper questionnaires
Some researchers have argued that many respondents feel they can be more candid.
But employees know that their e-mails are not secure.
Maintaining respondents’ anonymity is difficult because reply will have the
sender’s address.
Not all e-mail systems have the same capacity or settings, which limits the types of
questions and layout of the e-mail questionnaire.
Guidelines for printed mail surveys apply to e-mail surveys.
Should include a valid return e-mail address in the “from” box and reveal who is
conducting the survey.
E-mail should be addressed to a single person (the blind carbon copy, or BBC, field can
be used if the same message must be sent to an entire sample).
E-mail letters can be used as cover letter asking respondents to participate in an Internet
survey, and they usually include a password and a link to a unique Web site.
Internet Surveys
An Internet survey is a self-administered questionnaire posted on a website.
Speed and Cost-Effectiveness
Internet surveys allow researchers to reach a large audience (possibly a global
one), personalize individual messages, and secure confidential answers quickly
and cost-effectively.
Eliminates the costs of paper, postage, data entry, and other administrative costs.
Once developed, the incremental cost of reaching additional respondents is
minimal.
Even with large samples, surveys can be conducted in a week or less.
Visual Appeal and Interactivity
Surveys can be interactive.
Can use more sophisticated lines of questioning based on the respondent’s prior
answers.
Can use color, sound, and animation.
Can present visual materials.
Respondent Participation and Cooperation
Participation may occur because computer users intentionally navigate to a
particular website where questions are displayed.
In some cases, visitors to a site cannot venture beyond the survey page without
providing information.
Participants can initially be contacted via e-mail and given logon instructions and
a password, thus preventing access by individuals who are not part of the selected
sample.
Unique passwords also allow the researcher to track the responses of each
respondent to identify anyone who makes an effort to participate more than once.
The welcome screen should contain the name of the research company and
information about how to contact the organization if the respondent has a
problem or concern.
Representative Samples
Some individuals cannot access the Internet.
Not all people have the same level of technology.
Some people have minimal computer skills.
Internet surveys should be simple so that all respondents can interact at the same
level of technological sophistication.
Accurate Real-Time Data Capture
Each respondent’s answers are entered directly into the researcher’s computer as
soon as the questionnaire is submitted.
Software may be programmed to reject improper data entry.
Real-time data capture allows for real-time data analysis.
Callbacks
Panel members who have not completed the survey can be easily recontacted.
Software can automatically send e-mail reminders.
Can also identify respondents who completed only a portion of the questionnaire
and send those people customized messages.
Personalized and Flexible Questioning
Internet surveys are programmed in much the same way as computer-assisted
telephone interviews.
Major advantage is the ability to sequence questions based on previous
responses.
Another advantage is that it can prompt respondents when they skip over a
question.
Ability to customize questions and the low cost per recipient help researchers to
keep surveys short.
Designers of Internet questionnaires can be creative and flexible in the
presentation of questions using a variety of dialog boxes, or windows that
prompt the respondent to enter information.
Respondent Anonymity
The anonymity of the Internet encourages respondents to provide honest answers
to sensitive questions.
Response Rates
Methods to increase response rates are similar to those for other kinds of survey
research.
A personalized invitation may be important.
An e-mail message’s subject line is critical.
Incentives to respond must be in the form of a promise of a future reward.
Security Concerns
Hackers or competitors may access Web sites to discover new product concepts,
new advertising campaigns, and other top-secret ideas.
Respondents may worry whether personal information will remain private.
Kiosk Interactive Surveys
A computer with a touch screen may be installed in a kiosk at a trade show, at a
professional conference, in an airport, or in any other high-traffic location to administer
an interactive survey.
Self-selection often is a problem.
Computer-literate individuals are most likely to participate.
Temporary locations often require a fieldworker to be at the location to assist users.
Survey Research That Mixes Modes
Research objectives may dictate the use of some combination of telephone, mail, e-mail,
Internet, and personal interview (e.g., conduct a short telephone screening interview to
determine whether respondents are eligible for recontact in a more extensive personal
interview).
Mixed-mode surveys combine the advantages of the various methods.
Conducting research in two or more waves, however, creates the possibility that some
respondents will no longer cooperate or will be unavailable in the second wave.
Text-Message Surveys
These may use the SMS (short-message service) or MMS (Multi-Media Service).
Perhaps the newest survey approach.
Has all the advantages and disadvantages of mobile-phone surveys.
MMS messages can include graphic displays or even short videos.
VI. SELECTING THE APPROPRIATE SURVEY RESEARCH DESIGN
There is no best form of survey; each has advantages and disadvantages.
Exhibit 10.4 summarizes the major advantages and disadvantages the various types.
VII. PRETESTING
Pretesting involves a trial run with a group of respondents to iron out fundamental problems
in the instructions or design of a questionnaire.
This stage is sometimes eliminated because of costs or time pressures.
Three basic ways to pretest exist:
The first two involve screening the questionnaire with other research
professionals.
The third (i.e., the one often called pretesting) is a trial run with a group of
respondents.
An alternative type of screening might involve a client or the research manager
who ordered the research.
VII. ETHICAL ISSUES IN SURVEY RESAERCH
Many ethical issues apply to survey research (see Chapter 5), such as:
respondents’ right to privacy
use of deception
respondents’ right to be informed about the purpose of the research
need for confidentiality
need for honesty in collecting data
need for objectivity in reporting data