Chapter 15 – Data Collection: Types of Error and Response Rate Calculation
I. Learning Objectives:
Upon completing this chapter, the student should be able to:
1. Describe the five types of error that can enter a study.
2. Noncoverage errors – occur because part of the population of interest was
not included in the sampling frame.
3. Nonresponse errors – possible when some elements designated for
5. Office errors – occur when errors are introduced in the processing of the
data or in reporting the findings.
2. Give the general definition for response rate.
3. Discuss several ways in which response rates might be improved.
• Making the data collection instrument and procedure as interesting and as
short as possible
• Carefully choosing and training interviewers
• Asking respondents to complete a short task before asking them to
II. Chapter Outline:
A. Types of Error
Exhibit 15.1: Six Types of Error
2. Noncoverage Error
3. Nonresponse Error
4. Response Error
a. Does the respondent understand the question?
b. Does the respondent know the answer to the question?
c. Is the respondent willing to provide the true answer to the
question?
6. Office Errors
7. Total Error Is the Key
Manager’s Focus
Exhibit 15.3: Types of Errors and Methods for Handling Them
B. Calculating Response Rates
2. Telephone Interviews (No Eligibility Requirement)
4. Other Methods of Data Collection
B. Improving Response Rates
Exhibit 15.4: Tips for Increasing Response Rates on Online Surveys
Manager’s Focus
1. Survey Length
3. Interviewer Characteristics and Training
5. Response Incentives
6. Follow-Up Surveys
Manager’s Focus
D. Summary
E. Key Terms
F. Review Questions
III. Answers to Review Questions:
1. There are six basic types of errors: sampling errors, noncoverage errors,
nonresponse errors, response errors, recording errors, and office errors. Sampling
error occurs anytime that we work with a subset of the population instead of the
population itself. Noncoverage errors occur because part of the population of
interest was not included in the sampling frame. Nonresponse errors are possible
when some elements designated for inclusion in the sample did not respond and
were systematically different from those who did respond on key characteristics.
Response errors occur because inaccurate information was secured from the
sample elements. Recording errors occur when researchers or the technology they
employ fail to accurately record observational data. Office errors occur when errors
are introduced in the processing of the data or in reporting the findings.
2. Sampling error is potentially less troubling than other kinds of error because there
3. Noncoverage error is considered to be a sampling frame problem because it occurs
4. Nonresponse error is a potential source of error because there is the possibility that
5. The two primary sources of nonresponse error are refusals (designated
6. The nonresponse problem is important to the accuracy of most communication
7. The basic considerations underlying response error include the following: Does the
respondent know how to answer the question? Does the respondent understand
the question? Is the respondent willing to provide the true answer to the question?
8. Recording error is the mistakes made by humans or machines in the process of
recording respondents’ communication–or observation-based data.
9. The most frustrating type of error is office error because it can show up even after
10. Data merger errors are caused by the aggregation of data from or about a
12. For online and mail surveys with no eligibility requirements, the response rate
should be calculated by the number of usable questionnaires divided by the
number of contacts attempted minus the number of wrong addresses. For online
13. There are several approaches for improving response rates, including framing the
study to enhance respondent interest, keeping the survey as short as possible,
IV. Instruction Suggestions:
1. Begin by discussing the distinction between sampling error and nonsampling error.
Whereas sampling error is the difference between the observed values of a
variable and the long-run average of the observed values in repetitions of the
2. The distinction between nonsampling and sampling error can be more firmly
implanted by asking the students to associate each research plan with each sample
distribution in the following example:
Research Plan:
Chapter 15 – Data Collection: Enhancing Response Rates while Limiting Errors
5
Research Plan Sampling Distributions
A 2
B 1
C 4
D 3
Nonsampling errors cannot be estimated typically, as we quite often do
not even know their direction, much less their size; e.g., we “felt” the
students would round their GPAs upward when asked although it is possible
they might report them accurately or even round downward.
(b) Sampling errors can be reduced simply by increasing the sample size.
Nonsampling errors are not necessarily reduced this way and may in fact
increase with increases in sample size. Their reduction depends on better
methods rather than increased sample size.
3. Turn then to a discussion of the nonsampling errors. Paralleling the text, the
following points can all be made:
(a) The definition of each type of nonsampling error.
4. Discuss three possible methods of diagnosing nonresponse error. Instructors might
5. Discuss the different types and sources of response errors. Point out that response
error can originate with the respondent, the interviewer, and/or the situation. In
6. We think that it is important that the instructor (repeatedly) point out the value of
exploratory research and questionnaire pre-testing for (a) estimating the likely
7. Remind students that the goal is to reduce the total error in a project, not any
single type of error. Sampling error is only one of five major categories of error, and
8. Discuss the formulas for calculating response rate for various types of data
collection methods. Instructors may find it useful to work through one of the end-
Chapter 15 – Data Collection: Enhancing Response Rates while Limiting Errors
7
9. Instructors will probably want to spend a little time in class discussing some of the
various techniques that have been identified for improving response rates. One