Part Five
Sampling and Fieldwork
Chapter 16
Sampling Designs and Sampling Procedures
AT-A-GLANCE
I. Sampling Terminology
II. Why Sample?
A. Pragmatic reasons
B. Accurate and reliable results
C. Destruction of test units
III. Practical Sampling Concepts
A. Defining the target population
B. The sampling frame
Sampling frames for international research
C. Sampling units
IV. Random Sampling and Nonsampling Errors
A. Random sampling error
B. Systematic sampling error
C. Less than perfectly representative samples
V. Probability versus Nonprobability Sampling
VI. Nonprobability Samples
A. Convenience sampling
B. Judgment sampling
C. Quota sampling
Possible sources of bias
Advantages of quota sampling
D. Snowball sampling
VII. Probability Sampling
A. Simple random sampling
B. Systematic sampling
C. Stratified sampling
D. Proportional versus disproportional strata
E. Cluster sampling
F. Multistage area sampling
VIII. What Is the Appropriate Sample Design?
A. Degree of accuracy
B. Resources
C. Time
D. Advance knowledge of the population
E. National versus local project
IX. Internet Sampling Is Unique
A. Website visitors
B. Panel samples
C. Recruited ad hoc samples
D. Opt-in lists
LEARNING OUTCOMES
1. Explain reasons for taking a sample rather than a complete census
2. Describe the process of identifying a target population and selecting a sampling frame
3. Compare random sampling and systematic (nonsampling) errors
4. Identify the types of nonprobability sampling, including their advantages and disadvantages
5. Summarize the advantages and disadvantages of the various types of probability samples
6. Discuss how to choose an appropriate sample design, as well as challenges for Internet sampling
CHAPTER VIGNETTE: Changing Pocketbook Problems for Today’s
Families
Each quarter, the Gallup Corporation develops a representative sample of approximately 1,000 U.S. adults
to capture public perceptions on a variety of relevant topics, including financial concerns of the family.
The carefully selected sample serves as a representation of the population of adults in the U.S., and as a
result, researchers can be 95 percent confident that the responses of the sample are reflective of this
national population, with a sampling error of less than 3 percent. Using telephone interviews, respondents
are asked to describe “the most important financial problem facing your family today,” and these
open-ended responses are then coded based upon the theme of the response. Trends suggest that the most
important problem can often change over time, suggesting that the financial problems facing families
evolve over time.
SURVEY THIS!
Students are asked to look at the survey that asks a variety of questions of college students related to job
preferences. They are asked to answer the following questions:
1. How well do the results collected from this survey represent the population of entry-level,
business-oriented, recent college graduates?
2. If question one shown in the screenshot does not describe this population, describe one that you
believe is better represented by this data.
3. Can the data be stratified in a way that would allow it to represent more specific populations?
4. Does this particular respondent neatly represent a common population?
RESEARCH SNAPSHOTS
Finding Out About Work is a lot of Work!
What do people do for work? How long does it take them to get there? What do they earn?
These and many other questions are critically important for United States economists and social
scientists. The U.S. Census Bureau and the Bureau of Labor Statistics have jointly asked these
questions, every month, for almost 70 years. The Current Population Survey (CPS) uses a panel
sample of 60,000 households, surveying them for several months. The cost of conducting the
CPS is millions of dollars. This accuracy has resulted in the CPS to be considered one of the
standards by which other household surveys are conducted.
How Much Does Your Prescription Cost? It Depends on Who You Buy It From
Most people would expect that their prescriptions would cost about the same, no matter where
they buy them. The attorney general of the state of Michigan commissioned a targeted survey of
200 pharmacies to capture drug prescription costs around the state. Since the sample was drawn
purposely, there was confidence that the survey would lead to some fruitful insights. Prices for
the same prescription could vary as much as $100, even among pharmacies located near each
other. This led to a consumer alert from the attorney general’s office—encouraging customer to
shop carefully.
American Kennel Club Tries to Keep Pet Owners Out of the Doghouse
The American Kennel Club (AKC) is an organization dedicated to promoting purebred dogs and
their health and well-being as family companions. The organization commissioned a study to
investigate dog ownership and the acceptance of dogs in their neighborhoods, and quota sampling
was used. The sample was small—1,000 people—so the study design set quotas for completed
interviews in age, sex, and geographic categories. One half of the respondents owned dogs.
People without dogs tended to be concerned about dogs jumping and barking and owners not
“picking up after their dogs.” Dog owners were somewhat more laid-back and happy compared
to nonowners.
Who’s at Home? Different Ways to Select Respondents
A carefully planned telephone survey often involves multistage sampling where researchers first
select a sample of households and then select someone within each household to interview. One
researcher conducted an analysis of various selection procedures and found the following:
Full enumeration method – interviewer requests a list of all the adults living in the household,
generates a random number, uses the number to select a name from the list, and asks to speak
with that person.
Kish method – interviewer requests the number of males and females by age, and then uses
randomization to select either a male or female and a number (i.e., oldest male or third
female). This method did not seem to discourage respondents by being too intrusive, and it
was popular because it came close to being random.
Interview the person who last had a birthday – generated better cooperation rates.
New on Campus: Student Adjustment to College Life
A panel study was conducted where incoming students were assessed on their psychological traits
and coping behaviors upon entry and were resurveyed at the end of their first year. The results
indicate that students who engaged in negative coping behaviors or who had perfectionist
tendencies would more likely have poor adjustment outcomes after the first year. Students who
were optimistic and socially oriented were much more likely to adjust to the new college
environment. The use of a panel approach was necessary because the researchers were interested
in the change that occurred within a sample of students over time. These results can be used to
develop newcomer programs or experiences that students can use to adjust to their new
environment.
OUTLINE
I. SAMPLING TERMINOLOGY
A population (universe) is any complete group (i.e., people, sales territories, stores, etc.)
sharing some common set of characteristics.
The term population element refers to an individual member of the population.
A census is an investigation of all the individual elements making up the population—a total
enumeration rather than a sample.
A sample is a subset or some part of a larger population.
II. WHY SAMPLE?
Pragmatic Reasons
Applied research projects usually have budget and time constraints.
Sampling cuts costs, reduces labor requirements, and gathers vital information quickly.
Accurate and Reliable Results
Most properly selected samples give results that are quite accurate,
If the elements of a population are similar, only a small sample is necessary to accurately
portray the characteristic of interest.
A sample on occasion is more accurate than a census – interviewer mistakes, tabulation
errors, and other nonsampling errors may increase during a census because of the
increased volume of work.
Destruction of Test Units
Many research projects, especially those in quality-control testing, require the destruction
of the items being tested.
For example, if the manufacturer of firecrackers wished to find out whether each unit met
a specific production standard, there would be no product left after testing.
III. PRACTICAL SAMPLING CONCEPTS
Defining the Target Population
Once the decision to sample has been made, the first question concerns identifying the
target population.
What is the relevant population?
In many cases this is easy to answer, but in other cases, the decision may be difficult.
For consumer-related research, the appropriate population element frequently is the
household rather than an individual member of the household.
At the outset of the sampling process it is vitally important to carefully define the target
population so that the proper source from which the data are to be collected can be
identified.
To implement the sample in the field, tangible characteristics should be used to define the
population.
The Sampling Frame
In practice, the sample will be drawn from a list of population elements called a
sampling frame, which is a list of elements from which the sample may be drawn.
The sampling frame is also called the working population, because these units will
eventually provide units involved in the analysis.
Some firms, called sampling services or list brokers, specialize in providing lists or
databases that include the names, addresses, phone numbers, and e-mail addresses of
specific populations.
Lists offered by these companies are often compiled from subscriptions to profession
journals, credit card applications, warranty card registrations, and a variety of other
sources.
A reverse directory provides listings by city and street address or by phone number,
rather than alphabetical by last name, which is useful when a researcher wishes to survey
only a certain geographical area.
A sampling frame error occurs when certain sample elements are excluded or when the
entire population is not accurately represented in the sampling frame.
Population elements can be either under- or overrepresented in a sampling frame.
Sampling Frames for International Research
The availability of sampling frames around the globe varies dramatically.
Not every country’s government conducts a census of population.
Sampling Units
During the actual sampling process, the elements of the population must be selected
according to a certain procedure.
The sampling unit is a single element or group of elements subject to selection in the
sample.
If the target population has first been divided into units (i.e., airline flights), additional
terminology must be used.
The term primary sampling unit (PSU) is used to designate units selected in the
first stage of sampling.
If successive stages of sampling are conducted, sampling units are called secondary
sampling units, or tertiary sampling units.
When there is no list of population elements, the sampling unit is generally something
other than the population element. For example, in a random digit dialing study the
sampling unit will be telephone numbers.
IV. RANDOM SAMPLING AND NONSAMPLING ERRORS
If a difference exists between the value of a sample statistic of interest and the value of the
corresponding population parameter, a statistical error has occurred.
Two basic causes of differences between statistics and parameters:
1. random sampling errors
2. systematic (nonsampling) errors
Random Sampling Error
Random sampling error is the difference between the sample result and the result of a
census conducted using identical procedures.
Random sampling error occurs because of chance variation in the scientific selection of
sampling units.
Because random sampling errors follow chance variations, they tend to cancel one
another out when averaged.
This means that properly selected samples are generally good approximations of the
population.
Random sampling error is a function of sample size.
As sample size increases, random sampling error decreases
It is possible to estimate the random sampling error that may be expected with various
sample sizes.
Systematic Sampling Error
Systematic (nonsampling) errors result from nonsampling factors, primarily the nature of
a study’s design and the correctness of execution.
These errors are not due to chance fluctuations.
Sample biases account for a large portion of errors in research.
Nonsampling errors have already been discussed in Chapter 8.
Less than Perfectly Representative Samples
Random sampling errors and systematic errors associated with the sampling process may
combine to yield a sample that is less than perfectly representative of the population.
Additional errors will occur if individuals refuse to be interviewed or cannot be
contacted.
Such nonresponse error may also cause the sample to be less than perfectly
representative.
V. PROBABILITY VERSUS NONPROBABILITY SAMPLING
Several alternative ways to take a sample are available.
The main alternative sampling plans may be grouped into two categories:
1. probability techniques
2. nonprobability techniques.
In probability sampling, every element in the population has a known, nonzero probability
of selection.
The simple random sample, in which each member of the population has an equal
probability of being selected, is the best-known probability sample.
In nonprobability sampling, the probability of any particular member of the population
being chosen is unknown.
The selection of sampling units in nonprobability sampling is quite arbitrary, as
researchers rely heavily on personal judgment.
Technically, no appropriate statistical techniques exist for measuring random
sampling error from a nonprobability sample.
Therefore, projecting the data beyond the sample is, technically speaking, statistically
inappropriate.
Nevertheless, nonprobability samples are pragmatic and are used in business
research.