McDaniel & Gates – Marketing Research, 9th Edition Instructor’s Manual
Instructor’s Manual
CHAPTER 16
Statistical Testing of Differences and Relationships
LEARNING OBJECTIVES
1. To become aware of the nature of statistical significance.
3. To understand the difference between Type I and Type II errors.
5. To learn about analysis of variance.
KEY TERMS
Statistical significance
Hypothesis
Null hypothesis
Decision rule
Chi-square test
Z test
t test
Hypothesis test of proportions
CHAPTER SCAN
This chapter examines several methods of analyzing data. Examined first is the concept of
differences. There are three concepts of differences: mathematical differences, statistical
Instructor’s Manual
hypothesis when the null hypothesis actually is false. Then some basic terminology of hypothesis
testing is explored.
CHAPTER OUTLINE
1. Evaluating Differences and Changes
2. Statistical Significance
3. Hypothesis Testing
I. Hypothesis
A. Hypothesis Defined
B. Two Basic Explanations of Observed Differences
II. Steps in Hypothesis Testing
A. Step One: Stating the Hypothesis
B. Step Two: Choosing the Appropriate Test Statistic
4. Commonly Used Statistical Hypothesis Tests
I. Independent versus Related Samples
5. Goodness of Fit
6. Hypotheses about One Mean
I. ZTest
7. Hypotheses about Two Means
8. Hypotheses about Proportions
9. Analysis of Variance (ANOVA)
10. p-Values and Significance Testing
CHAPTER SUMMARY
1. EVALUATING DIFFERENCES AND CHANGES
1. Examples of Manager Questions
a. Our posttest measure of top-of-mind awareness (first brand mentioned unaided) is slightly
higher than the level recorded in the pretest. Did top-of-mind awareness really increase or is
there some other explanation for the increase?
b. Our overall customer satisfaction score increased from 92 percent 3 months ago to 93.5
percent today. Did customer satisfaction really increase?
c. Are the customers in Dallas really more satisfied with our customer service than those of
Cincinnati with 1.2 points higher?
2. STATISTICAL SIGNIFICANCE
I. Statistical Inference
A. Statistical Inference Defined–to generalize from sample results to population characteristics.
1. Fundamental tenet of statistical inference–possible for numbers to be different in a
mathematical sense but not significantly different in a statistical sense
B. Concepts of Differences
1. Mathematical Differences–if the numbers are not exactly the same, they are different–does
not imply statistical significance
2. Statistical Significance–particular difference is large enough to be unlikely to have occurred
because of chance or sampling error, the difference is statistically significant
3. Managerially Important Difference–the differences large enough to be meaningful to the
manager
See Practicing Marketing Research: Choosing the Right Test for the Right Situation (p
473)
When testing percentages with dependent groups, chi-squares should be used for three or more
groups, and Z tests should be used for two groups. When testing means, ANOVAs (Analysis of
Variance) are used in the case of three or more group, and t tests are used for the two group case.
Questions
1. Aside from the automatic settings in analytic software mentioned above, can you think of any
other procedural factors that might cause a researcher to misapply certain tests?
3. HYPOTHESIS TESTING
I. Hypothesis
A. Hypothesis Defined–assumption or guess that a researcher or manager makes about some
characteristic of the population being investigated
1. In hypothesis testing, a researcher determines whether a hypothesis concerning some
2. A statistical hypothesis test allows us to calculate the probability of observing a particular
1. Hypothesis is true and the observed difference is likely due to sampling error
2. Hypothesis is false and the true value is some other value
II. Steps in Hypothesis Testing
A. Step One: Stating the Hypothesis
1. Hypotheses are stated using two basic forms:
a. Ho–the null hypothesis (sometimes called the hypothesis of the status quo) that is tested
2. The null and the alternative hypothesis must be stated in a way that both cannot be true at the
same time.
B. Step Two: Choosing the Appropriate Test Statistic
1. Exhibit 16.1 Statistical Tests and Their Uses–provides a guide to selecting the appropriate
test for various situations
See Exhibit 16.1 Statistical Tests and Their Uses (p 475)
C. Step Three: Developing a Decision Rule
1. As discussed in an earlier chapter, we know it’s very unlikely for our sample estimate to
3. A Significance level ( ) that will determine whether to reject or fail to reject the null
hypothesis
Instructor’s Manual
D. Step Four: Calculating the Value of the Test Statistic
1. Use the appropriate formula to calculate the value of the statistic for the test chosen
3. Based on the comparison, determine to either reject or fail to reject the null hypothesis Ho
E. Step Five: Stating The Conclusion–the conclusion summarizes the results of the test. It
should be stated from the perspective of the original research question
III. Types of Errors in Hypothesis Testing
A. Two General Types of Errors in Hypothesis Testing
1. Type I ( ) Error involves situations in which the researcher rejects the null hypothesis when
2. Type II ( ) Error involves situations in which the researcher fails to reject the null
3. The value of is never set in advance– becomes larger when is made smaller–to
minimize type II error, then you choose a larger value for in order to make smaller
See Exhibit 16.2 Type I and Type II Errors (p 476)
4. The decision to reject or fail to reject is never 100% certain. There is a probability of being
correct and a probability of being incorrect.
5. Hence, Type I Error ( ) is set by the researcher after consulting with the client, considering
7. Thus in a hypothesis test, there are three critical elements: , , and the sample size, n.
8. When setting either or , it is critical to understand the implications.
a. Let’s assume we are testing for a disease using the following hypothesis:
McDaniel & Gates – Marketing Research, 9th Edition Instructor’s Manual
8. Typically, we do not set Type II error, , in advance. For a fixed sample size, when is made
smaller, becomes larger; and when is made larger, becomes smaller.
10. In the above example where Type II error is more serious than Type I, one would set = .1.
If the situation is that Type I error is more serious, than one would set = .01. If there is no real
difference between Type I and Type II error, than traditionally researchers set = .05.
See Practicing Marketing Research: Why in Medicine a Two-Tailed Test is Preferable to a
One-Tailed Test (p 477)
A. Loye, MD, and Alan T. N. Tita, MD, physicians at the University of Texas and Baylor
College of Medicine, respectively, argue that ethical researchers should studiously avoid one-
tailed or one-sided testing in medical research for both ethical and cost-efficiency reasons. Why?
Instructor’s Manual
measuring type I error. The trouble here is that one cannot tell what type I error is associated
Discussion Questions
1. Are there any circumstances in clinical research where a one-tailed test would be ethically
justified?
2. Are there any situations in market research where a one-tailed test might yield data that could
lead to harmful effects or conclusions?
IV. Decision to Reject or Fail to Reject
A. Accepting Ho or Failing to Reject (FTR) Ho–whether there is enough evidence in the data
to conclude that Ha is correct. There is either sufficient evidence to support Ha (reject Ho) or
there is not (fail to reject Ho).
V. One-Tailed versus Two-Tailed Test
A. Tests are either one-tailed or two tailed. This decision depends on the nature of the situation
and what you are trying to demonstrate.
B. When the researcher is concerned whether there is a difference in a specific direction only
See From the Front Line: More Tips on Significance Testing (p 479)
Paul Schmiege, Marketing Science, DSS Research
McDaniel & Gates – Marketing Research, 9th Edition Instructor’s Manual
Instructor’s Manual
You must be careful to distinguish the technical term statistical significance from more intuitive
terms such as practical significance or importance. Significance in the field of statistics means
that the difference is likely greater than we would expect due to sampling error.
The single most common test in marketing research is the two-sample t-test. Because the two-
sample t-test is so important, it is good to keep at least two points in mind about it:
For any given observed difference, there are sample sizes large enough that the difference will be
significant in a two-sample t-test (sample sizes go in the denominator of the equation). The larger
the sample size, smaller and smaller differences become statistically significant, but practical
significance remains the same. Is a 0.5 percent increase worth telling management about even if
it should happen to be statistically significant? Significance, or nonsignificance, is just another
VI. Example of Performing a Statistical Test
(Note: An example is presented–pages 527-531.)
See Exhibit 16.3 Shaded Area is Significance Level (p 481)
See Practicing Marketing Research: Does Statistical Precision Validate Results? (p 482)
Generally, two kinds of error affect the validity of statistical measurements. Random error
introduces error variance, but as it occurs randomly across respondents, it does not add statistical
Instructor’s Manual
Questions
1. Of the potential causes for error described above, which do you think would be easiest to
identify? Hardest? Explain your reasoning.
2. Can you think of any ways that could help researchers determine whether occurrences of
statistical significance in their results have managerial significance?
4. COMMONLY USED STATISTICAL HYPOTHESIS TESTS
I. Independent versus Related Samples
A. Independent Samples–involve situations in which measurement of the variable of interest in
one sample has no effect on (i.e., independent) the measurement of the variable in the other
sample (e.g., brand awareness of a product for men vs. women).
B. Related Samples–are those in which the measurement of the variable of interest in one
II. Degrees of Freedom
A. Many statistical tests refer to degrees of freedom.
B. Degrees of Freedom (D.F.)–the number of observations in a statistical problem that are not
restricted and or are free to vary.
C. The Number of Degrees of Freedom–equal to the number of observations minus the number
of assumptions or constraints necessary to calculate a statistic.
5. GOODNESS OF FIT
I. Chi-Square Test
A. Chi-square
1. Chi-square defined–chi-square ( ) test enables the research analyst to determine whether
an observed pattern of frequencies corresponds to or fits an “expected” pattern.
a. “goodness of fit”–test the observed distribution to the expected distribution
2. Chi-square Test of a Single Sample
1) The problem is discussing the impact of three different sales promotions on the customers
visiting a particular store
3. Chi-Square Test of Two Independent Samples
a. This is an example of a problem seeking to determine the nature a relationship between two
6. HYPOTHESES ABOUT ONE MEAN
One of the most common goals in marketing research is to make an inference for a single
population mean.
I. Z Test
A. Z Ttest Defined–When the sample size is large enough (n≥30), the most common test
statistic used for these types of hypotheses is the Z Ttest.
1. Example of how to apply this test page 538
B. Steps Involved in the Z Test
1. Specify the null and alternative hypotheses
3. Determine the sample standard deviation (S)
5. Calculate the test statistic
6. State the result
See Practicing Marketing Research: Taking the Z Test into the Classroom and
Courtroom—Two Practical Uses (p490-491)
Here are two practical applications of the Z-test as used by the California State Personnel Board
to assess differences in populations and by lawyers to analyze juror challenges. Testing Teacher
Population Statistics Hazelwood School District v. United States (1977) involved the issue of
Instructor’s Manual
whether appropriate numbers of black teachers had been hired for a school in St. Louis County,
Missouri, proportional to the black population of the county and state.
Legal researchers used a one-sample Z Test to determine whether the number of qualified black
teachers in a population compared to how many had actually been hired in a specific school
district. The hypothesis presumed that the two populations matched; the Z Test showed they did
not. The results set a research precedent and established a viable protocol for other school
districts, including the California State Personnel Board (CSPB), provided the ratio between
Discussion Questions
2. Should the Z Test be used routinely in jury selection?
II. t-Test
A. t-Test Defined– When the sample size is relatively small (n<30), the Student t-test is more
appropriate than the Z Test for hypothesis testing.
1. The t-test with n – 1 degrees of freedom is appropriate for making statistical inferences in this
2. The t distribution is theoretically correct for large samples (n≥30), however when the sample
size is large, the t-distribution and the z-distribution become indistinguishable.
4. Example of how to apply this test on page 542
B. Steps Involved in the t-Test
2. Specify the level of sampling error ( ) allowed
4. Calculate the estimated standard error of the mean
6. State the result
See SPSS Jump Start for t-Test (pp 493-494)
7. HYPOTHESES ABOUT TWO MEANS
I. Testing Differences between Groups
A. Marketers are frequently interested in testing differences between groups. (Note: Example of
how to apply this test on page 545)
II. Differences Between Groups
A. Steps for Testing the Differences between Groups
2. Specify the level of sampling error ( )
4. Calculate the test statistic Z
8. HYPOTHESES ABOUT PROPORTIONS
I. Proportion in One Sample
A. Hypothesis Test of Proportions Defined–test to determine whether the difference between
proportions is greater than would be expected because of sampling error
1. Example of how to apply this test pages 546-547
McDaniel & Gates – Marketing Research, 9th Edition Instructor’s Manual
Instructor’s Manual
B. The Procedure for the Hypothesis Test of Proportions
2. Specify the level of sampling error ( ) allowed
4. Calculate the test statistic
5. State the results
II. Two Proportions in Independent Samples
A. Difference between the Proportions in Two Different Groups–such as the proportions of
people in two different groups who engage in certain activity or have a certain characteristic
1. Specifications Required and the Procedure for Testing this Hypothesis pages 547-548
a. Specify the null and alternative hypotheses
9. ANALYSIS OF VARIANCE (ANOVA)
I. Analysis of Variance (ANOVA)
A. ANOVA Defined–test for the differences among the means of two or more independent
samples
2. It is a statistical technique that permits the researcher to determine whether the variability
among or across the C sample means is greater than expected because of sampling error.
3. The Z and t tests described earlier normally are used to test the null hypothesis when only two
4. Steps involved in an ANOVA (example pp 500-503)
McDaniel & Gates – Marketing Research, 9th Edition Instructor’s Manual
Instructor’s Manual
b. Sum the squared differences between each subsample mean (
X
j) and the overall sample mean
(
X
), weighted by sample size (n j). This is call the sum of squares among groups or among
group variation (SSA)
c. Calculate the variation among group means as measured by the mean sum of squares among
groups (MSA)
d. Sum the squared differences between each observation (Xjj) and its associated sample mean (
X
j), accumulated over all C levels (groups). Also called the sum of squares within groups or
See Practicing Marketing Research: 12 Coins, 3 Shops, 2 Microbiologists, and 1 Test—
ANOVA (p502)
ANOVA, or the analysis of variance, is a statistical test devised in the 1920s by the English
statistician, Ronald A. Fisher. British microbiologists Richard Armstrong and Anthony Hilton,
both at Aston University in Birmingham, England, recently found it to be the “most appropriate
method” for statistical analysis of complex data sets—in this case coins collected from a
butcher’s shop, news agent’s office, and a sandwich shop. The researchers took four coins
randomly from each premise and analyzed them for bacterial populations. Armstrong and Hilton
described their procedure as a one-way ANOVA with four replications performed in a
randomized test design. The procedure took into account the variation between the various
Instructor’s Manual
Discussion Questions
1. Can you devise a four-factor ANOVA test for the microbiologists?
2. The microbiologists applied their different ANOVA tests to food-service-related shops. How
might it be applied to a dentist’s office?
10. p VALUES AND SIGNIFICANCE TESTING
I. p Values and Significance Testing
A. p Value Defined–exact probability of getting a computed test statistic assuming the null
hypothesis is true.
1. The smaller the p value, the smaller the probability that the observed result occurred by
chance
2. The p value is the most demanding level of statistical (not managerial) significance that can be
met, based on the calculated value of the statistic.
3. An example is provided on page 553 using Exhibit 16.5.
See Exhibit 16.5 Sample t-Test Output (example of a p value calculation) (p 503)
11. SUMMARY
QUESTIONS FOR REVIEW AND CRITICAL THINKING
1. Explain the notions of mathematical differences, managerially important differences,
and statistical significance. Can results be statistically significant and yet lack managerial
importance. Explain your answer.
Mathematical differences occur when two measures are not exactly the same. This does not