Unlock document.
This document is partially blurred.
Unlock all pages and 1 million more documents.
Get Access
Part Four
Measurement Concepts
Chapter 13
Measurement and Scaling Concepts
AT-A-GLANCE
I. What Do I Measure?
A. Concepts
B. Operational definitions
Variables
Constructs
II. Levels of Scale Measurement
A. Nominal scale
B. Ordinal scale
C. Interval scale
D. Ratio scale
E. Mathematical and statistical analysis of scales
Discrete measures
Continuous measures
III. Index Measures
A. Indexes and composites
B. Computing scale values
IV. Three Criteria for Good Measurement
A. Reliability
Internal consistency
Test-retest reliability
B. Validity
Establishing validity
C. Reliability versus validity
D. Sensitivity
LEARNING OUTCOMES
1. Determine what needs to be measured to address a research question or hypothesis
2. Distinguish levels of scale measurement
3. Know how to form an index or composite measure
4. List the three criteria for good measurement
5. Perform a basis assessment of scale reliability and validity
CHAPTER VIGNETTE: Money Matters?
Griff Mitchell is the Vice-President of Customer Relationship Management (CRM) for one of the world’s
largest suppliers of industrial heavy equipment, and in this role he oversees all sales and service
operations. The company has decided to perform a CRM employee evaluation process that will allow an
overall ranking of all CRM employees, and the ranking will be used to single out the best performers to
be recognized at the annual CRM conference. The rankings will also be used to identify the lowest 20
percent of performers, who will be put on a probationary list with specific improvement goals. Griff’s
key question is, What is performance? So he calls a meeting of senior CRM managers to discuss how the
ranking decisions should be made. One manager argued that sales volume should be the sole criterion
because it is objective. Another said to use the manager’s opinion to classify an employee as a top
performer, a good performer, or an under performer. Another said to use margins. Finally, another said
that because they are CRM, customer satisfaction should be used. This did not help Griff much in his
quest to develop a valid performance measure that treated all employees fairly. So he seeks the help of an
outside research consultant, Robin Donald.
SURVEY THIS!
Students are asked to identify one of each of the four categories of scale measurement and to look at the
question shown. What scale measurement do these items represent? Each set of items is designed to
capture a single construct—the top portion assesses how much work-life interferes with nonwork-life and
the bottom portion assesses self-perceived performance. Compute a coefficient α for each scale to
estimate reliability and then create a composite scale by summing the items that make up that particular
scale.
RESEARCH SNAPSHOTS
Peer Pressure and Investing Behavior
While we all have experienced “peer pressure,” research has shown that some individuals are
more susceptible than others. This interpersonal influence is typically thought to be present in
conspicuous consumption and socially visible products, but recent research shows such influence
can occur even in the selection of less visible products and services, such as investments.
Researchers used the construct susceptibility to interpersonal influence (SCII) to study this. First,
this construct had to be conceptualized and measured and was thought to be composed of two
parts—susceptibility to informational influences and susceptibility to normative influences.
Questions were developed to measure both parts to capture SCII. Results of the study are given.
Recoding Made Easy
Most computer software makes scale recoding easy, and a screenshot from SPSS, perhaps the
most widely used statistical software in business-related research, is shown. The steps include:
(1) click on transform, (2) click on recode, (3) choose to recode into the same variable, (4) select
the variable(s) to be recoded, (5) click on old and new values, (6) use the menu that appears to
enter the old values and the matching new values, and click add after entering each pair, and (7)
click continue.
OUTLINE
I. WHAT DO I MEASURE?
The decision statement, corresponding research questions and research hypotheses can be
used to decide what concepts need to be measured.
Measurement is the process of describing some property of a phenomenon of interest usually
by assigning numbers in a reliable and valid way.
When numbers are used, the researcher must have a rule for assigning a number to an
observation in a way that provides an accurate description.
All measurement, particularly in the social sciences, contains error.
Concepts
A researcher has to know what to measure before knowing how to measure something.
A concept is a generalized idea that represents something of meaning.
Concepts such as age, sex, education and number of children are relatively concrete
properties and present few problems in either definition or measurement.
Concepts such as brand loyalty, corporate culture, and so on are more abstract and are
more difficult to both define and measure.
Operational Definitions
Researchers measure concepts through a process known as operationalization,
which is a process that involves identifying scales that correspond to variance in the
concept.
Scales provide a range of values that correspond to different values in the concept
being measured.
Scales provide correspondence rules that indicate that a certain value on a scale
corresponds to some true value of a concept, hopefully in a truthful way.
Variables
Researchers use variance in concepts to make diagnoses.
Variables capture different concept values.
Scales capture variance in concepts and as such, the scales provide the
researcher’s variables.
For practical purposes, once a research project is underway, there is little
difference between a concept and a variable.
Constructs
Sometimes a single variable cannot capture a concept alone.
Using multiple variables to measure one concept can often provide a more
complete account of some concept than could any single variable.
A construct is a term used for concepts that are measured with multiple
variables.
Can be very helpful in operationlizing a concept.
II. LEVELS OF SCALE MEASUREMENT
The level of scale measurement is important because it determines the mathematical
comparisons that are allowed.
The four levels of scale measurement are:
1. nominal
2. ordinal
3. interval
4. ratio
Nominal Scale
Nominal scales represent the most elementary level of measurement.
Assigns a value to an object for identification or classification purposes.
The value can be but does not have to be a number since no quantities are being
represented.
A qualitative scale.
Useful even though they can be considered elementary.
Business researchers use nominal scales quite often.
Nominal scaling is arbitrary in the sense that each label can be assigned to any of the
categories without introducing error.
Examples of nominal numbering are uniform numbers, airport terminals, and school bus
numbers.
Ordinal Scale
Ordinal scales have nominal properties, but they also allow things to be arranged based
on how much of some concept they possess.
A ranking scale.
Somewhat arbitrary, but not nearly as much as a nominal scale.
Example: “win,” “place,” and “show” in a horse race tells which horse was first,
second, and third, but does not tell by how much a horse won.
Interval Scale
Interval scales have both nominal and ordinal properties, but they also capture
information about differences in quantities of a concept.
Classic example is the Fahrenheit temperature scale:
80° F is hotter than 40° F, but you cannot conclude that the 40° is twice as cold as
80° because this is a scaling system.
To see the point above, convert the temperatures to the Celsius scale. 80° F =
26.7° C, and 40° F = 4.4° C.
The scale is not iconic, meaning that it does not exactly represent some
phenomenon.
Interval scales are very useful because they capture relative quantities in the form of
distances between observations.
Ratio Scale
Ratio scales represent the highest form of measurement in that they have all the
properties of interval scales with the additional attribute of representing absolute
quantities.
Interval scales represent only relative meaning while ratio scales represent absolute
meaning.
In other words, ratio scales provide iconic measurement.
Zero, therefore, has meaning in that it represents an absence of some concept.
An absolute zero is a defining characteristic in determining between ratio and
interval scales.
For example money is a way to measure economic value.
Mathematical and Statistical Analysis of Scales
Although you can put numbers into formulas and perform calculations with almost any
numbers, the researcher has to know the meaning behind the numbers before useful
conclusions can be drawn (e.g., averaging the numbers used to identify school busses is
meaningless).
Discrete Measures
Discrete measures are those that take on only one of a finite number of values.
Most often used to represent a classificatory variable and thus do not represent
intensity of measures, only membership.
Common discrete scales include any yes-no response, matching, color choice or
practically any scale that involves selecting from a small number of categories.
Nominal and ordinal scales are discrete measures.
Certain statistics are most appropriate for discrete measures (shown in Exhibit 13.5).
The central tendency of discrete measures is best captured by the mode (i.e., most
frequent level).
Continuous Measures
Continuous measures are those assigning values anywhere along some scale range
in a place that corresponds to the intensity of some concept.
Ratio measures are continuous measures.
Strictly speaking, interval scales are not necessarily continuous.
e.g., Likert item ranging from 1=strongly disagree to 5=strongly agree.
This is a discrete scale because only the values 1, 2, 3, 4, or 5 can be assigned.
The mean is not an appropriate way of stating central tendency and we really
shouldn’t use many common statistics on these responses.
However, as a scaled response of this type takes on more values, the error
introduced by assuming that the differences between the discrete points are equal
are smaller.
Therefore, business researchers generally treat interval scales containing 5 or
more categories of response as interval.
Researchers should keep in mind, however, the distinction between ratio and interval
measures.
Errors in judgment can be made when interval measures are treated as ratio.
e.g., An attitude of 0 means nothing as attitude only has meaning in a relative
sense (i.e., compared to another’s attitude).
The means and standard deviation may be calculated from continuous data.
Using the actual quantities from arithmetic operations is permissible with ratio
scales.
III. INDEX MEASURES
An attribute is a single characteristic or fundamental feature of an object, person, situation,
or issue.
Indexes and Composites
Multi-item instruments for measuring a construct are called index measures, or composite
measures.
An index measure assigns a value based on how much of the concept being measured is
associated with an observation.
An index is often formed by putting several variables together.
Composite measures also assign a value based on a mathematical derivation of multiple
variables.
For most practical applications, composite measures and indexes are computed in the
same way.
Computing Scale Values
Exhibit 13.6 demonstrates how a composite measure can be created from common rating
scales.
A summated scale is created by simply summing the response to each item making up
the composite measure.
A researcher may sometimes choose to average the scores rather than summing them
because the composite measure is expressed on the same scale as is the items that make it
up.
Reverse coding means that the value assigned for a response is treated oppositely from
the other items.
IV. THREE CRITERIA FOR GOOD MEASUREMENT
The three major criteria for evaluating measurements are:
1. reliability
2. validity
3. sensitivity
Reliability
Reliability is an indicator of a measure’s internal consistency.
A measure is reliable when different attempts at measuring something converge on the
same result.
Internal Consistency
Internal consistency represents a measure’s homogeneity.
The set of items that make up a measure are referred to as a battery of scale items.
Internal consistency of a multiple-item measure can be measured by correlating
scores on subsets of items making up a scale.
The split-half method of checking reliability is performed by taking half the items
from the scale and checking them against the results from the other half.
The two scale halves should correlate highly.
They should also produce similar scores.
Coefficient alpha (α) is the most commonly applied estimate of a multiple item
scale’s reliability.
Represents internal consistency by computing the average of all possible
split-half reliabilities for a multiple item scale.
The coefficient demonstrates whether or not the different items converge.
Ranges in value from 0 (no consistency) to 1 (complete consistency).
Generally, scales with a coefficient α:
0.80 - 0.95: very good reliability
0.70 - 0.80: good reliability
0.60 - 0.70: fair reliability
below 0.60: poor reliability
Test-Retest Reliability
The test-retest method of determining reliability involves administering the same
scale or measure to the same respondents at two separate times to test for stability.
If the measure is stable over time, the test, administered under the same conditions
each time, should obtain similar results.
Represents a measure’s repeatability.
Two problems common to all longitudinal studies:
the premeasure, or first measure, may sensitize the respondents to their
participation in a research project and subsequently influence the results of
the second measure.
If the time between measures is long, there may be an attitude change or
other maturation of the subjects.
Validity
Good measures should be both precise (i.e., reliable) and accurate (i.e., valid).
Validity is the accuracy of a measure or the extent to which a score truthfully represents a
concept.
Achieving validity is not a simple matter.
Establishing Validity
The four basic approaches to establishing validity are face validity, content validity,
criterion validity, and construct validity.
Face validity refers to the subjective agreement among professionals that a scale
logically reflects the concept being measured.
Content validity refers to the degree that a measure covers the domain of interest.
Criterion validity addresses the question: “Does my measure correlate with
measures of similar concepts or known quantities?”
May be classified as either concurrent validity or predictive validity
depending on the time sequence in which the new measurement scale
and the criterion measure are correlated.
If measures taken at the same time concurrent validity.
If measures taken at different times predictive validity.
Construct validity exists when a measure reliably measures and truthfully represents
a unique concept and consists of several components:
Face validity
Content validity
Criterion validity
Convergent validity – another way of expressing internal consistency;
highly reliable scales contain convergent validity.
Discriminant validity – represents how unique or distinct is a measure;
a scale should not correlate too highly with a measure of a different
construct.
Reliability versus Validity
The differences between the two are illustrated with rifle targets (see Exhibit 13.7).
A: The shots from the older gun are scattered low reliability.
B: The shots from the newer gun are closely clustered and on target high reliability
and validity.
C: The shots from a newer gun are closely clustered but off target high reliability
but low validity.
Sensitivity
The sensitivity of a scale is an important measurement concept, particularly when changes in
attitudes or other hypothetical constructs are under investigation.
Sensitivity refers to an instrument’s ability to accurately measure variability in a concept.
Sensitivity is generally increased by adding more response points or adding scale items.
