978-1111826925 Chapter 13 Lecture Note

Unlock access to all the studying documents.

View Full Document

Part Four

Measurement Concepts

Chapter 13

Measurement and Scaling Concepts

AT-A-GLANCE

I. What Do I Measure?

A. Concepts

B. Operational definitions

Variables

Constructs

II. Levels of Scale Measurement

A. Nominal scale

B. Ordinal scale

C. Interval scale

D. Ratio scale

E. Mathematical and statistical analysis of scales

Discrete measures

Continuous measures

III. Index Measures

A. Indexes and composites

B. Computing scale values

IV. Three Criteria for Good Measurement

A. Reliability

Internal consistency

Test-retest reliability

B. Validity

Establishing validity

C. Reliability versus validity

D. Sensitivity

LEARNING OUTCOMES

1. Determine what needs to be measured to address a research question or hypothesis

2. Distinguish levels of scale measurement

3. Know how to form an index or composite measure

4. List the three criteria for good measurement

5. Perform a basis assessment of scale reliability and validity

CHAPTER VIGNETTE: Money Matters?

Griff Mitchell is the Vice-President of Customer Relationship Management (CRM) for one of the world’s

largest suppliers of industrial heavy equipment, and in this role he oversees all sales and service

operations. The company has decided to perform a CRM employee evaluation process that will allow an

overall ranking of all CRM employees, and the ranking will be used to single out the best performers to

be recognized at the annual CRM conference. The rankings will also be used to identify the lowest 20

percent of performers, who will be put on a probationary list with specific improvement goals. Griff’s

key question is, What is performance? So he calls a meeting of senior CRM managers to discuss how the

ranking decisions should be made. One manager argued that sales volume should be the sole criterion

because it is objective. Another said to use the manager’s opinion to classify an employee as a top

performer, a good performer, or an under performer. Another said to use margins. Finally, another said

that because they are CRM, customer satisfaction should be used. This did not help Griff much in his

quest to develop a valid performance measure that treated all employees fairly. So he seeks the help of an

outside research consultant, Robin Donald.

SURVEY THIS!

Students are asked to identify one of each of the four categories of scale measurement and to look at the

question shown. What scale measurement do these items represent? Each set of items is designed to

capture a single construct—the top portion assesses how much work-life interferes with nonwork-life and

the bottom portion assesses self-perceived performance. Compute a coefficient α for each scale to

estimate reliability and then create a composite scale by summing the items that make up that particular

scale.

RESEARCH SNAPSHOTS

Peer Pressure and Investing Behavior

While we all have experienced “peer pressure,” research has shown that some individuals are

more susceptible than others. This interpersonal influence is typically thought to be present in

conspicuous consumption and socially visible products, but recent research shows such influence

can occur even in the selection of less visible products and services, such as investments.

Researchers used the construct susceptibility to interpersonal influence (SCII) to study this. First,

this construct had to be conceptualized and measured and was thought to be composed of two

parts—susceptibility to informational influences and susceptibility to normative influences.

Questions were developed to measure both parts to capture SCII. Results of the study are given.

Recoding Made Easy

Most computer software makes scale recoding easy, and a screenshot from SPSS, perhaps the

most widely used statistical software in business-related research, is shown. The steps include:

(1) click on transform, (2) click on recode, (3) choose to recode into the same variable, (4) select

the variable(s) to be recoded, (5) click on old and new values, (6) use the menu that appears to

enter the old values and the matching new values, and click add after entering each pair, and (7)

click continue.

OUTLINE

I. WHAT DO I MEASURE?

The decision statement, corresponding research questions and research hypotheses can be

used to decide what concepts need to be measured.

Measurement is the process of describing some property of a phenomenon of interest usually

by assigning numbers in a reliable and valid way.

When numbers are used, the researcher must have a rule for assigning a number to an

observation in a way that provides an accurate description.

All measurement, particularly in the social sciences, contains error.

Concepts

A researcher has to know what to measure before knowing how to measure something.

A concept is a generalized idea that represents something of meaning.

Concepts such as age, sex, education and number of children are relatively concrete

properties and present few problems in either definition or measurement.

Concepts such as brand loyalty, corporate culture, and so on are more abstract and are

more difficult to both define and measure.

Operational Definitions

Researchers measure concepts through a process known as operationalization,

which is a process that involves identifying scales that correspond to variance in the

concept.

Scales provide a range of values that correspond to different values in the concept

being measured.

Scales provide correspondence rules that indicate that a certain value on a scale

corresponds to some true value of a concept, hopefully in a truthful way.

Variables

Researchers use variance in concepts to make diagnoses.

Variables capture different concept values.

Scales capture variance in concepts and as such, the scales provide the

researcher’s variables.

For practical purposes, once a research project is underway, there is little

difference between a concept and a variable.

Constructs

Sometimes a single variable cannot capture a concept alone.

Using multiple variables to measure one concept can often provide a more

complete account of some concept than could any single variable.

A construct is a term used for concepts that are measured with multiple

variables.

Can be very helpful in operationlizing a concept.

II. LEVELS OF SCALE MEASUREMENT

The level of scale measurement is important because it determines the mathematical

comparisons that are allowed.

The four levels of scale measurement are:

1. nominal

2. ordinal

3. interval

4. ratio

Nominal Scale

Nominal scales represent the most elementary level of measurement.

Assigns a value to an object for identification or classification purposes.

The value can be but does not have to be a number since no quantities are being

represented.

A qualitative scale.

Useful even though they can be considered elementary.

Business researchers use nominal scales quite often.

Nominal scaling is arbitrary in the sense that each label can be assigned to any of the

categories without introducing error.

Examples of nominal numbering are uniform numbers, airport terminals, and school bus

numbers.

Ordinal Scale

Ordinal scales have nominal properties, but they also allow things to be arranged based

on how much of some concept they possess.

A ranking scale.

Somewhat arbitrary, but not nearly as much as a nominal scale.

Example: “win,” “place,” and “show” in a horse race  tells which horse was first,

second, and third, but does not tell by how much a horse won.

Interval Scale

Interval scales have both nominal and ordinal properties, but they also capture

information about differences in quantities of a concept.

Classic example is the Fahrenheit temperature scale:

80° F is hotter than 40° F, but you cannot conclude that the 40° is twice as cold as

80° because this is a scaling system.

To see the point above, convert the temperatures to the Celsius scale. 80° F =

26.7° C, and 40° F = 4.4° C.

The scale is not iconic, meaning that it does not exactly represent some

phenomenon.

Interval scales are very useful because they capture relative quantities in the form of

distances between observations.

Ratio Scale

Ratio scales represent the highest form of measurement in that they have all the

properties of interval scales with the additional attribute of representing absolute

quantities.

Interval scales represent only relative meaning while ratio scales represent absolute

meaning.

In other words, ratio scales provide iconic measurement.

Zero, therefore, has meaning in that it represents an absence of some concept.

An absolute zero is a defining characteristic in determining between ratio and

interval scales.

For example money is a way to measure economic value.

Mathematical and Statistical Analysis of Scales

Although you can put numbers into formulas and perform calculations with almost any

numbers, the researcher has to know the meaning behind the numbers before useful

conclusions can be drawn (e.g., averaging the numbers used to identify school busses is

meaningless).

Discrete Measures

Discrete measures are those that take on only one of a finite number of values.

Most often used to represent a classificatory variable and thus do not represent

intensity of measures, only membership.

Common discrete scales include any yes-no response, matching, color choice or

practically any scale that involves selecting from a small number of categories.

Nominal and ordinal scales are discrete measures.

Certain statistics are most appropriate for discrete measures (shown in Exhibit 13.5).

The central tendency of discrete measures is best captured by the mode (i.e., most

frequent level).

Continuous Measures

Continuous measures are those assigning values anywhere along some scale range

in a place that corresponds to the intensity of some concept.

Ratio measures are continuous measures.

Strictly speaking, interval scales are not necessarily continuous.

e.g., Likert item ranging from 1=strongly disagree to 5=strongly agree.

This is a discrete scale because only the values 1, 2, 3, 4, or 5 can be assigned.

The mean is not an appropriate way of stating central tendency and we really

shouldn’t use many common statistics on these responses.

However, as a scaled response of this type takes on more values, the error

introduced by assuming that the differences between the discrete points are equal

are smaller.

Therefore, business researchers generally treat interval scales containing 5 or

more categories of response as interval.

Researchers should keep in mind, however, the distinction between ratio and interval

measures.

Errors in judgment can be made when interval measures are treated as ratio.

e.g., An attitude of 0 means nothing as attitude only has meaning in a relative

sense (i.e., compared to another’s attitude).

The means and standard deviation may be calculated from continuous data.

Using the actual quantities from arithmetic operations is permissible with ratio

scales.

III. INDEX MEASURES

An attribute is a single characteristic or fundamental feature of an object, person, situation,

or issue.

Indexes and Composites

Multi-item instruments for measuring a construct are called index measures, or composite

measures.

An index measure assigns a value based on how much of the concept being measured is

associated with an observation.

An index is often formed by putting several variables together.

Composite measures also assign a value based on a mathematical derivation of multiple

variables.

For most practical applications, composite measures and indexes are computed in the

same way.

Computing Scale Values

Exhibit 13.6 demonstrates how a composite measure can be created from common rating

scales.

A summated scale is created by simply summing the response to each item making up

the composite measure.

A researcher may sometimes choose to average the scores rather than summing them

because the composite measure is expressed on the same scale as is the items that make it

up.

Reverse coding means that the value assigned for a response is treated oppositely from

the other items.

IV. THREE CRITERIA FOR GOOD MEASUREMENT

The three major criteria for evaluating measurements are:

1. reliability

2. validity

3. sensitivity

Reliability

Reliability is an indicator of a measure’s internal consistency.

A measure is reliable when different attempts at measuring something converge on the

same result.

Internal Consistency

Internal consistency represents a measure’s homogeneity.

The set of items that make up a measure are referred to as a battery of scale items.

Internal consistency of a multiple-item measure can be measured by correlating

scores on subsets of items making up a scale.

The split-half method of checking reliability is performed by taking half the items

from the scale and checking them against the results from the other half.

The two scale halves should correlate highly.

They should also produce similar scores.

Coefficient alpha (α) is the most commonly applied estimate of a multiple item

scale’s reliability.

Represents internal consistency by computing the average of all possible

split-half reliabilities for a multiple item scale.

The coefficient demonstrates whether or not the different items converge.

Ranges in value from 0 (no consistency) to 1 (complete consistency).

Generally, scales with a coefficient α:

0.80 – 0.95: very good reliability

0.70 – 0.80: good reliability

0.60 – 0.70: fair reliability

below 0.60: poor reliability

Test-Retest Reliability

The test-retest method of determining reliability involves administering the same

scale or measure to the same respondents at two separate times to test for stability.

If the measure is stable over time, the test, administered under the same conditions

each time, should obtain similar results.

Represents a measure’s repeatability.

Two problems common to all longitudinal studies:

the premeasure, or first measure, may sensitize the respondents to their

participation in a research project and subsequently influence the results of

the second measure.

If the time between measures is long, there may be an attitude change or

other maturation of the subjects.

Validity

Good measures should be both precise (i.e., reliable) and accurate (i.e., valid).

Validity is the accuracy of a measure or the extent to which a score truthfully represents a

concept.

Achieving validity is not a simple matter.

Establishing Validity

The four basic approaches to establishing validity are face validity, content validity,

criterion validity, and construct validity.

Face validity refers to the subjective agreement among professionals that a scale

logically reflects the concept being measured.

Content validity refers to the degree that a measure covers the domain of interest.

Criterion validity addresses the question: “Does my measure correlate with

measures of similar concepts or known quantities?”

May be classified as either concurrent validity or predictive validity

depending on the time sequence in which the new measurement scale

and the criterion measure are correlated.

If measures taken at the same time  concurrent validity.

If measures taken at different times  predictive validity.

Construct validity exists when a measure reliably measures and truthfully represents

a unique concept and consists of several components:

Face validity

Content validity

Criterion validity

Convergent validity – another way of expressing internal consistency;

highly reliable scales contain convergent validity.

Discriminant validity – represents how unique or distinct is a measure;

a scale should not correlate too highly with a measure of a different

construct.

Reliability versus Validity

The differences between the two are illustrated with rifle targets (see Exhibit 13.7).

A: The shots from the older gun are scattered  low reliability.

B: The shots from the newer gun are closely clustered and on target  high reliability

and validity.

C: The shots from a newer gun are closely clustered but off target  high reliability

but low validity.

Sensitivity

The sensitivity of a scale is an important measurement concept, particularly when changes in

attitudes or other hypothetical constructs are under investigation.

Sensitivity refers to an instrument’s ability to accurately measure variability in a concept.

Sensitivity is generally increased by adding more response points or adding scale items.