Test Bank
159
CHAPTER 7—EVALUATING HRD PROGRAMS
TRUE/FALSE
1. In order to effectively evaluate a training program the specific objectives of the program must be known?
2. HRD evaluation includes collecting information to make decisions about training activities.
3. According to Goldstein training evaluation has moved from simply collecting anecdotal evidence to now evaluating the entire
HRD process.
4. The CIPP framework for evaluation includes Context, Information, Process, Product
5. When writing a questionnaire you should limit the number of response options.
6. ROI stands for results of inventory.:
7. A control group is a group of people of the same age as the trainees.
8. The text author believes that a positive score on one of Kirkpatrick’s levels will mean a positive score on the measures of other
levels. Positive reactions will result in learning.
9. The reliability of data refers to the consistency of that data.
10. Using deception to do evaluation research is a very good idea – it is ethical and moral and employees generally like it when this is
used.
11. A control group is just a second group of trainees
Chapter 7—Evaluating HRD programs
160
12. Utility analysis is easy for trainers to calculate
13. Ideally, the control group and the training group have the same scores before training
14. Donald Kirkpatrick described the following four training evaluation criteria: reaction, learning, job behavior and results
15. Confidentiality in training evaluation is not important – the trainees work for the organization already.
16. A proposed modification to Kirkpatrick’s evaluation model is to add a fifth level beyond results to include ROI.
17. A dramatic extension beyond Kirkpatrick’s framework includes the idea that constituencies outside the organization should be
involved in the evaluation process.
18. Learning should be measured at two time:: pre training and post training.
19. A stakeholder is defined as ‘a person or group with an interest in seeing an endeavor succeeds but who do not need to support
that activity”.
20. According to Nickols each of Kirkpatrick’s levels is of similar importance to all stakeholder groups.
Test Bank
161
MULTIPLE CHOICE
1. The evaluation phase of the HRD process model includes all of the following except:
a.
Selecting evaluation criteria
b.
Conducting the evaluation
c.
Interpreting the results
d.
Publishing your results in the ASTD results book
2. According to Chris Bones:
a.
Evaluating what you know works is a waste of time
b.
Only evaluate if you are forced to do it
c.
Evaluation is too costly to bother with
d.
If you can’t see results stop it
3. 2. According to Phillips which of the following is not a purpose of training evaluation?
a.
determine the cost-benefit ration of an HRD program
b.
determine if the program was appropriate
c.
decide who should participate in future programs
d.
decide whether or not to fire the trainer
4. Based on ASTD surveys about evaluating training programs which of the following is true?
a.
Over 90% of companies surveyed conduct some form of evaluation
b.
Only about 10% of companies surveyed conduct some form of evaluation
c.
Results are the most commonly used method of evaluating training
d.
Reactions are almost never evaluated
5. The most popular and influential framework for evaluating training was developed by:
a.
Goldstein
c.
b.
Kirkpatrick
d.
6. Level 3 of Kirkpatrick’s framework for evaluation is:
a.
Reaction
c.
b.
Learning
d.
7. When collecting data which of the following issues is not important?
a.
Reliability
c.
b.
Validity
d.
Chapter 7—Evaluating HRD programs
162
8. The number of units produced is generally what kind of data?
a.
Plant data
c.
Systemwide data
b.
Economic data
d.
Individual performance data
9. Which of the following is not a common type of data collected for evaluation?
a.
Plant data
c.
Systemwide data
b.
Economic data
d.
Individual performance data
10. The Kraiger, Ford & Salas evaluation model uses all of the following classifications of learning outcomes except:
a.
Cognitive
c.
Skill based
b.
Intuitive
d.
Affective
11. A socially desirable response is
a.
Telling the researcher your true feelings
b.
Telling the researcher the truth
c.
Telling the researcher what you think they want to hear
d.
Telling the research an answer in sociological terms
12. Self-report data is:
a.
Data provided by an employee’s supervisor
b.
Data provided by the individual involved in the training
c.
Data from company archives
d.
Data provided by customers
13. If we provide the training and then collect evaluation measures we are:
a.
Sure that any change we see is due to the training
b.
Unsure as what caused the result we find, the training or ?
c.
OK – but only if we randomly select the trainees
d.
Following the best known research design
14. Collecting data in a pretest and posttest design allows us to:
a.
Be certain the training caused any change
b.
See what has changed after training
c.
Develop the cost effectiveness of our training
d.
Know whether or not we should revise the training design.
Test Bank
163
15. Questionnaires are often used for evaluation because:
a.
trainees like to fill them out
b.
they can be completed and analyzed quickly
c.
they are very interactive
d.
they measure all 4 levels quickly and easily
16. According to studies by ASTD, the most common form of evaluation is:
a.
c.
behavior
b.
d.
results
17. A pre/post comparison is used to evaluate learning so that:
a.
we know how much change took place during the program
b.
we can calculate a t-test to get nice-looking statistics
c.
we can compare the trained group to the control group
d.
we know where the group was at the start
18. Which of the following statements provides the best explanation for why training professionals often don’t evaluate the results of
their training programs:
a.
results don’t matter in training
b.
fear of criticism or program cuts if a program was not effective
c.
it can be done only if you majored in statistics in college
d.
management refuses to spend the money to do it right
19. Trainee reaction to a training program is:
a.
critical – if they don’t like it, they can’t learn
b.
limited because it does not tell us if the program met its objective
c.
in most cases, the only evaluation method that can be accomplished
d.
the only level of evaluation worth doing
20. A control group is:
a.
a second group of trainees
b.
of no value in helping us evaluate training programs
c.
often critical to determining whether changes were due to a training program rather than some other factor
d.
an added cost that is generally too expensive
21. Which of the following is not a purpose of training evaluation?
a.
deciding who should participate in future programs
b.
helping to design future training programs
c.
establishing a data base for management
d.
all of the above are purposes of training evaluation
Chapter 7—Evaluating HRD programs
164
22. Why is it a good idea to randomly assign people to the training group and the control group?
a.
it is the easiest way to select equivalent groups
b.
it is the easiest way to measure both pre and post training
c.
it guarantees that training evaluation will be believed by management
d.
it allows us to assume that attitude toward training is positive for all trainees
23. Confidentiality in training evaluation is:
a.
not important – the trainees work for the organization already
b.
up to the trainer – they should not discuss what they find out during training
c.
a good reason not to evaluate a training program
d.
assured by using code numbers, rather than names, to identify trainees
24. ROI stands for:
a.
return on inventory
c.
referral of information
b.
reduction of indirect costs
d.
return on investment
25. A cost-benefit analysis compares:
a.
The benefits of one program to those of another one
b.
The cost of training to its benefits
c.
The costs of two different HRD programs
d.
Actually – any two numbers you want to use would work
26. Donald Kirkpatrick described which of the following four training evaluation criteria?
a.
reaction, learning, job behavior, modification
b.
reaction , learning, job behavior and results
c.
context, input, process and product
d.
context, input, process and modification
27. Which of the following is NOT an issue to consider when determining which data collection method to use in HRD evaluation?
a.
reliability
c.
provability
b.
validity
d.
practicality
28. The most commonly used type of data in HR evaluation is:
a.
self-report data
c.
customer satisfaction surveys
b.
return on investment
d.
organizational report data
29. A method of evaluation, which measures the trainee’s Evaluation prior to and after training, is known as:
a.
return on investment
c.
pre-test, post-test
b.
variance improvement method
d.
gap closure method
Test Bank
165
30. All of the following are guidelines for writing effective questionnaires except:
a.
Combine several ideas into one question to save on paper
b.
Use clear and simple questions
c.
Limit the number of response options
d.
Match the mode of the response to the question
31. Which of the following is NOT an example of an ethical issue in evaluation research?
a.
confidentiality
c.
b.
informed consent
d.
32. Recent efforts to combine data from different research studies are known as:
a.
macro-analysis
c.
b.
meta-analysis
d.
33. Validity asks the question:
a.
Does the information evaluate a trainees reaction
b.
Is the measure consistent
c.
Does the data measure what it is supposed to measure
d.
Is the data legal
34. Practicality of a measure deals with:
a.
Is it consistent
b.
Does it measure what it is supposed to measure
c.
Is it legal
d.
Is there sufficient time, money and resources for this evaluation method
35. Which of the following is NOT an example of indirect training program costs?
a.
instructor preparation
b.
administrative support
c.
course materials
d.
all of the above are examples of indirect costs
36. Which of the following is NOT a factor used to compute a utility estimate?
a.
“A” ( available number of employees for training
b.
“N” ( number of employees trained
c.
“T” ( length of time benefits of training is expected to last
d.
“C” ( cost of conducting the training
Chapter 7—Evaluating HRD programs
166
37. Ideally, the control group and the training group have:
a.
A lot in common
c.
The same supervisors
b.
The same scores before training
d.
The exact same job description
38. Informed consent:
a.
Informs participants about any risks
b.
Must be obtained before any training is done
c.
Is completed at time of hire for all employees
d.
Is never needed
39. Utility analysis:
a.
Is easy for trainers to calculate
b.
Compares training gain to training costs
c.
Is done using a computer program like Blackboard
d.
Has never been used for training
40. Goldstein has suggested that training evaluation has moved through ______ stages since the 1960’s.
a.
4
c.
2
b.
5
d.
22
Test Bank
167
APPENDIX 7-1
MORE ON RESEARCH DESIGN
TRUE/FALSE
1. A control group is a group of people of the same age as the trainees.
2. A control group is just a second group of trainees
3. Ideally, the control group and the training group have the same scores before training
4. Internal validity deals with the question – could something besides the training program have caused the observed change to
occur.
5. External validity means you are comparing your results to those of similar size organizations.
6. History means that unrelated events occur during the training process that influences the training measurements.
7. Statistical power is the probability of concluding there is a difference between the training and the control groups when such a
difference actually exists.
8. A case study involves training followed by post training measurement.
9. Relational research involves measuring two or more variables to see if they are measuring the same thing.
10. Sackett and Mullen begin their analysis of non-experimental research design by asking the question “How much change has
occurred”.
Appendix 7-1 More on Research Design
168
MULTIPLE CHOICE
1. Which of the following is not a form of research design validity?
a.
Internal
c.
Statistical conclusion
b.
External
d.
Statistical initial
2. Internal validity can be affected by all of the following, except:
a.
History
c.
Differential selection
b.
Maturation
d.
Statistical abnormality
3. Internal validity can be affected by all of the following, except:
a.
Instrumentation
c.
Differential investigation
b.
Testing
d.
Experimental mortality
4. Maturation refers to:
a.
Gaining job experience
b.
The effect of a pretest on a posttest score
c.
Differential loss of respondents from various groups
d.
Using different procedures to select individuals for experimental and control groups
5. Differential selection refers to:
a.
Gaining job experience
b.
The effect of a pretest on a posttest score
c.
Differential loss of respondents from various groups
d.
Using different procedures to select individuals for experimental and control groups
6. Testing refers to:
a.
Gaining job experience
b.
The effect of a pretest on a posttest score
c.
Differential loss of respondents from various groups
d.
Using different procedures to select individuals for experimental and control groups
7. A case study research design:
a.
Is easy to use to evaluate training
b.
Is questionable since there is no pre training information
c.
Measures only results
d.
Is very commonly used to evaluate training
Test Bank
169
8. A quasi-experimental research design could be:
a.
A non equivalent control group
b.
A time series
c.
A pre post test
d.
Both A and B are quasi-experimental designs
9. Collecting data in a pretest and posttest design allows us to:
a.
Be certain the training caused any change
b.
See what has changed after training
c.
Develop the cost effectiveness of our training
d.
Know whether or not we should revise the training design.
10. A pre/post comparison is used to evaluate learning so that:
a.
we know how much change took place during the program
b.
we can calculate a t-test to get nice-looking statistics
c.
we can compare the trained group to the control group
d.
we know where the group was at the start
11. A control group is:
a.
a second group of trainees
b.
of no value in helping us evaluate training programs
c.
often critical to determining whether changes were due to a training program rather than some other factor
d.
an added cost that is generally too expensive
12. Ideally, the control group and the training group have:
a.
A lot in common
c.
b.
The same scores before training
d.
13. Relational research typically involves the use of what type of statistical measurement?
a.
Regression
c.
b.
Correlation
d.
14. According to the text what is true about experimental design evaluations:
a.
They are easy to use in almost any organization
c.
b.
They are cheaper to do than case studies
d.
15. To control costs while increasing statistical power Yang et. al. recommend
a.
Training more people
c.
b.
Placing more people in the control group
d.