1. Which of the following is NOT related to test worthiness?
a. Reliability
b. Statistical formation
c. Cross-cultural fairness
d. Validity
e. Practicality
2. Which of the following has the strongest correlation?
a. −.87
b. +.86
c. + 0
d. + .3
3. Which of the following has a correlation of approximately .70?
a. b.
c. d.
Chapter5—TestWorthiness:Validity,Reliability,CrossCulturalFairness,andPracticality
4. A correlation coefficient of .60 was found when correlating the scores of 1,000 randomly chosen individuals who
took a test that measures verbal reasoning with a test that measures problem solving abilities. Which statement is
true about this correlation coefficient?
a. The coefficient of determination is .60.
b. The shared variance is .49.
c. The shared variance is .36.
d. The shared variance is 1.2.
e. None of these are true.
5. Which of the following is an example of negative correlation?
a. Caloric intake and a person’s weight
b. Exercise and a person’s weight
c. Height and a person’s weight
d. All of these are examples.
e. None of these are examples.
6. Which of the following scatterplots would best identify the most reliable test-retest correlation?
a. b. c.
7. Which of the following scatterplots would best define the relationship between intelligence (IQ) and height?
a. b. c.
8. Which of the following does NOT describe validity?
a. It is the degree to which all of the accumulated evidence supports the intended interpretation of test scores for
the intended purpose.
b. It is a unitary concept.
c. It examines how well a test measures what it is supposed to measure.
d. It tells you whether or not test scores are consistent.
9. Which of the following is NOT a step in determining the content validity of an instrument?
a. The test developer adequately surveyed the content domain.
b. The content of the test was shown to match what was found in the survey of the domain.
c. The test items accurately reflect the content.
d. The test is shown to be highly correlated with another, similar test.
e. The number of items for each content area matches their relative importance.
10. Which of the following is a type of criterion-related validity?
a. Content validity
b. Predictive validity
c. Convergent validity
d. Discriminant validity
e. None of these are criterion-related.
11. Concurrent and predictive validity are types of:
a. content validity.
b. criterion-related validity.
c. construct validity.
d. test-retest reliability.
e. internal consistency reliability.
12. If you are interested in examining whether or not the GREs can accurately tell how students perform in graduate
school, you would most likely want to investigate the ________ validity of the test.
a. content
b. concurrent
c. predictive
d. construct
e. convergent
13. A test developer asks five clinicians who are experts in depression to rank 100 depressed clients on a 1 to 5 scale
from minor depression to severe depression. The rankings are then correlated with test scores on a new test of
depression. What kind of validity is being measured?
a. Experimental
b. Concurrent
c. Predictive
d. Construct
e. Convergent
14. Which of the following is NOT a kind of construct validity?
a. Experimental design validity
b. Factor analysis validity
c. Convergent validity
d. Criterion-related validity
e. Discriminant validity
15. In assessing the validity of a test, you randomly assign 100 individuals to two groups: one group assesses intelligence
through a newly developed test, and a second group assesses intelligence through a traditional intelligence test that
has been shown to be valid. You then conduct a statistical analysis to see if there are significant differences between
the groups. Your hypothesis states that there will be no significant differences. This type of validity is called:
a. experimental design validity.
b. factor analysis validity.
c. convergent validity.
d. criterion-related validity.
e. discriminant validity.
16. In developing a multiple aptitude test, the test developers wanted to assure that the various subtests do not correlate
strongly with one another, thus assuring the integrity of each subtest. They conduct a number of intercorrelations
among subtests. This type of validity is called:
a. experimental design validity.
b. factor analysis validity.
c. convergent validity.
d. criterion-related validity.
e. predictive validity.
17. A test developer creates a new instrument to measure psychotic tendencies. She correlates this instrument with an
existing test that measures psychotic tendencies that has been shown to be valid. This type of validity is called:
a. experimental design validity.
b. factor analysis validity.
c. convergent validity.
d. criterion-related validity.
e. discriminant validity.
18. A test developer creates a new instrument to measure depression. He correlates this instrument with an existing test
that measures anxiety. The test developer hopes to not find a high correlation, thus assuring the integrity of the
construct he is measuring. This type of validity is called:
a. experimental design validity.
b. factor analysis validity.
c. convergent validity.
d. criterion-related validity.
e. discriminant validity.
19. A newly developed instrument for screening depression suggests that an individual is not depressed; however, later
information reveals that in fact he is depressed. This problem can be attributed to:
a. parallel forms.
b. internal consistency.
c. a false positive.
d. a false negative.
e. sampling error.
20. When a test has been shown to produce consistent test results, it has which of the following?
a. Reliability
b. Validity
c. Practicality
d. Cross-cultural fairness
e. None of these
21. Which of the following does NOT describe reliability?
a. The degree to which test scores are free from errors of measurement
b. The degree to which test scores are consistent and accurate
c. The degree to which a test measures what it’s supposed to measure
d. All of these describe reliability.
22. Which of the following is NOT a type of reliability?
a. Test-retest
b. Alternate forms
c. Parallel forms
d. Internal consistency
e. All of these are types of reliability.
23. Which of the following is NOT a type of reliability?
a. Split-half
b. Internal consistency
c. Cronbach’s Coefficient Alpha
d. Kuder-Richardson
e. All of these are types of reliability.
24. In assessing the reliability of a new test, a sample of 1,000 examinees take the test, and one week later are asked to
take the same test again. This is an example of which type of reliability?
a. Test-retest
b. Alternate forms
c. Internal consistency
d. Odd-even
e. None of these
25. One problem with giving two forms of the same test is that, despite the publisher’s attempt to make them similar,
there could be major differences. This problem is most closely associated with which type of reliability?
a. Test-retest
b. Alternate forms
c. Internal consistency
d. Odd-even
e. Kuder-Richardson
26. “A determination is made as to how scores on individual items relate to each other or to the test as a whole.” This
statement refers to which kind of reliability?
a. Test-retest
b. Alternate forms
c. Parallel forms
d. Internal consistency
e. None of these
27. The Spearman-Brown formula is for which purpose?
a. To make adjustments due to concerns that examinees might look up answers between taking the same test
twice
b. To make adjustments to reliability estimate due to concerns of shortness of the test (limited test items)
c. To adjust correlations during factor analysis to assure accurate representation of all items
d. To calculate the coefficient of determination (shared variance) when making adjustments to reliability
estimates
28. A type of reliability in which half of the test items are correlated with the other half is called:
a. test-retest.
b. alternate forms.
c. parallel forms.
d. odd-even.
e. none of these.
29. When one “attempts to estimate the reliability of all the possible split half combinations” of a test, one is assessing
what kind of reliability?
a. Test-retest
b. Alternate forms
c. Parallel forms
d. Coefficient Alpha
e. None of these
30. Theoretically, if several individuals who score high on a test have a substantial number of low responses, and
individuals who score low on the test have a substantial number of high responses, then the test is likely to have
which of the following?
a. High validity
b. Low validity
c. High reliability
d. Low reliability
31. Which of the following is NOT a type of internal consistency reliability?
a. Odd-even
b. Parallel forms
c. Coefficient Alpha
d. Kuder-Richardson
e. All of these are types of internal consistency reliability.
32. Which of the following is true regarding Classical Test Theory and/or Item Response Theory (IRT)?
a. Classical Test Theory provides more sophisticated information regarding test items.
b. IRT examines items individually to assess their ability to measure the trait.
c. In IRT, test items are viewed as a whole in an attempt to reduce measurement error.
d. As reliability increases in Classical Test Theory, the measurement error increases.
33. Which is true regarding the item characteristic curve, or “S” curve, in Item Response Theory (IRT)?
a. It assumes that as people’s abilities increase, their probability of answering the item correctly decreases.
b. A tall “S” means that an item has difficulty discriminating across ability.
c. IRT and the S curve assist in writing items to target specific ranges of ability.
d. It is useful in predicting the Standard Error of the Estimate.
34. Griggs v. Duke Power Company asserted:
a. that accommodations must be made for individuals who are taking tests for employment and that testing must
be shown to be relevant to the job in question.
b. the right of all individuals to their school records, including test records.
c. that individuals with disabilities, or who are disadvantaged, have access to vocational assessment, counseling,
and placement.
d. that tests used for hiring and advancement at work must show that they can predict job performance for all
groups.
e. that any instrument used to measure appropriateness for a program or service must be measuring the
individual’s ability, not be a reflection of his or her disability.
35. The Americans with Disabilities Act asserted:
a. that accommodations must be made for individuals who are taking tests for employment and that testing must
be shown to be relevant to the job in question.
b. that any test used for employment or promotion must be shown to be suitable and valid for the job in question.
c. that tests used for hiring and advancement at work must show that they can predict job performance for all
groups.
d. the right of individuals to access their federal records, including test records.
e. the right of students to be tested, at a school system’s expense, if they are suspected of having a disability that
interferes with learning.
36. The Buckley Amendment, otherwise known as FERPA, asserted:
a. that accommodations must be made for individuals who are taking tests for employment and that testing must
be shown to be relevant to the job in question.
b. that tests used for hiring and advancement at work must show that they can predict job performance for all
groups.
c. the right of all individuals to their school records, including test records.
d. that individuals with disabilities, or who are disadvantaged, have access to vocational assessment, counseling,
and placement.
e. the right of students to be tested, at a school system’s expense, if they are suspected of having a disability that
interferes with learning.
37. The Carl Perkins Act (PL 98-524) asserted:
a. that any test used for employment or promotion must be shown to be suitable and valid for the job in question.
b. that tests used for hiring and advancement at work must show that they can predict job performance for all
groups.
c. the right of individuals to access their federal records, including test records.
d. the right of students to be tested, at a school system’s expense, if they are suspected of having a disability that
interferes with learning.
e. that individuals with disabilities, or who are disadvantaged, have access to vocational assessment, counseling,
and placement.
38. PL 94-142 and the IDEA asserted:
a. that accommodations must be made for individuals who are taking tests for employment and that testing must
be shown to be relevant to the job in question.
b. the right of all individuals to their school records, including test records.
c. that individuals with disabilities, or who are disadvantaged, have access to vocational assessment, counseling,
and placement.
d. the right of students to be tested, at a school system’s expense, if they are suspected of having a disability that
interferes with learning.
39. The Freedom of Information Act asserted:
a. the right of students to be tested, at a school system’s expense, if they are suspected of having a disability that
interferes with learning.
b. that individuals with disabilities, or who are disadvantaged, have access to vocational assessment, counseling,
and placement.
c. that any instrument used to measure appropriateness for a program or service must be measuring the
individual’s ability, not be a reflection of his or her disability.
d. the right of individuals to access their federal records, including test scores.
e. the right of students to be tested, at a school system’s expense, if they are suspected of having a disability that
interferes with learning.
40. Section 504 of the Rehabilitation Act asserted:
a. that any instrument used to measure appropriateness for a program or service must be measuring the
individual’s ability, not be a reflection of his or her disability.
b. the right of individuals to access their federal records, including test records.
c. the right of students to be tested, at a school system’s expense, if they are suspected of having a disability that
interferes with learning.
d. that tests used for hiring and advancement at work must show that they can predict job performance for all
groups.
e. that accommodations must be made for individuals who are taking tests for employment and that testing must
be shown to be relevant to the job in question.
41. Rather than giving individualized tests to assess learning disabilities to thousands of students, a school system decides
to first use a shortened version of the test that can be given in a group format. This decision is mostly concerned
with which test worthiness issue?
a. Reliability
b. Statistical formation
c. Cross-cultural fairness
d. Validity
e. Practicality
42. Practicality has to do with all of the following EXCEPT:
a. time.
b. cost.
c. format and readability.
d. ease of administration, scoring, and interpretation.
e. Practicality has to do with all of these.
43. According to the text, which of the following is NOT a step to selecting and administering a good test?
a. Determining the goals of your assessment
b. Accessing information about possible instruments
c. Examining the validity, reliability, cross-cultural fairness, and practicality of possible instruments
d. Making a wise choice when choosing an instrument
e. Running a “pilot study” to assess the instruments you have chosen
44. In selecting a test, one often goes to existing sources including all of the following EXCEPT:
a. experts in the field.
b. journals in the field.
c. The Assessment Catalog and Resource Index.
d. Mental Measurement Yearbook.
e. Buros.
45. A correlation coefficient of .22 is stronger than a correlation coefficient of −.87.
a. True
b. False
46. The square root of a correlation coefficient gives the coefficient of determination, otherwise known as the shared
variance.
a. True
b. False
47. Probably, the most basic form of validity is content validity.
a. True
b. False
48. The most important form of content validity is face validity.
a. True
b. False
49. Concurrent validity is sometimes called “here and now” validity.
a. True
b. False
50. Concurrent validity is to criterion-related validity as discriminant validity is to construct validity.
a. True
b. False
51. One problem with parallel forms reliability is that individuals may look up questions between the time they take the
two tests.
a. True
b. False
52. Alternate forms reliability eliminates some of the problems found in test-retest reliability but raises other concerns.
a. True
b. False
53. One problem with split-half reliability is that due to fact that you are correlating two shorter halves, you can create
greater error in your correlation coefficient.
a. True
b. False
54. Item Response Theory (IRT) has the advantage over Classical Test Theory in that it provides more detailed
information regarding each item in a test.
a. True
b. False
55. Cross-cultural fairness in testing has always been a critical factor in the development of tests.
a. True
b. False
Match each description to the term or concept listed below.
a. Validity
b. Reliability
c. Correlation
d. SEest
REFERENCES: Correlation Coefficient
Validity
Reliability
56. Uses the score of one variable to predict a range of scores for a second variable.
57. The degree to which an instrument’s scores are free from error of measurement.
58. What the instrument measures and how well it does it.
59. The statistic used to express a reliability estimate.
60. A statistical expression of the relationship between sets of scores.
61. The degree to which test scores are consistent, dependable, and repeatable.
Chapter5—TestWorthiness:Validity,Reliability,CrossCulturalFairness,andPracticality
Match the terms with their correct descriptive statement.
a. The extent to which the instrument measures a theoretical or hypothetical trait or concept
b. Tests scores from one version of a test are correlated with another “alternate” version
c. The extent to which an instrument correlates with an outcome criterion in the present
d. Correlating one test administration with another administration of the same test at a later date
e. The degree the evidence shows items and questions represent the proper domain
f. The correlation of odd numbered items on a test with even numbered items
g. The extent to which the instrument correlates with future outcome criterion
h. A statistical analysis of each test item against all of the other test items
REFERENCES: Validity
Reliability
62. Split-half reliability
63. Parallel form reliability
64. Kuder-Richardson reliability
65. Test-retest reliability
66. Content validity
67. Concurrent validity
68. Predictive validity
69. Construct validity
Chapter5—TestWorthiness:Validity,Reliability,CrossCulturalFairness,andPracticality
Choosing the Correct Item
In developing the national certification exam for counselors, the test developers wanted to assure adequate validity
and reliability. For the following items, place the letter that represents the kind of validity or reliability described.
a. Content validity
b. Concurrent validity
c. Construct validity
d. Discriminant validity
e. Parallel or equivalent form reliability
f. Internal consistency
g. Predictive validity
h. Odd-even reliability
i. Test-retest reliability
REFERENCES: Validity
Reliability
70. All the possible split-half correlations are correlated and then the mean, or average, of all of these scores is obtained.
71. The test is correlated with the national comprehensive exams to assure that the two tests are not measuring the
same thing.
72. Students who are graduating from their master’s degrees take the test, and, at the same time, their test scores are
correlated with professor rankings of how good a student they are.
73. Student scores on the exam are correlated with how well supervisors rate them two years later.
74. Half of the test is correlated with the other half of the test.
75. In creating the test, they hypothesized that students who finished a master’s degree in counseling would do
significantly better than students who received a master’s in psychology. Then, a study is conducted examining this
hypothesis.
Chapter5—TestWorthiness:Validity,Reliability,CrossCulturalFairness,andPracticality
76. The test is given to a group of students and then, one day later, they take the same test again.
77. In developing the test, counseling books were surveyed and experts were contacted in an effort to create items.
78. Students take two different versions of the same test and their scores are correlated.