CHAPTER 3
Validity of Assessment Results
ITEMS
1. Validity of assessment refers to the soundness of
(a) the assessment instrument.
(b) your interpretation and use of the assessment results.
(c) the reliability of the interpretation and use of the results.
2. Which of the following conditions is sufficient for the validation of a classroom test in social
studies?
(a) Evidence that the assessment results are internally consistent.
(b) Evidence that the assessment covers the curriculum.
(c) Evidence that the content covers what the teacher wants to assess.
(d) None of the above is sufficient.
3. Spelling test results that are valid for ranking students in spelling contests are likely to be equally
valid for evaluating the types of word patterns students can spell.
(a) True
(b) False
4. In order to judge the validity of assessment results, one should look primarily for
(a) a single sound research study demonstrating the appropriateness of the interpretation.
(b) a combination of sound research and logical explanation demonstrating the appropriateness
of the interpretations.
(c) several studies demonstrating the consistency of the scores.
5. How can a teacher attain a high level of consistency in scoring his/her assessment tasks?
(a) Make the tasks relatively easy for students
(b) Prepare and use scoring rubrics
(c) Score the tasks using personal knowledge of each student’s past performance
6. Evidence that a social studies test adequately samples a specific area of the social studies
curriculum is called
(a) content-related evidence.
(b) substantive evidence.
(c) external structure evidence.
7. Test scores that are highly valid are necessarily reliable.
(a) True
(b) False
8. Test scores that are highly reliable are necessarily valid.
(a) True
(b) False
9. Which of the following procedures should a general science teacher use to help validate her end-
of-year classroom grades?
(a) Use the end-of-year test as a criterion.
(b) Compare the grades with students’ test performance on several different assessments over
the year.
(c) Correlate the scores from the first half of the end-of-year test with scores from the second
half.
10. The argument-based approach to validation implies that rhetoric is just as important as evidence in
demonstrating the validity of interpretations and uses of assessment results.
(a) True
(b) False
11. A table of specifications is a table showing the
(a) specific tasks or items in a test.
(b) difficulty level of each task or item.
(c) the content areas and skills covered in a test.
12. Which of the following is the best explanation of why the results of a grade three standardized
achievement test in mathematics cannot be validly used by a grade three teacher for assigning
grades at the end of the semester?
(a) The items are likely to be too difficult for the students.
(b) The items are not likely to emphasize what the teacher taught the students.
(c) The items are likely to be ambiguous for many students.
13. What should be included as evidence to support the claim that the results of an assessment
procedure validly reflect important mathematics thinking skills and processes?
(a) Expert judgments on the thinking processes and skills involved in students’ solutions of the
tasks.
(b) A clear description of the thinking processes and skills purported to be assessed by the
tasks.
(c) Teachers’ judgments that the assessment covers the topics listed in the curriculum.
(d) Both (a) and (b) should be included.
14. A school district wants to use assessments to evaluate its social studies program. Which of the
following would be potentially valid types of assessment instruments?
(a) Standardized tests in social studies that covered desirable content and skills.
(b) Tasks crafted specifically to match the social studies curriculum of the district.
(c) Tasks developed based on the social studies curriculum of one of the best schools in them
district.
(d) All of the above would be potentially valid sources of assessment instruments for program
evaluation.
(e) Only (b) and (c) would be potentially valid sources of assessment instruments for program
evaluation.
15. Evidence on the internal structure of an assessment instrument concerns how the tasks on the
instrument are
(a) structured from low to high difficulty levels.
(b) interrelated in measuring the same construct.
(c) organized into sections according to format of task.
16. Which type of validity evidence is weighted most heavily in college admissions assessment?
(a) Content representativeness
(b) External structure
(c) Internal structure
Items 17 – 19 refer to the instructions that follow.
Read each of the numbered statements and decide which type of validity evidence it illustrates. Write
the letter of your choice of validity evidence on your answer sheet.
A — if the validity evidence illustrates content representativeness
B — if the validity evidence illustrates external structure of the test results
C — if the validity evidence illustrates generalization of the test results
D — if the validity evidence illustrates the internal structure of the task scores
17. An English language instructor gave a verbal aptitude test to a class of 50 students. Later, the
instructor compared these scores to their grades at the end of the first term.
18. After administering a test of 100 multiple-choice items to a class of 50 students, the teacher
correlated the students’ item scores with their total scores on the test.
19. When crafting her test items for the end-of-term examination in social studies, Ms. Morrison
checked each item to see if it matched the material that she taught the class.
20. What index quantifies the degree to which two variables are related?
(a) Standard error of measurement
(b) Standard error of estimate
(c) Correlation coefficient
(d) Coefficient of relationship
Item 21 refers to the information below.
Below are scores for five students from three tests.
Student
Mathematical
Reasoning
Verbal
Reasoning
Spatial
Reasoning
John
Jane
Mauves
Ken
Lori
80
77
75
70
65
75
80
78
75
70
75
72
73
70
68
21. Which pair of the tests has the highest positive correlation?
(a) Mathematical Reasoning and Verbal Reasoning
(b) Mathematical Reasoning and Spatial Reasoning
(c) Verbal Reasoning and Spatial Reasoning
22. The correlation between Test A scores and mathematics grades is –.85 while the correlation
between Test B scores and mathematics grades is .70. Which of the two test scores will give a
better prediction of a person’s mathematics grades?
(a) Test A
(b) Test B
23. Suppose it has been established in a particular high school that the correlation between scores on
Test X and scores on Test Y scores –0.75. This means that the score of a student on Test X is
likely to be high if his score on Test Y is low.
(a) True
(b) False
24. The correlation coefficient of 0.95 between mathematics and physics scores of a group of students
means that high performance in mathematics causes high performance in physics.
(a) True
(b) False
25. The same test was administered twice to two groups of students. The students were similar in
ability and achievement. Group I was administered the test on two consecutive days. Group II
was administered the test at the beginning and the end of the month. Which groups’ test-retest
scores will have a lower correlation?
(a) I
(b) II
(c) We cannot infer from the information given.
26. A teacher administered a 100-item multiple-choice language aptitude test to two groups of
students. All the students in one group have nearly the same language aptitude. The students in
the other group differ greatly in their language aptitude. If the aptitude test scores are correlated
with the students’ French course grades, for which group would a higher correlation be obtained?
(a) the group in which the students differ greatly in their language aptitude
(b) the group in which the students have nearly the same language aptitude
(c) It is impossible to infer the outcome without knowing the actual scores of the students.
27. One of the factors that positively affects the correlation coefficient between two tests is the
similarity of the constructs being measured by the assessments.
(a) True
(b) False
Items 28 through 30 refer to the table below.
Expectancy Table Predicting College Math Performance from Aptitude Test Scores
Aptitude Test Scores
Poor
Average
Good
Outstanding
80-99
60-79
40-59
20-39
10-19
1-9
15 %
17%
25%
33%
45%
63%
25%
28%
35%
32%
28%
23%
25%
32%
23%
20%
18%
12%
35%
23%
17%
15%
9%
2%
28. What percentage of students with aptitude scores less than 10 were above average in college math
performance?
(a) 12%
(b) 14%
(c) 23%
(d) 37%
29. What is the probability that a student with an aptitude test score of 80 or higher would perform
below average in college math?
(a) 0.15
(b) 0.25
(c) 0.35
(d) 1.00
30. Suppose all students who obtained aptitude scores below 10 were refused admission to college
level math courses. For what percentage of them would have this have been a poor decision?
(a) 12%
(b) 37%
(c) 63%
(d) 100%
31. Which of the following is the most important characteristic of a good criterion measure?
(a) Practicality of the instrument
(b) Validity of the scores in assessing the tasks to be predicted
(c) Usability of the measure
32. Which of the following can be used as evidence to judge the external structure of test results?
(a) The reliability of the test
(b) The correlation coefficient between the test and a criterion
(c) The practicality of the test in terms of cost and efficiency
(d) The judgments of external content experts
33. Which of the following is LEAST important for a criterion measure in a validation study to have?
(a) freedom from bias against individuals or groups
(b) high reliability
(c) norm group comparisons
(d) relevance to long-term performance
34. An algebra teacher wants to see how well the grades she assigns her students correlate with their
recent scores on a standards-based math assessment. Which of the following should she be
interested in?
(a) concurrent validity evidence
(b) internal consistency evidence
(c) predictive validity evidence
35. Petra took a standardized, norm-referenced achievement test with two accommodations: extra time
and the use of a study carrel. How should you report her results?
(a) Report her norm-referenced scores in the same manner as for any student.
(b) Report her norm-referenced scores and explain the accommodations used.
(c) Explain the accommodations and give examples of some of the questions she could answer.
CHAPTER 3
Answers and Classification of Items