Deck 13: How to Evaluate a Test

Question 1

Tests used in making important educational decisions for students should have reliability estimates of at least

A) 0.90.
B) 0.60.
C) 0.70.
D) 0.95.

Answer 1

A

Question 2

Better norm groups are selected on the basis of

A) all levels of socioeconomic status.
B) urban and suburban communities.
C) the highly populated areas of the United States.
D) the two largest racial and ethnic groups.

Answer 2

A

Question 3

If an evaluator finds that the technical adequacy of a test is generally good, then

A) one can use it with all students.
B) one should consider if it seems appropriate to use it for the specific student to be assessed.
C) the inferences can be trusted.
D) it has strong reliability and validity, the key elements to be concerned with.

Answer 3

B

Question 4

Good technical adequacy of a test at minimum includes

A) general validity and a sizable norm sample.
B) reliable scores and general validity.
C) representative norms, reliable scores, and evidence for general validity.
D) clear purposes, reliable scores, and evidence for general validity.

Question 5

If a test has limited reliability, then

A) authors should bolster content validity.
B) authors should present convincing evidence of general validity.
C) its validity must also be lacking.
D) the test should be abandoned.

Question 6

When test authors allow certain accommodations to be used, they

A) must also give separate norms for students with disabilities.
B) should provide evidence that the accommodations produce scores with the same meaning.
C) should include certain groups of students with disabilities in their norm groups.
D) should provide more flexible materials and directions.

Question 7

Your book authors recommend that there should be less than a_____ discrepancy between a majority group's proportion in the norm sample as compared to its proportion in the general population

A) 15%
B) 10%
C) 30%
D) 5%

Question 8

Test manuals should include

A) tight testing procedures.
B) detailed directions and procedures.
C) flexible testing procedures.
D) brief descriptions of the domains covered.

Question 9

To evaluate tests, reviewers were advised to use the

A) Standards for Educational and Psychological Testing .
B) National Council on Measurement in Education's website.
C) American Psychological Association's bylaws.
D) Test Evaluation Guidelines published by AERA.

Question 10

Test users must determine if a test will result in __________ and __________ inferences about the specific students who will be assessed.

Question 11

Test evaluators must be careful to assess the __________ of the evidence presented.

Question 12

A test score in a normative sample may be dropped for all of the reasons except if

A) it is an outlier.
B) demographic data is complete.
C) the student was too young for the sample requirements.
D) the student failed to complete the test.

Question 13

Unbeknownst to many,

A) the number of students tested generally equals the norm group.
B) the norm group is typically less than the number of students tested.
C) the norm group is typically greater than the number of students tested.
D) None of the answers are correct.

Question 14

Test content can be confirmed by

A) reasoned rationale for the exclusion of certain content.
B) comparisons with other new similar tests.
C) comparisons with an internal standard.
D) your professional opinion.

Question 15

If the purpose of a test is to compare students' progress in comparison to other students, ​

A) the test should have at least 1,000 students in the norm sample.
B) the test should provide criterion-referenced scores.
C) the test must provide same gender comparison scores.
D) the test should provide normative scores.

Question 16

Test reviewers should do all of the following except

A) use tests that were published 15 years ago.
B) review the most recent version of the test.
C) acquire all the relevant materials.
D) review all available tests in the domain of interest.

Question 17

In general, test publishers

A) report everything there is to know about their test.
B) admit when their test is poorly normed.
C) must prove that their test is technically adequate.
D) provide exciting product development information.

Question 18

It is more costly and time-consuming to collect

A) internal consistency reliability.
B) item reliability.
C) stability information.
D) interscorer agreement.

Question 19

Most importantly, test reviewers should

A) define the five components of a test review.
B) consider the quality of the evidence presented.
C) look at the "best face" of the test.
D) trust test authors to provide accurate educational data.

Question 20

The topics one should review in evaluating tests include:

A) purposes, materials, scores, norms, reliability, and validity.
B) validity, internal consistency, norms, reliability, and procedures.
C) purposes, content and assessment procedures, scores, norms, reliability and validity.
D) test-retest reliability, item reliability, validity, purposes, materials, norms, and scores.

Question 21

Each test must have sufficient reliability for every __________ for which the test was intended.

Question 22

Test reviewers often find that evidence of a test's __________ is often incomplete.

Question 23

__________ means each normative group to which a person's score on a test could be compared.

Question 24

Once you use a test, you become responsible for test-based  __________.

Question 25

At a minimum, the technical adequacy of a test should include representative norms, reliable scores and general __________.

Question 26

The proportions of students from each racial and ethnic group should be approximately __________ the proportions found in the general population.

Question 27

Why is it most important to assess quality of the evidence presented in addition to checking to see if specific information relating to standards is provided?

Question 28

The American Educational Research Association, American Psychological Association and the National Council on Measurement in Education produced a set of test review guidelines called __________.

Question 29

Norms should include students from across the United States as well as students from urban, __________, and __________ communities.

Question 30

Item reliability and __________ must be presented.

Question 31

When reviewing the representativeness of norms, which areas are important to consider?

Question 32

If you discover that a specific test has adequate technical properties, what else should you be concerned about?  Explain your reasoning.

Question 33

A test with four domain scores and six norm groups should have ____ reliability estimates.

Question 34

A student's score is usually compared to the scores of same __________, same __________, and/or same __________.

Question 35

Generally, the more flexible a test's materials and directions, the more __________ will be the test results for students with severe disabilities.

Question 36

__________ are scores for each domain or subdomain on a test that can be interpreted appropriately.

Question 37

As a rule, tests that were published _____ or more years ago should not be used unless absolutely necessary.

Question 38

Explain the four steps of reviewing a test.

Question 39

List the six components of a test to address in a thorough test review.

Question 40

Describe two of four reasons why scores from a normative sample may be dropped.