Deck 5: Reliability and Validity

Question 1

The reliabilities of different forms of a test are not important in equating tests.

Answer 1

False

Question 2

A test can be valid without being reliable, but it cannot be reliable without being valid.

Answer 2

False

Question 3

Test reliability varies directly with the magnitude of the errors of measurement.

Answer 3

False

Question 4

The magnitude of a test-retest reliability coefficient is generally less when the interval between initial testing and retesting is short than when it is long.

Question 5

Coefficient alpha represents a generalization of the Kuder -Richardson internal consistency method of determining test reliability.

Question 6

The general Spearman-Brown formula is used to estimate the reliability of a lengthened test.

Question 7

Test-retest, parallel forms, and internal consistency reliability formulas are appropriate for both norm-referenced and criterion-referenced tests.

Question 8

The most useful procedure for determining the reliability of a speeded test is to apply one of the internal consistency formulas.

Question 9

The reliabilities of oral tests tend to be lower than those of written tests.

Question 10

The standard error of measurement is used to construct a percentile band around an examinee?s observed test score.

Question 11

The reliability of a criterion-referenced test should not be determined in the same manner as that of a norm-referenced test.

Question 12

Predictive validity is of greatest concern on tests of achievement and personality.

Question 13

Standard error of estimate is to predictive validity as standard error of measurement is to reliability.

Question 14

In general, tests having greater score variability have lower reliabilities than tests with less variability.

Question 15

Convergent and discriminant validation are necessary characteristics of a test that has construct validity.

Question 16

Content validity is to achievement tests as predictive validity is to aptitude tests.

Question 17

Predictive validity and concurrent validity are both types of criterion-related validity.

Question 18

Computation of the standard error of estimate requires knowing the reliability coefficient.

Question 19

Convergent and discriminant validation are concepts that are applicable to the determination of the content validity of a test.

Question 20

The type of validity of greatest concern in personnel decisions is criterion-referenced validity.

Question 21

A test designed to predict a particular criterion is most valid when the base rate is 50%.

Question 22

False positive errors are more serious than false negative errors.

Question 23

Prediction of a characteristic having a low base rate is less accurate than that of a test having a high base rate.

Question 24

The selection ratio is low in a tight labor market and high in an open labor market.

Question 25

In the multiple-cutoff procedure, high scores on one selection variable can compensate for low scores on another selection variable.

Question 26

The multiple correlation coefficient is an index of the accuracy of predicting a criterion variable from a combination of two or more predictor variables.

Question 27

Use of a Taylor-Russell table assists in estimating the validity of a selection test.

Question 28

The extent to which a test measures anything consistently is a definition of

A) normality.
B) objectivity.
C) reliability.
D) validity.

Question 29

According to classical test theory, s2obs = s2tru + s2err, and r11 = s2tru/s2obs. If the observed variance of a test is 50 and the error variance is 10, what is the estimated reliability of the test?

A) .20
B) .25
C) .45
D) .80

Question 30

The definition of reliability as the ratio of true score variable to observed score variance comes from

A) dependability theory.
B) classical test theory.
C) generalizability theory.
D) item response theory.

Question 31

If 40% of a test?s observed variance is due to errors of measurement, then what is the reliability coefficient of the test?

A) .20
B) .40
C) .60
D) .80

Question 32

Which of the following types of reliability coefficients is a coefficient of equivalence?

A) coefficient alpha
B) Kuder-Richardson
C) parallel forms
D) split-half
E) test-retest

Question 33

Which type of reliability takes into account error variance due to both different samples of test items and different conditions of administration?

A) internal consistency
B) parallel forms
C) split-half
D) test-retest

Question 34

The standard deviation of a particular WISC-III subtest is 3, and its reliability coefficient is .84. What is the standard error of measurement of the subtest?

A) 1.2
B) 2.0
C) 2.5
D) 3.0

Question 35

Assuming that the correlation between the odd-numbered and the even-numbered items on a test is .74, the corrected split-half reliability of the test (using the Spearman-Brown prophecy formula) is approximately

A) .80.
B) .85.
C) .90.
D) .95.

Question 36

The Kuder-Richardson method of determining reliability yields an average
Coefficient.

A) alternate tests
B) parallel forms
C) split-half
D) test-retest

Question 37

Which of the following statistics enables an examiner to establish confidence limits for the true scores of examinees having a given observed score on a test?

A) Kuder-Richardson predictive index
B) Spearman-Brown prophecy coefficient
C) standard error of estimate
D) standard error of measurement

Question 38

A test can usually be made more reliable by increasing the

A) correlation between test and criterion scores.
B) length of time for administering the test.
C) number of items on the test.
D) observed variance relative to true variance.

Question 39

The most general formula for computing an internal consistency reliability coefficient is the

A) Cronbach coefficient alpha formula.
B) Kuder-Richardson formula 21.
C) Spearman-Brown prophecy formula.
D) Wherry-Doolittle consistency formula.

Question 40

Which of the following types of reliability does not belong in the same category as the other three?

A) coefficient alpha
B) Kuder-Richardson
C) split-half
D) test-retest

Question 41

An interrater reliability coefficient is computed in determining the reliability of__________ tests.

A) completion and matching
B) essay and oral
C) multiple-choice and true-false
D) objective and projective

Question 42

The standard error of measurement is always zero whenever the reliability coefficient equals

A) -1.00.
B) .00.
C) .50.
D) 1.00.

Question 43

Suppose that Jane makes a score of 60 on a test having a standard deviation of 5 and a reliability coefficient of .85. Between what two values can we be 95% confidant that Jane's true score on the test falls?

A) 59-61
B) 57-63
C) 56-64
D) 55-65

Question 44

Percentile bands for a true score on a test are computed by determining the percentile rank equivalents of scores that are one standard __________ on either side of the examinee?s obtained score.

A) deviation
B) error of the difference
C) error of estimate
D) error of measurement

Question 45

Increasing the true variance and the observed variance by the same amount will increase the test's

A) arithmetic mean.
B) error variance.
C) reliability.
D) validity.

Question 46

The reliability of a certain test is .70. Approximately how much longer will the test have to be made, by adding more items of the same general type, to increase the reliability of the test
To .90?

A) two times as long
B) three times as long
C) four times as long
D) five times as long

Question 47

Analysis of variance techniques are used in the reliability estimation procedure known as

A) classical reliability theory.
B) generalizability theory.
C) split-half theory.
D) true score theory.

Question 48

A test consisting of 25 items has a reliability of .80. Approximately how many items of the same type as the original ones must be added to the test in order to increase its reliability to .95?

A) 25
B) 50
C) 75
D) 95

Question 49

What type of validity is of greatest important for an achievement test?

A) concurrent
B) construct
C) content
D) predictive

Question 50

The type of validity that is used most often in selection programs in education and industry, in which criteria of success are specified, is

A) concurrent validity.
B) construct validity.
C) content validity.
D) predictive validity.

Question 51

A confidence interval for an examinee?s obtained score on a criterion measure can be determined by using the standard error of

A) estimate.
B) measurement.
C) the mean.
D) the sample.

Question 52

The standard error of estimate is most relevant to a test's __________ validity

A) concurrent
B) construct
C) content
D) predictive

Question 53

The larger the validity coefficient, the smaller the

A) coefficient of regression.
B) standard error of estimate.
C) standard error of measurement of X.
D) standard error of measurement of Y.

Question 54

Assuming that the correlation between test X and test Y is .60 and the standard deviation of test Y is 10, if Johnny?s predicted Y score is 75 the chances are 95 out of 100 that he is one of a group of people whose scores on test Y are between

A) 59 and 91.
B) 63 and 87.
C) 67 and 83.
D) 69 and 81.

Question 55

The criterion-related validity of a test is affected by

A) criterion contamination.
B) moderator variables.
C) the length of the test.
D) all of the above.

Question 56

The most comprehensive type of validity, in that it comprises all of the others, is __________validity.

A) concurrent
B) construct
C) content
D) predictive

Question 57

Assigning applicants to jobs on the basis of their scores on more than one test is most appropriately referred to as

A) classification.
B) placement.
C) screening.
D) selection.

Question 58

Consider the following empirical expectancy table:

$\quad$ $\quad$ $\quad$ $\quad$ $\quad$ $\quad$$\text { Performance Rating}$
$\begin{array}{l|cccc|c}\begin{array}{l} \text{Selection} \\\text{Test Scores}\end{array} &0-1 & 2-3 & 4-5 & 6-7 &\text{ Totals} \\\hline 76-100 & & 1 & 3 & 4 & 8 \\ 51-75 & 1 & 3 & 6 & 7 & 17 \\ 26-50 & 3 & 9 & 4 & & 16 \\0-25 & 5 & 3 & 1 & & 9 \\\hline \text{Totals }& 9 & 16 & 14 & 11 &\end{array}$

-If an applicant's score on the selection test is 30, what is the probability that his or her performance rating will be 4 or higher?

A) .08
B) .25
C) .30
D) .80

Question 59

Consider the following empirical expectancy table:

$\quad$ $\quad$ $\quad$ $\quad$ $\quad$ $\quad$$\text { Performance Rating}$
$\begin{array}{l|cccc|c}\begin{array}{l} \text{Selection} \\\text{Test Scores}\end{array} &0-1 & 2-3 & 4-5 & 6-7 &\text{ Totals} \\\hline 76-100 & & 1 & 3 & 4 & 8 \\ 51-75 & 1 & 3 & 6 & 7 & 17 \\ 26-50 & 3 & 9 & 4 & & 16 \\0-25 & 5 & 3 & 1 & & 9 \\\hline \text{Totals }& 9 & 16 & 14 & 11 &\end{array}$

-If the cutoff score on the selection test is set at 26, what percent of the applicants will be rejected?

A) 18%
B) 32%
C) 50%
D) 84%

Question 60

When high scores on one predictor variable compensate for low scores on another predictor variable, then the __________ procedure is more appropriate.

A) multiple cutoff
B) multivariate method
C) multiple regression
D) successive hurdles

Question 61

The use of a linear equation such as Ypred = .002 VAR-A +.001 VAR-B + .003 VAR-C - 2.00 to select job applicants is referred to as a __________ procedure.

A) multiple cutoff
B) multiple regression
C) sequential screening
D) successive hurdles

Question 62

The validity of a test for identifying people with a rare disorder would be greatly limited by a low

A) base rate.
B) selection ratio.
C) standard deviation.
D) standard error of measurement.

Question 63

What is meant by test reliability, and what are some procedures for obtaining information concerning it?

Question 64

What is meant by test validity, and what are some procedures for obtaining information concerning it?

Question 65

Differentiate between the standard error of measurement and the standard error of estimate and the ways in which each statistic is used.

Question 66

How can a test be made more reliable, and how can it be made more valid?

Question 67

For what purposes and how are the Taylor-Russell tables used?