Deck 4: Reliability of Selection Measures

Question 1

A split-half reliability overestimates actual reliability.Therefore,a special formula,the Spearman-Brown prophecy formula,is used to make the correction.

Answer 1

False

Question 2

Error score represents errors of measurement.

Answer 2

True

Question 3

The higher the test-retest reliability coefficient,the greater the true score and the less the error.

Answer 3

True

Question 4

The higher the value of a reliability coefficient,the less the measurement error.

Question 5

If all respondents on a selection measure remember their previous answers to an initial administration of a measure and then on the retest respond according to their memory,the reliability coefficient will decrease.

Question 6

Selection measures involving traits of personality,attitudes,or interests are usually considered to be fairly static yielding high reliability coefficients.

Question 7

Selection measures that are designed to assess job-related characteristics are more precise than measures of physical characteristics.

Question 8

A split-half reliability estimate is NOT a pure measure of internal consistency.

Question 9

Reliability is generally determined by examining the relationship between two sets of measures measuring the same thing.

Question 10

With a long time interval between administrations of a measure,test-retest reliability may underestimate reliability.

Question 11

To control the effects of memory on test-retest reliability estimates,the same measure should be used the second time.

Question 12

Reliability coefficients computed between parallel forms tend to be conservative estimates.

Question 13

With increasing time intervals,test-retest reliability coefficients will generally decrease.

Question 14

Surprisingly,increasing the lengths of time between administrations does not reduce the impact of memory effects on reliability.

Question 15

When a measure is perfectly reliable,its obtained score is higher than its true score.

Question 16

In general,the amount of measurement error has little effect on how high the reliability of measurement error will be.

Question 17

Because of the way it is calculated,a higher reliability coefficient is desirable.

Question 18

A selection measure is internally consistent or homogeneous when individuals' responses on one part of the measure are unrelated to their responses on other parts.

Question 19

Reliability of measurement in selection is synonymous with dependability,consistency,or stability of measurement.

Question 20

To achieve a parallel forms reliability estimate,at least two equal versions of a measure must exist.

Question 21

Although interrater agreement indices have their limitations,they are still widely used in selection research.

Question 22

As the number of response options or categories on a measure increases,reliability also increases.

Question 23

Split-half reliability procedures tend to produce a conservative estimate of reliability.

Question 24

The standard error of measurement is another approach for estimating reliability.

Question 25

Kuder-Richardson reliability estimates are usually lower than those obtained from split-half estimates.

Question 26

Reliability is a necessary but not sufficient condition for validity.

Question 27

A good rule of thumb is that reliability must be .90 or higher.

Question 28

Interrater agreement indices are generally restricted to interval or ratio data.

Question 29

If variability or individual differences increase among respondents while variation within individuals remains the same,reliability will increase.

Question 30

Unreliable performance by a respondent on a reliable measure is possible,but reliable performance on an unreliable measure is impossible.

Question 31

K-R20 reliability is a general version of coefficient alpha reliability.

Question 32

In the context of personnel selection,the reliability of criterion measures need not be as high as predictor measures.

Question 33

Reliability is a group-based statistic.

Question 34

In general,as the length of a measure decreases,its reliability also increases.

Question 35

The standard error of measurement is affected by variability within the group of respondents to whom a measure has been administered.

Question 36

If our standard error is 3.16 and the difference between two applicants' scores is 3,then it is possible that the difference in scores is due to chance.

Question 37

Interrater reliability estimates test the hypothesis that ratings are determined by characteristics of the rater rather than by what is being rated.

Question 38

Tests with many items that are very difficult are more reliable than tests containing many items of moderate difficulty.

Question 39

Selection measures are not simply "reliable" or "not reliable," there are degrees of reliability.

Question 40

If coefficient alpha reliability is unacceptably low,then the items on the selection measure may be assessing more than one characteristic.

Question 41

What is a true score?

A)the score obtained for a person under normal conditions
B)the score obtained because of the presence of external factors
C)the mean/average score made by a person on many different administrations of tests
D)the standard deviation on many different administrations of the same test on the same individual

Question 42

What reliability estimate consists of administering the same selection measure twice and correlating the two sets of scores?

A)parallel forms
B)internal consistency
C)split-half
D)test-retest

Question 43

What factor determines the choice of KR-20 or coefficient alpha as appropriate for estimating internal consistency reliability?

A)the number of items
B)how the items are scored
C)the duration of the test
D)the content of the test

Question 44

As the coefficient approaches 1.00,the set of measures is viewed as:

A)equivalent.
B)identical.
C)very different.
D)unrelated.

Question 45

For a test with a time limit (i.e.,a speed test),which reliability estimation procedure is not appropriate?

A)test-retest
B)split-half
C)parallel or equivalent forms (immediate administration)
D)parallel or equivalent forms (long-term administration)

Question 46

Calculation of reliability estimates results in a coefficient ranging from _____ to _____.

A)0, 1.96
B)0.00; 1.00
C)-1.00; 1.00
D)-1.00; 0.00

Question 47

What is the difference between interclass and intraclass correlations (reliability estimates)?

A)minimum number of targets being rated
B)minimum number of equivalent forms being used
C)minimum number of raters needed for calculation
D)minimum number of attributes measured

Question 48

What is a correlation coefficient calculated between two sets of scores over time called?

A)coefficient of stability
B)coefficient alpha
C)coefficient of equivalence
D)coefficient of dependability

Question 49

Test-retest reliability estimation is most appropriate for which of the following?

A)mental ability
B)attitudes
C)self-esteem
D)self-concept

Question 50

An obtained score consists of which two components?

A)controllable and uncontrollable
B)systematic and unsystematic
C)true and error
D)true and predictive

Question 51

In order to calculate an intraclass correlation,how many raters are necessary?

A)2
B)2 or more
C)more than 2
D)any number will do

Question 52

What impact does memory have on a test-retest reliability estimate?

A)It is not possible to determine its effect on reliability.
B)It has no effect on test-retest reliability.
C)It will underestimate the true reliability of obtained scores.
D)It will overestimate the true reliability of obtained scores.

Question 53

In order to calculate an interclass correlation,how many raters are necessary?

A)2
B)2 or more
C)more than 2
D)more than 3

Question 54

Generally speaking,the greater the variability or standard deviation of scores on the characteristic measured,the higher the reliability of the measure of that characteristic.

Question 55

How many test administrations do you need in order to calculate a split-half reliability estimate?

A)1/2
B)1
C)2
D)1/4

Question 56

For which of the following selection measures is it most appropriate to use equivalent forms for reliability estimation?

A)vocabulary
B)biographical inventory
C)personality inventory
D)physical fitness

Question 57

Which of the following is NOT one of the categories of statistical procedures for estimating interrater reliability?

A)interclass correlation
B)intraclass correlation
C)interrater agreement
D)underclass correlation

Question 58

With a long time interval between administrations of a measure (test-retest),what could cause scores to change resulting in an underestimate of the reliability?

A)reasoning
B)thinking
C)memory
D)learning

Question 59

Which of the following is not a method for estimating internal consistency reliability?

A)parallel or equivalent forms reliability
B)Kuder-Richardson reliability
C)Cronbach's coefficient alpha reliability
D)split-half reliability

Question 60

Which of the following would NOT be a likely cause of interrater disagreement?

A)Raters view the same behavior differently.
B)Raters interpret the same behavior differently.
C)Error in rating or recording each impression.
D)Length of time behavior is displayed.

Question 61

Why is reliability important in human resource selection?

Question 62

Describe (verbally)how each of the following reliability estimates is calculated:
a.test-retest
b.parallel or equivalent forms
c.internal consistency (in general)
d.interrater

Question 63

A male scored three points lower than a female on a selection test.The female was hired.The male filed a reverse discrimination case on the basis of unfair use of test scores for selection.What evidence would he have to provide to support his claim (or alternatively what evidence would the company have to provide to support its hiring decision)?

Question 64

Explain why coefficient alpha estimations of reliability are not appropriate for speed tests.

Question 65

If we have a test called "x" and r_xx = .80,this means

A)80% of the differences in test scores is due to error and only 20% is due to true variance.
B)20% of the test scores were used to obtain the reliability estimate.
C)20% of the differences in test scores is due to error and 80% is due to true variance.
D)the test average is in the low 'B' range.

Question 66

Why are there so many different ways to estimate reliability?

Question 67

If r_xx = .85,and the standard deviation of x is 10,then the standard error of measurement for measure x is

A)3.873
B)3.16
C)3.50
D)3.30

Question 68

.The difference between two individuals' scores should not be considered significant unless the difference is at least ___________ the standard error of measurement of the measure.

A)equal to
B)twice
C)three times
D)four times

Question 69

Research has shown that the reliability of rating scales can be improved by offering from _____ to _____ rating categories:

A)1, 4
B)1, 5
C)3, 7
D)5, 9