Deck 6: Understanding Reliability and Validity: Just the Truth

Question 1

In experiment,the treatment is called ______.

A)dependent variable
B)independent variable
C)intervening variable
D)mediating variable

Answer 1

independent variable

Question 2

Political affiliation depends on political orientation.Which variable is political orientation?

A)dependent variable
B)independent variable
C)mediation variable
D)intervening variable

Answer 2

independent variable

Question 3

What is the key to establishing criterion validity?

A)number of criteria
B)quality of criterion
C)type of correlation
D)predictive ability

Answer 3

quality of criterion

Question 4

Which of the following explore(s)the question,"How do I know that the test,scale,instrument,and so on,I use works every time I use it?"

A)reliability
B)validity
C)both reliability and validity
D)neither reliability nor validity

Question 5

Which of the following is concerned with monitoring estimates of future performance?

A)content validity
B)construct validity
C)concurrent validity
D)predictive validity

Question 6

Reliability and validity specifically relate to the ______.

A)correlations between variables
B)appropriateness of the statistical tests chosen
C)qualitative methodology
D)quality of the data

Question 7

A researcher examined the effect of different reading programs on comprehension.What was the dependent variable?

A)reading programs
B)comprehension
C)differences
D)researcher

Question 8

If you correlate scores from your test with some other valid measure that assesses the same set of abilities,what type of validity evidence are you collecting?

A)criterion validity
B)content validity
C)construct validity
D)internal consistency

Question 9

What type of validity is concerned with the adequate representation of test items?

A)content validity
B)predictive validity
C)concurrent validity
D)construct validity

Question 10

If you want to know that a test measures some underlying psychological concept,what type of validity evidence would you want to collect?

A)content validity
B)predictive validity
C)construct validity
D)concurrent validity

Question 11

The EATS test (of culinary skills)has been shown to be correlated with being a fine chef 2 years after culinary school.This is an example of which type of validity?

A)content validity
B)criterion validity
C)concurrent validity
D)construct validity

Question 12

Which of the following explore(s)the question,"How do I know that the test,scale,instrument,etc.I use measures what it is supposed to?"

A)reliability
B)validity
C)both reliability and validity
D)neither reliability nor validity

Question 13

If I can say that my weekly statistics quiz fairly assesses the material covered,what source of validity evidence should I have collected?

A)criterion validity
B)concurrent validity
C)construct validity
D)content validity

Question 14

It is TRUE that men who participate in body contact sports score higher on the TEST (osterone)test of aggression.This is an example of which type of validity?

A)content validity
B)criterion validity
C)concurrent validity
D)construct validity

Question 15

The more violent television programing an individual watches,the more likely the individual develops aggressive behavior.What is watching violent television programming?

A)a dependent variable
B)an independent variable
C)a mediation variable
D)an intervening variable

Question 16

What is the outcome variable in an analysis is called?

A)predictor variable
B)treatment variable
C)dependent variable
D)independent variable

Question 17

At what point in the research process should reliability and validity be considered?

A)before you start analyzing data
B)after data have been analyzed
C)when data are being collected
D)when researching human subjects only

Question 18

A researcher examines how comprehension affects test scores.What is the independent variable?

A)comprehension
B)researcher
C)test scores
D)examination

Question 19

If data are not reliable or not valid,the results of any test or hypothesis ______.

A)must be true
B)are inconclusive
C)are valid in only certain situations
D)must be interpreted

Question 20

If you wanted to examine the impact of fast-food consumption on weight gain,what would your measurement of weight gain be?

A)dependent variable
B)treatment variable
C)predictor variable
D)independent variable

Question 21

Which coefficient reflects the occurrence of perfect reliability?

A)1.00
B)0.5
C)0.100
D)2.00

Question 22

What the type of validity examines how well a test outcome is consistent with a criterion that occurs in the future?

A)content validity
B)test-retest validity
C)concurrent criterion validity
D)predictive validity

Question 23

What type of validity examines how well a test outcome is consistent with a criterion that occurs in the present?

A)content validity
B)predictive validity
C)concurrent criterion validity
D)test-retest validity

Question 24

The correlation between two forms of the same test is called ______.

A)parallel forms reliability
B)inverse reliability
C)test-retest reliability
D)internal consistency reliability

Question 25

The number of agreements between your two raters divided by the total number of possible agreements is the way to calculate ______.

A)parallel forms reliability
B)multiple judges reliability
C)interrater reliability
D)multiple judges validity

Question 26

What type of validity is established by consultation with an expert on the topic focused upon by your instrument?

A)content validity
B)criterion validity
C)parallel forms validity
D)construct validity

Question 27

All but which of the following is a measure of validity?

A)content validity
B)parallel forms validity
C)criterion validity
D)construct validity

Question 28

What is the difference between the observed and true scores?

A)score differential
B)score difference
C)error score
D)error of approximation

Question 29

The correlation between scores from Time 1 and Time 2 is called ______.

A)test-retest reliability
B)parallel forms reliability
C)internal consistency reliability
D)similar-forms reliability

Question 30

In terms of the reliability of test scores,there are multiple elements to each person's score.The score that is actually recorded is the ______.

A)error score
B)true score
C)false score
D)observed score

Question 31

______ is based on the judgment of how well a test reflects an underlying idea.

A)Construct validity
B)Criterion validity
C)Internal consistency validity
D)Content validity

Question 32

All but which of the following are measures of reliability?

A)test-retest reliability
B)multiple-forms reliability
C)internal consistency reliability
D)interrater reliability

Question 33

If the measures associated with a test are said to be consistent,you might conclude that the measure is ______.

A)standard
B)valid
C)reliable
D)concurrent

Question 34

"A measure of how stable a test is over time" is an example of which of the following types of reliability?

A)interrater
B)test-retest
C)parallel forms
D)internal consistency

Question 35

High-school class rank is highly correlated with college GPA.This is an example of what type of validity?

A)concurrent validity
B)predictive validity
C)content validity
D)construct validity

Question 36

What is the score that you would receive if the test contained no error?

A)observed score
B)actual score
C)true score
D)error score

Question 37

What type of validity is determined by the association between the test scores and some specified present or future criterion.

A)construct validity
B)criterion validity
C)content validity
D)internal consistency validity

Question 38

Two trained professionals observe the behavior of children in a classroom.They each rate observed behaviors using the same form,and the number of items that were rated the same is calculated.This is an example of which type of reliability?

A)parallel forms
B)test-retest
C)interrater
D)internal consistency

Question 39

Cronbach's alpha is used to calculate ______.

A)internal consistency reliability
B)internal consistency validity
C)parallel forms reliability
D)internal validity

Question 40

The actual or measured score is called the ______.

A)true score
B)observed score
C)error score
D)real score

Question 41

Test-retest reliability is an example of which of the following?

A)measure of consistency
B)measure of reliability over time
C)measure of agreement
D)measure of equivalence

Question 42

For a total of 18 possible agreements,there are seven times where Duck and Cow agreed on whether students in college fell asleep at their desks or not.What is the interrater reliability coefficient?

A)56
B)0.39
C)3.90
D)0.003

Question 43

What would the interrater reliability be for a 100-item measure in which the number of agreements between Rater 1 and Rater 2 was 80?

A)0.45
B)80%
C)0.90
D)95%

Question 44

Parallel forms reliability is an example of which of the following?

A)measure of consistency
B)measure of stability
C)measure of agreement
D)measure of equivalence

Question 45

Interrater reliability is an example of which of the following?

A)measure of consistency
B)measure of stability
C)measure of agreement
D)measure of equivalence

Question 46

What is the difference between the TRUE score and the observed score?

A)critical score
B)error score
C)expected score
D)reliable score

Question 47

The maximum level of validity possible is equal to ______.

A)the square of the reliability coefficient
B)the square root of the reliability coefficient
C)the cube of the reliability coefficient
D)the square root of the reliability coefficient,plus 1

Question 48

If you want to know if several different forms of a test are reliable,what type of reliability do you use?

A)parallel forms reliability
B)test-retest reliability
C)interrater reliability
D)internal consistency reliability

Question 49

For a total of 80 possible agreements,there are 60 times where Teacher A and Teacher B agreed on whether students in college checked their cellphone in the classroom.What is the interrater reliability coefficient?

A)1.33
B)0.80
C)25
D)0.75

Question 50

If your test is not reliable,you must try to lower your error.

Question 51

What would the interrater reliability be for a 50-item measure in which the number of agreements between Rater 1 and Rater 2 was 45?

A)0.45
B)80%
C)0.90
D)95%

Question 52

Which of the following is the Greek letter associated with Cronbach's alpha,sometimes known as coefficient alpha?

A)k
B)α
C)Σ
D)s

Question 53

If you want to know if the items on a test assess one and only one dimension,what type of reliability do you use?

A)parallel forms reliability
B)test-retest reliability
C)interrater reliability
D)internal consistency reliability

Question 54

If you want to know whether a test is reliable over time,what type of reliability do you use?

A)parallel forms reliability
B)test-retest reliability
C)interrater reliability
D)internal consistency reliability

Question 55

Interrater reliability measures consistency ______.

A)over time
B)from form to form
C)across different tests
D)from rater to rater

Question 56

For a total of 10 possible agreements,there are eight times where Bear and Cat agreed on whether children in ice cream commercials closed their eyes while eating ice cream or not.What is the interrater reliability coefficient?

A)0.80
B)0.08
C)80.0
D)8.0

Question 57

When we calculate reliability,we know the observed score.What are the two unknown components of the reliability equation?

A)method and error scores
B)means and standard deviations
C)test-retest and interrater scores
D)true and error scores

Question 58

If you want to know whether there is consistency in the rating of some outcome,what type of reliability do you use?

A)parallel forms reliability
B)test-retest reliability
C)interrater reliability
D)internal consistency reliability

Question 59

Observed scores are commonly exactly the same as true scores in the social and behavioral sciences.

Question 60

For high reliability,you want reliability coefficients to be positive and as close as possible to 1.

Question 61

When should we measure internal consistency reliability?

Question 62

Construct validity is the most difficult of all the validities to establish.

Question 63

The observed score equals the true score minus error score.

Question 64

How does error impact reliability? Write the general equation for reliability that demonstrates this concept.

Question 65

How do reliability and validity work together?

Question 66

What is the difference between reliability and validity?

Question 67

Internal consistency reliability tests how each individual item score correlates with the total score.

Question 68

To use archery practice as a metaphor for reliability and validity,hitting the bull's-eye of a target demonstrates ______,whereas repeatedly hitting the target in the same small area demonstrates ______.

Question 69

Why can we not know the true score for a test?

Question 70

When should you use construct validity?

Question 71

A test can be valid but not reliable,but it is impossible to have a test that is reliable and not valid.

Question 72

Criterion validity assesses whether a test reflects a set of abilities in a future setting.

Question 73

Interrater reliability examines the percentage of agreement between raters.

Question 74

Name four types of reliability and what they measure.

Question 75

What are the traditional types of validity?

Question 76

Explain the formula for understanding observed scores.What is included in any observed score?