Deck 9: Internal Validity

Question 1

If something other than the treatment is responsible for the effect, there is a problem with the study's

A) construct validity
B) internal validity
C) external validity
D) all of the above

Answer 1

internal validity

Question 2

To establish internal validity, you have to show that

A) changes in the first variable are followed by changes in the second variable
B) the changes in the second variable couldn't be caused by any variable other than the first variable
C) the measures used are valid
D) both a and b

Answer 2

both a and b

Question 3

Campbell and Stanley (1966) identified ___ threats to internal validity.

A) two
B) four
C) six
D) eight

Answer 3

eight

Question 4

Which of the following would eliminate selection error?

A) self-assignment
B) matching
C) arbitrary assignment
D) none of the above

Question 5

To test the effect of a new teaching procedure, a professor asks some of her students to volunteer to enter an experimental section of the course. The experimental section does better than the standard section. There is a strong possibility that this result may be due to ________ .

A) testing
B) regression
C) selection
D) mortality

Question 6

Which is the most serious source of error in a study that compares a treatment group to a no-treatment group?

A) mortality
B) regression
C) selection
D) testing

Question 7

Which of the following is the most serious threat for the typical two-group design?

A) selection
B) maturation
C) history
D) instrumentation

Question 8

The most serious problem with arbitrary assignment of participants to groups is:

A) mortality
B) regression
C) selection
D) testing

Question 9

In terms of internal validity, which of the following is NOT a problem with matching?

A) You can't find participants who are identical in every respect.
B) You can't measure and match subjects on every single variable.
C) You probably won't know every single relevant variable that subjects should be matched on.
D) You have to throw out subjects who don't match up with any of your other subjects, resulting in mortality.

Question 10

Which of the following is a threat to internal validity in studies that match on pretest scores?

A) selection by maturation
B) regression
C) mortality
D) all of the above

Question 11

Why do the authors say that matching on pretest scores only makes your groups APPEAR to be identical rather than actually making the groups identical?

A) Participants are matched on measures of characteristics rather than on the characteristics themselves.
B) Participants are not matched on every single variable.
C) both a and b
D) neither a nor b

Question 12

The fact that participants are matched on measures of characteristics rather than on the characteristics themselves makes matching vulnerable to:

A) Testing effects
B) Selection by maturation effects
C) Regression
D) Maturation

Question 13

The fact that participants are not matched on every single variable makes matching most vulnerable to:

A) Testing effects
B) Selection by maturation
C) Regression
D) Maturation

Question 14

Matching on pretest scores makes the study vulnerable to:

A) selection
B) selection by maturation
C) maturation
D) history

Question 15

Matching on pretest scores makes a study vulnerable to:

A) regression effects
B) selection by maturation effects
C) both a and b
D) neither a nor b

Question 16

Participants who scored very high on a test, will tend to score ____ on the retest

A) the same
B) a little higher
C) a little lower
D) below the mean

Question 17

Participants who scored very low on a test, will tend to score ____ on the retest:

A) the same
B) a little higher
C) a little lower
D) above the mean

Question 18

Mary scored 100% on her first biology test but 96% on the second, even though she studied harder for the second test. Which of the following best explains Mary's deteriorating pattern of performance?

A) improper matching
B) selection
C) mortality
D) regression toward the mean

Question 19

If a test had no random measurement error (was perfectly reliable), which of the following would not be a threat?

A) history
B) regression
C) selection
D) selection by maturation

Question 20

In a two-group study, in which participants are matched on a measure that has much random measurement error (had poor reliability), which of the following would be the most serious threat to the study's validity?

A) history
B) regression
C) testing
D) instrumentation

Question 21

If participants in a two-group study are matched on only a few of the relevant characteristics, which of the following is the most serious threat to the study's internal validity?

A) history
B) regression
C) selection by maturation
D) instrumentation

Question 22

Suppose that an investigator tested the treatment and no-treatment groups at very different times and in different places. In addition to the typical threats to the two-group design, the investigator would have to worry about ____, a threat to internal validity that usually doesn't trouble researchers who use a two-group design.

A) instrumentation
B) selection
C) regression
D) history

Question 23

Why might participants change between pretest and posttest?

A) maturation
B) history
C) testing
D) all of the above

Question 24

Why might participants' scores change between pretest and posttest?

A) regression
B) instrumentation
C) testing
D) all of the above

Question 25

If you don't realize that scores may be affected by random error, you may fail to detect ____ effects.

A) instrumentation
B) selection by maturation
C) selection
D) regression

Question 26

Which is a likely threat to validity for BOTH pretest-posttest designs and matched groups designs?

A) selection
B) regression
C) history
D) maturation

Question 27

Shortly after seeing a quack, several people have recovered from an illness, regaining their former health. The best explanation for their recovery is

A) testing
B) mortality
C) selection
D) regression

Question 28

The instrumentation threat is most frequently due to changes in

A) participants
B) the recording equipment
C) human raters
D) temperature and humidity

Question 29

Regression toward the mean is most serious as a threat to internal validity when

A) people are included in a treatment group on the basis of extreme scores on a pretest
B) retarded or behaviorally disordered subjects are used
C) the treatment group scores near the mean of the population on the pretest
D) two group designs are used

Question 30

In a two-group study, a highly unusual thing happens: None of the no-treatment group subjects drop out, but 90% of the treatment group subjects drop out. The most obvious problem with this study is

A) mortality
B) selection by maturation interaction
C) history
D) maturation

Question 31

When trying to establish that a treatment causes an effect, researchers ALWAYS try to

A) eliminate all extraneous factors
B) account for the effects of extraneous factors
C) increase the number of extraneous factors
D) sacrifice external validity

Question 32

In a two-group study, 10 participants drop out of each group before the treatment is administered. This poses the greatest threat to the study's

A) construct validity
B) internal validity
C) external validity
D) content validity

Question 33

External validity is to internal validity as _______ is to _______.

A) rigor; relevance
B) cause; generality
C) generality; cause
D) control; measurement

Question 34

The goals of internal and external validity

A) sometimes conflict
B) always conflict
C) are always mutually supporting
D) conflict with the goal of construct validity

Question 35

Studying only twins might help an investigation's

A) external validity
B) internal validity
C) both a and b
D) neither a nor b

Question 36

Studying participants in a highly controlled lab situation might help an investigation's

A) external validity
B) internal validity
C) both a and b
D) neither a nor b

Question 37

Studying participants in a real life situation would be most likely to help an investigation's

A) external validity
B) internal validity
C) both a and b
D) neither a nor b

Question 38

In order to be confident that your treatment caused an observed effect, you must establish that the study has internal validity.

Question 39

The best way to rule out selection bias is to have research participants select which group they will be in.

Question 40

Matching is the best way of making sure that your groups are equivalent before the treatment is administered.

Question 41

In a longitudinal study that compares the effects of a memory training in a group of 9-year-olds and a group of 70-year-olds, selection by maturation could be a major threat to the internal validity of the study.

Question 42

Regression towards the mean would suggest that a group that was formed based on extremely low scores on a pretest, would have even lower scores on a second testing regardless of the effects of any treatment.

Question 43

The pretest-posttest design is an imperfect substitute for random assignment to condition.

Question 44

Maturation refers to the effects of outside events, other than the treatment, that might influence participants' scores.

Question 45

The fact that taking a pretest may affect posttest scores is called testing.

Question 46

The degree to which the results of a study can be generalized to other populations is called external validity.

Question 47

When you increase external validity you will usually also improve internal validity.

Question 48

Distinguish between testing and instrumentation.

Question 49

Distinguish between maturation and history.

Question 50

List, define, and give an example of each of Campbell and Stanley's threats to validity.

Question 51

Explain why matching is not a perfect way of making two groups equal.

Question 52

Give an example of regression affecting the results of a two group study. Then, give an example of regression affecting the results of a pretest/posttest study.

Question 53

What steps taken to improve internal validity may hurt external validity? Why? How could you justify boosting internal validity at the expense of external validity?