Deck 7: Validity Issues in Research: the Heart of It All

Question 1

In the context of threats to internal validity, differential regression is said to exist when more than one group is involved in a study and the attrition across the groups is not comparable.

Answer 1

False

Question 2

Instrumentation refers to changes in the measuring device or procedure over the course of a study.

Answer 2

True

Question 3

Range restriction is one of the threats to statistical conclusion validity.

Answer 3

True

Question 4

Construct underrepresentation occurs when:

A) test results show less correlation between variables than expected.
B) a construct plays less of a role in the correlation between variables than expected.
C) irrelevant aspects are included as part of a construct in designing a study.
D) all of the important aspects of a construct are not incorporated.

Question 5

Identify a way to minimize the mono-method bias in a study.

A) By examining associations between the constructs and finding ways to neutralize them
B) By detecting the effects of novelty and disruptions on the constructs
C) By utilizing multiple exemplars to assess a construct
D) By using multiple ways to measure a construct

Question 6

In the context of threats to construct validity, mono-operation bias can be minimized by using a single exemplar or a single measure to assess a construct.

Question 7

The internal validity of a study refers to:

A) the confidence one can have in inferring a causal relationship among the variables.
B) the extent to which the variables represent what they were intended to measure.
C) the statistical accuracy of the constructs of the study.
D) the degree of meaning that can be placed upon a set of test results of the study.

Question 8

In statistical tests, an alternative hypothesis states that there is no relationship between the variables in the study.

Question 9

The threat of compensatory equalization of treatments to internal validity occurs when participants in a control group attempt to outperform participants in the treatment group.

Question 10

The threat of unreliable treatment implementation to statistical conclusion validity of a study is defined as the increase in error variance due to any aspect of the experimental setting that leads to variability in responding.

Question 11

The construct validity of a study is concerned with:

A) how relevant the constructs of a study are to the general public.
B) the generalizability of the results of an experiment.
C) how well the variables chosen represent and capture the essence of a hypothetical construct.
D) the degree to which a researcher has come to the correct conclusion about the relationship between the variables.

Question 12

In a research study, the degree to which inferences reflect how things actually are is referred to as validity of the research.

Question 13

The threat of experimental expectations to the construct validity arises when experimenters consciously or unconsciously affect results through the way they communicate and interact with participants.

Question 14

Regression toward the mean can occur when:

A) participants who score low on the pretest score higher on the posttest.
B) participants who score high on the pretest score higher on the posttest.
C) participants in the control group score higher than those in the treatment group.
D) participants in the treatment group score higher than those in the control group.

Question 15

Which of the following is an example of a threat to construct validity?

A) Regression
B) Resentful demoralization
C) Range restriction
D) Ambiguous temporal precedence

Question 16

In a true experimental design:

A) every threat to the validity of the conclusions reached is eliminated.
B) treatments to different samples are implemented with significant variations.
C) the validity of a conclusion is confirmed if a threat is plausible.
D) an independent variable is manipulated by the researcher to assess the effect of the manipulation on a dependent variable.

Question 17

A Type II error is committed when:

A) results obtained with one set of measures do not generalize to outcomes that would be assessed by a different set of measures.
B) one incorrectly fails to reject the null hypothesis when in reality it should be rejected.
C) a researcher incorrectly concludes that a true relationship exists between the variables in a study.
D) a statistically significant result containing a sampling error is accepted.

Question 18

Statistical power is 1 minus the Type II error rate.

Question 19

Statistical conclusion validity refers to:

A) the degree of certainty with which one can make statements about the existence of a causal relationship between the variables in a study.
B) the degree to which the measured variables used in a study represent the hypothesized constructs.
C) the degree to which a researcher reaches the correct conclusion about the relationships among the variables in a research question.
D) the degree to which the causal relationship is generalizable across units, treatments, outcomes, and settings.

Question 20

In the context of threats to internal validity, history refers to:

A) the time between pretest and posttest results that may affect the results of a study.
B) an event that occurs during the administration of a treatment and that may affect a researcher's observations.
C) the differences between groups that exist before the implementation of a treatment and that may indirectly affect the conclusion drawn by the researcher.
D) a developmental change in participants between the pretest and the posttest that might affect the results.