Question 1

Single-session, short-term laboratory experimentation virtually precludes ______ as a problem. But in the field, where time-series data and a succession of follow-ups are more likely, this problem increases in relevance.
A) compensatory equalization
B) compensatory rivalry
C) resentful demoralization
D) Mortality

Accepted Answer

Mortality refers to the loss of participants over time, which is not a concern in single-session, short-term laboratory experiments but becomes relevant in longer-term field studies where participants may drop out or be lost to follow-up.

Question 2

Alicia is conducting a study on marathon runners by interviewing them at multiple points during the marathon, however, some individuals she interviewed leave the marathon without completing. What threat to internal validity is this?
A) compensatory equalization
B) compensatory rivalry
C) resentful demoralization
D) mortality

Accepted Answer

Mortality refers to the loss of participants from a study before it is completed, which can threaten the internal validity by potentially skewing the results if the reasons for leaving affect the outcome measures.

Question 3

According to Weiss, which of the following is NOT one of the three major areas of political consideration should evaluation researchers be aware of in their research?
A) Programs are political creatures.
B) the personal stance of the evaluator
C) politics of the decision-making process
D) the political stance of the evaluation itself

Accepted Answer

Weiss points out that evaluation research is inherently political due to the nature of programs being political entities, the politics involved in the decision-making process, and the political implications of the evaluation itself. The personal stance of the evaluator is not listed as one of the major political considerations.

Question 4

______ is the view that a better world can only be created by empirically driven evaluators.
A) Empirical morality
B) Resentful demoralization
C) Evaluation imperialism
D) Empirical imperialism

Accepted Answer

Evaluation imperialism is the perspective that emphasizes the importance of empirically driven evaluations in creating a better world, suggesting that rigorous assessment and evidence are central to improving conditions or outcomes.

Question 5

Weiss offers three pieces of advice for dealing with politics in evaluation research. Which of the following is NOT included in her advice?
A) put program goals "in sensible perspective"
B) minimize the involvement of politicians
C) evaluate a particularly strong version of the program before, or along with, the evaluation of the ordinary levels at which it functions"
D) do something other than evaluation research

Accepted Answer

Weiss does not advise minimizing the involvement of politicians; instead, she suggests understanding and navigating the political landscape effectively.

Question 6

The three criteria of causality should not be considered when assessing quasi-experimentation.

Accepted Answer

The three criteria of causality (association, temporal precedence, and non-spuriousness) are still important in assessing quasi-experimental designs to infer causality, despite the lack of random assignment.

Question 7

Rival plausible explanations are not a concern in traditional laboratory experimentation.

Accepted Answer

In traditional laboratory experimentation, controlling for rival plausible explanations is crucial to ensure that the observed effects can be attributed to the manipulated variables rather than other factors.

Question 8

When we cannot randomly assign people, we should start looking for "nonequivalent" control groups to allow us to make "reasonable guesses."

Accepted Answer

When random assignment is not possible, using nonequivalent control groups is a common method to attempt to account for differences between groups, allowing for more reasonable comparisons and guesses about the effects of an intervention or variable of interest.

Question 9

Time-series designs should not be used in quasi-experimentation.

Accepted Answer

Time-series designs can be used in quasi-experimental research to examine the effects of an intervention over time, allowing for the observation of trends before and after the intervention without the need for a control group.

Question 10

In traditional laboratory research, it is easy to isolate experimental and control groups from each other.

Accepted Answer

The answer of In traditional laboratory research, it is easy...

Question 11

Quasi-experimentation does not threaten internal validity.

Accepted Answer

Quasi-experimental designs often lack random assignment, which can lead to selection bias and confounding variables, thereby threatening internal validity.

Question 12

Mortality refers to a situation in which some individuals drop out of the research before it's completed

Accepted Answer

Mortality refers to the loss of participants from a research study before its completion, which can occur due to various reasons such as withdrawal, dropout, or death.

Question 13

Evaluation researchers should be aware of political considerations.

Accepted Answer

Evaluation research often takes place within political contexts, and political considerations can influence the focus, methodology, and dissemination of findings, making it important for researchers to be aware of these factors.

Question 14

Compare and contrast how rival plausible explanations are eliminated in traditional laboratory experimentation compared to in quasi-experimentation.

Accepted Answer

The answer of Compare and contrast how rival plausible explanations...

Question 15

Summarize and distinguish between the five threats to internal validity relevant to quasi-experimentation.

Accepted Answer

The answer of Summarize and distinguish between the five threats...

Question 16

Identify the key differences between manipulative control and analytic control and discuss when each approach should be used.

Accepted Answer

The answer of Identify the key differences between manipulative control...

Question 17

Summarize what it means to have a John Henry effect in your research and discuss how it may threaten internal validity.

Accepted Answer

The answer of Summarize what it means to have a...

Question 18

Consider the logic of quasi-experimentation and discuss the advantages and disadvantages to the use of quasi-experimentation. Provide an example of a study in which quasi-experimentation should be used.

Accepted Answer

The answer of Consider the logic of quasi-experimentation and discuss...

Question 19

Consider the politics of evaluation research and discuss how and why politics should be considered in evaluation research. Now imagine that your local government wants to implement a basic income of $1,000 per month for citizens earning less than $200,000 per year and they want you to evaluate the programs efficacy. Discuss the following:
a. name the independent variable in this study.
b. propose a possible dependent variable for this study.
c. What are two rival plausible explanations?
d. evaluate the study's internal validity, identify what threat(s) to internal validity you believe might be present, and state how you might safeguard the study to control for those threats.
e. discuss the potential use of time-series data and why it may (or may not) be helpful to this study.
f. discuss the role of politics in this process

Accepted Answer

The answer of Consider the politics of evaluation research and...

Quiz 7: From Manipulative to Analytic Control: Quasi-Experimentation

Single-session, short-term laboratory experimentation virtually precludes ______ as a problem. But in the field, where time-series data and a succession of follow-ups are more likely, this problem increases in relevance.

Alicia is conducting a study on marathon runners by interviewing them at multiple points during the marathon, however, some individuals she interviewed leave the marathon without completing. What threat to internal validity is this?

According to Weiss, which of the following is NOT one of the three major areas of political consideration should evaluation researchers be aware of in their research?

______ is the view that a better world can only be created by empirically driven evaluators.

Weiss offers three pieces of advice for dealing with politics in evaluation research. Which of the following is NOT included in her advice?