Deck 12: Evaluation and Policy Analysis

Question 1

Applied research refers to which of the following?

A) research that involves mixed methods
B) interdisciplinary collaborations between researchers
C) research that has an impact on social policies
D) research that involves secondary data

Answer 1

C

Question 2

Evaluation research is ______.

A) a method of data collection
B) a type of sampling
C) a way to collect data
D) research conducted to investigate social programs

Answer 2

D

Question 3

The goal of evaluation research is to ______.

A) advance scientific knowledge
B) improve society
C) encourage interdisciplinary research
D) increase the ethical treatment of research subjects

Answer 3

B

Question 4

Which piece of legislation was a catalyst for evaluation research?

A) War on Drugs
B) Vietnam War
C) War on Poverty
D) New Deal

Question 5

A researcher is evaluating the success of an anger management program. In evaluation research terms, attending sessions comprise the ______.

A) inputs
B) outputs
C) program process
D) feedback

Question 6

All the following are examples of stakeholders EXCEPT ______.

A) program staff
B) clients
C) researchers
D) clients' families

Question 7

Which type of method is commonly used in process evaluations?

A) quantitative
B) qualitative
C) quantitative and qualitative
D) comparative

Question 8

Which of the following would be an example of a finding produced from a process evaluation?

A) An assessment could be done within the available timeframe and funding.
B) There is an issue in how clients are receiving the intervention.
C) The program is successful.
D) There is a need for this type of program.

Question 9

All the following are general types of evaluation research EXCEPT ______.

A) need
B) process
C) impact
D) improvement

Question 10

A juvenile court is assessing the impact of a diversion program. This diversion program curtails the use of secure confinement and increases the use of informal sanctions for minor delinquents. The evaluation shows that keeping minor delinquents out of secure confinements would not significantly increase the crime rate. The crime rate would remain stable because there are very few juveniles who would be eligible to participate in the diversion program. A citizen who is concerned about wasting financial resources argues that, before conducting an evaluation of impact, the researcher should have conducted an evaluation of ______.

A) need
B) process
C) efficiency
D) knowledge

Question 11

A local community hires a team of researchers to implement and evaluate a drug prevention program. After determining the needs of the community, the researchers suggest four successful drug prevention programs. The community selects one to implement. In their evaluations, the researchers critique the community for only selecting one drug prevention program rather than all four. The researchers further demonstrate that rates of drug use will only be affected if the community makes a more concerted effort in preventing use. The researchers' final conclusion is that the community's efforts to reduce drug use were not successful. These researchers conducted an evaluation of ______.

A) need
B) process
C) impact
D) efficiency

Question 12

A domestic violence reduction program is found to be ineffective. Researchers find the curriculum fails to consider how victims from different cultural backgrounds perceive victimization. As a result, they recommend greater cultural awareness in future iterations of the program. This is an example of a(n) ______.

A) need evaluation
B) process evaluation
C) impact evaluation
D) evaluability assessment

Question 13

A politician is interested in implementing a statewide welfare reform program. By providing welfare recipients with extensive job training, she believes that the number of welfare recipients will decrease. As a side benefit, she also believes that the crime rate will drop. In response, a researcher argues that the crime rate would indeed drop; however, the cost of this drop would be too high. Other cheaper alternatives could be utilized to reduce the level of crime. This researcher's work is an evaluation of ______.

A) need
B) process
C) impact
D) efficiency

Question 14

A cost-benefit analysis is an analysis in which ______.

A) inputs are estimated in terms of actual impacts
B) inputs and outputs are estimated in monetary terms
C) inputs are estimated terms of actual impacts and outputs are estimated in monetary terms
D) inputs and outputs are estimated in actual impacts

Question 15

A cost-effectiveness analysis is an analysis in which ______.

A) inputs are estimated in terms of actual impacts
B) inputs are estimated in monetary terms and outputs are estimated in terms of actual impacts
C) inputs are estimated terms of actual impacts and outputs are estimated in monetary terms
D) inputs and outputs are estimated in actual impacts

Question 16

A researcher examines the effectiveness of a drug rehabilitation program for severely addicted users. She finds that most of them manage to successfully quit their drug habits. What would the researcher need to know in order to perform a cost-effectiveness analysis?

A) inputs in monetary terms
B) outputs in monetary terms
C) inputs in terms of actual impacts
D) outputs in terms of actual impacts

Question 17

The strongest research design for assessing the impacts and outcomes of programs and other interventions is the ______.

A) nonequivalent control group design
B) longitudinal design
C) true experimental design
D) quasi-experimental design

Question 18

All the following are strengths of randomized experimental designs in impact evaluations EXCEPT which of the following?

A) Time order is correctly specified.
B) Causal mechanisms are easily identified.
C) The influence of other variables is ruled out.
D) Internal validity is generally high.

Question 19

Cost-benefit and cost-effectiveness analyses ______.

A) focus on internal and external validity, respectively
B) focus on external and internal validity, respectively
C) are examples of the evaluation of need
D) are examples of the evaluation of efficiency

Question 20

Compared with true experimental designs, nonexperimental designs ______.

A) have a higher internal validity
B) have a lower external validity
C) have a lower internal validity
D) are more likely to rule out the influence of other variables

Question 21

Nonexperimental designs ______.

A) offer very little in scientific value
B) compensate for their inability to establish external validity by establishing internal validity
C) are being used more frequently in evaluation research
D) are the second-best designs, after true experimental designs

Question 22

Braga et al.'s (1999) evaluation of the effectiveness of problem-oriented policing to decrease the incidence of violent street crime revealed which of the following?

A) Experimental designs are unethical to use in police work.
B) Problem-oriented policing can successfully control and prevent crime.
C) There was no relationship between problem-oriented policing and crime rates.
D) Problem-oriented policing led to an increase in crime.

Question 23

Braga et al.'s (1999) evaluation of the effectiveness of problem-oriented policing lacked which feature of experimental design?

A) mechanism
B) nonspuriousness
C) association
D) time order

Question 24

Which of the following is an accurate statement of Taylor and Woods' evaluation of conducted energy devices (CEDs)?

A) CEDs were found to reduce both officer and citizen injuries.
B) CEDs were found to reduce officer injuries.
C) CEDs were found to reduce citizen injuries.
D) CEDs were found to increase citizen injuries.

Question 25

In which type of research design is the experimental group studied only once after the treatment has been delivered?

A) experimental
B) quasi-experimental
C) non-experimental
D) semi-experimental

Question 26

Which of the following is NOT a part of the simple systems model of evaluation research?

A) inputs
B) outputs
C) outcomes
D) shareholders

Question 27

All the following are instances when qualitative methods would be preferable for evaluation research EXCEPT ______.

A) when there is an interest in comparing outcomes between experimental and control groups
B) when learning how different individuals react to a treatment is important
C) when the researcher wants to understand how social programs actually operate
D) when the study is investigating program process

Question 28

Most published evaluation studies are which type?

A) impact
B) need
C) process
D) evaluability

Question 29

Goal-free evaluation is an extreme form of which type of approach?

A) social sciences
B) stakeholder
C) integrative
D) experimental

Question 30

Before proposing or conducting his study, a researcher examines the projected costs of his study on inmate rehabilitation and compares them against expected program effectiveness. He is conducting a prestudy ______.

A) efficiency analysis
B) cost-benefit analysis
C) cost-effectiveness analysis
D) micromanagement analysis

Question 31

All the following are accurate statements about evaluability assessment EXCEPT which of the following?

A) Qualitative methods are commonly used.
B) Evaluability assessment makes a preliminary determination of whether the program can be assessed.
C) Evaluability assessment is used to determine if the program is cost-effective.
D) Alienated staff may be distrustful of assessment.

Question 32

Which of the following programs would be appropriate for evaluation research?

A) welfare programs
B) criminal justice programs
C) employment programs
D) all of these

Question 33

In what era did scientific methods emerge as it relates to examining the workings of government with the goal of improving society?

A) 1700s
B) 1850s
C) 1900s
D) 1950s

Question 34

Which of the following constituents are considered as practitioners?

A) government officials
B) corporations
C) non-profit agencies
D) all of these

Question 35

What are clients, customers, or students referred to in evaluation research?

A) inputs
B) outcomes
C) outputs
D) none of these

Question 36

Which of the following could be considered as possible outcomes in evaluation research?

A) improved test scores
B) higher rates of job retention
C) fewer criminal offenses
D) all of these

Question 37

What are clients served, case managers trained, or food parcels received referred to in evaluation research?

A) inputs
B) outcomes
C) outputs
D) none of these

Question 38

What type of assessment is appropriate to determine if a new program should be implemented?

A) risk assessment
B) needs assessment
C) evaluability assessment
D) none of these

Question 39

What type of assessment is appropriate to determine if a change should be made in program operations?

A) risk assessment
B) needs assessment
C) evaluability assessment
D) none of these

Question 40

What type of assessment is appropriate to determine if a study to identify the effects of a particular program may or may not be possible within the available time and resources?

A) risk assessment
B) needs assessment
C) evaluability assessment
D) none of these

Question 41

Evaluation research was not conducted prior to the 1950s.

Question 42

Evaluation research could use either quantitative or qualitative methodologies.

Question 43

An evaluation of need should be performed before new social programs are developed rather than after.

Question 44

Whenever possible, nonexperimental designs should not be used for evaluation research because they are "of almost no scientific value."

Question 45

In a theory-driven evaluation, a researcher addresses the process of the program.

Question 46

One issue with using an experimental research design in evaluation studies is the inability to randomly assign people into the control or experimental groups.

Question 47

A reason that a social program may not be able to be assessed is because it lacks a clear purpose.

Question 48

Evaluation research is designed to test the implications of criminological theory.

Question 49

A one-shot design is considered to be a credible research design.

Question 50

In a responsive evaluation, the stakeholders are responsive to the needs of the researcher.

Question 51

In basic science, the motivation is to improve society.

Question 52

Efficiency evaluations indicate that therapeutic communities are ineffective as the costs to operate them exceed the benefits.

Question 53

In a black box evaluation, a researcher addresses the process of the program.

Question 54

In a social science approach, the researcher independently specifies the nature and focus of the evaluation.

Question 55

Most published evaluation studies in criminal justice are devoted to some form of need assessment.

Question 56

Evaluation research will be pointless if the program itself cannot be evaluated.

Question 57

A needs assessment could determine if the program is not clearly distinct from other services delivered from the agency and so cannot be evaluated by itself.

Question 58

A process evaluation is a systematic attempt by evaluation researchers to examine program coverage and delivery.

Question 59

Assessing program delivery consists of estimating the extent to which a program is reaching its intended target population.

Question 60

Assessing program coverage consists of measuring the degree of congruence between the plan for providing services and treatments and the ways they are actually provided.

Question 61

What was one of the main initiatives in the United States in the 1960s that spawned the growth of evaluation research?

Question 62

What are the major differences between basic science and applied research?

Question 63

The National Institute of Justice would like to request proposals for the evaluation of an offender reintegration program. Describe what type of evaluation program would be most useful, and why.

Question 64

Briefly describe the main components of a true experiment to determine if proactive policing reduces crime.

Question 65

Describe a research project using a one-shot design.

Question 66

Identify and explain the major ethical issues in evaluation research.

Question 67

What is a needs assessment?

Question 68

Describe the difference between a black box evaluation and a theory-driven evaluation.

Question 69

Discuss the advantages of using stakeholder approaches in evaluation research.

Question 70

Using an experimental design, create an impact evaluation of a program that allows prisoners to attend classes at a local college. The program is designed to reduce the recidivism of participants as compared with nonparticipants.

Question 71

Identify several questions that evaluation projects typically focus on.

Question 72

What are the differences between an efficiency analysis, a cost-benefit analysis, and a cost-effectiveness analysis?

Question 73

What are the disadvantages to both stakeholder and social science approaches to program evaluation?

Question 74

Define and describe the three criteria for establishing a causal relationship between an independent variable and a dependent variable.

Question 75

Describe what the Campbell Collaboration is and what purpose they serve.