Question 1

Justice agencies have come to rely more on evidence-based policy, in which the actions of justice agencies are linked to evidence used for planning and evaluation.

Accepted Answer

Justice agencies have indeed come to rely more on evidence-based policy in recent years, in which decisions about policies and programs are made based on solid evidence and data. This approach allows for more effective planning and evaluation, and can help ensure that justice interventions are both cost-effective and evidence-based.

Question 2

Treatment integrity or treatment consistency refers to whether an experimental intervention is delivered as intended. It is almost equivalent to _____. Experimental designs in applied studies often suffer from problems related to treatment inconsistencies.

Accepted Answer

measurement reliability

Question 3

Problem analysis is another type of evaluation research. It helps public officials plan and select alternative actions. Not all types of designs, measures, and data collection techniques can be used in evaluation research and problem analysis.

Accepted Answer

This is false because virtually all types of designs, measures, and data collection techniques can be used in evaluation research and problem analysis. Problem analysis is another type of evaluation research. It helps public officials plan and select alternative actions.

Question 4

Increased federal requirements for program evaluations to accompany the implementation of new programs, and the availability of research funds to meet that requirement are some of the main reasons for the growth of _____ research. However, increased interest in program evaluation and problem analysis has followed heightened concern for the accountability of public officials and public policy.

Accepted Answer

A. This is an incorrect answer because the heightened concern for the accountability of public officials and public policy have resulted in the increased federal requirements for program evaluations to accompany the implementation of new programs, and the availability of research funds to meet that requirement are some of the main reasons for the growth of evaluation research. This can be qualitative or quantitative.
B. This is an incorrect answer because the heightened concern for the accountability of public officials and public policy have resulted in the increased federal requirements for program evaluations to accompany the implementation of new programs, and the availability of research funds to meet that requirement are some of the main reasons for the growth of evaluation research. This can be qualitative or quantitative.
C. This is an incorrect answer because the heightened concern for the accountability of public officials and public policy have resulted in the increased federal requirements for program evaluations to accompany the implementation of new programs, and the availability of research funds to meet that requirement are some of the main reasons for the growth of evaluation research. This can be qualitative or quantitative.
D. The correct answer is "D" because the heightened concern for the accountability of public officials and public policy have resulted in the increased federal requirements for program evaluations to accompany the implementation of new programs, and the availability of research funds to meet that requirement are some of the main reasons for the growth of evaluation research. This can be qualitative or quantitative.

Question 5

Specifying program goals is a fundamental first step in conducting evaluation
Studies. If officials are not certain about _____, it is not possible to determine whether goals are reached. If different stakeholders embrace different goals, evaluators must specify different ways to assess those goals.

Accepted Answer

A. This is an incorrect answer because if the officials do not know what a program is expected to achieve, then it is difficult to evaluate what is achieved. So, the first and foremost step in evaluation process is to determine program goals that specify what the program is expected to achieve. For example, if we want to reduce fear of crime, we need to be able to measure fear of crime. If we want to increase consistency in sentences for drug offenses, we need to be able to measure that.
B. This is an incorrect answer because if the officials do not know what a program is expected to achieve, then it is difficult to evaluate what is achieved. So, the first and foremost step in evaluation process is to determine program goals that specify what the program is expected to achieve. For example, if we want to reduce fear of crime, we need to be able to measure fear of crime. If we want to increase consistency in sentences for drug offenses, we need to be able to measure that.
C. This is an incorrect answer because if the officials do not know what a program is expected to achieve, then it is difficult to evaluate what is achieved. So, the first and foremost step in evaluation process is to determine program goals that specify what the program is expected to achieve. For example, if we want to reduce fear of crime, we need to be able to measure fear of crime. If we want to increase consistency in sentences for drug offenses, we need to be able to measure that.
D. The correct answer is "D" because if the officials do not know what a program is expected to achieve, then it is difficult to evaluate what is achieved. So, the first and foremost step in evaluation process is to determine program goals that specify what the program is expected to achieve. For example, if we want to reduce fear of crime, we need to be able to measure fear of crime. If we want to increase consistency in sentences for drug offenses, we need to be able to measure that.

Question 6

Problem analysis is another type of evaluation research. It helps public officials plan and select alternative actions. Almost all types of designs, measures, and data collection techniques can be used in _____ and problem analysis.

Accepted Answer

A. This is an incorrect answer because problem analysis is helps public officials plan and select alternative actions. Almost all types of designs, measures, and data collection techniques can be used in evaluation research and problem analysis Problem analysis is another type of evaluation research. It helps public officials plan and select alternative actions.
B. This is an incorrect answer because problem analysis is helps public officials plan and select alternative actions. Almost all types of designs, measures, and data collection techniques can be used in evaluation research and problem analysis Problem analysis is another type of evaluation research. It helps public officials plan and select alternative actions.
C. This is an incorrect answer because problem analysis is helps public officials plan and select alternative actions. Almost all types of designs, measures, and data collection techniques can be used in evaluation research and problem analysis Problem analysis is another type of evaluation research. It helps public officials plan and select alternative actions.
D. The correct answer is "D" because problem analysis is helps public officials plan and select alternative actions. Almost all types of designs, measures, and data collection techniques can be used in evaluation research and problem analysis Problem analysis is another type of evaluation research. It helps public officials plan and select alternative actions.

Question 7

Evaluation research does not play any significant role in program development and policy making.

Accepted Answer

Evaluation research is an important tool in program development and policy making. It helps to determine the effectiveness of current programs and policies, identify areas for improvement, and inform decision making about future programs and policies. Without evaluation research, programs and policies may not be based on evidence and may not effectively address the needs of the population they are designed to serve.

Question 8

Increased federal requirements for program evaluations to accompany the implementation of new programs, and the availability of research funds to meet that requirement are some of the main reasons for the growth of evaluation research.

Accepted Answer

Increased federal requirements for program evaluations and availability of research funds have indeed contributed to the growth of evaluation research.

Question 9

Program evaluation is different from problem analysis. Program evaluation is the process that seeks to link the _____ of criminal justice policy to empirical evidence that policies are being carried out as planned and are having the desired effects.

Accepted Answer

A. This is an incorrect answer because problem analysis takes place in the earlier stages of the policy process. In contrast, program evaluation studies are conducted in later stages and seek answers to two types of questions: 1) Are policies being implemented as planned? and 2) Are policies achieving their intended goals? Thus, program evaluation is the process that seeks to link the intended actions and goals of criminal justice policy to empirical evidence that policies are being carried out as planned and are having the desired effects.
B. This is an incorrect answer because problem analysis takes place in the earlier stages of the policy process. In contrast, program evaluation studies are conducted in later stages and seek answers to two types of questions: 1) Are policies being implemented as planned? and 2) Are policies achieving their intended goals? Thus, program evaluation is the process that seeks to link the intended actions and goals of criminal justice policy to empirical evidence that policies are being carried out as planned and are having the desired effects.
C. The correct answer is "C" because problem analysis takes place in the earlier stages of the policy process. In contrast, program evaluation studies are conducted in later stages and seek answers to two types of questions: 1) Are policies being implemented as planned? and 2) Are policies achieving their intended goals? Thus, program evaluation is the process that seeks to link the intended actions and goals of criminal justice policy to empirical evidence that policies are being carried out as planned and are having the desired effects.
D. This is an incorrect answer because problem analysis takes place in the earlier stages of the policy process. In contrast, program evaluation studies are conducted in later stages and seek answers to two types of questions: 1) Are policies being implemented as planned? and 2) Are policies achieving their intended goals? Thus, program evaluation is the process that seeks to link the intended actions and goals of criminal justice policy to empirical evidence that policies are being carried out as planned and are having the desired effects.

Question 10

An evaluability assessment is a "pre-evaluation," in which a researcher determines whether conditions necessary for conducting an evaluation are present. One obvious condition is _____ that will be evaluated.

Accepted Answer

A. This is an incorrect answer because one condition of the evaluability assessment is to determine whether there is support for the study from organizations delivering program components that will be evaluated. Sometimes it is difficult to obtain cooperation and support of the organizations delivering program components because the program leaders may fear that their own job performance is being evaluated and evaluation research can disrupt routine agency operations.
B. The correct answer is "B" because one condition of the evaluability assessment is to determine whether there is support for the study from organizations delivering program components that will be evaluated. Sometimes it is difficult to obtain cooperation and support of the organizations delivering program components because the program leaders may fear that their own job performance is being evaluated and evaluation research can disrupt routine agency operations.
C. This is an incorrect answer because one condition of the evaluability assessment is to determine whether there is support for the study from organizations delivering program components that will be evaluated. Sometimes it is difficult to obtain cooperation and support of the organizations delivering program components because the program leaders may fear that their own job performance is being evaluated and evaluation research can disrupt routine agency operations.
D. This is an incorrect answer because one condition of the evaluability assessment is to determine whether there is support for the study from organizations delivering program components that will be evaluated. Sometimes it is difficult to obtain cooperation and support of the organizations delivering program components because the program leaders may fear that their own job performance is being evaluated and evaluation research can disrupt routine agency operations.

Question 11

The policy process begins with a demand that normally appears as support for a new course of action or opposition to existing policy. Such demands can emerge from _____ sources. For example, widely publicized cases of racial discrimination in drug sentencing eventually led to the revision of federal sentencing guidelines for drug offenses.

Accepted Answer

A. This is an incorrect answer because the policy process begins with a demand for support for a new course of action or opposition to existing policy. These demands can emerge from within a public organization or from outside sources. For example, widely publicized cases of racial discrimination in drug sentencing eventually led to the revision of federal sentencing guidelines for drug offenses. Or a local prosecutor may decide to review all sentence recommendations made by deputies who prosecute drug cases.
B. The correct answer is "B" because problem analysis is helps public officials plan and select alternative actions. Almost all types of designs, measures, and data collection techniques can be used in evaluation research and problem analysis Problem analysis is another type of evaluation research. It helps public officials plan and select alternative actions.
C. This is an incorrect answer because problem analysis is helps public officials plan and select alternative actions. Almost all types of designs, measures, and data collection techniques can be used in evaluation research and problem analysis Problem analysis is another type of evaluation research. It helps public officials plan and select alternative actions.
D. This is an incorrect answer because problem analysis is helps public officials plan and select alternative actions. Almost all types of designs, measures, and data collection techniques can be used in evaluation research and problem analysis Problem analysis is another type of evaluation research. It helps public officials plan and select alternative actions.

Question 12

A growing number of law enforcement agencies are using computerized crime maps to _____. Producing computer-generated maps that display reported crimes within days after they have occurred is one of the most important policy planning tools for the New York City Police
Department.

Accepted Answer

A. The correct answer is "A" because using computerized crime maps to detect emerging patterns in crime and develop appropriate responses is one of the newest techniques law enforcement agencies are using. Producing computer-generated maps that display reported crimes within days after they have occurred is one of the most important policy planning tools for the New York City Police Department.
B. This is an incorrect answer because using computerized crime maps to detect emerging patterns in crime and develop appropriate responses is one of the newest techniques law enforcement agencies are using. Producing computer-generated maps that display reported crimes within days after they have occurred is one of the most important policy planning tools for the New York City Police Department.
C. This is an incorrect answer because using computerized crime maps to detect emerging patterns in crime and develop appropriate responses is one of the newest techniques law enforcement agencies are using. Producing computer-generated maps that display reported crimes within days after they have occurred is one of the most important policy planning tools for the New York City Police Department.
D. This is an incorrect answer because using computerized crime maps to detect emerging patterns in crime and develop appropriate responses is one of the newest techniques law enforcement agencies are using. Producing computer-generated maps that display reported crimes within days after they have occurred is one of the most important policy planning tools for the New York City Police Department.

Question 13

Evaluation research-sometimes called program evaluation-refers to a research purpose rather than a specific research method. Its special purpose is to evaluate the effects of policies.

Accepted Answer

This is true because the special purpose of evaluation research is to evaluate the effects of policies. For example research conducted to examine the effectiveness of mandatory arrest for domestic violence, innovations in probation, and new sentencing laws etc. will be evaluation research.

Question 14

Evaluation research entails special logistical, ethical, and political problems because it is
Embedded in the day-to-day events of _____. Although some ethical norms have political aspects-for example, not harming subjects relates to protection of civil liberties-no one has developed a set of political norms that can be agreed on by all criminal justice researchers.

Accepted Answer

A. The correct answer is "A" because day-to-day events of public policy and real life are integral part of criminal justice research. So, evaluation research entails special logistical, ethical, and political problems. Although some ethical norms have political aspects-for example, not harming subjects relates to protection of civil liberties-no one has developed a set of political norms that can be agreed on by all criminal justice researchers.
B. This is an incorrect answer because day-to-day events of public policy and real life are integral part of criminal justice research. So, evaluation research entails special logistical, ethical, and political problems. Although some ethical norms have political aspects-for example, not harming subjects relates to protection of civil liberties-no one has developed a set of political norms that can be agreed on by all criminal justice researchers.
C. This is an incorrect answer because day-to-day events of public policy and real life are integral part of criminal justice research. So, evaluation research entails special logistical, ethical, and political problems. Although some ethical norms have political aspects-for example, not harming subjects relates to protection of civil liberties-no one has developed a set of political norms that can be agreed on by all criminal justice researchers.
D. This is an incorrect answer because day-to-day events of public policy and real life are integral part of criminal justice research. So, evaluation research entails special logistical, ethical, and political problems. Although some ethical norms have political aspects-for example, not harming subjects relates to protection of civil liberties-no one has developed a set of political norms that can be agreed on by all criminal justice researchers.

Question 15

Evaluation research and problem analysis play a significant role in program development and policy making since criminal justice agencies are expected to _____ of their actions. If traditional approaches to probation supervision, for example, do not deter future law-breaking, new approaches should be developed and their effectiveness assessed.

Accepted Answer

A. This is an incorrect answer because evaluation research and problem analysis play a significant role in program development and policy making since criminal justice agencies are expected to justify the effectiveness and cost of their actions. For example, if traditional approaches to probation supervision do not deter future law-breaking, new approaches should be developed and their effectiveness assessed.
B. The correct answer is "B" because evaluation research and problem analysis play a significant role in program development and policy making since criminal justice agencies are expected to justify the effectiveness and cost of their actions. For example, if traditional approaches to probation supervision do not deter future law-breaking, new approaches should be developed and their effectiveness assessed.
C. This is an incorrect answer because evaluation research and problem analysis play a significant role in program development and policy making since criminal justice agencies are expected to justify the effectiveness and cost of their actions. For example, if traditional approaches to probation supervision do not deter future law-breaking, new approaches should be developed and their effectiveness assessed.
D. This is an incorrect answer because evaluation research and problem analysis play a significant role in program development and policy making since criminal justice agencies are expected to justify the effectiveness and cost of their actions. For example, if traditional approaches to probation supervision do not deter future law-breaking, new approaches should be developed and their effectiveness assessed.

Question 16

Policies are formulated to produce an effect or impact. In this sense, a policy can be
Viewed as a hypothesis in which _____ is expected to produce change in a dependent variable.

Accepted Answer

A. This is an incorrect answer because policies are formulated to produce an effect or impact. In this sense, a policy can be viewed as a hypothesis in which an independent variable is expected to produce change in a dependent variable. For example sentence review procedures are expected to produce a reduction in sentence disparities. Goal-directed public policies may therefore be viewed as if-then statements: if some policy action is taken, then we expect some result to be produced.
B. This is an incorrect answer because policies are formulated to produce an effect or impact. In this sense, a policy can be viewed as a hypothesis in which an independent variable is expected to produce change in a dependent variable. For example sentence review procedures are expected to produce a reduction in sentence disparities. Goal-directed public policies may therefore be viewed as if-then statements: if some policy action is taken, then we expect some result to be produced.
C. The correct answer is "C" because policies are formulated to produce an effect or impact. In this sense, a policy can be viewed as a hypothesis in which an independent variable is expected to produce change in a dependent variable. For example sentence review procedures are expected to produce a reduction in sentence disparities. Goal-directed public policies may therefore be viewed as if-then statements: if some policy action is taken, then we expect some result to be produced.
D. This is an incorrect answer because policies are formulated to produce an effect or impact. In this sense, a policy can be viewed as a hypothesis in which an independent variable is expected to produce change in a dependent variable. For example sentence review procedures are expected to produce a reduction in sentence disparities. Goal-directed public policies may therefore be viewed as if-then statements: if some policy action is taken, then we expect some result to be produced.

Question 17

Outcome measures are not the same as program goals. They are derived from goals, Program goals represent desired outcomes, whereas _____ of whether those desired outcomes are achieved. Identifying program goals and objectives results in a conceptual definition.

Accepted Answer

A. This is an incorrect answer because outcome measures are empirical indicators of whether those desired outcomes are achieved. Identifying program goals and objectives results in a conceptual definition. Outcomes measures are not the same as program goals. They are derived from program goals.
B. The correct answer is "B" because outcome measures are empirical indicators of whether those desired outcomes are achieved. Identifying program goals and objectives results in a conceptual definition. Outcomes measures are not the same as program goals. They are derived from program goals.
C. The correct answer is "B" because outcome measures are empirical indicators of whether those desired outcomes are achieved. Identifying program goals and objectives results in a conceptual definition. Outcomes measures are not the same as program goals. They are derived from program goals.
D. This is an incorrect answer because outcome measures are empirical indicators of whether those desired outcomes are achieved. Identifying program goals and objectives results in a conceptual definition. Outcomes measures are not the same as program goals. They are derived from program goals.

Question 18

Random assignment of subjects is often impossible in criminal justice evaluations. Rather than forgo evaluation altogether in such instances, it is usually possible to create and execute research designs that will permit evaluation of the program using quasi-experiments design. Quasi experiments design uses _____ to the experimental and control group.

Accepted Answer

A. This is an incorrect answer because random assignment of subjects is often impossible in criminal justice evaluations. Rather than forgo evaluation altogether in such instances, it is usually possible to create and execute research designs that will permit evaluation of the program using quasi-experiments design. Quasi experiments design uses random assignment to the experimental and control group.
B. This is an incorrect answer because random assignment of subjects is often impossible in criminal justice evaluations. Rather than forgo evaluation altogether in such instances, it is usually possible to create and execute research designs that will permit evaluation of the program using quasi-experiments design. Quasi experiments design uses random assignment to the experimental and control group.
C. The correct answer is "C" because random assignment of subjects is often impossible in criminal justice evaluations. Rather than forgo evaluation altogether in such instances, it is usually possible to create and execute research designs that will permit evaluation of the program using quasi-experiments design. Quasi experiments design uses random assignment to the experimental and control group.
D. This is an incorrect answer because random assignment of subjects is often impossible in criminal justice evaluations. Rather than forgo evaluation altogether in such instances, it is usually possible to create and execute research designs that will permit evaluation of the program using quasi-experiments design. Quasi experiments design uses random assignment to the experimental and control group.

Question 19

Policy outputs refer to what is actually produced, in much the same manner that a manufacturer of office supplies produces paper clips and staples. For example, the prosecutor's policy produces the routine review of sentence recommendations in drug cases or a selective traffic enforcement program intended to reduce auto accidents on a particular roadway may produce a _____ together with traffic citations for speeding.

Accepted Answer

A. This is an incorrect answer because to understand applications of evaluation to different stages of the policy process, it is important to distinguish between policy outputs and their impacts. Impacts are fundamentally related to policy goals while outputs embody the means to achieve desired policy goals. For example a prosecutor seeks to achieve equality in sentence recommendations - that is impact, so a review process is produced as a means to achieve that goal - that is output.
B. The correct answer is "B" because to understand applications of evaluation to different stages of the policy process, it is important to distinguish between policy outputs and their impacts. Impacts are fundamentally related to policy goals while outputs embody the means to achieve desired policy goals. For example a prosecutor seeks to achieve equality in sentence recommendations - that is impact, so a review process is produced as a means to achieve that goal - that is output.
C. This is an incorrect answer because to understand applications of evaluation to different stages of the policy process, it is important to distinguish between policy outputs and their impacts. Impacts are fundamentally related to policy goals while outputs embody the means to achieve desired policy goals. For example a prosecutor seeks to achieve equality in sentence recommendations - that is impact, so a review process is produced as a means to achieve that goal - that is output.
D. This is an incorrect answer because to understand applications of evaluation to different stages of the policy process, it is important to distinguish between policy outputs and their impacts. Impacts are fundamentally related to policy goals while outputs embody the means to achieve desired policy goals. For example a prosecutor seeks to achieve equality in sentence recommendations - that is impact, so a review process is produced as a means to achieve that goal - that is output.

Quiz 11: Section 2: Evaluation Research and Problem Analysis Glossary