Question 1

If an evaluator believes that a potential client has requested an evaluation that will not meet the agency's real needs, the evaluator should
A) simply do the best job she can.
B) attempt to redefine the form of evaluation to be done.
C) tell the client what is to be done.
D) seek to gain power by going to the client's supervisor or board of directors.

Accepted Answer

It is important for the evaluator to ensure that the evaluation meets the agency's real needs. Attempting to redefine the form of evaluation to be done can help ensure that the evaluation is more useful and relevant to the agency. However, simply doing the best job possible may not address the underlying issue of the evaluation not meeting the agency's needs. Telling the client what to do or seeking to gain power may harm the evaluator's relationship with the client and damage the overall evaluation process.

Question 2

Ethical issues in program evaluation seem more compelling than in basic research because
A) evaluators are less skilled than basic researchers.
B) program evaluations are often replicated, thus assuring that errors will be detected.
C) program evaluations are seldom replicated and the findings may be applied even if in error.
D) service program staff members are notoriously unethical.

Accepted Answer

Program evaluations are often used to inform decision-making, such as whether to continue funding or implement a certain program. Therefore, errors in program evaluations can have real-world consequences and potentially harm individuals who rely on these programs. Additionally, program evaluations may involve working with vulnerable populations, raising additional ethical considerations. While basic research also raises ethical issues, the potential impact of errors is not always as immediate or significant. The other options (A, B, and D) are incorrect because they do not accurately reflect why ethical issues in program evaluation may be more compelling than in basic research.

Question 3

Program evaluators stop thinking about ethical issues
A) once a project is planned in an ethical manner.
B) after it is clear that the data have been gathered ethically.
C) after a report of an ethically conducted evaluation has been written.
D) at their peril because ethical standards can be violated at any stage or even after the report is finished.

Accepted Answer

Ethical standards must be considered throughout the entire evaluation process, from planning to report writing and beyond. Ignoring ethical issues can result in serious ramifications, both for the evaluator and for the individuals or groups being evaluated. Therefore, program evaluators should never stop thinking about ethical issues.

Question 4

"Informed consent" refers to
A) the contract used to employ an evaluator.
B) obtaining the agreement from people to supply data for an evaluation after they have been adequately informed about what is expected of them.
C) the consent that program staff must give to the evaluator who has been requested to evaluate their program; without such consent, the evaluation cannot be done.
D) participation of people in the service program.

Accepted Answer

Informed consent refers to obtaining consent from individuals to participate in an evaluation after providing them with adequate information about what is expected of them. This ensures that participants understand the purpose of the evaluation and what they are agreeing to before they participate. The other options are not accurate definitions of informed consent.

Question 5

Evaluators are especially concerned about the possibility of invalid conclusions because
A) evaluations are more likely to have an impact on people's lives than basic research reports do.
B) what they conclude cannot be applied.
C) they work in applied settings in which mistakes cannot be tolerated.
D) evaluation is a young field that has yet to prove its worth.

Accepted Answer

Evaluations often have direct implications for real-world decisions, policies, and actions, so it is crucial for evaluators to ensure that their conclusions are valid and reliable. Invalid conclusions could potentially harm individuals or communities affected by the program, policy, or intervention being evaluated. Therefore, evaluators are especially concerned about the possibility of invalid conclusions.

Question 6

If after beginning an evaluation the evaluator comes to feel that the evaluation plan cannot be carried out in a way that will yield a valid evaluation, the best alternative is to
A) do the best one can do, recognizing that no evaluation is perfect.
B) seek to renegotiate the evaluation plan so that it describes a plan that can be carried out well.
C) provide a favorable evaluation to the stakeholder paying for the project.
D) refuse to complete the project on the grounds that the findings will not be valid.

Accepted Answer

If the evaluator comes to the conclusion that the evaluation plan cannot be carried out in a way that will yield a valid evaluation, it is important to seek to renegotiate the plan with the stakeholder. This will involve finding ways to modify the plan so that it is more feasible and will produce valid results. It would not be appropriate to simply do the best one can do or provide a favorable evaluation when the findings may not be valid. Refusing to complete the project without attempting to renegotiate the plan could also be seen as unprofessional and unproductive.

Question 7

In conducting an evaluation of an innovative program, the most important responsibility of the evaluator is to
A) conduct a valid evaluation.
B) consider the generalizability of any positive findings.
C) be sure that the most needy individuals get into the program.
D) be sure that the evaluation does not harm the participants.

Accepted Answer

The answer of In conducting an evaluation of an innovative...

Question 8

Evaluators prefer to have informed consent from all participants in an evaluation. When a full description of the program groups (for example, innovative program and control groups) threatens to undermine the internal validity of the evaluation, a good way to decide if full
 disclosure is necessary is to
A) ask the major stakeholders what is the best procedure.
B) argue that a valid evaluation is more important than program participants having full information.
C) consider the cost to the participants of each group if they did not have such information.
D) insist that full disclosure is always required.

Accepted Answer

Considering the cost to the participants of each group if they did not have such information would help in weighing the importance of informed consent against the internal validity of the evaluation. This would enable evaluators to make an informed decision about whether full disclosure is necessary or not. Simply insisting on full disclosure or arguing that a valid evaluation is more important than program participants having full information may not be ethical or practical in all cases. Asking stakeholders' opinion may also be helpful, but ultimately, the decision should be based on ethical and practical considerations.

Question 9

Prior agreement from people who are to be sources of data in program evaluations can be called "informed consent" only when
A) an Informed Consent form has been signed.
B) the evaluator is present when the form is signed.
C) agreement has been given only after sufficient information has been provided to permit understanding the requirements of the evaluation.
D) the sponsor of the program fully understands the evaluation and what is expected of program participants.

Accepted Answer

Informed consent requires that the person has given agreement only after being provided with sufficient information to understand the requirements of the evaluation. The signing of a form or the presence of an evaluator is not necessary for informed consent to occur, although both may be helpful in documenting the consent. The understanding of the sponsor of the program is not a factor in determining informed consent.

Question 10

Problems that develop when an evaluator becomes an advocate for the program being evaluated include:
A) the loss of objectivity needed to conduct an evaluation.
B) the temptation to design evaluations that are too complex.
C) being forced by the staff to produce only favorable evaluations.
D) the development of hostility between evaluator and program staff.

Accepted Answer

When an evaluator becomes an advocate for the program being evaluated, they may lose their objectivity in conducting an evaluation, which is crucial for producing accurate and unbiased results. The evaluator may also feel pressure from staff to produce only favorable evaluations, potentially leading to the development of hostility between the evaluator and program staff. The temptation to design evaluations that are too complex may be a separate issue not necessarily related to becoming an advocate for the program being evaluated.

Question 11

The term stakeholder refers to
A) a tool used in barbecue restaurants.
B) the evaluators of a program.
C) anyone affected by the program being evaluated.
D) all people who are not affected by the program evaluation.

Accepted Answer

Stakeholders are anyone who is affected by the program being evaluated. They can include program participants, staff, funders, community members, and anyone else who has a vested interest in the program's outcomes.

Question 12

The evaluator needs to be concerned about serving the information needs of stakeholders because
A) these groups determine whether an evaluator will be paid.
B) stakeholders who feel that they have been ignored will seldom use the evaluation.
C) they must be mollified even if their input is not essential.
D) the evaluator will learn little from them.

Accepted Answer

Stakeholders are the ones who are directly affected by the evaluation and its outcomes, and hence their information needs are important to consider. If stakeholders feel that their information needs are ignored, they may not use the evaluation's findings or recommendations, which can affect the evaluation's overall effectiveness. Also, by serving the information needs of stakeholders, evaluators can gain a better understanding of the context in which the evaluation is taking place, which can lead to more useful and relevant recommendations.

Question 13

A problem with using measurement tools that do not validly measure the outcome of a program is that
A) the evaluation is not adequately scientific.
B) the stakeholders are less likely to understand the evaluation report.
C) a biased evaluation favoring the program is likely.
D) the actual outcome may be obscured.

Accepted Answer

If the measurement tools do not validly measure the outcome of a program, then the actual outcome may be obscured, leading to inaccurate conclusions about the effectiveness of the program. This can result in misguided decisions and wasted resources. Therefore, it is important to use measurement tools that accurately measure the intended outcomes of a program.

Question 14

Using a general school achievement test when a measurement of a specific skill is needed
A) can only lead to a favorable, but invalid report.
B) will lead to very dissatisfied stakeholders.
C) is not likely to be sensitive to the effect of the program.
D) will increase the likelihood that the report will be accepted.

Accepted Answer

Using a general school achievement test when a measurement of a specific skill is needed is not likely to be sensitive to the effect of the program because it does not focus on the specific skill being measured. This could lead to inaccurate and unreliable results. It is important to use a test that is specifically designed to measure the skill being assessed.

Question 15

Interviewing to gather data for an evaluation
A) is usually used in low-budget jobs.
B) indicates that the evaluator does not understand scientific measurement.
C) is seldom necessary for conducting a valid evaluation.
D) requires considerable training and skill.

Accepted Answer

Interviewing to gather data for an evaluation requires considerable training and skill. It is a widely used method for gathering qualitative data and can provide valuable insights into individuals' experiences, attitudes, and behaviors. However, to ensure validity and reliability, interviewers must use appropriate techniques to gather and analyze the data gathered. Therefore, it is not a low-budget job, nor does it indicate a lack of understanding of scientific measurement. Furthermore, in some cases, interviewing may be the only way to gather the necessary data for a valid evaluation.

Question 16

When considering evaluation designs, ethical evaluators seek to
A) match the research design and the informational needs of the stakeholders.
B) design the most scientifically valid outcome evaluation.
C) convince stakeholders that standardized tests are always better than other measurement procedures.
D) trace causal relationships between the program elements and the outcome even when examining a new program.

Accepted Answer

Ethical evaluators prioritize matching the research design with the informational needs of the stakeholders. It is important to consider the perspectives and needs of all stakeholders when designing an evaluation, and to make sure the evaluation is accessible and useful to them. The other options, such as prioritizing scientific validity above all else or advocating for standardized tests, may not necessarily align with ethical principles and could potentially harm stakeholder relationships. The fourth option of tracing causal relationships is important, but it is not the sole consideration for ethical evaluation design.

Question 17

The most scientifically sound evaluation design can fail to be useful if
A) the evaluator never checked to learn if the program elements were actually implemented.
B) interviews are relied upon for the data.
C) complex statistical analyses are not done.
D) the evaluation results are not published in scientifically respectable journals.

Accepted Answer

If the evaluator never checked to learn if the program elements were actually implemented, then the evaluation design and data collection methods would be irrelevant since the program being evaluated wasn't even properly implemented.

Question 18

It would be unusual for an evaluator to discover that
A) there were conflicts in the values of different stakeholder groups.
B) there were no conflicts in the values of different stakeholder groups.
C) the various stakeholder groups have difficulty articulating how the program is supposed to affect participants.
D) the various stakeholder groups draw the same interpretation from the results of an evaluation.

Accepted Answer

It is rare for an evaluator to find no conflicts in the values of different stakeholder groups as each group has its own goals and interests which may not align with those of other groups. Therefore, option B is an unlikely scenario. Options A, C, and D are more probable outcomes.

Question 19

A stakeholder group that is seldom consulted in planning an evaluation is the
A) management of the facility sponsoring the program.
B) staff of the program being evaluated.
C) leadership of the program being evaluated.
D) program participants or clients.

Accepted Answer

Program participants or clients are often excluded from the planning and implementation of program evaluations, even though their perspectives and experiences are essential to understanding the impact of the program. Including them in the evaluation process can provide valuable insights and help ensure that the evaluation outcomes are responsive to their needs and interests. The other stakeholder groups listed (management, staff, and leadership) are typically more involved in the planning and implementation of evaluations.

Question 20

Sources of different interpretations of the meaning of an evaluation that are largely beyond the control of an evaluator are disagreements over the
A) relative values placed on the outcomes of a program.
B) ways variables should be measured.
C) sources of evaluation data to be used.
D) ways to measure the cost of the program.

Accepted Answer

Different stakeholders may have different priorities and values when it comes to evaluating a program, which can lead to disagreements over the relative importance of particular outcomes. This can result in different interpretations of the meaning of the evaluation, and it may be difficult for the evaluator to control or influence these differing perspectives.  While variables measurement, data sources, and cost measurement can also contribute to differing interpretations, they are more within the control of the evaluator to address and manage.

Quiz 5: Ethics in Program Evaluation

If an evaluator believes that a potential client has requested an evaluation that will not meet the agency's real needs, the evaluator should

Ethical issues in program evaluation seem more compelling than in basic research because

Program evaluators stop thinking about ethical issues

"Informed consent" refers to

Evaluators are especially concerned about the possibility of invalid conclusions because

If after beginning an evaluation the evaluator comes to feel that the evaluation plan cannot be carried out in a way that will yield a valid evaluation, the best alternative is to

In conducting an evaluation of an innovative program, the most important responsibility of the evaluator is to

Prior agreement from people who are to be sources of data in program evaluations can be called "informed consent" only when

Problems that develop when an evaluator becomes an advocate for the program being evaluated include:

The term stakeholder refers to

The evaluator needs to be concerned about serving the information needs of stakeholders because

A problem with using measurement tools that do not validly measure the outcome of a program is that

Using a general school achievement test when a measurement of a specific skill is needed

Interviewing to gather data for an evaluation

When considering evaluation designs, ethical evaluators seek to

The most scientifically sound evaluation design can fail to be useful if

It would be unusual for an evaluator to discover that

A stakeholder group that is seldom consulted in planning an evaluation is the

Sources of different interpretations of the meaning of an evaluation that are largely beyond the control of an evaluator are disagreements over the