Deck 7: Evaluating HRD Programs

Question 1

HRD evaluation includes collecting information to make decisions about training activities.

Answer 1

True

Question 2

ROI stands for results of inventory.:

Answer 2

False

Question 3

Using deception to do evaluation research is a very good idea - it is ethical and moral and employees generally like it when this is used.

Answer 3

False

Question 4

Confidentiality in training evaluation is not important - the trainees work for the organization already.

Question 5

In order to effectively evaluate a training program the specific objectives of the program must be known?

Question 6

Utility analysis is easy for trainers to calculate

Question 7

A dramatic extension beyond Kirkpatrick's framework includes the idea that constituencies outside the organization should be involved in the evaluation process.

Question 8

The text supports separating behaviors into two segments: training and per training.

Question 9

A proposed modification to Kirkpatrick's evaluation model is to add a fifth level beyond results to include ROI.

Question 10

Donald Kirkpatrick described the following four training evaluation criteria: reaction , learning, job behavior and results

Question 11

According to Nickols each of Kirkpatrick's levels is of similar importance to all stakeholder groups.

Question 12

The reliability of data refers to the consistency of that data.

Question 13

A stakeholder is defined as 'a person or group with an interest in seeing an endeavor succeed but who do not need to support that activitiy".

Question 14

A control group is just a second group of trainees

Question 15

Ideally, the control group and the training group have the same scores before training

Question 16

According to Goldstein training evaluation has moved from simply collecting anecdotal evidence to now evaluating the entire HRD process.

Question 17

The CIPP framework for evaluation includes Context, Information, Process, Product

Question 18

When writing a questionnaire you should limit the number of response options.

Question 19

A control group is a group of people of the same age as the trainees.

Question 20

The text author believes that a positive score on one of Kirkpatrick's levels will mean a positive score on the measures of other levels. Positive reactions will result in learning.

Question 21

Which of the following is not a common type of data collected for evaluation?

A)Plant data
B)Economic data
C)Systemwide data
D)Individual performance data

Question 22

Trainee reaction to a training program is:

A)critical - if they don't like it, they can't learn
B)limited because it does not tell us if the program met its objective
C)in most cases, the only evaluation method that can be accomplished
D)the only level of evaluation worth doing

Question 23

According to Phillips which of the following is not a purpose of training evaluation?

A)determine the cost-benefit ration of an HRD program
B)determine if the program was appropriate
C)decide who should participate in future programs
D)decide whether or not to fire the trainer

Question 24

Collecting data in a pretest and posttest design allows us to:

A)Be certain the training caused any change
B)See what has changed after training
C)Develop the cost effectiveness of our training
D)Know whether or not we should revise the training design.

Question 25

The evaluation phase of the HRD process model includes all of the following except:

A)Selecting evaluation criteria
B)Conducting the evaluation
C)Interpreting the results
D)Publishing your results in the ASTD results book

Question 26

Self-report data is:

A)Data provided by an employee's supervisor
B)Data provided by the individual involved in the training
C)Data from company archives
D)Data provided by customers

Question 27

A control group is:

A)a second group of trainees
B)of no value in helping us evaluate training programs
C)often critical to determining whether changes were due to a training program rather than some other factor
D)an added cost that is generally too expensive

Question 28

Which of the following statements provides the best explanation for why training professionals often don't evaluate the results of their training programs:

A)results don't matter in training
B)fear of criticism or program cuts if a program was not effective
C)it can be done only if you majored in statistics in college
D)management refuses to spend the money to do it right

Question 29

If we provide the training and then collect evaluation measures we are:

A)Sure that any change we see is due to the training
B)Unsure as what caused the result we find, the training or ?
C)OK - but only if we randomly select the trainees
D)Following the best known research design

Question 30

The Kraiger, Ford & Salas evaluation model uses all of the following classifications of learning outcomes except:

A)Cognitive
B)Intuitive
C)Skill based
D)Affective

Question 31

Questionnaires are often used for evaluation because:

A)trainees like to fill them out
B)they can be completed and analyzed quickly
C)they are very interactive
D)they measure all 4 levels quickly and easily

Question 32

According to Zenger and Hargis HRD should be also be evaluated because:

A)Senior managers want this information and HRD budgets may be cut if it is not developed effectively.
B)Senior managers usually don't care - but line supervisors do care
C)It usually impact the trainer's bonus
D)Trainees need to know whether or not the program was effective.

Question 33

The most popular and influential framework for evaluating training was developed by:

A)Goldstein
B)Kirkpatrick
C)Phillips
D)Brinkerhoff

Question 34

According to studies by ASTD, the most common form of evaluation is:

A)reaction
B)learning
C)behavior
D)results

Question 35

Based on ASTD surveys about evaluating training programs which of the following is true?

A)Over 90% of companies surveyed conduct some form of evaluation
B)Only about 10% of companies surveyed conduct some form of evaluation
C)Results are the most commonly used method of evaluating training
D)Reactions are almost never evaluated

Question 36

Level 3 of Kirkpatrick's framework for evaluation is:

A)Reaction
B)Learning
C)Behaviors
D)Results

Question 37

When collecting data which of the following issues is not important?

A)Reliability
B)Validity
C)Effectiveness
D)Practicality

Question 38

A pre/post comparison is used to evaluate learning so that:

A)we know how much change took place during the program
B)we can calculate a t-test to get nice-looking statistics
C)we can compare the trained group to the control group
D)we know where the group was at the start

Question 39

A socially desirable response is

A)Telling the researcher your true feelings
B)Telling the researcher the truth
C)Telling the researcher what you think they want to hear
D)Telling the research an answer in sociological terms

Question 40

The number of units produced is generally what kind of data?

A)Plant data
B)Economic data
C)Systemwide data
D)Individual performance data

Question 41

Why is it a good idea to randomly assign people to the training group and the control group?

A)it is the easiest way to select equivalent groups
B)it is the easiest way to measure both pre and post training
C)it guarantees that training evaluation will be believed by management
D)it allows us to assume that attitude toward training is positive for all trainees

Question 42

All of the following are guidelines for writing effective questionnaires except:

A)Combine several ideas into one question to save on paper
B)Use clear and simple questions
C)Limit the number of response options
D)Match the mode of the response to the question

Question 43

Ideally, the control group and the training group have:

A)A lot in common
B)The same scores before training
C)The same supervisors
D)The exact same job description

Question 44

Donald Kirkpatrick described which of the following four training evaluation criteria?

A)reaction, learning, job behavior, modification
B)reaction , learning, job behavior and results
C)context, input, process and product
D)context, input, process and modification

Question 45

Informed consent:

A)Informs participants about any risks
B)Must be obtained before any training is done
C)Is completed at time of hire for all employees
D)Is never needed

Question 46

Practicality of a measure deals with:

A)Is it consistent
B)Does it measure what it is supposed to measure
C)Is it legal
D)Is there sufficient time, money and resources for this evaluation method

Question 47

Goldstein has suggested that training evaluation has moved through ______ stages since the 1960's.

A)4
B)5
C)2
D)22

Question 48

Validity asks the question:

A)Does the information evaluate a trainees reaction
B)Is the measure consistent
C)Does the data measure what it is supposed to measure
D)Is the data legal

Question 49

A method of evaluation, which measures the trainee's knowledge prior to and after training, is known as:

A)return on investment
B)variance improvement method
C)pre-test, post-test
D)gap closure method

Question 50

Which of the following is NOT a factor used to compute a utility estimate?

A)"A" ( available number of employees for training
B)"N" ( number of employees trained
C)"T" ( length of time benefits of training is expected to last
D)"C" ( cost of conducting the training

Question 51

Recent efforts to combine data from different research studies are known as:

A)macro-analysis
B)meta-analysis
C)combined analysis
D)collaborative analysis

Question 52

Utility analysis:

A)Is easy for trainers to calculate
B)Compares training gain to training costs
C)Is done using a computer program like Blackboard
D)Has never been used for training

Question 53

ROI stands for:

A)return on inventory
B)reduction of indirect costs
C)referral of information
D)return on investment

Question 54

Confidentiality in training evaluation is:

A)not important - the trainees work for the organization already
B)up to the trainer - they should not discuss what they find out during training
C)a good reason not to evaluate a training program
D)assured by using code numbers, rather than names, to identify trainees

Question 55

Which of the following is NOT an example of indirect training program costs:

A)instructor preparation
B)administrative support
C)course materials
D)all of the above are examples of indirect costs

Question 56

Which of the following is NOT an example of an ethical issue in evaluation research?

A)confidentiality
B)informed consent
C)adverse impact
D)use of deception

Question 57

Which of the following is not a purpose of training evaluation?

A)deciding who should participate in future programs
B)helping to design future training programs
C)establishing a data base for management
D)all of the above are purposes of training evaluation

Question 58

A cost-benefit analysis compares:

A)The benefits of one program to those of another one
B)The cost of training to its benefits
C)The costs of two different HRD programs
D)Actually - any two numbers you want to use would work

Question 59

The most commonly used type of data in HR evaluation is:

A)self-report data
B)return on investment
C)customer satisfaction surveys
D)organizational report data

Question 60

Which of the following is NOT an issue to consider when determining which data collection method to use in HRD evaluation.

A)reliability
B)validity
C)provability
D)practicality

Question 61

Ideally, the control group and the training group have the same scores before training

Question 62

A case study research design:

A)Is easy to use to evaluate training
B)Is questionable since there is no pre training information
C)Measures only results
D)Is very commonly used to evaluate training

Question 63

A pre/post comparison is used to evaluate learning so that:

A)we know how much change took place during the program
B)we can calculate a t-test to get nice-looking statistics
C)we can compare the trained group to the control group
D)we know where the group was at the start

Question 64

Internal validity deals with the question - could something besides the training program have caused the observed change to occur.

Question 65

Differential selection refers to:

A)Gaining job experience
B)The effect of a pretest on a posttest score
C)Differential loss of respondents from various groups
D)Using different procedures to select individuals for experimental and control groups

Question 66

Statistical power is the probability of concluding there is a difference between the training and the control groups when such a difference actually exists.

Question 67

A control group is just a second group of trainees

Question 68

Testing refers to:

A)Gaining job experience
B)The effect of a pretest on a posttest score
C)Differential loss of respondents from various groups
D)Using different procedures to select individuals for experimental and control groups

Question 69

Which of the following is not a form of research design validity?

A)Internal
B)External
C)Statistical conclusion
D)Statistical initial

Question 70

A control group is a group of people of the same age as the trainees.

Question 71

A quasi-experimental research design could be:

A)A non equivalent control group
B)A time series
C)A pre post test
D)Both A and B are quasi-experimental designs

Question 72

External validity means you are comparing your results to those of similar size organizations.

Question 73

History means that unrelated events occur during the training process that influence the training measurements.

Question 74

Relational research involves measuring two or more variables to see if they are measuring the same thing.

Question 75

Sackett and Mullen begin their analysis of nonexperimental research design by asking the question "How much change has occurred".

Question 76

Collecting data in a pretest and posttest design allows us to:

A)Be certain the training caused any change
B)See what has changed after training
C)Develop the cost effectiveness of our training
D)Know whether or not we should revise the training design.

Question 77

Internal validity can be affected by all of the following, except:

A)History
B)Maturation
C)Differential selection
D)Statistical abnormality

Question 78

Internal validity can be affected by all of the following, except:

A)Instrumentation
B)Testing
C)Differential investigation
D)Experimental mortality

Question 79

Maturation refers to:

A)Gaining job experience
B)The effect of a pretest on a posttest score
C)Differential loss of respondents from various groups
D)Using different procedures to select individuals for experimental and control groups

Question 80

A case study involves training followed by post training measurement.