Deck 7: Evaluating HRD Programs

Question 1

Learning should be measured at two time:: pre training and post training.

Answer 1

True

Question 2

The reliability of data refers to the consistency of that data.

Answer 2

True

Question 3

HRD evaluation includes collecting information to make decisions about training activities.

Answer 3

True

Question 4

Donald Kirkpatrick described the following four training evaluation criteria: reaction,learning,job behavior and results

Question 5

A control group is just a second group of trainees

Question 6

The text author believes that a positive score on one of Kirkpatrick's levels will mean a positive score on the measures of other levels.Positive reactions will result in learning.

Question 7

According to Goldstein training evaluation has moved from simply collecting anecdotal evidence to now evaluating the entire HRD process.

Question 8

A control group is a group of people of the same age as the trainees.

Question 9

When writing a questionnaire you should limit the number of response options.

Question 10

According to Nickols each of Kirkpatrick's levels is of similar importance to all stakeholder groups.

Question 11

A stakeholder is defined as 'a person or group with an interest in seeing an endeavor succeeds but who do not need to support that activity".

Question 12

A proposed modification to Kirkpatrick's evaluation model is to add a fifth level beyond results to include ROI.

Question 13

In order to effectively evaluate a training program the specific objectives of the program must be known?

Question 14

The CIPP framework for evaluation includes Context,Information,Process,Product

Question 15

ROI stands for results of inventory.:

Question 16

Using deception to do evaluation research is a very good idea - it is ethical and moral and employees generally like it when this is used.

Question 17

Utility analysis is easy for trainers to calculate

Question 18

Confidentiality in training evaluation is not important - the trainees work for the organization already.

Question 19

Ideally,the control group and the training group have the same scores before training

Question 20

A dramatic extension beyond Kirkpatrick's framework includes the idea that constituencies outside the organization should be involved in the evaluation process.

Question 21

According to Chris Bones:

A) Evaluating what you know works is a waste of time
B) Only evaluate if you are forced to do it
C) Evaluation is too costly to bother with
D) If you can't see results stop it

Question 22

The number of units produced is generally what kind of data?

A) Plant data
B) Economic data
C) Systemwide data
D) Individual performance data

Question 23

When collecting data which of the following issues is not important?

A) Reliability
B) Validity
C) Effectiveness
D) Practicality

Question 24

Based on ASTD surveys about evaluating training programs which of the following is true?

A) Over 90% of companies surveyed conduct some form of evaluation
B) Only about 10% of companies surveyed conduct some form of evaluation
C) Results are the most commonly used method of evaluating training
D) Reactions are almost never evaluated

Question 25

If we provide the training and then collect evaluation measures we are:

A) Sure that any change we see is due to the training
B) Unsure as what caused the result we find, the training or ?
C) OK - but only if we randomly select the trainees
D) Following the best known research design

Question 26

A control group is:

A) a second group of trainees
B) of no value in helping us evaluate training programs
C) often critical to determining whether changes were due to a training program rather than some other factor
D) an added cost that is generally too expensive

Question 27

A pre/post comparison is used to evaluate learning so that:

A) we know how much change took place during the program
B) we can calculate a t-test to get nice-looking statistics
C) we can compare the trained group to the control group
D) we know where the group was at the start

Question 28

Which of the following statements provides the best explanation for why training professionals often don't evaluate the results of their training programs:

A) results don't matter in training
B) fear of criticism or program cuts if a program was not effective
C) it can be done only if you majored in statistics in college
D) management refuses to spend the money to do it right

Question 29

Questionnaires are often used for evaluation because:

A) trainees like to fill them out
B) they can be completed and analyzed quickly
C) they are very interactive
D) they measure all 4 levels quickly and easily

Question 30

A socially desirable response is

A) Telling the researcher your true feelings
B) Telling the researcher the truth
C) Telling the researcher what you think they want to hear
D) Telling the research an answer in sociological terms

Question 31

Collecting data in a pretest and posttest design allows us to:

A) Be certain the training caused any change
B) See what has changed after training
C) Develop the cost effectiveness of our training
D) Know whether or not we should revise the training design.

Question 32

Trainee reaction to a training program is:

A) critical - if they don't like it, they can't learn
B) limited because it does not tell us if the program met its objective
C) in most cases, the only evaluation method that can be accomplished
D) the only level of evaluation worth doing

Question 33

The evaluation phase of the HRD process model includes all of the following except:

A) Selecting evaluation criteria
B) Conducting the evaluation
C) Interpreting the results
D) Publishing your results in the ASTD results book

Question 34

The Kraiger,Ford & Salas evaluation model uses all of the following classifications of learning outcomes except:

A) Cognitive
B) Intuitive
C) Skill based
D) Affective

Question 35

Which of the following is not a common type of data collected for evaluation?

A) Plant data
B) Economic data
C) Systemwide data
D) Individual performance data

Question 36

According to studies by ASTD,the most common form of evaluation is:

A) reaction
B) learning
C) behavior
D) results

Question 37

Level 3 of Kirkpatrick's framework for evaluation is:

A) Reaction
B) Learning
C) Behaviors
D) Results

Question 38

The most popular and influential framework for evaluating training was developed by:

A) Goldstein
B) Kirkpatrick
C) Phillips
D) Brinkerhoff

Question 39

Self-report data is:

A) Data provided by an employee's supervisor
B) Data provided by the individual involved in the training
C) Data from company archives
D) Data provided by customers

Question 40

Which of the following is not a purpose of training evaluation?

A) deciding who should participate in future programs
B) helping to design future training programs
C) establishing a data base for management
D) all of the above are purposes of training evaluation

Question 41

Donald Kirkpatrick described which of the following four training evaluation criteria?

A) reaction, learning, job behavior, modification
B) reaction , learning, job behavior and results
C) context, input, process and product
D) context, input, process and modification

Question 42

Confidentiality in training evaluation is:

A) not important - the trainees work for the organization already
B) up to the trainer - they should not discuss what they find out during training
C) a good reason not to evaluate a training program
D) assured by using code numbers, rather than names, to identify trainees

Question 43

Which of the following is NOT an example of indirect training program costs?

A) instructor preparation
B) administrative support
C) course materials
D) all of the above are examples of indirect costs

Question 44

Practicality of a measure deals with:

A) Is it consistent
B) Does it measure what it is supposed to measure
C) Is it legal
D) Is there sufficient time, money and resources for this evaluation method

Question 45

A control group is a group of people of the same age as the trainees.

Question 46

All of the following are guidelines for writing effective questionnaires except:

A) Combine several ideas into one question to save on paper
B) Use clear and simple questions
C) Limit the number of response options
D) Match the mode of the response to the question

Question 47

Ideally,the control group and the training group have:

A) A lot in common
B) The same scores before training
C) The same supervisors
D) The exact same job description

Question 48

Goldstein has suggested that training evaluation has moved through ______ stages since the 1960's.

A) 4
B) 5
C) 2
D) 22

Question 49

Validity asks the question:

A) Does the information evaluate a trainees reaction
B) Is the measure consistent
C) Does the data measure what it is supposed to measure
D) Is the data legal

Question 50

Informed consent:

A) Informs participants about any risks
B) Must be obtained before any training is done
C) Is completed at time of hire for all employees
D) Is never needed

Question 51

Which of the following is NOT an example of an ethical issue in evaluation research?

A) confidentiality
B) informed consent
C) adverse impact
D) use of deception

Question 52

Which of the following is NOT an issue to consider when determining which data collection method to use in HRD evaluation?

A) reliability
B) validity
C) provability
D) practicality

Question 53

Recent efforts to combine data from different research studies are known as:

A) macro-analysis
B) meta-analysis
C) combined analysis
D) collaborative analysis

Question 54

The most commonly used type of data in HR evaluation is:

A) self-report data
B) return on investment
C) customer satisfaction surveys
D) organizational report data

Question 55

A cost-benefit analysis compares:

A) The benefits of one program to those of another one
B) The cost of training to its benefits
C) The costs of two different HRD programs
D) Actually - any two numbers you want to use would work

Question 56

Utility analysis:

A) Is easy for trainers to calculate
B) Compares training gain to training costs
C) Is done using a computer program like Blackboard
D) Has never been used for training

Question 57

Which of the following is NOT a factor used to compute a utility estimate?

A) "A" ( available number of employees for training
B) "N" ( number of employees trained
C) "T" ( length of time benefits of training is expected to last
D) "C" ( cost of conducting the training

Question 58

Why is it a good idea to randomly assign people to the training group and the control group?

A) it is the easiest way to select equivalent groups
B) it is the easiest way to measure both pre and post training
C) it guarantees that training evaluation will be believed by management
D) it allows us to assume that attitude toward training is positive for all trainees

Question 59

ROI stands for:

A) return on inventory
B) reduction of indirect costs
C) referral of information
D) return on investment

Question 60

A method of evaluation,which measures the trainee's Evaluation prior to and after training,is known as:

A) return on investment
B) variance improvement method
C) pre-test, post-test
D) gap closure method

Question 61

Collecting data in a pretest and posttest design allows us to:

A) Be certain the training caused any change
B) See what has changed after training
C) Develop the cost effectiveness of our training
D) Know whether or not we should revise the training design.

Question 62

A quasi-experimental research design could be:

A) A non equivalent control group
B) A time series
C) A pre post test
D) Both A and B are quasi-experimental designs

Question 63

Statistical power is the probability of concluding there is a difference between the training and the control groups when such a difference actually exists.

Question 64

Maturation refers to:

A) Gaining job experience
B) The effect of a pretest on a posttest score
C) Differential loss of respondents from various groups
D) Using different procedures to select individuals for experimental and control groups

Question 65

A control group is just a second group of trainees

Question 66

History means that unrelated events occur during the training process that influences the training measurements.

Question 67

Differential selection refers to:

A) Gaining job experience
B) The effect of a pretest on a posttest score
C) Differential loss of respondents from various groups
D) Using different procedures to select individuals for experimental and control groups

Question 68

Ideally,the control group and the training group have the same scores before training

Question 69

Testing refers to:

A) Gaining job experience
B) The effect of a pretest on a posttest score
C) Differential loss of respondents from various groups
D) Using different procedures to select individuals for experimental and control groups

Question 70

A case study involves training followed by post training measurement.

Question 71

Relational research involves measuring two or more variables to see if they are measuring the same thing.

Question 72

Sackett and Mullen begin their analysis of non-experimental research design by asking the question "How much change has occurred".

Question 73

A case study research design:

A) Is easy to use to evaluate training
B) Is questionable since there is no pre training information
C) Measures only results
D) Is very commonly used to evaluate training

Question 74

A control group is:

A) a second group of trainees
B) of no value in helping us evaluate training programs
C) often critical to determining whether changes were due to a training program rather than some other factor
D) an added cost that is generally too expensive

Question 75

Internal validity can be affected by all of the following,except:

A) History
B) Maturation
C) Differential selection
D) Statistical abnormality

Question 76

Internal validity can be affected by all of the following,except:

A) Instrumentation
B) Testing
C) Differential investigation
D) Experimental mortality

Question 77

Which of the following is not a form of research design validity?

A) Internal
B) External
C) Statistical conclusion
D) Statistical initial

Question 78

Internal validity deals with the question - could something besides the training program have caused the observed change to occur.

Question 79

A pre/post comparison is used to evaluate learning so that:

A) we know how much change took place during the program
B) we can calculate a t-test to get nice-looking statistics
C) we can compare the trained group to the control group
D) we know where the group was at the start

Question 80

External validity means you are comparing your results to those of similar size organizations.