Deck 7: Utility

Question 1

"Will the use of police-worn body cameras reduce use-of-force complaints?" According to the textbook,this question is a question regarding

A) reliability.
B) validity.
C) utility.
D) None of these.

Answer 1

C

Question 2

Ariel et al.(2015)found that police use-of-force rates were ______ that in the No Camera shifts as compared to the Camera shifts.

A) less than half
B) more than half
C) less than twice
D) more than twice

Answer 2

D

Question 3

Which of the following tests was mentioned by name by Dr.Delphine Courvoisier as an instrument that she uses in her daily work?

A) SFQ
B) HAQ
C) OPQ-2
D) All of these

Answer 3

B

Question 4

According to Dr.Delphine Courvoisier,psychologists who raise compelling research questions must understand that the road to satisfactory answers is paved with psychometric essentials such as

A) a sound research design.
B) the use of appropriate measures.
C) accurate analysis and interpretation of findings.
D) All of these.

Question 5

According to Dr.Delphine Courvoisier,when a research project demands that subjects respond to a series of telephone calls,it would be instructive to understand how _______ affects the other variables in the study.

A) anxiety
B) compliance
C) intelligence
D) distraction

Question 6

In Chapter 7's Meet an Assessment Professional,you met Dr.Delphine Courvoisier,whose Ph.D.is in _______ and who works as a ______.

A) biometrics; research consultant
B) clinical psychology; counselor
C) health psychology; psychometrician
D) psychometrics; biostatistician

Question 7

In the Chapter 7 Everyday Psychometrics in your textbook,the vignettes at the beginning were used to illustrate the fact that the police can do what they were trained to do and

A) the result will be "win-win."
B) the result will be "lose-lose."
C) and no one ever has to get hurt.
D) error can still find its way into the situation.

Question 8

According to Dr.Delphine Courvoisier,quality-of-life research for patients with a chronic disease

A) is best conducted at the time the disease is first diagnosed.
B) is best conducted one-year after onset of the disease.
C) is of limited applied value five years after the first diagnosis of the disease.
D) may be conducted at different points in time through the course of the disease.

Question 9

For the purposes of the Ariel et al.(2015)study,"use of force" was coded as being present on any occasion that a police verbal confrontation with a citizen escalated to the point of

A) shouting back-and-forth.
B) any physical contact.
C) the citizen being restrained by the officer.
D) shots fired.

Question 10

The Ariel et al.(2015)study of police use of force was conducted using officers from the police department of

A) San Francisco, California.
B) Rialto, California.
C) Los Angeles, California.
D) San Dimas, California.

Question 11

According to Dr.Delphine Courvoisier,contrary to what many may hold as an intuitive truth,success in the world of psychometrics cannot be measured by

A) a psychological test.
B) zeroes in a bank statement.
C) publication citations.
D) numbers alone.

Question 12

According to Dr.Delphine Courvoisier,ecological momentary assessment (EMA)is a tool of assessment that researchers can use to examine behaviors and subjective states in the settings in which they naturally occur,and at a frequency that can capture their

A) validity.
B) volume.
C) variability.
D) volatility.

Question 13

In the Ariel et al.(2015)study,the research protocol required officers to

A) wear cameras only during Camera shifts.
B) keep cameras on throughout their entire Camera shift.
C) issue verbal warnings during the Camera shifts to advise citizens confronted that the interaction was being videotaped by a camera attached to the officer's uniform.
D) All of these

Question 14

As used in utility analyses,the term cost refers to

A) insurance payments.
B) mortgage payments.
C) payments for test protocols.
D) All of these

Question 15

The Ariel et al.(2015)study was conducted over the course of

A) one year.
B) two years.
C) three years.
D) four years.

Question 16

In the Ariel et al.(2015)study,police officers were assigned to the Camera or No Camera condition

A) by the commander on duty at the time.
B) using the Cambridge Randomizer.
C) on the basis of case history data.
D) None of these

Question 17

In Chapter 7 of your textbook,if the illustrative case of the man who complained of leg pain while playing basketball was a story,the "moral" of the story is BEST captured by:

A) "Never play basketball, or other contact sport, if you are on the wrong side of 30."
B) "The utility of playing the game must be weighed against that of not playing."
C) "There are costs associated with testing, and costs associated with not testing."
D) "What goes up, eventually comes down."

Question 18

In a typical work day,Dr.Delphine Courvoisier might

A) help out one team of researchers in conceptualizing initial hypotheses.
B) assist a research team in selecting the most appropriate outcome measure.
C) help a research team with data analysis or interpretation.
D) All of these

Question 19

Dr.Delphine Courvoisier described her use of one test in health outcome research and the proprietary "DAS" it yielded.What did she mean by "DAS" in this context?

A) disease activity score
B) drug after-effect score
C) disability adjustment score
D) differential ability scaled score

Question 20

Ariel et al.(2015)found that body cameras worn by police have utility in reducing use-of-force incidents,as well as use-of-force complaints by citizens.However,given the procedures used in their study,questions remain regarding whether changes in the participants' behavior was more a function of the camera or

A) the police officer's verbal warning.
B) the ten directives in the experimental protocol.
C) officers attempting to give citizens two or more chances to comply with commands.
D) All of these

Question 21

Test validity

A) has nothing whatever to do with test utility.
B) sets a ceiling on test reliability.
C) sets a ceiling on test utility.
D) None of these

Question 22

The textbook authors caution that in addition to formulas,tables,and other formal methodologies,_____ must also be factored into any utility analysis.

A) common sense
B) physical property
C) intellectual property
D) All of these

Question 23

High morale on a university campus can be ____________ resulting from a utility analysis of student selection procedures.

A) an economic cost
B) a non-economic cost
C) an economic benefit
D) a non-economic benefit

Question 24

Which is the BEST general statement regarding a self-report test of integrity?

A) It is a valid test.
B) It is a reliable test.
C) It is not a psychometrically sound test.
D) It is a test of questionable utility.

Question 25

A utility analysis may be undertaken for many different purposes.Which of the following reasons is LEAST LIKELY to be one of them?

A) to determine if one test is preferable to another test
B) to determine if one method of intervention is preferable to another
C) to evaluate the validity of a test publisher's claim
D) to evaluate whether maintaining a training program is better than not having one.

Question 26

A group of testtakers all fail to follow the directions for taking a particular test.Which is TRUE?

A) The test results could still have great utility.
B) The test could still be psychometrically sound.
C) The test results could still be valid.
D) The reason for this must have to do with the test and not the testtakers.

Question 27

In the now classic utility analysis for the Bell system telephone company conducted in the 1980s,Cascio and Ramos concluded that

A) the use of a particular approach to assessment in selecting managers could save the company millions of dollars.
B) the use of a new approach to assessment in selecting managers was of little utility and would cost millions of dollars.
C) charging for 411 calls to "Information" could result in millions of dollars in new revenue for the company.
D) customers would make more calls if they could understand their phone bills.

Question 28

This is a tool that can be used to conduct a utility analysis.It is

A) expectancy tables.
B) Taylor-Russell tables.
C) Naylor-Shine tables.
D) All of these

Question 29

An index of utility can be distinguished from both an index of reliability and an index of validity in that an index of utility can tell us something about

A) how consistently a test measures what it measures in a particular context.
B) whether a test measures what it purports to measure in a particular context.
C) the practical value of the information derived from what a test measures.
D) None of these

Question 30

A limitation of the Taylor-Russell tables is that

A) the relationship between the test and the rating of performance must be linear.
B) the relationship between the predictor and the criterion must be linear.
C) criterion score difficulties in differentiating "successful" from "not successful."
D) All of these

Question 31

The textbook defines a utility analysis as

A) a proprietary technique.
B) a family of techniques.
C) an exclusionary procedure.
D) a multidisciplinary effort.

Question 32

A utility analysis of a test may BEST be thought of as

A) a model for understanding the costs of producing the test.
B) a way of envisioning new uses for the test.
C) an evaluation of the costs and benefits of the test.
D) an amalgam of reliability and validity data on the test.

Question 33

As noted by the Hansen et al.study cited in Chapter 7 of your text,the addition of two more x-ray views in addition to the conventional two is advisable because

A) it may make a more expensive procedure such as a CAT scan unnecessary.
B) the additional radiation exposure was shown to be insignificant.
C) it may be helpful in diagnosing whether physical abuse has occurred.
D) the procedure can do "double duty" as a drug screening tool.

Question 34

The tools of assessment used to make clinical judgments regarding the need for involuntary hospitalization

A) benefit society at large.
B) may cause some to be unjustly denied their freedom.
C) include tests, case history data, and interviews.
D) All of these

Question 35

Generally speaking,the specific objective of a utility analysis will dictate what sort of information will be required,as well as the specific

A) methods to be used.
B) expectancy tables to be used.
C) Naylor-Shine tables to be used.
D) Rise-and-Shine tables to be used.

Question 36

Thinking of the illustration of a utility analysis in the Close-up in Chapter 7,what type of error would have been made if a driver who was hired was actually not qualified for the job?

A) a false positive
B) a false negative
C) a near outlier
D) none of these

Question 37

If undertaken for the purpose of evaluating a particular intervention,a utility analysis can be helpful in making decisions about whether

A) one training program is preferable to another training program.
B) any intervention is better than no intervention.
C) one tool of assessment is more practical than another.
D) All of these

Question 38

The selection policies of the admissions office of a university can be instrumental in

A) building a good reputation for the university.
B) developing a good learning environment for students.
C) building high morale for the university's faculty.
D) All of these

Question 39

The end-point of a utility analysis is typically an educated decision about

A) limits on monetary expenditures for a new test or testing program.
B) which of many possible courses of action is optimal.
C) where to fix a cut point on a new test.
D) how to structure an intervention so that it is most cost efficient.

Question 40

In industrial settings,there are many non-economic benefits to be derived for the company that runs an effective testing program.Which of the following is NOT one such benefit?

A) increase in quality of workers' performance
B) decrease in time to train workers
C) reduction of work turnover
D) decrease in worker healthcare benefits

Question 41

The term multiple cut scores refers to

A) the use of two or more cut scores with reference to one predictor for the purpose of categorizing testtakers.
B) the use of two or more cut scores with reference to a multi-stage evaluation process that employs several predictors.
C) Both the use of two or more cut scores with reference to one predictor for the purpose of categorizing testtakers and the use of two or more cut scores with reference to a multi-stage evaluation process that employs several predictors.
D) None of these

Question 42

Multiple hurdles as used in a decision-making process regarding a selection decision refers to

A) the use of two or more cut scores with reference to one predictor for the purpose of categorizing testtakers.
B) the multiple stages each applicant must successfully complete in order to get to the next stage in the evaluation process.
C) the obstacles to success placed before each of the raters judging a competitive event.
D) All of these

Question 43

The known groups method for setting cut scores is also known as

A) the method of contrasting groups.
B) an IRT-based method.
C) the method of discriminant analysis.
D) the Angoff method.

Question 44

A hospital uses a compensatory model of selection in hiring surgeons.In their hiring evaluations,ratings regarding past safety record is given more weight than ratings regarding the surgeon's "bedside manner." From this,one could reasonably conclude that the people who are in charge of hiring surgeons believe that:

A) bedside manner is not very important for surgical staff members.
B) surgical safety is an art and a skill that is amenable to training.
C) a safe, bedside manner is essential for all staff members.
D) bedside manner is more amenable to training than surgical safety.

Question 45

When a cut score is set based on norm-related considerations rather than on the relationship of test scores to a criterion,the cut score is referred to as

A) a relative cut score.
B) a fixed cut score.
C) an absolute cut score.
D) a referential cut score.

Question 46

The term bookmark method refers to an IRT-based method

A) of marking items with regard to difficulty in a book of items.
B) of setting cut scores based on expert judgment.
C) with possible drawbacks such as floor or ceiling effects.
D) All of these

Question 47

The term item-mapping refers to an IRT-based method of

A) setting cut scores that entails expert judgments based, in part, upon how culturally fair items are deemed to be.
B) setting cut scores that entails the use of experts rearranging items placed on maps by level of difficulty.
C) setting cut scores that entails a histographic representation of test items.
D) test construction that was first used for a high school geography achievement test.

Question 48

The Angoff method of setting cutting scores relies heavily on

A) data and empirical findings.
B) the judgment of experts.
C) researching the scholarly literature.
D) Both data and empirical findings and researching the scholarly literature.

Question 49

The Brogden-Cronbach-Gleser formula was developed by

A) the team of Brogden, Cronbach, and Gleser working together.
B) Brogden, Cronbach, and Gleser, each working independently.
C) the work of Brogden and later, the work of Cronbach and Gleser.
D) the work of Cronbach, and later, the work of Brogden and Gleser.

Question 50

Decision theory is used to determine a test's utility for hiring employees who need to have high attention to detail to be successful on the job.In this context,to what does the term hit rate refer?

A) the proportion of people the test accurately identified as having this characteristic
B) the proportion of people the test failed to identify as having this characteristic
C) the number of people the test incorrectly identified as having this characteristic
D) the number of people having this characteristic that the test did not identify

Question 51

A potential non-economic benefit of a well-run evaluation program is

A) an increase in quality of workers' on-the-job performance.
B) a decrease in time it takes to train new workers.
C) a reduction in the number of workplace accidents.
D) All of these

Question 52

Which of the following is a direct economic cost that could result as a consequence of NOT evaluating personnel for employment positions within a large corporation?

A) the cost of public confidence in the corporation
B) the cost of accounting services and other routine costs of doing business
C) the cost in terms of lowered morale for employees of the corporation
D) the cost of lawsuits against the corporation

Question 53

In the context of utility analysis,which is NOT a direct economic benefit to a company?

A) increase in worker morale
B) less product being trashed as waste
C) increase in international sales
D) higher worker productivity

Question 54

Which is an example of a false negative in the context of employee selection?

A) hired applicants who scored at or above the cut-off score on the employment test and then went on to fail on the job
B) hired applicants who scored at or above the cut-off score on the employment test and then went on to succeed on the job
C) rejected applicants who scored below the cut-off score on the employment test who would have never gone on to succeed on the job
D) rejected applicants who scored below the cut-off score on the employment test who would have succeeded on the job had they been hired

Question 55

The "Achilles heel" of the Angoff method is

A) incremental validity.
B) test-retest reliability.
C) inter-rater reliability.
D) non-economic costs.

Question 56

A study that explored the utility of diagnostic X-rays in routine screening of children at risk for child abuse concluded that there was most utility in

A) a four-view series of X-rays.
B) a two-view series of X-rays.
C) one X-ray combined with visual examination.
D) visual examination only.

Question 57

Which is NOT an economic cost typically factored in a test utility analysis?

A) legal costs of doing business
B) fees paid by testtakers for testing services
C) computerized test processing services
D) cost of supplies of blank test protocols

Question 58

When the purpose of a utility analysis is to answer some finance-related question with a dollars-and-cents answer,the utility analysis will most likely employ

A) the Cascio-Ramos formula.
B) the Brogden-Cronbach-Gleser formula.
C) Kuder-Richardson Formula 20.
D) Taylor-Naylor Formula 1.

Question 59

The television program Dancing With the Stars

A) has a multiple hurdle selection model in place.
B) employs an absolute cut score for selection.
C) uses the known groups method to validate results.
D) once encouraged William Angoff to compete.

Question 60

A problem with using the known group method of setting cut scores is that

A) there is no consistent method of obtaining contrasting groups.
B) strong deterrents to test user acceptance of the data are in place.
C) no standards are in place for choosing contrasting groups.
D) test users must be personally familiar with each member in the known group.

Question 61

When the selection ratio for new personnel at a corporation goes down,

A) top-down selection policy can become discriminatory.
B) hiring becomes less selective.
C) competition for the position is likely to increase.
D) Both top-down selection policy can become discriminatory and hiring becomes less selective.

Question 62

The more complex the job,

A) the more people differ on how well they do it.
B) the less complex the decision process in hiring.
C) the less selective the hiring process.
D) the more the need for item-mapping procedures.

Question 63

Over the last year,a personnel manager hired 20 new employees but only 12 of them performed successfully.Here,the base rate of successful performance is

A) 0.375.
B) 0.625.
C) 0.600.
D) 0.750.

Question 64

Which is an example of a false positive in the context of employee selection?

A) hired applicants who scored at or above the cut-off score on the employment test went on to fail on the job
B) hired applicants who scored at or above the cut-off score on the employment test went on to succeed on the job
C) rejected applicants who scored below the cut-off score on the employment test were rejected but would have gone on to succeed on the job had they been given a chance
D) rejected applicants who scored below the cut-off score on the employment test were rejected but went on to succeed at another, totally different job

Question 65

An instructor assigns a grade of "A" to all students who earn 900 or more points out of a total of 1000 points during the semester.In this case,900 points is equal to
A)

A) the cut score for an
B) the success rate.
C) the selection ratio.
D) the base rate of A-level students.

Question 66

In Chapter 7 of your textbook,a pedagogical tool was presented to help you remember the implications of a low selection ratio.It was:

A) "Selection ratio low, more employees to know."
B) "Selection ratio high, watch workers wave good-bye."
C) "Selection ratio down, less employees around."
D) "Selection ratio low, long road to hoe."

Question 67

The term used to describe the proportion of people in a population who are distinctive due to their exhibition of a particular trait is

A) success rate.
B) base rate.
C) target rate.
D) cut rate.

Question 68

A large corporation scrupulously avoids any possibility of discrimination and adverse impact in its hiring practices.The selection procedure it probably has in place with regard to its entry-level test is one that entails

A) personnel selection based on a cut score.
B) a top-down selection policy based on test score.
C) personnel interviews using translators, if necessary.
D) None of these

Question 69

An educational psychologist conducts a utility analysis of a teaching program used to improve the handwriting of very young children.The measure of utility in this analysis will MOST likely be a variable related to

A) an increase in performance level.
B) a decrease in the cost of the program.
C) a reduction in program-related accidents.
D) an increase in program-related revenue.

Question 70

Consider an employment test that is highly accurate (about 98% correct in classifications),but very costly (about $5,000 per test).For which of the following positions would use of the test be most warranted?

A) Translator for the United States Ambassador to the U.N.
B) Short-order cook for the President of the United States
C) Assistant buyer for Sam's Club
D) All of these

Question 71

When setting a cut score on a predictor,

A) the Brogden-Cronbach-Gleser is usually used.
B) the utility of the test must first be quantitatively determined.
C) the goal of selection must be taken into account.
D) All of these

Question 72

Using a cut score of 50 on a predictor test a researcher finds a base rate of 1.00.This means that when a cut score of 50 is used,

A) 50% of applicants will perform successfully on a criterion measure.
B) 100% of applicants will perform successfully on a criterion measure.
C) 100% of applicants will fail on a criterion measure.
D) 50% of applicants will fail on a criterion measure.

Question 73

Which of the following is NOT an assumption of utility analysis?

A) the value of people and their performance can be estimated.
B) psychological tests are always preferred over other means of assessment.
C) the performance of people in organizations can affect organizational viability.
D) large amounts of information can be integrated to make good decisions.

Question 74

Having a personnel selection policy in place that scrupulously seeks to avoid the hiring of unqualified personnel,even at the expense of not hiring qualified personnel,

A) is patently discriminatory.
B) usually has costs attached to it that tend to outweigh the benefits.
C) can be justified if the position is one of great responsibility.
D) usually has benefits attached to it that tend to outweigh the costs.

Question 75

The term bookmark method refers to an IRT-based method

A) of estimating item difficulty using Las Vegas style odds.
B) of setting cut scores based on scholarly book reviews.
C) derived by researchers on sabbatical in Monte Carlo.
D) None of these

Question 76

When setting a cut score on a predictor,an objective is to attain

A) the highest false positive rate.
B) the lowest false negative rate.
C) a moderate hit rate.
D) All of these

Question 77

When the selection ratio goes up,

A) the bookmark procedure has less validity.
B) hiring becomes less selective.
C) hiring becomes more selective.
D) hiring is unaffected by the selection ratio.

Question 78

Offering permanent positions to only top-performing applicants is a strategy that can backfire because

A) competing companies could offer these same applicants positions.
B) these applicants, once hired, might not stay.
C) it may be discriminatory.
D) All of these

Question 79

Validity is to ____________ as utility is to ____________.

A) accuracy; consistency
B) usefulness; consistency
C) usefulness; accuracy
D) accuracy; usefulness

Question 80

In Chapter 7 of your textbook,a pedagogical tool was presented to help you remember the implications of a high selection ratio.It was:

A) "Selection ratio up, more coffee in your cup."
B) "Selection ratio up, the bigger the cut."
C) "Selection ratio high, more employees say 'Goodbye.'"
D) "Selection ratio high, more employees say 'Hi.'"