Deck 8: Test Development

Question 1

In the course of developing their asexuality measure,Brotto and Yule were able to identify about ____% of self-identified asexual individuals.

A) 88
B) 93
C) 94
D) 97

Answer 1

B

Question 2

As illustrated in the sample item-characteristic curve published in your textbook,the vertical axis on the graph lists the

A) values of the score on the test ranging from 0 to 100.
B) values of the characteristic of the items on a scale of 1 to 10.
C) heteroscedasity of the item curve in values ranging from 0 to infinity.
D) probability of correct response in values ranging from 0 to 1.

Answer 2

D

Question 3

It is an online community of asexual individuals which has become a source of recruitment of subjects for asexuality research.It is called the

A) Asexuality and Visibility Education Network.
B) Friends of Asexuality.
C) League of Asexual and Non-Sexual Individuals.
D) American Society of Affiliated Individuals for Asexuality.

Answer 3

A

Question 4

Estimates suggest that approximately __% of the population might be asexual.

A) 1
B) 2
C) 3
D) 4

Question 5

Brotto and Yule established the discriminant validity of their measure of asexuality by comparing scores on it with scores on

A) the Childhood Trauma Questionnaire.
B) the Short-Form Inventory of Interpersonal Problems-Circumplex scales.
C) the Big-Five Inventory.
D) All of these

Question 6

Asexuality

A) is a sexual orientation.
B) is not a sexual orientation.
C) is considered by some to be a sexual orientation and by others not.
D) was de-listed as a sexual orientation in DSM-5.

Question 7

The concept of asexuality was first introduced by

A) William Masters.
B) Alfred Kinsey.
C) Virginia Johnson.
D) William Masters and Virginia Johnson.

Question 8

Brotto and Yule reported that the development of their measure of asexuality was developed in four stages.Which best characterizes Stage 1?

A) literature search for definitions of asexuality
B) development of open-ended questions
C) literature search for correlates of asexuality
D) writing and submission of a research grant request

Question 9

Brotto and Yule expressed their belief that their new measure of asexuality

A) does not depend on one's self-identification as asexual.
B) is not capable of identifying the individual who exhibits characteristics of a lifelong lack of sexual attraction in the absence of personal distress.
C) should be used with caution as a tool of recruitment with members of the asexuality population.
D) All of these

Question 10

Which statement is TRUE regarding test development and testtaker guessing?

A) Methods have been designed to detect guessing.
B) Methods have been designed to statistically correct for guessing.
C) Methods have been designed to minimize the effects of guessing.
D) All of these

Question 11

Brotto and Yule reported that the development of their measure of asexuality was developed in four stages.Which best characterizes what they did during Stages 2 and 3?

A) analysis of variance
B) regression analysis
C) factor analysis
D) meta-analysis

Question 12

A disadvantage of recruiting asexual research subjects from a single online community is that

A) the persons belonging to the online community may constitute a unique group within the asexual population.
B) the persons belonging to the online community have already acknowledged their asexuality as an identity.
C) asexual individuals who do not belong to the community will be systematically omitted.
D) All of these.

Question 13

Item banks

A) were once a profit center for the Wells Fargo Company.
B) originated as a result of investments made by Morgan-Stanley.
C) originated as a result of investments made by Morgan Freeman.
D) None of these

Question 14

Many asexual individuals refer to themselves as

A) "selfies".
B) "ace".
C) "lone rangers".
D) "gender-neutral".

Question 15

In response to the need for an instrument to help identify individuals who have experienced a lifelong lack of sexual attraction,but who have never heard the term "asexual," Yule et al.(2015)developed a test called the

A) Asexuality Evaluation Schedule.
B) Asexuality Identification Scale.
C) Asexual Research Subject Selector.
D) None of these

Question 16

The test of asexuality developed by Yule et al.(2015)contains ___ items.

A) 12
B) 18
C) 36
D) 48

Question 17

In order to determine whether their new measure of asexuality was useful over and above already-available measures of sexual orientation,Brotto and Yule compared it to a previously established measure of sexual orientation called the

A) Sexual Desire Inventory.
B) Solitary Desire subscale of the Sexual Desire Inventory.
C) Abernathy Measure of Sexual Orientation.
D) Klein Scale.

Question 18

An analysis of a test's item may take many forms.Thinking of the descriptions cited in your text,which is NOT one of those forms?

A) item validity analysis
B) item discrimination analysis
C) item tryout analysis
D) item reliability analysis

Question 19

According to Brotto and Yule,their new measure of asexuality performed satisfactorily on

A) a measure of incremental validity.
B) a measure of convergent validity.
C) a measure of discriminant validity.
D) All of these

Question 20

Human asexuality is generally defined as

A) the absence of sexual attraction to anything at all.
B) a sexual attraction only to other asexual people.
C) an unwillingness or inability to experience sexual arousal.
D) the absence of sexual attraction to anyone at all.

Question 21

The elements of a multiple-choice item include

A) a stem.
B) a distractor.
C) a foil.
D) All of these

Question 22

If 100 people take a test and 20 of those testtakers answer a particular item correctly,then the p value of the item is

A) .25.
B) .20.
C) .40.
D) .04.

Question 23

Test items that contain alternatives with five points ranging from "strongly agree" to "strongly disagree" are characterized as using this approach to scaling:

A) Guttman scaling.
B) Likert scaling.
C) Nielson scaling.
D) Opinion scaling.

Question 24

With regard to the test tryout phase of test development,

A) test conditions should be as similar to the actual administration as possible.
B) at least 500 subjects should be included to ensure accurate results.
C) the sample used must be nationally representative.
D) All of these

Question 25

Multiple-choice items draw primarily on which testtaker ability?

A) recognition.
B) organization.
C) planning.
D) perceptual-motor skills.

Question 26

An ADVANTAGE of applying item response theory (IRT)in test development is that

A) the principles underlying IRT make its application easy and appealing.
B) sample sizes used to test the utility of test items can be relatively small.
C) assumptions underlying IRT usage are weak.
D) item statistics are independent of the samples administered the test.

Question 27

Guttman scales

A) are typically used with nominal categories.
B) typically are constructed so that agreement with one statement may predict agreement with another statement.
C) typically are constructed so that agreement with one statement should not be correlated with agreement with any other statement.
D) were originally developed by a Peace Corps task force.

Question 28

The idea for a new test may come from

A) social need.
B) review of the available literature.
C) common sense appeal.
D) All of these

Question 29

An item bank is

A) a computerized system whereby test items "pay dividends" only when used.
B) the optimum combination of reliability and validity in an item.
C) a set of items from which a test can be constructed.
D) a statistical "IRA" for data relating to high and low scorers on a test.

Question 30

Item branching refers to

A) administering certain test items on a test depending on the testtakers' responses to previous test items.
B) the creation of alternate and parallel forms of tests based on a group of testtakers' responses to the original test.
C) statistical efforts to ensure that items translated into foreign languages are of the same difficulty.
D) re-using items in an original test that were originally developed for use in a parallel test.

Question 31

Which is an example of the selected-response item format?

A) a multiple-choice item
B) a fill-in-the-blank item
C) Both a multiple-choice item and a fill-in-the-blank item
D) None of these

Question 32

An example of a selected-response type of item is

A) a multiple-choice item.
B) an essay item.
C) a matching item.
D) Both a multiple-choice item and a matching item.

Question 33

According to your textbook,the minimum sample for a test tryout is

A) one-half of the number of testtakers in the standardization sample.
B) 25 testtakers.
C) 50 testtakers.
D) 500 testtakers.

Question 34

According to the text,which statement is TRUE of scaling?

A) There is only one best approach to scaling and only one best type of scale.
B) Ratio scaling leads to the least scoring drift.
C) Ratio scaling was first developed in the Republic of Samoa.
D) None of these

Question 35

Sorting techniques can be employed to develop

A) nominal scales.
B) ordinal scales.
C) interval scales.
D) All of these

Question 36

Item analysis is conducted to evaluate

A) item reliability.
B) item validity.
C) item difficulty.
D) All of these

Question 37

Scoring drift refers to

A) the tendency of scorers to give higher scores to testtakers with certain characteristics (such as age and gender) that is similar to themselves.
B) differences between the typical scoring of an item during standardization and subsequent, more authoritative scoring of an item.
C) a gradual decline in inter-scorer reliability after 95% of the examinations have been scored due to scorer fatigue.
D) a flexible method of scoring test items for populations other than that of the standardization sample.

Question 38

A well-written true-false item

A) includes multiple ideas.
B) has a correct response that is either true or false, and not subject to debate.
C) typically contains irrelevant information as a distracter.
D) Both includes multiple ideas and has a correct response that is either true or false, and not subject to debate.

Question 39

Ideally,the first draft of a test should include at least how many items as compared with the final version of the test?

A) about twice the number of the final version
B) about half the number of the final version
C) about three times the number of the final version
D) roughly the same number as the final version

Question 40

An anchor protocol is

A) a previously developed test with known validity that can be used as a comparison for newly developed tests.
B) a statistical procedure in which weights are assigned to each item of a model test to maximize predictive validity.
C) a list of guidelines for a standardized test used to ensure that all testtakers are similar in key ways to the population of the original standardization sample.
D) a model for scoring and a mechanism for resolving scoring discrepancies.

Question 41

As a distribution of scores gets flatter,what happens to the optimal boundary line for determining higher- and lower-scoring groups for item-discrimination indices?

A) the optimal boundary line gets smaller
B) the optimal boundary line gets larger
C) the optimal boundary line does not change
D) the optimal boundary line ceases to be optimal

Question 42

Which statement best describes the relationship between item difficulty and a "good" item?

A) The difficulty level is not a factor in determining a "good" item.
B) An item with a high difficulty level is likely to be "good."
C) An item with a mid-range difficulty level is likely to be "good."
D) An item with a low difficulty level is likely to be "good."

Question 43

An item-characteristic curve includes all of the following EXCEPT

A) information that can be used to judge item bias.
B) information that can be used to judge item fairness.
C) item-discrimination information.
D) item-difficulty information.

Question 44

What is the value of the item-discrimination index for an item that all the students in the higher-scoring group answered correctly but that no one in the lower-scoring group answered correctly?

A) -1
B) +1
C) .50
D) .25

Question 45

An item-difficulty index can range from

A) 0 to 1.
B) .10 to .99.
C) .25 to .75.
D) 0 to 100.

Question 46

The higher the item-difficulty index,the ________ the item.

A) easier
B) harder
C) more robust
D) less robust

Question 47

An item-reliability index provides a measure of a test's

A) test-retest reliability.
B) internal consistency.
C) stability.
D) All of these

Question 48

Item-discrimination indexes can range from

A) .001 to 1.00.
B) -1 to +1.
C) 0% to 100%.
D) 1 to 100.

Question 49

A negative item-discrimination index results for a particular item when

A) more high scorers than low scorers on a test get the item correct.
B) more low scorers than high scorers on a test get the item correct.
C) an item is found to be biased and unfair.
D) most testtakers do not enter the response keyed correct for the particular item.

Question 50

What is the optimal item-difficulty level for a true-false item?

A) .500
B) .625
C) .755
D) 1.000

Question 51

An item-endorsement index is most likely to be used in which type of test?

A) a cognitive test
B) an achievement test
C) a vocational aptitude test
D) a personality test

Question 52

An item-discrimination index typically compares

A) high scorers' performances with low scorers' performances on a particular item.
B) medium scorers' performances with low and high scorers' performances on a particular item.
C) low scorers' performances with lower scorers' performances on a particular item.
D) one group of scorers' performances on the item with any other groups of scorers' performances on the same item.

Question 53

In item analysis,the term item endorsement refers to the percent of testtakers who

A) responded correctly to a particular item.
B) indicate that they agree with a particular item.
C) passed the item on a pass/fail test of ability.
D) consented to answer an optional item.

Question 54

An item-difficulty index of 1 occurs when

A) all examinees answer the item incorrectly.
B) all examinees answer the item correctly.
C) examinees are evenly divided between correct and incorrect responses.
D) None of these

Question 55

It is needed to calculate the item-validity index.It is

A) the point-biserial correlation between the item score and the criterion score.
B) the mean of the item-score distribution.
C) the item-score standard deviation.
D) All of these

Question 56

To calculate an item-reliability index,one must have previously calculated

A) the correlation between the item score and the criterion.
B) the correlation between the item score and the total score.
C) the item-score standard deviation.
D) All of these

Question 57

What is the value of the item-discrimination index for an item answered correctly by an equal number of students in the higher- and lower-scoring groups?

A) -1
B) +1
C) .50
D) 0

Question 58

The item-validity index is key in determining

A) construct validity.
B) criterion-related validity.
C) content validity.
D) All of these

Question 59

Which statement is TRUE regarding an item-discrimination index?

A) It has been used by e-Harmony.com and other dating sites for matchmaking.
B) There is more than one formula for calculating an item-discrimination index.
C) Tetrachoric correlation is most frequently used in any formula for an item-discrimination index.
D) All of these.

Question 60

The greater the magnitude of the item-discrimination index,the more testtakers in the higher-scoring group answered the item correctly,as compared to testtakers

A) who served as the non-test-taking control group.
B) in the lower-scoring group.
C) who participated in the test standardization.
D) None of these

Question 61

Co-validation is:

A) highly recommended and encouraged by test professionals.
B) also referred to as co-norming.
C) a strategy that can save time and money for the test publisher.
D) Both also referred to as co-norming and a strategy that can save time and money for the test publisher.

Question 62

A student makes the following complaint after taking an exam: "I spent all night studying Chapter 7 and there wasn't even one test question from that chapter!" From a psychometric perspective,this student is concerned about the exam's

A) error variance.
B) test-retest reliability.
C) rater error.
D) None of these

Question 63

In general,what can be said about an item analysis of a speeded test?

A) Results are often misleading and difficult to interpret.
B) Item-difficulty levels are higher toward the end of the test.
C) Item-discrimination levels are higher for later items.
D) All of these

Question 64

Which is TRUE with regard to latent-trait models?

A) The latent trait is multidimensional.
B) The latent trait is unidimensional.
C) The latent trait cannot be measured by traditional models.
D) The latent trait surfaces before age 12.

Question 65

A student raises concern that a professor has given different grades to two essay answers that are very similar.From a psychometric perspective,the student is expressing concerns about

A) criterion-related validity.
B) rater error.
C) test-retest reliability.
D) parallel forms reliability.

Question 66

All of the following are methods of evaluating item bias EXCEPT

A) noting differences between the item-characteristic curves.
B) noting differences in the item-difficulty levels.
C) noting differences in item-discrimination indexes.
D) noting differences in validity shrinkage.

Question 67

Ideally,psychological or educational tests are revised

A) every decade.
B) when the test is no longer useful.
C) as a function of annual test sales.
D) None of these

Question 68

The best type of item yields an item-characteristic curve that

A) has a positive slope.
B) has a negative slope.
C) is leptokurtic.
D) has few, if any, outliers.

Question 69

Which of the following conditions may lead to the decision to revise a psychological or educational test?

A) item content, including the vocabulary used in instructions and pictures, has become dated
B) test norms no longer represent the population for which the test is designed
C) reliability and validity of a test can be improved by a revision
D) All of these

Question 70

Which is TRUE of item-characteristic curves?

A) They determine which items are fair.
B) They may be used as an aid in assessing whether or not items are biased.
C) They determine which items are most reliable under specified conditions.
D) They may be used as an aid in determining the kurtosis of a distribution of test scores.

Question 71

With regard to the test revision process,it typically

A) takes about one year to complete.
B) includes all of the steps that the initial test development included.
C) is much less expensive than the original development of a test.
D) All of these

Question 72

A test manual for a commercially prepared test should ideally include

A) a description of the test development procedures used.
B) test-retest reliability data.
C) internal-consistency reliability data.
D) All of these

Question 73

As part of the test development process,a test revision may entail

A) re-wording, deletion, or development of new items.
B) development of a new edition of a test.
C) the reprinting of a test.
D) Both re-wording, deletion, or development of new items and development of a new edition of a test.

Question 74

Ability tests are typically standardized on a sample that is representative of the general population and selected on the basis of variables such as

A) age.
B) gender.
C) geographic region.
D) All of these

Question 75

The term used to describe the decrease in item validities that typically occurs during cross-validation is

A) validity detriment.
B) validity decrement.
C) validity shrinkage.
D) cross-validation devaluation.

Question 76

Which is TRUE of cross-validation of a test after standardization has occurred?

A) Cross-validation creates confusion regarding the meaning of the original standardization data.
B) The cross-validation sample is composed of the same testtakers that participated in the original test standardization.
C) Cross-validation often results in validity shrinkage.
D) All of these

Question 77

Generous time limits are typically associated with

A) speeded conditions.
B) power conditions.
C) untimed conditions.
D) hazardous conditions.

Question 78

A student complains that a midterm examination did not include items from a particular in-class lecture.From a psychometric perspective,the students is expressing concern about the midterm's

A) test-retest reliability.
B) internal consistency reliability.
C) content validity.
D) cross-validation.

Question 79

Which statement is TRUE of guessing?

A) It occurs more often on achievement than personality tests.
B) It posts methodological problems for the testtaker.
C) Most testtakers guess based on little knowledge of the subject matter.
D) It poses methodological problems for the test developer.

Question 80

During the norming of a new intelligence test,a test publisher administers to all of the testtakers not only the new intelligence test,but a vision test using an eye chart.The publisher has engaged in

A) test conceptualization.
B) cross-validation.
C) shared validation.
D) None of these