Question 1

Which of the following would not be considered an example of best practice for gathering evidence about the validity of a test?

Accepted Answer

Merely looking over the assessment questions to determine if the test is valid is not a sufficient or comprehensive method for gathering evidence about the validity of a test. Other methods such as evaluating validation evidence, questioning examinees, and analyzing the internal structure of the instrument are more effective in assessing test validity.

Question 2

If an instrument measures consistently but does not measure what it was designed to measure, the instrument is:

Accepted Answer

If an instrument measures consistently but does not measure what it was designed to measure, it is considered reliable (consistent) but not valid (accurate). Therefore, choice B is the correct answer.

Question 3

In using an assessment with subscales, a counselor should examine:

Accepted Answer

It is important for a counselor to examine the validation evidence regarding the internal structure of the instrument with populations that correspond to the counselor's clients to ensure that the subscales are valid and reliable for use with their specific clients. The standard error of estimate and expectancy tables may also be useful to examine, but they should not be the only considerations. The number of items per subscale is not necessarily indicative of best practice in assessment.

Question 4

The difference between concurrent and predictive validity is whether the:

Accepted Answer

Concurrent validity is the extent to which a measure is related to a criterion outcome measured at the same time, while predictive validity is the extent to which a measure predicts a future criterion outcome. Therefore, the main difference between concurrent and predictive validity is the timing of the prediction.

Question 5

A multitrait-multimethod matrix involves:

Accepted Answer

The answer of A multitrait-multimethod matrix involves:
...

Question 6

A high correlation between a self-report inventory that measures depression and a depression measure completed by the therapist is an example of:

Accepted Answer

A high correlation between two measures that are supposed to measure the same construct (depression) is a form of convergent evidence, indicating that the measures are assessing the same thing.

Question 7

Validation evidence based on instrument content is related to the concept of:

Accepted Answer

Validation evidence based on instrument content is related to the concept of content validity. Content validity refers to the extent to which an assessment tool measures the relevant content or domain adequately. It is assessed based on the expert judgment and agreement of the content domain that the tool intends to measure. Therefore, the evidence from the expert review of the instrument's content provides evidence of content validity. Neither criterion-related validity nor reliability are related to instrument content in the same way as content validity.

Question 8

A low, non-significant correlation between an observation-based assessment of symptoms of autism and a teacher-report of ADHD symptoms provides ______________________ of the validity of the autism measure.

Accepted Answer

Discriminant evidence suggests that the measure is distinct from other constructs, implying that it is a valid construct. A low correlation between measures of autism and ADHD symptoms suggests that they are distinct constructs and supports the validity of the autism measure. Convergent evidence refers to the correlation between measures of the same construct, multimethod evidence refers to the use of multiple methods to assess the same construct, and differential evidence refers to the ability of the measure to discriminate between different groups. None of these apply in this scenario where the focus is on assessing the validity of the autism measure.

Question 9

It is necessary for all good tests and instruments to:

Accepted Answer

Reliability and validity are crucial qualities for tests and instruments to have in order to ensure accurate and consistent results. National norms, a normalized distribution of scores, and a review in the Mental Measurements Yearbook may also be helpful, but they are not as fundamental as reliability and validity.

Question 10

A statistical technique often used to examine internal structure of an instrument is:

Accepted Answer

Factor analysis is a statistical technique commonly used to examine the internal structure of an instrument, such as a questionnaire or survey, by identifying underlying factors or dimensions that explain the patterns of responses. It is a powerful tool for analyzing data and can help researchers better understand the structure and validity of their measures. Meta-analysis, on the other hand, is a statistical technique used to combine the results of multiple studies to draw conclusions about a particular research question. Internal analysis and structural analysis are not commonly used terms in the context of instrument development or validation.

Question 11

In examining validation evidence, Messick suggests that clinicians should consider the meaning of the score and the :

Accepted Answer

Messick emphasizes the importance of considering the social value or consequence of using an instrument as a part of validation evidence. While the other options are also important factors in validation evidence, they do not directly address the concept of social value or consequence.

Question 12

An instructor announces that an examination will cover the topics of counseling theories and techniques.You spend hours studying both concepts, but all of the test questions are only on techniques.You might correctly conclude that the examination:

Accepted Answer

Content validity refers to whether the questions on the test are representative of the entire content area being tested. In this case, since the test only covered techniques and not counseling theories, it lacked content validity. Criterion-related validity refers to whether the test is related to an external criterion, such as performance on the job. This is not applicable in this scenario. Reliability refers to the consistency or stability of measurements taken with a test, and is not relevant to the issue in this scenario.

Question 13

Test specifications:

Accepted Answer

Test specifications are a detailed plan that includes the organizational framework used for instrument development, identification of assessment goals, and the process of how objectives were developed. Therefore, all of the above options are correct.

Question 14

A validity coefficient is best illustrated by the correlation between:

Accepted Answer

A validity coefficient is the correlation between a test score and performance on a criterion measure. Therefore, the best choice to illustrate a validity coefficient is the correlation between test scores and performance on a criterion (option C). Option A describes the reliability coefficient, which is the correlation between test scores on two different occasions. Option B describes the correlation between scores on two different tests, which is not necessarily an indicator of validity. Option D is the item-total correlation, which is used to assess the quality of individual items on a test, but is not directly related to validity.

Question 15

If the validity evidence focuses on the degree to which the evidence indicates the items, questions, or tasks adequately represent the intended behavior domain, which traditional category of validity would it be most related to?

Accepted Answer

Content-related validity is concerned with how well the items, questions, or tasks represent the intended behavior domain, so it would be the most related traditional category of validity to the validity evidence described in the question.

Question 16

The most comprehensive type of validity that incorporates a gradual accumulation of evidence is __________ validity.

Accepted Answer

Construct validity is the type of validity that incorporates a gradual accumulation of evidence. It involves testing a hypothesis about the theoretical constructs that underlie observable behaviors or measures, and requires multiple sources of evidence to support its validity. This can include convergent validity (correlations between different measures of the same construct), discriminant validity (demonstrating that measures of different constructs are not highly correlated), and criterion-related validity (testing a measure against an external criterion). Concurrent, content, and predictive validity are all important types of validity, but do not necessarily involve a gradual accumulation of evidence.

Question 17

An example of gathering evidence based on response processes is:

Accepted Answer

Gathering evidence based on response processes involves examining how individuals are interpreting and responding to assessment items. Asking individuals to "think aloud" while completing an inventory allows researchers to gain insight into the individual's cognitive processes and understand how they are interpreting and responding to the items. This provides valuable information for evaluating the validity of the assessment.

Question 18

In evaluating an instrument's validity, practitioners should:

Accepted Answer

While high validity coefficients are important, practitioners should also consider factors such as the appropriateness of the instrument for the population being assessed, the purpose of the assessment, and the context in which the assessment will be used. In addition, practitioners should be convinced by the preponderance of evidence on the appropriate use of the instrument, which may involve reviewing research studies and other sources of information. The use of factor analysis and multitrait-multimethod matrix can be helpful in establishing validity, but they are not necessary for all instruments.

Question 19

In order to determine if an instrument can be used appropriately in a certain situation, the counselor must:

Accepted Answer

Examining the validation evidence is the best way to determine if an instrument can be used appropriately in a certain situation. Validation evidence includes determining if the instrument measures what it is intended to measure, if it is reliable, and if it has been validated for use in the specific population and setting. Reliability coefficients and validity coefficients are both important pieces of validation evidence, but they alone do not provide a comprehensive picture of the instrument's appropriateness for a specific situation. Consulting a clinician who has used the instrument may be helpful, but it should not be the only basis for determining appropriateness.

Question 20

Of the three historical types of validation evidence, which one is believed by many experts to be the most pertinent with assessments used by counselors?

Accepted Answer

Construct-related validation is often considered the most pertinent for assessments used by counselors because it focuses on whether the test measures the theoretical construct or trait it claims to measure, which is crucial in counseling settings where understanding underlying psychological constructs is key.

Quiz 4: Validity and Item Analysis