Question 1

When we measure the same thing on more than one occasion to see if we get the same score, which type of reliability are we testing?

Accepted Answer

Test-retest reliability is assessed by measuring the same individuals at two points in time and then correlating the two sets of measurements. This type of reliability checks for consistency over time.

Question 2

Kris has created a new scale to measure provider satisfaction in nursing homes. When the measure is completed, Kris asks her friend Dave to review it and subjectively assess if the content appears to be related to what she has intended to measure (provider satisfaction). What type of validity is Kris asking Dave to assess?

Accepted Answer

Kris is asking Dave to assess content validity, which refers to the extent to which a measure represents all facets of a given construct. In this case, Dave is evaluating whether the items on the scale appear to cover the concept of provider satisfaction adequately.

Question 3

Which threat to validity asks the question, "Are the results yours, or have multiple treatments by yourself and others combined to give the effect?"

Accepted Answer

This question refers to the threat of "interaction of different treatments," which concerns whether the observed effects are due to the treatment under investigation or a combination of multiple treatments.

Question 4

Which type of reliability testing is concerned with whether or not a person's score on a new measure can be used to predict future scores on a different measure?

Accepted Answer

Predictive reliability refers to the extent to which a measure can predict future outcomes or scores on different measures. This type of reliability is concerned with the predictive power of a test, meaning how well a score obtained at one time can forecast an outcome at a future time.

Question 5

John has created a scale that measures "extraterrestrial communication competence." John is torn between which type of validity testing he should employ. Which approach to validity should John utilize?

Accepted Answer

John should employ all approaches to validity that can be employed because each type of validity testing offers unique insights into the scale's effectiveness and applicability in different contexts. Face validity ensures the scale appears to measure what it's supposed to, predictive validity checks if the scale can predict future outcomes related to extraterrestrial communication competence, concurrent validity compares the scale's results with those of an already established measure, and construct validity examines whether the scale truly measures the theoretical construct of extraterrestrial communication competence. Utilizing all these approaches provides a comprehensive validation of the scale.

Question 6

John then takes his Extraterrestrial Communication Competence Scale and has participants fill it out along with Weiner's Communication Competence Scale because he theorizes the two should be related. What type of validity is this?

Accepted Answer

John is assessing the validity of his scale by comparing it with another established scale at the same time, which is an example of concurrent validity. This method checks if two measures that should theoretically be related are indeed related when measured at the same time.

Question 7

John also conducts a series of interviews with abductees and determines their level of communication competence based on their interviews. He then has them fill out the Extraterrestrial Communication Competence Scale. What type of validity is this?

Accepted Answer

The answer of John also conducts a series of interviews...

Question 8

After initially validating his Extraterrestrial Communication Competence Scale, John realizes that he has not really determined what the scale was supposed to measure in the first place. Which validity threat is John facing?

Accepted Answer

John is facing a threat of inadequate preoperational explication of concepts because he has not clearly defined or determined what his scale is supposed to measure before validating it.

Question 9

Which type of single administration reliability test is most commonly used by social scientists?

Accepted Answer

Cronbach's alpha (α) is the most commonly used method for assessing the reliability of a scale or test in social sciences. It measures the internal consistency of a test or scale, indicating how well the items that make up the scale are positively correlated to one another.

Question 10

Zacary has just finished running Cronbach's alpha for his newly developed touch avoidance scale. During his reliability analysis, Zacary finds out that his scale's alpha reliability is .72. How would we characterize this reliability?

Accepted Answer

Cronbach's alpha of .72 is generally considered respectable, indicating that the scale has acceptable internal consistency for research purposes, though it's not in the excellent or good range.

Question 11

While filling out a scale on drug use, Charlie states that he has never done drugs even though he gets high three or four times daily. Which validity problem does this represent?

Accepted Answer

Charlie's response is an example of socially desirable bias responding, where individuals provide answers that they believe will be viewed favorably by others, rather than answers that are truthful or reflective of their actual behavior.

Question 12

Which validity problem describes a research participant who attempts to give the research responses indicative of the exact opposite of what he or she wants or to give completed random responses?

Accepted Answer

The "screw-you effect" describes a situation where research participants deliberately give responses that are contrary to what they actually believe or completely random, often out of spite or a desire to sabotage the research findings. This can significantly compromise the validity of the research results.

Question 13

Johan is testing a new measure of public speaking anxiety. The scale consists of 10 Likert-type items. To test for reliability, Johan compares the even and odd numbered questions. What type of reliability test is Johan completing?

Accepted Answer

Johan is conducting a split-half reliability test, which involves dividing the test into two parts (in this case, even and odd numbered questions) and comparing the scores to assess the consistency of the measure.

Question 14

Jenny is running a study examining people's perceptions of "fake news." Her newly created measure for "fake news susceptibility" had a Cronbach's alpha of .86. How would you characterize the alpha reliability of Jenny's measure?

Accepted Answer

A Cronbach's alpha of .86 is generally considered to indicate good reliability, as it is above the commonly accepted threshold of .70 for acceptable reliability.

Question 15

In his study, Joshua reports the following in his instrumentation section:
"The alpha reliability found for the Perceptions of Religious Communication in the current study was 0.75 (M = 32.53, SD = 7.9)."
Which of the following statements is TRUE?

Accepted Answer

Alpha reliability, or Cronbach's alpha, is a measure of internal consistency. A value of 0.75 is generally considered respectable or acceptable in social science research, indicating a good level of reliability for the scale being used.

Question 16

Which of the following is NOT a way to improve the alpha reliability of a measure?

Accepted Answer

Alpha reliability, or Cronbach's alpha, is a measure of the internal consistency of a test or scale, indicating how closely related a set of items are as a group. It is improved by strategies that enhance the homogeneity and clarity of the items, such as adding more items (A), avoiding ambiguous items (B), and writing clear instructions (C). Ensuring the concurrent validity (D) relates to how well a test correlates with a measure of the same construct taken at the same time, which does not directly affect the internal consistency or alpha reliability of a measure.

Question 17

The Personal Report of Communication Apprehension-24 (PRCA-24) is not a _________ measure of ethnocentrism.

Accepted Answer

The PRCA-24 is a measure of communication apprehension, not ethnocentrism, so it is not a valid measure for assessing ethnocentrism.

Question 18

Amy created a new measure of media bias. She first shows one of her colleagues the new measure to see if the measure appears valid. What type of validity is Amy ensuring?

Accepted Answer

Face or content validity refers to whether a test appears to measure what it's supposed to measure, based on the judgment of experts or knowledgeable individuals. By showing the measure to a colleague for their opinion, Amy is ensuring that the measure seems valid on the surface, which is what face validity is about.

Question 19

___________ validity occurs when a researcher has previously measured the criterion and then attempts to relate it to a newly developed measure at a later point.

Accepted Answer

Retrospective validity occurs when a researcher uses previously measured criteria to validate a new measure at a later time.

Question 20

A scale can be reliable but not valid.

Accepted Answer

A scale can consistently measure something (reliable) but might not accurately measure what it's supposed to measure (valid).

Quiz 8: Reliability and Validity