Question 1

Which one of the following formulas is used when two halves of one test is used to adjust the reliability coefficient?
A) Spearman Brown
B) coefficient alpha
C) Pearson product moment correlation
D) KR-20

Accepted Answer

The Spearman Brown formula is used when adjusting the reliability coefficient for situations where two halves of one test are used. It provides an estimate of the reliability coefficient for a full-length test based on the correlation coefficient between two halves of the test.

Question 2

Which one of the following is used to provide an index of the strength and direction of the linear relationship between the two sets of scores?
A) coefficient alpha
B) correlation
C) KR-20
D) Spearman Brown

Accepted Answer

Correlation is used to provide an index of the strength and direction of the linear relationship between two sets of scores. Coefficient alpha and KR-20 are measures of internal consistency reliability, while Spearman Brown is a correction formula used to estimate the reliability of a test after altering the length of the test.

Question 3

What is the unexplained difference between the true score (T) and the obtained score (X) called?
A) systematic error
B) random error
C) internal consistency
D) coefficient alpha

Accepted Answer

The unexplained difference between the true score and the obtained score is called random error. This is because it is unpredictable and can vary from test to test. Systematic error, on the other hand, is a consistent bias in the measurement process that affects all scores in a particular way. Internal consistency and coefficient alpha are both measures of reliability, but they do not refer specifically to the difference between true and obtained scores.

Question 4

The Spearman Brown formula is used when calculating ______.
A) only internal reliability
B) only split halves reliability
C) only scorer reliability
D) all forms of reliability

Accepted Answer

The Spearman Brown formula is specifically designed for split halves reliability, so the best choice is B. It is not used for other types of reliability (such as internal or scorer reliability) and thus D is not correct.

Question 5

Jonathan wanted to estimate the internal consistency of a multiple choice test. Which one of the following would be most appropriate for him to use?
A) Spearman Brown
B) coefficient alpha
C) Cohen's kappa
D) KR-20

Accepted Answer

The answer of Jonathan wanted to estimate the internal consistency...

Question 6

Judy wants to estimate the internal consistency of a survey on which the respondent marks (1) Not at all, (2) Sometimes, (3) Most of the time, or (4) Always. Which one of the following is most appropriate for her to use?
A) Spearman Brown
B) coefficient alpha
C) Cohen's kappa
D) KR-20

Accepted Answer

Coefficient alpha is the most appropriate choice for estimating internal consistency of a survey that includes items with Likert-type responses. This coefficient measures how well the items within the survey are related to each other and can indicate the reliability of the survey as a whole. Spearman Brown and KR-20 are alternative methods for estimating reliability, but do not take into account the specific characteristics of surveys with Likert-type responses. Cohen's kappa is used to measure inter-rater agreement and is not appropriate for this situation.

Question 7

Which one of the following statements is TRUE about error?
A) Systematic error increases the reliability of a test.
B) Systematic error lowers the reliability of a test.
C) Random error lowers the reliability of a test.
D) Both random and systematic error lower the reliability of a test.

Accepted Answer

Random error lowers the reliability of a test because it introduces unpredictability and inconsistency in the results, making them less repeatable.

Question 8

Theoretically, if a test taker took a test, an infinite number of times and an average score was calculated from those administrations, the average test score would ______.
A) equal the true test score
B) contain the true score and error
C) contain only error
D) have no meaning

Accepted Answer

The average score from an infinite number of test administrations would theoretically equal the true test score, as the random errors of measurement would cancel each other out over time, leaving only the true score.

Question 9

Cohen's kappa is a statistical method for ______.
A) estimating test-retest reliability
B) estimating interrater agreement
C) estimating internal consistency
D) correlating ratings by two judges

Accepted Answer

Cohen's kappa is specifically geared towards assessing the level of agreement between two raters or judges, making it a useful tool for estimating interrater agreement. It takes into consideration the agreement that would be expected by chance, making it a more robust measure than a simple percentage agreement calculation.

Question 10

Which one of the following statements regarding test reliability is TRUE?
A) Researchers must choose one method of estimating the reliability/precision of test scores.
B) Researchers can identify the causes of random error.
C) No measurement instrument is perfectly reliable or consistent.
D) A measurement instrument should be perfectly reliable or consistent.

Accepted Answer

No measurement instrument is perfectly reliable or consistent. Reliability is the degree to which a measurement instrument gives consistent results over time or across different raters. However, there will always be some degree of error or inconsistency, known as random error, that cannot be completely eliminated. It is important for researchers to estimate and report the reliability of their measures to determine the extent to which their results can be trusted. There are various methods for estimating reliability, and researchers should choose the most appropriate method given the nature of their instrument and design.

Question 11

What do we call the statistic that reflects the amount of inconsistency or error expected in an individual's test score?
A) reliability coefficient
B) standard deviation
C) standard error of measurement
D) correlation coefficient

Accepted Answer

The statistic that reflects the amount of inconsistency or error expected in an individual's test score is the standard error of measurement. This takes into account factors such as the reliability of the test and the individual's own variability in performance. The reliability coefficient (A) measures the consistency of scores on a test and is often used to estimate the standard error of measurement. The standard deviation (B) measures the variability of a set of scores, but does not specifically account for error or inconsistency in individual scores. The correlation coefficient (D) measures the strength and direction of a relationship between two variables, but is not directly related to the consistency or error in individual test scores.

Question 12

Julio wants to calculate the interrater agreement for an essay exam that is scored by giving each essay a pass (2) or fail (1) mark. Which one of the following would be most appropriate for Julio to use?
A) Cohen's kappa
B) coefficient alpha
C) KR-20
D) Correlation

Accepted Answer

Cohen's kappa is a statistical measure specifically designed to assess interrater agreement for categorical data, which is the case for the pass/fail marks given to the essays. Therefore, it is the most appropriate choice for Julio to use in this situation. Coefficient alpha and KR-20 are measures of internal consistency, and correlation is used to measure the strength of the relationship between two continuous variables.

Question 13

Scorer reliability and agreement concerns the consistency of what?
A) test scores
B) scorer judgments
C) different test forms
D) test takers' performance

Accepted Answer

Scorer reliability and agreement concerns the consistency of scorer judgments, specifically whether different scorers are evaluating test takers' responses consistently and accurately.

Question 14

What do we call a single source of error that always increases or decreases the true score by the same amount?
A) the true score
B) the average score
C) random error
D) systematic error

Accepted Answer

A single source of error that always increases or decreases the true score by the same amount is called systematic error. Option A (the true score) is not a source of error, but rather the ideal score we aim for. Option B (the average score) is not necessarily affected by a single source of error, but can be impacted by both random and systematic errors. Option C (random error) is not the correct answer as it refers to errors that occur due to chance and are not consistent in their impact on scores. Therefore, the correct answer is D (systematic error).

Question 15

Chris wants to calculate the interscorer reliability for an essay test that had two scorers. Which one of the following would be most appropriate for Chris to use?
A) Cohen's kappa
B) coefficient alpha
C) KR-20
D) Correlation

Accepted Answer

Correlation is appropriate for calculating interscorer reliability when there are two scorers, as it measures the degree of agreement between their scores.

Question 16

As the reliability/precision of a test decreases, which one of the following items increases?
A) standard deviation
B) standard error of measurement
C) internal consistency
D) difficulty

Accepted Answer

As the reliability or precision of a test decreases, the standard error of measurement increases. This is because the standard error of measurement is a function of the test's reliability; lower reliability leads to higher measurement error.

Question 17

The range of scores that we feel comfortable and includes the true score is called a ______.
A) standard deviation
B) standard error of measurement
C) confidence interval
D) normal curve

Accepted Answer

The range of scores that we feel comfortable and includes the true score is called a confidence interval. Option A, standard deviation, is a measure of the spread of scores around the mean and does not directly relate to the true score. Option B, standard error of measurement, is an estimate of the variability that can be expected in the scores due to measurement error and does not describe the range of scores that includes the true score. Option D, normal curve, is a mathematical model that describes the distribution of scores in a population, but it does not address the range of scores that includes the true score.

Question 18

Which one of the following is most helpful to test developers who wish to increase the reliability/precision of a test's scores?
A) Spearman Brown formula
B) coefficient alpha
C) correlation
D) Cohen's kappa

Accepted Answer

The Spearman Brown formula is used to estimate the reliability of a test when its length is changed, making it helpful for increasing the reliability/precision of test scores.

Question 19

If a meteorologist uses a thermometer that always reads 1 degree higher than the actual temperature, then the error that results is ______. If a meteorologist is nearsighted and he reads the thermometer with a different amount and direction of inaccuracy each time, the error that results will be ______.
A) reliable; unreliable
B) unreliable; reliable
C) random; systematic
D) systematic; random

Accepted Answer

If the thermometer always reads 1 degree higher than the actual temperature, it is a systematic error. However, if the meteorologist's nearsightedness causes different amounts and directions of inaccuracy each time, it is a random error. Therefore, the first statement corresponds to systematic error and the second statement corresponds to random error, making option D the correct answer.

Question 20

What is the amount of consistency among scorers' judgments called?
A) test-retest agreement
B) interscorer agreement
C) intrascorer agreement
D) internal agreement

Accepted Answer

The answer of What is the amount of consistency among...

Quiz 5: What Is Test Reliabilityprecision

Which one of the following formulas is used when two halves of one test is used to adjust the reliability coefficient?

Which one of the following is used to provide an index of the strength and direction of the linear relationship between the two sets of scores?

What is the unexplained difference between the true score (T) and the obtained score (X) called?

The Spearman Brown formula is used when calculating ______.

Jonathan wanted to estimate the internal consistency of a multiple choice test. Which one of the following would be most appropriate for him to use?

Judy wants to estimate the internal consistency of a survey on which the respondent marks (1) Not at all, (2) Sometimes, (3) Most of the time, or (4) Always. Which one of the following is most appropriate for her to use?

Which one of the following statements is TRUE about error?

Theoretically, if a test taker took a test, an infinite number of times and an average score was calculated from those administrations, the average test score would ______.

Cohen's kappa is a statistical method for ______.

Which one of the following statements regarding test reliability is TRUE?

What do we call the statistic that reflects the amount of inconsistency or error expected in an individual's test score?

Julio wants to calculate the interrater agreement for an essay exam that is scored by giving each essay a pass (2) or fail (1) mark. Which one of the following would be most appropriate for Julio to use?

Scorer reliability and agreement concerns the consistency of what?

What do we call a single source of error that always increases or decreases the true score by the same amount?

Chris wants to calculate the interscorer reliability for an essay test that had two scorers. Which one of the following would be most appropriate for Chris to use?

As the reliability/precision of a test decreases, which one of the following items increases?

The range of scores that we feel comfortable and includes the true score is called a ______.

Which one of the following is most helpful to test developers who wish to increase the reliability/precision of a test's scores?

If a meteorologist uses a thermometer that always reads 1 degree higher than the actual temperature, then the error that results is . If a meteorologist is nearsighted and he reads the thermometer with a different amount and direction of inaccuracy each time, the error that results will be .

What is the amount of consistency among scorers' judgments called?