Exhibit 9-8: The following data were obtained when two forms of a criterion-referenced test were given to a group of 10 students.There were 40 items on each form and a student had to receive75 percent correct on each form.
$ \begin{array}{lll}\text { Examinee } & \text { Form 1 } & \text { Form 2 } \ 1 & 30 & 30 \ 2 & 28 & 29 \ 3 & 34 & 37 \ 4 & 38 & 39 \ 5 & 32 & 40 \ 6 & 26 & 28 \ 7 & 28 & 35 \ 8 & 35 & 36 \ 9 & 38 & 29 \ 10 & 36 & 36\end{array} $
-Refer to Exhibit 9-8.What would the score reliability of the test be if we take chance agreement into consideration (Kappa)?
A) .50
B) .52
C) .58
D) .60
E) .72

Question

Exhibit 9-8: The following data were obtained when two forms of a criterion-referenced test were given to a group of 10 students.There were 40 items on each form and a student had to receive75 percent correct on each form.
 $ \begin{array}{lll}\text { Examinee } & \text { Form 1 } & \text { Form 2 } \ 1 & 30 & 30 \ 2 & 28 & 29 \ 3 & 34 & 37 \ 4 & 38 & 39 \ 5 & 32 & 40 \ 6 & 26 & 28 \ 7 & 28 & 35 \ 8 & 35 & 36 \ 9 & 38 & 29 \ 10 & 36 & 36\end{array} $
-Refer to Exhibit 9-8.What would the score reliability of the test be if we take chance agreement into consideration (Kappa)?
A) .50
B) .52
C) .58
D) .60
E) .72

Quizplus · Accepted Answer

To calculate Kappa, we first need to create a contingency table that shows the number of agreements and disagreements between the two forms of the test. In this case, the contingency table is:

| Form 1 | Form 2 | Total
-------+-------+-------+-------
Agree  |   5   |   2   |   7
-------+-------+-------+-------
Disagree|   1   |   2   |   3
-------+-------+-------+-------
Total  |   6   |   4   |  10

Using this table, we can calculate Kappa:

Observed Agreement = (5 + 2) / 10 = 0.7
Expected Agreement = [(7/10) * (6/10)] + [(3/10) * (4/10)] = 0.52
Kappa = (Observed Agreement - Expected Agreement) / (1 - Expected Agreement) = (0.7 - 0.52) / (1 - 0.52) = 0.346
Therefore, the score reliability of the test would be 0.52, or choice B.

Exhibit 9-8: the Following Data Were Obtained When Two Forms