Deck 13: Interpreting the Results of Hypothesis Testing: Effect Size, Type I and Type II Errors, and Power

Question 1

On the basis of the results for a large random sample of males from his university, Professor Flagg reports: "The mean height for my sample fell significantly below 6 feet."
(a)Explain in precise statistical terms the meaning of Professor Flagg's statement.
(b)Does the statement indicate that the mean height for males at his university falls below 6 feet by an important amount (i.e., that he is teaching at a university of "shorties")
(c)What is one possible reason (other than very short students) for the "significance" of his results. Explain.

Answer 1

(a)?
tested and rejected; conclusion:
(b)?no; it indicates nothing about how far below?
(c)?a very large sample will result in a small
and rejection of
even for a small discrepancy between
and the hypothesized value of 6.

Question 2

A researcher plans to test

Her alternative hypothesis is

Complete the following sentences:
(a)A Type I error is possible only if the population mean is ____________.
(b)A Type II error is possible only if the population mean is ____________.

Answer 2

(a)50
(b)not 50.

Question 3

Mr. Jones tends to worry about drawing incorrect conclusions. He plans to select a sample of size 20 and to use the .01 level of significance in testing

against

(a)If he obtains a

which type of error would he be worried about? Why?
(b) If he obtains a

which type of error would he be worried about? Why?

Answer 3

(a) Type II;

and thus he retains

; for a decision to retain

, only Type II is possible
(b)Type I; here he rejects

; for a decision to reject, only Type I is possible.

Question 4

_X =20.0

n = 100

Question 5

In statistical work, a significant difference is one that is large enough

A) to be meaningful to the experimenter
B) that it would rarely be expected to occur by chance
C) that chance cannot affect it
D) that it leads to retention of the null hypothesis

Question 6

If sample size is quite small and the decision is to reject the null hypothesis,

A) the test should probably be repeated
B) the true difference is more likely to be substantial
C) the level of significance should be lowered
D) one should shift from a one-tailed test to a two-tailed test

Question 7

If sample size is very large and the decision is to reject the null hypothesis,

A) the test should probably be repeated
B) one may put special confidence in the correctness of the conclusion
C) the true difference may be inconsequential from a practical importance point of view
D) the level of significance should be raised

Question 8

If sample size is very small and the decision is to retain the null hypothesis,

A) there is greater likelihood that a real difference may have been overlooked
B) one may place special confidence in the correctness of the conclusion
C) if the conclusion is in error, the error is probably inconsequential
D) the level of significance should be raised

Question 9

A Type II error is committed when

A) a true null hypothesis is rejected
B) a false null hypothesis is retained
C) a true alternative hypothesis is retained
D)
is higher than it should be

Question 10

An investigator retains all null hypotheses; some are true and some are false. What is

A) 1
B) 1/2
C) 0
D) cannot be determined

Question 11

is

A)
B) the probability of a Type I error
C) the probability of overlooking a true difference
D) the probability of claiming a true difference when none exists

Question 12

The smaller the level of significance

A) the less the probability of a Type I error
B) the less the probability of a Type II error
C) the greater the probability of a Type I error
D) the less confident we may be in whatever statistical conclusion is reached

Question 13

When
is .05, the probability of a Type I error is

A) .05
B) .025
C) .95
D) not determinable from the information given

Question 14

When
is .05, the probability of a Type II error is

A) .05
B) .025
C) .95
D) not determinable from the information given

Question 15

When
is .05, the power of the test is

A) .05
B) .025
C) .95
D) not determinable from the information given

Question 16

When
is .05, the probability of a Type II error is

A) .05
B) .025
C) .95
D) not determinable from the information given

Question 17

When
is .05, the power of the test is

A) .05
B) .025
C) .95
D) not determinable from the information given

Question 18

A man rolls a six-sided die to determine whether or not to reject a hypothesis. If the die comes up "1," he rejects the hypothesis; otherwise he retains it. Some hypotheses are true, and some are false. What is
?

A) 1/6
B) 1/3
C) 5/6
D) impossible to determine from the information given

Question 19

The concept of power requires

A)
to be true
B)
to be false
C) to be false
D) (a) or (b) above

Question 20

The power of a test is most closely related to

A) the level of significance
B) procedures for estimation
C) the probability of committing a Type I error
D)

Question 21

If
is true,

A)
is zero
B) is zero
C) the power of the test is .95 if
D) the probability of a Type II error is

Question 22

If the value of
is not known

A)
cannot be established
B) a proper estimate of
cannot be made
C)
is indeterminate
D) it is not possible to decide whether a one-tailed test or a two-tailed test should be made

Question 23

If
is false,

A) the probability of a Type II error is zero
B) the probability of a Type II error is
C) the power of the test is zero
D) the power of the test is

Question 24

Under the conditions we plan to employ for our experiment, the power of our t test ( $\alpha$ = .01) is .16. If we were to repeat our experiment 30 times under these identical conditions, we should expect to retain
approximately ____________ times.

A) 3
B) 25
C) 32
D) 5

Question 25

Suppose that 18 out of 54 identical repetitions of an experiment comparing two methods of instruction lead to non-significant results. If method of instruction does, in fact, have an effect, our best estimate of power for this situation is

A) .67
B) .18
C) .33
D) .36

Question 26

The greater the difference between
and

the

A) less the power of the test
B) greater the power of the test
C) larger n must be to discover the difference
D) smaller $\sigma$ must be to discover the difference

Question 27

If
is selected to be .05 rather than .01,

A) the probability of committing a Type I error will be greater
B) the power of the test will be lower
C)
will be increased
D) there is greater likelihood of claiming that
is false when it is really true

Question 28

In general, reducing the risk of committing a Type I error

A) reduces the risk of committing a Type II error
B) reduces the power of the test
C) increases the power of the test
D) has no effect on any of the above

Question 29

In general, the larger the sample size, the

A) less the risk of a Type II error
B) greater the power of the test
C) more likely
will be rejected when it should be
D) more likely all of the above are to be true

Question 30

As compared to a two-tailed test, a one-tailed test

A) carries less risk of retaining
when it is false
B) requires a larger n
C) carries less risk of rejecting
when

is fixed
D) has a greater probability of a false conclusion, of whatever type

Question 31

We will more readily detect a difference between what exists and that which is hypothesized when

A)
is smaller
B)
is larger
C) a two-tailed test is employed
D) n is larger

Question 32

We will less readily detect a difference between that which has been hypothesized and that which is true if

A)
is high
B)
is low
C) n is low
D) any (or all) of the above is true

Question 33

The question most directly related to adequate power is

A) Should I use hypothesis testing or estimation?
B) Is my sample large enough?
C) Is sampling random or systematic?
D) Is the population standard deviation known or unknown?

Question 34

In general, the larger the sample size, the

A) lower the value of
B) lower the power of the test
C) greater the power of the test
D) more meaningful is a finding of a significance difference

Question 35

Sample size affects power because of its effect on

A) random sampling variation
B) d
C) sample standard deviations
D) distribution shapes

Question 36

The effect size d does not depend on which of the following

A)
B)
C)
D) it depends on all of the above

Question 37

The effect size d depends on which of the following?

A)
B) n
C) power
D)

Question 38

In practice we can never determine the actual power value for a given experimental situation, because

A) the power curves are only hypothetical
B)
is seldom exactly true
C) the assumptions underlying the test are rarely fulfilled
D) effect size is unknown

Question 39

Consideration of which of the following factors illustrates most clearly the distinction between significance and importance

A) n
B) level of significance
C) Type I error
D) p-value

Question 40

In a statistical study, importance becomes a matter of

A) statistical significance
B) judgment and values
C) rejection of
D) effect size

Question 41

The ability to ascertain appropriate size of sample(s) required to test a particular hypothesis depends on the specification of

A)
B)
C) the size of difference that would be considered to be important
D) all of the above

Question 42

A Type II error may well be desirable when

A) power is very large
B) d is very small
C) the probability of a Type II error is very large
D) is very small