Question 1

Why is it important, when reading the results of a hypothesis test, to determine whether the test was one or two sided?
A)If a test is one sided, and details aren't reported, you could be misled into thinking there is no difference, when in fact there was one in the opposite direction than was hypothesized.
B)If the researchers planned to conduct a two-sided test, then later changed to a one-sided test, you should be very suspicious.
C)It gives you some insight as to what the researchers originally intended to investigate, and/or the direction in which they originally hypothesized the relationship to be.
D)All of the above.

Accepted Answer

It is important to determine whether a test was one or two sided because it affects the interpretation of the results. If a test is one sided and the details aren't reported, it could be misleading to assume there is no difference when there was one in the opposite direction than hypothesized. Additionally, understanding whether researchers planned to conduct a one or two sided test gives insight into their original hypothesis and the direction of the relationship they were investigating.

Question 2

When it comes to sample size, "results may be larger (more significant) than they appear" is a good way to describe one of the possible problems with a hypothesis test.Explain how this can happen.

Accepted Answer

The answer of When it comes to sample size, "results...

Question 3

The word 'significant' is often used to try to convince you that there is an important effect or relationship.Explain how this word can take on different meanings and how you need to be aware of that when consuming statistical information.

Accepted Answer

The answer of The word 'significant' is often used to...

Question 4

Which of the following statements is false?
A)A small enough sample size could miss even the strongest of relationships that exists in a population.
B)Small sample sizes make it harder to rule out the null hypothesis.
C)A test based on a small sample size has less statistical power than a test using a large sample size, even if the percentages, means, and other statistics found in the sample are the same in both situations.
D)None of the above.

Accepted Answer

All of the statements are true, therefore none of them are false.

Question 5

What is the danger of researchers only reporting on the tests that came out significant (versus reporting on all the tests they conducted)?
A)If nothing interesting is happening in the population, then 1 in 20 tests are expected to achieve statistical significance just by chance.
B)The researchers could have been 'data snooping.'
C)Results that are not statistically significant give valuable information to the public as well; not just those that are statistically significant.
D)All of the above.

Accepted Answer

If researchers only report on the tests that came out significant, they might have engaged in data snooping or may have missed reporting on relevant results that are not statistically significant. Additionally, reporting on all the tests conducted may provide important context and valuable information for the public, even if they did not produce statistically significant outcomes. It is important for researchers to be transparent in reporting and not selectively present only the significant results.

Question 6

Suppose a pilot study tells you that there is a great deal of natural variability in the population.How will this impact the design of your actual, full blown study?
A)It will cause you to take a larger sample than you may have originally planned on.
B)It will cause you to abandon the bigger study; you won't be able to detect any differences or relationships even if they did exist.
C)Since there is so much variability, you might as well use a smaller sample size, because smaller sample sizes have more variability as well.
D)The pilot study shouldn't impact your design at all; that would be 'data snooping.'

Accepted Answer

If there is a great deal of natural variability in the population, it is likely that a larger sample size will be needed in order to detect significant differences or relationships. This is because a larger sample size will provide a more representative sample of the population, which in turn will increase the accuracy of any conclusions drawn from the study.

Question 7

Suppose a researcher conducted a 2-sided hypothesis test for a population mean (significance level .05) and his test statistic was +1.84.(The significance level was .05.) He knows this test statistic is not large enough to reject the null hypothesis, because the test statistic was not greater than 1.96.Then he realizes that if he had done a one-sided test to begin with (alternative hypothesis ">" ) his results would have been statistically significant, because his test statistic was greater than 1.65.At this point, he decides to change to a one-sided hypothesis test, and reports his results as being statistically significant.Is this acceptable? Why or why not.

Accepted Answer

The answer of Suppose a researcher conducted a 2-sided hypothesis...

Question 8

If 100 independent hypothesis tests are conducted using a significance level of 0.10, and if all of the null hypotheses are true, what is the expected number of false positives (type I errors)?
A)0
B)5
C)10
D)There is not enough information to say.

Accepted Answer

With a significance level of 0.10, we expect to see 10% of tests resulting in false positives for true null hypotheses. Therefore, out of 100 tests, we can expect 10 false positives.

Question 9

Suppose you got a p-value with a small study that was less than .05.Had you gotten the same sample results with a larger study done in a similar way, the p-value from the larger study would have been _______________ (choose: less than, greater than, equal to) the p-value from the smaller study.

Accepted Answer

The answer of Suppose you got a p-value with a...

Question 10

The simplest and most conservative method for handling type 1 error probabilities in multiple comparisons is the _____________________ method.

Accepted Answer

The answer of The simplest and most conservative method for...

Question 11

The natural variability in the population can affect a test's ability to detect a difference or relationship in the population that actually does exist.Explain how this is taken into account in the formula for the test statistic for testing a population mean.

Accepted Answer

The answer of The natural variability in the population can...

Question 12

Choosing beforehand to conduct a two-sided alternative will make it __________to detect a real difference in one single direction (choose: harder, easier, just as hard or easy).

Accepted Answer

The answer of Choosing beforehand to conduct a two-sided alternative...

Question 13

When it comes to sample size, "results may be smaller (less significant) than they appear" is a good way to describe one of the possible problems with a hypothesis test.Explain how this can happen.

Accepted Answer

The answer of When it comes to sample size, "results...

Question 14

Imagine you wish to test 4 independent hypotheses so that the total type 1 error probability is 0.10.According to the Bonferroni method, each test should be conducted at what significance level?
A)0.01
B)0.025
C)0.05
D)0.10

Accepted Answer

According to the Bonferroni method, the significance level for each test should be divided by the number of independent hypotheses being tested in order to control the overall type 1 error probability. In this case, since there are 4 independent hypotheses and the total type 1 error probability should be 0.10, each test should be conducted at a significance level of 0.10/4 = 0.025. Therefore, the best choice is B.

Question 15

Which of the following conclusions should make you suspicious as an educated consumer of statistical information?
A)"Based on our sample results we know there is no relationship between these two variables in the population."
B)"We looked at all possible correlations between these 10 variables, and this was the only one that was significant, signifying its tremendous importance."
C)"There is a relationship between these two variables in the population.Our statistics proved it."
D)All of the above.

Accepted Answer

A) This conclusion is suspicious because it is impossible to know for certain that there is no relationship in the population based solely on sample results. There is always a chance that the sample may not be representative of the entire population, or there may be other factors or variables that were not considered.

B) This conclusion is suspicious because just because a correlation is significant does not necessarily mean that it is important. It is possible that the correlation is simply due to chance or other factors that were not considered. Additionally, just because no other significant correlations were found does not mean that they do not exist.

C) This conclusion is suspicious because while the statistics may suggest a relationship, it is important to remember that correlation does not necessarily imply causation. Other factors or variables may be influencing the relationship, and it is important to consider these before making any definitive conclusions.

Overall, it is important to approach statistical information with a critical eye and carefully evaluate the methods used and the conclusions drawn.

Question 16

Large samples make it easier to detect real relationships or differences in the population than small samples (assuming everything else is equal).Explain how this is taken into account in the formula for the test statistic for testing a population mean.

Accepted Answer

The answer of Large samples make it easier to detect...

Question 17

Suppose you examined a possible relationship between gender and opinion on an issue in a population using two different studies.The percentages found were exactly the same for both studies, but one study had a much larger sample size than the other.Which of the following is not a possible outcome in this scenario?
A)Both studies find the relationship to be statistically significant.
B)Neither study finds the relationship to be statistically significant.
C)The larger study finds a statistically significant relationship but the smaller study does not.
D)The smaller study finds a statistically significant relationship but the larger study does not.

Accepted Answer

If the percentages found are exactly the same in both studies, then the relationship between gender and opinion on the issue is the same in both studies. However, the larger sample size in one study may increase the power of the study to detect a statistically significant relationship. Therefore, it is possible for both studies to find the relationship to be statistically significant (choice A) or neither study to find the relationship to be statistically significant (choice B).

It is also possible for the larger study to find a statistically significant relationship but the smaller study does not (choice C), as the larger study may have enough power to detect the relationship while the smaller study may not.

The only scenario that is not possible is for the smaller study to find a statistically significant relationship but the larger study does not (choice D). If the relationship is present in the smaller study, it should also be present in the larger study as they both have the same percentages.

Question 18

Suppose researchers tell you that they did not find a statistically significant relationship between two variables, and the p-value was .84.Which of the following must be true?
A)The sample size must have been small.
B)There must not be a relationship in the population.
C)There must not have been a relationship observed in their sample.
D)None of the above.

Accepted Answer

A p-value of .84 simply means that the observed data are very consistent with the null hypothesis, indicating that any difference or relationship observed in the sample is likely due to chance. It does not necessarily imply anything about the sample size (A), nor does it definitively prove the absence of a relationship in the population (B). It also does not mean that no relationship was observed in the sample; a relationship might have been observed, but it was not statistically significant (C).

Quiz 24: Significance, Importance, and Undetected Differences

Why is it important, when reading the results of a hypothesis test, to determine whether the test was one or two sided?

When it comes to sample size, "results may be larger (more significant) than they appear" is a good way to describe one of the possible problems with a hypothesis test.Explain how this can happen.

The word 'significant' is often used to try to convince you that there is an important effect or relationship.Explain how this word can take on different meanings and how you need to be aware of that when consuming statistical information.

Which of the following statements is false?

What is the danger of researchers only reporting on the tests that came out significant (versus reporting on all the tests they conducted) ?

Suppose a pilot study tells you that there is a great deal of natural variability in the population.How will this impact the design of your actual, full blown study?

If 100 independent hypothesis tests are conducted using a significance level of 0.10, and if all of the null hypotheses are true, what is the expected number of false positives (type I errors) ?

Suppose you got a p-value with a small study that was less than .05.Had you gotten the same sample results with a larger study done in a similar way, the p-value from the larger study would have been _______________ (choose: less than, greater than, equal to) the p-value from the smaller study.

The simplest and most conservative method for handling type 1 error probabilities in multiple comparisons is the _____________________ method.

The natural variability in the population can affect a test's ability to detect a difference or relationship in the population that actually does exist.Explain how this is taken into account in the formula for the test statistic for testing a population mean.

Choosing beforehand to conduct a two-sided alternative will make it __________to detect a real difference in one single direction (choose: harder, easier, just as hard or easy).

When it comes to sample size, "results may be smaller (less significant) than they appear" is a good way to describe one of the possible problems with a hypothesis test.Explain how this can happen.

Imagine you wish to test 4 independent hypotheses so that the total type 1 error probability is 0.10.According to the Bonferroni method, each test should be conducted at what significance level?

Which of the following conclusions should make you suspicious as an educated consumer of statistical information?

Large samples make it easier to detect real relationships or differences in the population than small samples (assuming everything else is equal).Explain how this is taken into account in the formula for the test statistic for testing a population mean.

Suppose researchers tell you that they did not find a statistically significant relationship between two variables, and the p-value was .84.Which of the following must be true?