Question 1

If an effect is significant but the effect size for the difference between the two means is medium (according to Cohen's conventions), about how much overlap will there be between the two distributions?

Accepted Answer

According to Cohen's conventions, a medium effect size corresponds to a Cohen's d of approximately 0.5. This means that the means of the two groups will differ by about half a standard deviation, which results in approximately 67 percent overlap between the two distributions.

Question 2

If a researcher performs a meta-analysis and finds that the mean d = 0.45, and the 95 percent confidence interval around this mean is (0.34, 0.56), what can the researcher conclude?

Accepted Answer

The best choice is D because the meta-analysis found a medium effect size of d=0.45, and the 95% confidence interval around this mean does not include zero. This means there is a statistically significant effect and it is medium in size. Therefore, we can conclude that averaging across all of the literature, there is a medium effect, and this effect is statistically significant. Choice A is incorrect because the 95% CI does not imply that all future effect sizes will fall within this range. Choice B is incorrect because the mean effect size is d=0.45, which is not zero. Choice C is incorrect because we know the direction of the effect is positive (since the CI only includes positive values).

Question 3

A high degree of overlap between two distributions of approximately 95 percent is likely to result in a(n) _____ effect size.

Accepted Answer

When there is a high degree of overlap between two distributions, it indicates that the difference between them is small. Therefore, the effect size is likely to be small.

Question 4

An overlap between two distributions of approximately 39 percent is likely to result in a(n) _____ effect size.

Accepted Answer

The answer of An overlap between two distributions of approximately...

Question 5

Meta-analysis involves:

Accepted Answer

Meta-analysis involves finding all studies published on a topic, calculating the effect size for each of those studies, and averaging the effect sizes together to find the average size of the effect across all studies. This allows for a more comprehensive understanding of the overall effect of a particular intervention or factor. Option A is not always possible as authors may not have data available or may not respond to requests for data. Option C is not practical as it would require access to every possible study on the topic, which is unlikely. Option D is not typical for meta-analysis as it involves recreating experimental conditions rather than analyzing existing data from multiple sources.

Question 6

The minimum acceptable level of estimated power for a study is:

Accepted Answer

A minimum acceptable level of power for a study is typically considered to be 0.80, meaning that there is an 80% chance of detecting a true effect if it exists. A power level of 0.95 would be ideal, but it may not always be feasible or necessary depending on the study design and research question. Power levels below 0.80 are generally considered too low to reliably detect meaningful effects. Choice A and C are incorrect as they do not reflect the typical minimum acceptable level of power. Choice D is incorrect as 0.10 is an extremely low power level that would be unlikely to provide meaningful results.

Question 7

Statistical power is calculated as 0.93.This means that if the null hypothesis is _____, there is a _____ percent chance of rejecting the null hypothesis.

Accepted Answer

Statistical power is the probability of rejecting the null hypothesis when the alternative hypothesis is true. Therefore, if the null hypothesis is false, and the statistical power is 0.93, there is a 93 percent chance of rejecting the null hypothesis.

Question 8

One of the roles of the researcher performing a meta-analysis is to:

Accepted Answer

One of the key roles of the researcher performing a meta-analysis is to determine the criteria for the inclusion of studies in the analysis. This involves defining the study population, interventions, comparators, outcomes, and study designs that qualify for inclusion. The inclusion criteria should be clearly and transparently defined, and should be based on the research question and the available evidence. This ensures that the meta-analysis is focused and meaningful, and that the results are robust and generalizable.

Question 9

If a researcher performs a meta-analysis and finds that the mean d = 0.11, and that the 95 percent confidence interval around this mean is (-0.04, 0.26), what could the researcher conclude?

Accepted Answer

The confidence interval includes zero (-0.04, 0.26), suggesting that the effect might not be statistically significant and that there is a possibility of no real effect across the studies analyzed.

Question 10

When Cohen's d is large (based on Cohen's conventions), the amount of overlap between the two distributions being compared is _____ percent.

Accepted Answer

The answer of When Cohen's d is large (based on...

Question 11

Imagine that a study of memory and aging finds that younger participants correctly recall 55 percent of studied words, older participants correctly recall 42 percent of studied words, and the size of this effect is Cohen's d = 0.49.According to Cohen's conventions for interpreting d, this effect is:

Accepted Answer

Cohen's conventions define a d of 0.2 as a small effect, a d of 0.5 as a medium effect, and a d of 0.8 or above as a large effect. Therefore, a d of 0.49 is considered a medium effect.

Question 12

Following a meta-analysis, the researcher might decide to perform a(n) _____ to determine the number of null results that would have to exist to overturn any statistically significant effect found in the meta-analysis.

Accepted Answer

A file drawer analysis would be the appropriate choice to determine the number of null results that would have to exist to overturn any statistically significant effect found in the meta-analysis. This type of analysis involves examining the possibility of publication bias, where studies with null results may not have been published or included in the meta-analysis, potentially skewing the overall effect size. By assessing the number of unpublished studies that would need to exist to diminish the observed effect size, the researcher can determine if the results of the meta-analysis are robust or if publication bias may be a concern.

Question 13

If there is less than a(n) _____ percent chance of rejecting the null hypothesis when it is false, there is insufficient power.

Accepted Answer

A power of 80% or higher is generally considered acceptable in hypothesis testing. This means that there is at least an 80% chance of correctly rejecting the null hypothesis when it is false. Any lower than 80% and there is insufficient power, meaning that the results may not be reliable or valid.

Question 14

An overlap between two distributions of approximately 99 percent is likely to result in a(n) _____ effect size.

Accepted Answer

An overlap of approximately 99 percent between two distributions indicates that there is very little difference between the two groups. Therefore, the effect size would be considered small.

Question 15

Statistical power is a measure of the ability to reject the null hypothesis when:

Accepted Answer

Statistical power measures the ability to reject the null hypothesis when it is false, which is the scenario where we want to have high power.

Question 16

An overlap between two distributions of approximately 55 percent is likely to result in a(n) _____ effect size.

Accepted Answer

The answer of An overlap between two distributions of approximately...

Question 17

The ability to reject the null hypothesis given that the null hypothesis is false is:

Accepted Answer

Statistical power refers to the ability to detect a significant difference or effect when one truly exists, meaning the null hypothesis is false. Therefore, statistical power is the ability to reject the null hypothesis in these situations. A Type II error occurs when the null hypothesis is false but is not rejected, a false alarm refers to a Type I error where the null hypothesis is wrongly rejected when it is actually true.

Question 18

A degree of overlap between two distributions of approximately 50 percent is likely to result in a(n) _____ effect size.

Accepted Answer

The answer of A degree of overlap between two distributions...

Question 19

If an effect is significant but the effect size for the difference between the two means is small (according to Cohen's conventions), about how much overlap will there be between the two distributions?

Accepted Answer

When the effect size for the difference between two means is small according to Cohen's conventions, there is still a considerable amount of overlap between the two distributions, approximately 85 percent. This indicates that despite the statistical significance, the practical difference between the groups might be minimal.

Question 20

The statistical convention for the minimal acceptable power is:

Accepted Answer

The minimal acceptable power in statistical tests is conventionally set at 0.80. This means there is an 80% chance of correctly rejecting the null hypothesis when it is false.

Quiz 8: Confidence Intervals, Effect Size, and Statistical Power

If an effect is significant but the effect size for the difference between the two means is medium (according to Cohen's conventions) , about how much overlap will there be between the two distributions?

If a researcher performs a meta-analysis and finds that the mean d = 0.45, and the 95 percent confidence interval around this mean is (0.34, 0.56) , what can the researcher conclude?