Question 1

A difference between means that is statistically significant is
A)due to error variance.
B)likely to be a Type II error.
C)a null finding.
D)unlikely to be due to error variance.

Accepted Answer

A statistically significant difference between means indicates that the observed difference is unlikely to be due to random chance or error variance, suggesting a real effect or difference in the population.

Question 2

Power refers to the degree to which a study is likely to
A)discover large effects.
B)detect effects of the independent variable.
C)be replicated.
D)produce Type I errors.

Accepted Answer

Power in statistical terms refers to the likelihood that a study will detect an effect if there is an actual effect of the independent variable to be detected. It does not directly relate to discovering large effects, replication, or producing Type I errors.

Question 3

Type II errors are more likely when
A)a Type I error occurs.
B)the results are statistically significant.
C)power is low.
D)the effect size is large.

Accepted Answer

Type II errors, also known as false negatives, are more likely to occur when the power of a study is low. Power is the probability that a test will correctly reject a false null hypothesis (i.e., detect a true effect). Low power means the study is less likely to detect an effect if there is one, increasing the risk of a Type II error.

Question 4

Why is it possible for condition means to differ at the end of an experiment even if the independent variable had no effect?
A)A Type II error has occurred.
B)Error variance has created differences between conditions.
C)A null finding was obtained.
D)The power of the study was low.

Accepted Answer

Error variance refers to the variability among scores caused by variables other than the independent variable. This can lead to differences between condition means even if the independent variable had no effect, as it introduces random fluctuations in the data.

Question 5

A power analysis is often used to determine
A)the number of participants needed for a study.
B)the likelihood of making a Type I error.
C)the strength of the independent variable.
D)whether to reject the null hypothesis.

Accepted Answer

A power analysis is primarily used to calculate the sample size required to detect an effect of a given size with a certain degree of confidence, thus determining the number of participants needed for a study.

Question 6

Failing to reject the null hypothesis when it is false is
A)confounding.
B)Type I error.
C)Type II error.
D)alpha.

Accepted Answer

Failing to reject the null hypothesis when it is actually false is known as a Type II error. This occurs when the test does not have enough power to detect an effect that is actually there.

Question 7

Researchers use null hypothesis significance testing to
A)understand why null findings were obtained.
B)test whether confounding has occurred.
C)test whether condition means differ more than they would if the difference were due to error variance.
D)separate the effects of the independent from the effects of participant variables.

Accepted Answer

Null hypothesis significance testing is used to determine if the observed differences between groups or conditions are statistically significant and not just due to random chance or error variance. This involves comparing the means of the groups to see if they differ more than would be expected by error alone.

Question 8

A Type I error can occur only when a researcher
A)accepts the null hypothesis.
B)rejects the null hypothesis.
C)fails to reject the null hypothesis.
D)rejects the experimental hypothesis.

Accepted Answer

A Type I error occurs when a researcher incorrectly rejects a true null hypothesis, meaning they find evidence for a difference or effect that does not actually exist.

Question 9

A Type I error occurs when a researcher
A)rejects a null hypothesis that is false.
B)fails to reject a null hypothesis that is false.
C)accepts a null hypothesis that is true.
D)rejects a null hypothesis that is true.

Accepted Answer

A Type I error occurs when a researcher incorrectly rejects a true null hypothesis, meaning they believe they have found a significant effect or difference when there is none.

Question 10

The probability that a researcher is willing to make a Type I error is
A)the alpha-level.
B)beta.
C)power.
D)the confidence interval.

Accepted Answer

The alpha-level (often denoted as α) is the threshold value that a researcher sets as the maximum probability they are willing to accept for making a Type I error, which is the error of rejecting a true null hypothesis.

Question 11

An effect size of .40 indicates that
A)the size of the difference between two means is .40 of a standard deviation.
B)the odds of a response in one condition is .40 of the odds in another condition.
C)40% of the variability in the dependent variable is due to the independent variable.
D)the answer depends on which effect size indicator is being used.

Accepted Answer

The interpretation of an effect size of .40 can vary depending on the specific effect size measure being used (e.g., Cohen's d, odds ratio, eta squared). Each measure has its own way of quantifying the magnitude of an effect, making the interpretation context-dependent.

Question 12

The null hypothesis states that
A)null findings will not be obtained.
B)there is no error variance in the data.
C)differences between conditions will be due only to the independent variable.
D)the independent variable did not have an effect.

Accepted Answer

The null hypothesis typically posits that there is no effect or difference as a result of the independent variable, meaning it suggests the independent variable did not have an effect.

Question 13

The proportion of the variance in one variable that can be accounted for by another variable is the
A)power.
B)statistical significance.
C)beta.
D)effect size.

Accepted Answer

The proportion of the variance in one variable that can be accounted for by another variable is referred to as the effect size. This concept is often used in statistical analysis to describe the strength of the relationship between two variables.

Question 14

Why is p-hacking a questionable research practice?
A)It increases Type I errors.
B)It results in excessive null findings.
C)It requires discarding data.
D)It reduces the power of a study.

Accepted Answer

P-hacking involves selectively reporting results that are statistically significant, which increases the likelihood of Type I errors, where a true null hypothesis is incorrectly rejected.

Question 15

Null hypothesis significance testing has been criticized because
A)no hypothesis is ever truly false.
B)the conclusion that a result is significant is based on an arbitrary criterion.
C)even when an effect is significant, Type I error is always possible.
D)the lack of significant results causes studies to fail.

Accepted Answer

The criticism is that the significance level (often p < 0.05) is an arbitrary threshold, which can lead to misinterpretation of results.

Question 16

The power of an experiment was .87. This indicates that
A)there is an 87% chance of making a Type I error.
B)87% of the participants in one condition will score higher than the average participant in the other condition.
C)beta is .87.
D)the experiment has an 87% chance of detecting any effect of the independent variable that might occur.

Accepted Answer

The power of an experiment, which is 1 - beta (β), represents the probability of correctly rejecting the null hypothesis when it is false, or in other words, the likelihood of detecting an effect if there is one. A power of .87 means there is an 87% chance of detecting an effect of the independent variable if it exists.

Question 17

Researchers reject the null hypothesis when
A)statistical analyses showed that the p-value is large.
B)the difference between condition means is larger than would be expected on the basis of error variance alone.
C)the effect size is smaller than .05.
D)they want to avoid a Type I error.

Accepted Answer

Researchers reject the null hypothesis when the observed data show a statistically significant difference, which is often indicated by the difference between condition means being larger than what would be expected by chance (error variance) alone. This is typically assessed through statistical tests that yield a p-value; if the p-value is small enough (usually < .05), it suggests that the observed data are unlikely under the null hypothesis, leading to its rejection. Options A, C, and D do not accurately describe conditions for rejecting the null hypothesis.

Question 18

If statistical analyses show that the difference between condition means is larger than would be expected on the basis of error variance alone, a researcher will
A)accept the null hypothesis.
B)reject the null hypothesis.
C)fail to reject the null hypothesis.
D)reject the experimental hypothesis.

Accepted Answer

When statistical analyses indicate that the difference between condition means is larger than what would be expected by error variance alone, it suggests that the observed effect is statistically significant. Therefore, the researcher will reject the null hypothesis, which typically posits that there is no effect or no difference.

Question 19

Cohen's d expresses the size of an effect in terms of
A)correlations.
B)standard deviations.
C)amount of variance accounted for.
D)the odds of a response.

Accepted Answer

Cohen's d is a measure of effect size that expresses the difference between two means in terms of standard deviations, not correlations, variance accounted for, or odds of a response.

Question 20

The alpha-level that researchers generally use is designed to ensure Type I errors occur on less than _____ of statistical decisions.
A).5%.
B)1%.
C)5%.
D)50%.

Accepted Answer

Researchers typically use an alpha level of 0.05, which corresponds to a 5% chance of making a Type I error, meaning they aim to ensure that Type I errors occur in less than 5% of statistical decisions.

Quiz 11: Analyzing Experimental Data: Basic Concepts

A difference between means that is statistically significant is

Power refers to the degree to which a study is likely to

Type II errors are more likely when

Why is it possible for condition means to differ at the end of an experiment even if the independent variable had no effect?

A power analysis is often used to determine

Failing to reject the null hypothesis when it is false is

Researchers use null hypothesis significance testing to

A Type I error can occur only when a researcher

A Type I error occurs when a researcher

The probability that a researcher is willing to make a Type I error is

An effect size of .40 indicates that

The null hypothesis states that

The proportion of the variance in one variable that can be accounted for by another variable is the

Why is p-hacking a questionable research practice?

Null hypothesis significance testing has been criticized because

The power of an experiment was .87. This indicates that

Researchers reject the null hypothesis when

If statistical analyses show that the difference between condition means is larger than would be expected on the basis of error variance alone, a researcher will

Cohen's d expresses the size of an effect in terms of

The alpha-level that researchers generally use is designed to ensure Type I errors occur on less than _____ of statistical decisions.