Question 1

Evaluating the Experiment from the Inside: Internal Validity While several mistakes can threaten statistical conclusion validity, the two most common ones are
A)computing too many statistical tests and achieving insufficient statistical power.
B)confounding by extraneous variables and using too many subjects.
C)selecting too extreme a significance level and using the wrong test for experimental data.
D)using too many subjects and achieving insufficient statistical power.

Accepted Answer

Computing too many statistical tests can lead to an increased risk of Type I errors (false positives), while insufficient statistical power (due to a small sample size, for example) increases the risk of Type II errors (false negatives). Both of these issues directly impact the statistical conclusion validity of an experiment.

Question 2

Evaluating the Experiment from the Inside: Internal Validity ____ verifies how successfully the experimenter created values of the independent variable.
A)A manipulation check
B)Aggregation across subjects
C)Aggregation across trials
D)Pilot testing

Accepted Answer

A manipulation check is a method that evaluates if the experimenter successfully manipulated the independent variable. Therefore, it verifies the internal validity of the experiment. Aggregation across subjects, aggregation across trials, and pilot testing do not directly assess internal validity.

Question 3

Taking a Broader Perspective: The Problem of External Validity An experiment must be ____ to be externally valid.
A)internally valid
B)practically significant
C)replicable
D)internally valid and replicable

Accepted Answer

For an experiment to be externally valid, it must first be internally valid, meaning that the data is accurate and trustworthy within the context of the study. Additionally, the results must be replicable, meaning that other researchers are able to duplicate the study and obtain similar results. Therefore, the best choice is D, which includes both internal validity and replicability.

Question 4

Evaluating the Experiment from the Inside: Internal Validity Statistical tests permit us to
A)directly test the experimental hypothesis.
B)make probability statements.
C)prove the experimental hypothesis.
D)rule out the possibility of Type 1 error.

Accepted Answer

Statistical tests allow us to make probability statements about the likelihood that the results we observed could have occurred by chance alone. They do not directly test the experimental hypothesis, prove the hypothesis, or rule out the possibility of Type 1 error.

Question 5

Taking a Broader Perspective: The Problem of External Validity When researchers replicate statistically significant research findings, this increases our confidence that these findings were not produced by
A)demand characteristics.
B)experimenter bias.
C)Type 1 error.
D)Type 2 error.

Accepted Answer

Replicating statistically significant research findings helps to ensure that the results were not due to a Type 1 error, which occurs when researchers incorrectly reject a true null hypothesis, mistakenly concluding that there is a significant effect when there is not.

Question 6

Taking a Broader Perspective: The Problem of External Validity Amanda replicated a main effect of type of packaging on product recognition.This increases our confidence that her initial findings were not due to
A)demand characteristics.
B)the Rosenthal effect.
C)Type 1 error.
D)Type 2 error.

Accepted Answer

Replicating a main effect increases confidence that the initial findings were not due to Type 1 error, which is the incorrect rejection of a true null hypothesis (i.e., finding a statistically significant effect when there is none). Replication helps ensure the effect is not a statistical fluke.

Question 7

Evaluating the Experiment from the Inside: Internal Validity ____ evaluates whether a sad film really made the subjects feel sad.
A)A manipulation check
B)Aggregation across subjects
C)Aggregation across trials
D)Pilot testing

Accepted Answer

Internal validity refers to the extent to which a study can establish a cause-and-effect relationship between the independent and dependent variables. In this case, evaluating whether a sad film really made the subjects feel sad is a test of the manipulation (showing the sad film) of the independent variable. A manipulation check measures whether the manipulation of the independent variable was successful in producing the intended effect on the dependent variable, which is an important aspect of internal validity.

Question 8

Evaluating the Experiment from the Inside: Internal Validity An experiment's internal validity depends on whether
A)probability sampling was used.
B)researchers have correctly interpreted statistical tests.
C)the experiment is free of confounding.
D)the research findings can be extended to other samples and situations.

Accepted Answer

Internal validity refers to whether the experiment accurately captures the relationship between the independent and dependent variables, without the influence of other variables. Confounding variables, which are extraneous variables that may affect the dependent variable and that are not under experimental control, can distort the results and reduce internal validity. Therefore, it is crucial for an experiment to be free of confounding to achieve high internal validity. While probability sampling and the generalizability of findings are also important considerations in research design, they are not directly related to internal validity. The correct interpretation of statistical tests is important for drawing accurate conclusions from data, but it does not guarantee internal validity.

Question 9

Taking a Broader Perspective: The Problem of External Validity ____ means that we can accurately extend experimental results to other situations.
A)External validity
B)Internal validity
C)Practical significance
D)Research significance

Accepted Answer

External validity refers to the ability to generalize findings from a study to other situations or populations. It is important to consider external validity when interpreting research results and making decisions based on those findings. Internal validity, practical significance, and research significance are all important concepts in research but do not refer specifically to the ability to extend results to other situations.

Question 10

Evaluating the Experiment from the Inside: Internal Validity Jack just completed a study of the effects of caffeine and rave music on recall.He is excited about finding a significant interaction and says, "This is so cool! I've proved my research hypothesis.Jolt cola with rave music is the way to study." Jack's experiment cannot prove his research hypothesis because his results could be due to
A)confounding.
B)Type 1 error.
C)Type 2 error.
D)confounding and/or Type 1 error.

Accepted Answer

Jack's experiment cannot prove his research hypothesis because his results could be due to confounding and/or Type 1 error. Confounding occurs when an extraneous variable that was not controlled for affects the results. Type 1 error occurs when the null hypothesis is mistakenly rejected, indicating a significant effect when there actually is none. Therefore, Jack's excitement about proving his research hypothesis is premature and his results must be interpreted with caution.

Question 11

Evaluating the Experiment from the Inside: Internal Validity Which variable confounded the Ritchey (1982) study of the effects of age (third-grade, sixth-grade, and adult) and stimulus (words, outline drawings, and detailed drawings) on recall?
A)attention
B)detail
C)distinctiveness
D)visual memory

Accepted Answer

The variable that confounded the Ritchey (1982) study was distinctiveness. This was because the detailed drawings were more distinctive than the other stimuli, making it more likely for participants to recall them. Therefore, the results could not be attributed solely to the effects of age and stimulus on recall.

Question 12

Evaluating the Experiment from the Inside: Internal Validity Orne's (1969) "pact of ignorance" shows how validity issues can be hard to resolve because of
A)confounding.
B)demand characteristics.
C)experimenter bias.
D)reactive measurements.

Accepted Answer

Orne's "pact of ignorance" refers to a situation where participants agree not to reveal their suspicions or knowledge about the experimental hypothesis, in order to avoid interfering with the experimenter's expectations. This can create demand characteristic issues that affect the internal validity of the study. Confounding refers to a situation where the experimental design does not control for extraneous variables that can affect the dependent variable. Experimenter bias refers to situations where the experimenter's expectations influence the results of the study. Reactive measurements refer to situations where the measurement of the dependent variable influences the participant's behavior.

Question 13

Evaluating the Experiment from the Inside: Internal Validity When a researcher computes too many a priori comparisons, this threatens
A)external validity.
B)internal validity.
C)statistical conclusion validity.
D)Type 2 error.

Accepted Answer

Computing too many a priori comparisons increases the risk of finding a statistically significant difference purely by chance, which threatens statistical conclusion validity, not the other aspects of validity or error types mentioned.

Question 14

Evaluating the Experiment from the Inside: Internal Validity If your experimental results were barely significant even though you tested 400 subjects, you should be concerned about
A)committing a Type 2 error.
B)confounding by extraneous variables.
C)insufficient statistical power.
D)statistical conclusion validity.

Accepted Answer

Statistical conclusion validity is concerned with the proper statistical treatment of data and the correct interpretation of statistical outcomes. If experimental results are barely significant despite a large sample size (400 subjects), it suggests that the statistical analysis might be on the edge of detecting an effect, raising concerns about whether the statistical conclusions drawn are valid. This scenario does not directly imply issues with Type 2 errors (which would be more of a concern with non-significant results), confounding by extraneous variables (which affects internal validity but is not directly related to the statistical significance of the results), or insufficient statistical power (since a large sample size typically increases power).

Question 15

Evaluating the Experiment from the Inside: Internal Validity Andrew designed an experiment that controlled extraneous variables so that the effects on the dependent variable could only be attributed to the independent variable.Based on this description, his experiment achieved satisfactory
A)construct validity.
B)ecological validity.
C)external validity.
D)internal validity.

Accepted Answer

Internal validity refers to the extent to which a study's design and execution can rule out alternative explanations for the observed results, thus attributing the effects on the dependent variable to the independent variable. Therefore, by controlling extraneous variables, Andrew's study achieved internal validity. Construct validity refers to the extent to which the operationalization of constructs (i.e., variables) matches the theoretical concepts being studied. Ecological validity refers to the extent to which findings can be generalized to real-life settings. External validity refers to the extent to which findings can be generalized to other populations or settings. While these are all important types of validity, based on the description provided, the experiment achieved internal validity primarily.

Question 16

Evaluating the Experiment from the Inside: Internal Validity An experiment 's validity is threatened by ____ when subjects do not reveal all they know about the purpose of an experiment and researchers accept subject reports at face value.
A)a pact of ignorance
B)confounding
C)experimenter bias
D)history threat

Accepted Answer

A pact of ignorance occurs when subjects withhold information about their awareness of the experiment's purpose, and researchers mistakenly take their reports at face value, potentially compromising the experiment's internal validity.

Question 17

Evaluating the Experiment from the Inside: Internal Validity Experiments by Ritchey and Armstrong (1982) showed that subjects recalled ____ when the degree of ____ was controlled.
A)detailed drawings better than less elaborate drawings; attention
B)detailed drawings better than less elaborate drawings; distinctiveness
C)less elaborate drawings better than detailed drawings; attention
D)less elaborate drawings better than detailed drawings; distinctiveness

Accepted Answer

The study by Ritchey and Armstrong (1982) controlled for the degree of distinctiveness, not attention. The study found that subjects recalled detailed drawings better than less elaborate drawings when the degree of distinctiveness was controlled.

Question 18

Evaluating the Experiment from the Inside: Internal Validity Many researchers attempt to discover whether subjects understood and followed experimental instructions by
A)asking a panel of judges to rate instructions on clarity.
B)informally interviewing subjects at the end of an experiment.
C)performing a replication study.
D)statistically analyzing discarded data.

Accepted Answer

Informally interviewing subjects at the end of an experiment can provide insight into whether they understood and followed experimental instructions. This can help researchers evaluate the internal validity of their experiment by identifying any potential sources of confusion or error. A panel of judges rating instructions on clarity or statistically analyzing discarded data may also be helpful, but they do not directly address whether subjects followed instructions during the actual experiment. Performing a replication study can help establish external validity but does not address internal validity.

Question 19

Evaluating the Experiment from the Inside: Internal Validity Dr.Moss privately worries that his subjects may have guessed his research hypothesis, but doesn't press his subjects for information.Several of his subjects believe that they know the experiment's purpose, but do not reveal this to Dr.Moss.What problem does this situation describe?
A)a pact of ignorance
B)confounding
C)experimenter bias
D)history threat

Accepted Answer

This situation describes a "pact of ignorance," where subjects believe they know the experiment's purpose but choose not to reveal this to the experimenter, potentially affecting the study's internal validity.

Question 20

Evaluating the Experiment from the Inside: Internal Validity A well-designed experiment cannot prove that your research hypothesis is correct because there is always a chance of
A)insufficient statistical power.
B)Type 1 error.
C)Type 2 error.
D)all of these

Accepted Answer

The answer of Evaluating the Experiment from the Inside: Internal...

Quiz 15: Drawing Conclusions: the Search for the Elusive Bottom Line

Evaluating the Experiment from the Inside: Internal Validity While several mistakes can threaten statistical conclusion validity, the two most common ones are

Evaluating the Experiment from the Inside: Internal Validity ____ verifies how successfully the experimenter created values of the independent variable.

Taking a Broader Perspective: The Problem of External Validity An experiment must be ____ to be externally valid.

Evaluating the Experiment from the Inside: Internal Validity Statistical tests permit us to

Taking a Broader Perspective: The Problem of External Validity When researchers replicate statistically significant research findings, this increases our confidence that these findings were not produced by

Taking a Broader Perspective: The Problem of External Validity Amanda replicated a main effect of type of packaging on product recognition.This increases our confidence that her initial findings were not due to

Evaluating the Experiment from the Inside: Internal Validity ____ evaluates whether a sad film really made the subjects feel sad.

Evaluating the Experiment from the Inside: Internal Validity An experiment's internal validity depends on whether

Taking a Broader Perspective: The Problem of External Validity ____ means that we can accurately extend experimental results to other situations.

Evaluating the Experiment from the Inside: Internal Validity Which variable confounded the Ritchey (1982) study of the effects of age (third-grade, sixth-grade, and adult) and stimulus (words, outline drawings, and detailed drawings) on recall?

Evaluating the Experiment from the Inside: Internal Validity Orne's (1969) "pact of ignorance" shows how validity issues can be hard to resolve because of

Evaluating the Experiment from the Inside: Internal Validity When a researcher computes too many a priori comparisons, this threatens

Evaluating the Experiment from the Inside: Internal Validity If your experimental results were barely significant even though you tested 400 subjects, you should be concerned about

Evaluating the Experiment from the Inside: Internal Validity Andrew designed an experiment that controlled extraneous variables so that the effects on the dependent variable could only be attributed to the independent variable.Based on this description, his experiment achieved satisfactory

Evaluating the Experiment from the Inside: Internal Validity An experiment 's validity is threatened by ____ when subjects do not reveal all they know about the purpose of an experiment and researchers accept subject reports at face value.

Evaluating the Experiment from the Inside: Internal Validity Experiments by Ritchey and Armstrong (1982) showed that subjects recalled when the degree of was controlled.

Evaluating the Experiment from the Inside: Internal Validity Many researchers attempt to discover whether subjects understood and followed experimental instructions by

Evaluating the Experiment from the Inside: Internal Validity A well-designed experiment cannot prove that your research hypothesis is correct because there is always a chance of