Deck 7: External Validity of Laboratory Experiments

Question 1

The use of extreme politically left- or right-wing participants has the potential to bias, or undermine the generalizability of, the results in which of the following investigations? (circle all that apply)

A) investigating the role of political attitudes on volunteering behavior
B) investigating the effects of caffeine on people's neurological processes (e.g., the level of serotonin and dopamine in the brain)
C) using a prime that induces feelings of individualism and autonomy as an effective treatment for low self-esteem
D) investigating the role of communication in couple's sexual satisfaction

Answer 1

A,C,D

Question 2

Which participants have the potential to bias the results of a study:

A) voluntary participants
B) involuntary participants
C) nonvoluntary participants
D) A and B
E) B and C
F) all of the above

Answer 2

D

Question 3

Providing an explanation of the goals of a study, as well as participants' role in reaching that goal, is often used to reduce the risk of __________participants actively trying to sabotage the results of a study.

A) voluntary participants
B) involuntary participants
C) nonvoluntary participants
D) faithful participants

Answer 3

B

Question 4

In a field experiment, researchers experimentally manipulated a sign next to the trash cans in a park. In one condition, the sign read "DON'T LITTER," in the other condition, the sign read "Please keep your park clean." Researchers found that on average, park goers tended to throw less trash in the trash can (i.e., they littered more) when the DON'T LITTER sign was posted. The park goers in this experiment would be considered what type of participants?

A) apprehensive participants
B) nonvoluntary participants
C) involuntary participants
D) voluntary participants

Question 5

Which types of participants are likely to yield the greatest degree of generalizability or external validity? (circle all that apply)

A) voluntary participants
B) involuntary participants
C) nonvoluntary participants
D) good participants
E) none of the above

Question 6

What types of participants are most likely to try to figure out the hypotheses of an experiment so that they can respond accordingly? (circle all that apply)

A) good participants
B) faithful participants
C) apprehensive participants
D) negative participants

Question 7

In a double-blind study examining the effects of a new cola-flavored energy drink on people's physiological arousal, participants in the experimental drink condition demonstrated significantly greater arousal than the control group. However, upon further investigation, a third, placebo group (using flavored Coca-Cola), also showed significantly greater arousal than the control group. Which of the following may have influenced the results of this study?

A) participants' expectations about the effects of an experimental energy drink
B) experimenters' expectations about the effects of the experimental energy drink
C) the drink that was used as a placebo
D) A and B
E) A and C
F) all of the above

Question 8

Experimenters can unknowingly influence the results of their own studies through very subtle, unconscious cues. Which of the following techniques would be least likely to reduce the risk of experimenter bias in a study?

A) using a double-blind procedure
B) video-recording the experimenter's instructions prior to anyone participating
C) using a computer program to administer the experimental treatments
D) having another equally qualified experimenter monitor and record the interactions between experimenter and participants.

Question 9

Researchers wanted to investigate the effects of having plants in the laboratory (plants present, plants absent) and giving participants a snack prior to participating (cookie, carrots) influenced participants' aggressiveness after completing 30 minutes of a tedious and seemingly unending task (e.g., rotating each of 64 dowels in a grid a quarter-turn, and when finished, repeating the process). Which of the following could be considered demand characteristics in the context of this study? (circle all that apply)

A) the screen-savers on the lab computers are all of serene landscape photographs
B) the presence of lush flora on the grounds surrounding the research building
C) during the boring experimental task, the researcher says to a research assistant, loud enough for the participant to hear: "I need a snack. If I don't eat, I get cranky"
D) a poster of the food pyramid on the far wall of the laboratory

Question 10

Study 1 investigated whether lack of sleep influences people's information processing using a college sample and paper and pencil measures. Study 2 replicated the study using a convenience sample gathered from a sleep clinic (of middle-aged and elderly adults), and measured information processing using fMRI images. Study 3 conducted a third study on sleep and information processing in school children, using behavioral observations and verbal responses to several information processing questions. Studies 1 and 2 were consistent with hypotheses, while Study 3 found no differences. This series of studies would likely be high in which of the following?

A) robustness
B) ecological validity
C) relevance
D) A and B
E) A and C
F) none of the above

Question 11

Researchers were interested in testing whether a new experimental therapeutic technique would reduce people's interest in Internet pornography. To do so, they recruited all of the residents of an assisted living facility (for the elderly) and randomly assigned them to the experimental treatment or control group. The treatment worked! People in the therapy group scored 0.05 on interest in Internet pornography, while people in the control group scored significantly higher (mean = 0.50). This study would likely be high in which of the following?

A) robustness
B) ecological validity
C) relevance
D) A and B
E) B and C
F) none of the above

Question 12

Researchers were interested in determining whether feelings of self-uncertainty actually cause people to seek out and join more extreme groups (a relationship that had not previously been established). For this study, which of the following would likely be the most important?

A) robustness
B) ecological validity
C) relevance
D) internal validity

Question 13

The type(s) of external validity that are important in an experiment sometimes vary depending on the ultimate goals of that study. For example, in descriptive research of a social phenomenon, _____(a)_____ would be crucial. Determining the extent to which an experimental effect can be found in different experimental settings is important for establishing the _____(b)_____ of that effect.

A) robustness; relevance
B) relevance; ecological validity
C) ecological validity; robustness
D) ecological validity; relevance

Question 14

_________ and _________ participants are most likely to threaten the generalizability of a laboratory study:

A) involuntary, nonvoluntary
B) nonvoluntary, voluntary
C) involuntary, voluntary
D) none of the above

Question 15

A person who adjusts his or her responses to a scale designed to measure racist attitudes in order to not appear racist to the experimenter is an example of a __________ participant.

A) negative
B) good
C) apprehensive
D) faithful

Question 16

In Milgram's (1963; 1965) obedience experiments, participants who proceeded to administer shocks to confederates to the highest level of shock possible, were most likely being ________ participants.

A) apprehensive
B) good
C) negative
D) faithful

Question 17

If a participant believes that the treatment they have received will cause anxiety, and thus they report higher levels of anxiety, this is an example of:

A) demand characteristics
B) response bias
C) experimenter bias
D) an experimenter expectancy effect
E) a placebo effect

Question 18

Experimenter expectancy bias:

A) is a form of random error
B) is a form of systematic error
C) can lead to Type II error
D) results from having "good" or "faithful" participants in a study
E) does not affect generalizability

Question 19

Which of the following would NOT be an effective means of avoiding experimenter bias:

A) having participant responses recorded by a computer program
B) informing experimenters of the experimental conditions
C) explaining expected results to experimenters before conducting the study
D) making study results unavailable to experimenters for the duration of the study
E) using blind procedures

Question 20

Laboratory experiments conducted with college student samples are likely to have low:

A) robustness
B) ecological validity
C) experimental realism
D) relevance
E) response rates

Question 21

____________ refers to whether or not a phenomenon of interest occurs across a variety of settings and people.

A) representativeness
B) relevance
C) ecological validity
D) robustness
E) none of the above

Question 22

Explain what it means to have a nonrepresentative sample. Describe the situations where a pool of such participants risks dramatically limiting the extent to which results can be generalized to the rest of the population, and when using such participants is unlikely to influence results much, if at all. Provide an example study (i.e., list of 1 IV and 1 DV) that would be vulnerable to nonrepresentative-participant bias, as well as an example study that would not be vulnerable to this bias, and be sure to explain why and how those examples demonstrate what you claim that they are demonstrating.

Question 23

Describe the concept of demand characteristics, and provide one example of a demand characteristic in a study with perception of the threat of physical harm (high, low) as the IV and aggression as the DV. In your response, be sure to (a) describe demand characteristics; (b) provide your example; and (c) explain how your example reflects a demand characteristic in the context of the threat-aggression study.

Question 24

Describe the three major aspects of external validity. How do they differ, and how are they similar? What aspects of a study's findings does each "face" address? In your response, be sure to (a) define each aspect, (b) explain how each aspect is similar and/or different from the others, and (c) describe what about a study would be externally valid if it were high in each aspect of external validity.

Question 25

What is meant by Webber and Cook's (1972) categories of participants (i.e., good, negative, faithful, apprehensive)? How do these relate to voluntary, involuntary, and nonvoluntary participants? Relate these two models of participants. In your response, be sure to (a) briefly define each term, and (b) explain how Webber and Cook's categories would correspond with the degrees of "voluntary" participation.

Question 26

Researchers in Los Angeles, CA, were interested in the predictors of aggressive driving. To investigate the potential antecedents of aggressive driving, these researchers developed a full-emersion driving simulator (i.e., in a real car adapted for a simulator, and with screens in all of the windows to resemble the available visual field one has in a car) where participants were subjected to several driving scenarios that reflected everyday driving situations in Los Angeles, including morning rush-hour, moderate traffic on a weekend, and busy nighttime driving. In other words, the driving simulator was designed to be as close to the real experience of driving as possible. Furthermore, the researchers were painstakingly thorough in making sure they recruited a sample of drivers that was representative of Los Angeles. The dependent variables were various observational and unobtrusive measures throughout the simulation (e.g., distance between the front of the participant's car to the rear of the car ahead of the participant; number and severity of fluctuations in speed; the amount of pressure applied to the gas and brake pedal). Is this study high in all three major aspects of external validity? Why or why not? Specifically, (a) identify and describe the major aspects of external validity, and (b) explain how and why the present study would be high or low on these three dimensions of external validity.

Question 27

You are conducting a study using mood (happy, sad) and the presence of plants (plants present, plants not present) as the IVs and donation to charity as your DV. Specifically, (a) create a research design for this study that is high in relevance but low in ecological validity; (b) be sure to explain how your design demonstrates high relevance and low ecological validity. Furthermore, (c) what is robustness, and (d) what would you do with this study to increase its robustness, and why would that increase its robustness?

Question 28

How does the use of college student samples in social research affect external validity?

Question 29

Participants bring many things with them to participate in a lab experiment. How can the experiences of a participant influence the course of a study? Consider types participants (e.g., good, faithful) and factors within the experiment itself (i.e., demand characteristics).

Question 30

What are some ways to reduce the possibility of experimenter expectancy effects?

Question 31

You have developed a new form of behavioral therapy for people with various conduct disorders, and you think it is a relatively robust treatment. Specifically, the therapy has been successful when it was practiced in your private clinical office and in a public park, and it has been successful across various age groups (e.g., children, teens, middle-age, seniors). However, whenever another researcher tries to replicate your treatment implementation, or when individuals attempt to reproduce what they have seen you do on television, no one seems to be able to achieve the same levels of success that you do (and your argument is that others have less success because "they aren't doing it right"). That is not to say your results are fabricated; indeed, your therapy works, just only when you administer it.
In this example, how would you classify the validity and generalizability of your behavioral therapy? Is it an externally valid therapeutic technique if only you can implement it successfully? How generalizable is it?

Question 32

If you conduct a field study in a place where people are known to behave slightly to extremely differently than they would otherwise behave (e.g., in Las Vegas), how ecologically valid are the results of that study? In other words, is what happens in Las Vegas truly representative of what would likely happen in "real life" situations outside of Las Vegas? Would the goal of a Vegas field study be to generalize to other similar contextual environments (e.g., Atlantic City) rather than to general "real life" contexts? If you are trying to achieve a high degree of ecological validity with your field study, should you plan to conduct it somewhere that is more likely to reflect people's everyday interactions and experiences, or does it matter where your field study is conducted as long as your sample is large enough?

Question 33

If naïve researchers and participants alike are prone to try to figure out what a study is trying to investigate, how do we know the extent to which participants' responses are due solely to our experimental treatments rather than to some sort of informal cues by researchers and good guesses by participants? Even if participants are unable to articulate their suspicions regarding what they thought the study was about, does that mean they were not actively trying to figure it out while participating? Does the process of random assignment solve, or at least help to address, this problem, and if so, why and how? Do biased responses by "good participants" essentially get balanced out by biased responses by "negative participants?"