Deck 25: Meta-Analysis: Resolving Inconsistencies Across Studies

Question 1

A meta-analysis combines data from separate studies, and hence can be prone to the statistical phenomenon known as _______________.

Answer 1

simpson's paradox

Question 2

If the statistically significant studies of a relationship are the ones that are more likely to be available, what effect will this have on the resulting meta-analysis?

Answer 2

It may overestimate the size of the relationship.

Question 3

Which of the following cautions must be applied with a meta-analysis?

A)Beware of confounding variables.
B)Be careful about concluding causal relationships.
C)More isn't necessarily better; a single analysis from a well-conducted study is better than a meta-analysis from several poor studies.
D)All of the above.

Answer 3

All of the above.

Question 4

Which of the following can be a direct benefit of a meta-analysis for future research?

A)Designing better experiments.
B)A change in the focus of the research.
C)Finding patterns in single studies that were not apparent before.
D)All of the above.

Question 5

Which of the following statements comparing the vote-count approach and the meta-analysis approach is false?

A)A meta-analysis focuses on the magnitude of the effect in each study, and vote-count does not.
B)Vote-count only works for studies involving categorical variables, while a meta-analysis can be used on any type of study.
C)A vote-count approach typically weights each study the same, regardless of sample size, while a meta-analysis typically does not.
D)None of the above.

Question 6

Because a meta-analysis by definition combines the results of different studies, what problem is it particularly prone to?

A)Simpson's Paradox.
B)Collapsing results over an important third variable.
C)Weakening or even reversing a relationship once groups of data are combined.
D)All of the above.

Question 7

Give two criteria that researchers often use to decide whether or not a certain study should be included in a meta-analysis.

Question 8

Under what conditions (if ever) should data from all the studies in a meta-analysis be combined into one single hypothesis test or confidence interval?

Question 9

Name one way in which a meta-analysis improves the situation where many small studies were conducted, but no large studies.

Question 10

What is meant by the 'file drawer problem' regarding meta-analysis?

Question 11

Why do we characterize a meta-analysis as being an observational study (versus an experiment)?

Question 12

Which of the following is true about meta-analysis?

A)Meta-analysis has made it possible to find definitive answers to questions about small and moderate relationships by taking into account data from a number of different studies.
B)Meta-analysis is a non-controversial approach to research.
C)Meta-analysis just means "a very big analysis of one very big data set."
D)All of the above statements are true.

Question 13

Name three possible problems with meta-analysis.

Question 14

Name two benefits of meta-analysis.

Question 15

Suppose you are doing a meta-analysis using 20 studies, and 2 of them show a much larger relationship between the variables studied, compared to all the others.Which of the following is an appropriate way to address this issue?

A)Throw those two studies out.
B)Consider those two studies to be the most important and weight them accordingly in the meta-analysis.
C)Conduct one meta-analysis that includes the two studies and another meta-analysis that excludes the two studies and compare the results.
D)None of the above.

Question 16

Reports are often published that appear to conflict with earlier results, confusing the public and researchers alike.How can sample size play a role in this?

Question 17

If the original studies were flawed or biased, what is true about the resulting meta-analysis?

A)It will also be flawed or biased.
B)It will be able to minimize or even eliminate the flaws and bias.
C)It will be unaffected.
D)The impact is unknown - the flaws or biases may enter into the meta-analysis or may not.

Question 18

One problem of meta-analysis is that two different studies may involve subtle differences in treatments but call the treatments by the same name.Give an example.

Question 19

Reports are often published that appear to conflict with earlier results, confusing the public and researchers alike.How can a meta-analysis help address this problem?

Question 20

When reading the results of a meta-analysis, what two important things do you need to know?

Question 21

Meta-analysis is particularly likely to find results that __________ (choose: are, are not) statistically significant.

Question 22

Can a meta-analysis erroneously reach the conclusion that there is no difference or relationship?

A)Yes, when there are small sample sizes and/or possible methodological flaws.
B)Yes, because of its typically extra-large sample size.
C)No, a meta-analysis can never erroneously reach a conclusion of no relationship because a meta-analysis is so comprehensive.
D)No.If there is no relationship, then there is no erroneous conclusion to come to in the first place.

Question 23

Meta-analysis can miss detecting results that are actually statistically significant due to __________ samples and/or possible methodological __________.