Publication bias

Educational level: this is a tertiary (university) resource. Completion status: this resource is ~25% complete. Publication bias arises from the tendency for researchers, editors, and companies to handle the reporting of experimental results that are positive (i.e., they show a significant finding) differently from results that are negative (i.e. supported the null hypothesis) or inconclusive. This page provides an undergraduate-level introduction to the problem of publication bias.

Two counter-acting biases

• Low Power: under-estimation of real effects
• Publication Bias or File-drawer effect: over-estimation of real effects

What is publication bias?

• Publication of results depends on their nature and direction.
• Studies that show significant effects are more likely to be published.
• Type I publication errors are underestimated to the extent that they are: “frightening, even calling into question the scientific basis for much published literature.” (Greenwald, 1975, p. 15)

Funnel plots

• A scatterplot of treatment effect against study size.
• Precision in estimating the true treatment effect increases as N increases.
• Small studies scatter more widely at the bottom of the graph.
• As studies become less precise, results should be more variable, scattered to both sides of the more precise larger studies … unless there is publication bias.
• In the absence of bias the plot should resemble a symmetrical inverted funnel.
• If there is publication bias this will cause meta-analysis to overestimate effects.
• The more pronounced the funnel plot asymmetry, the more likely it is that the amount of bias will be substantial.

File drawer effects

1. Tendency for non-sig. results to be ‘filed away’ (hidden) and not published.
2. of null studies which would have to ‘filed away’ in order for a body of significant published effects to be considered doubtful.

Countering the bias

1. Journal Articles in Support of the Null Hypothesis

See also

• Publication bias (Wikipedia)
• Academic integrity

External links

• Publication bias (delicious.com/jtneill bookmarks)