Extended definition
Publication bias is the systematic distortion of the literature that results from studies being published according to the direction and strength of their findings, not the quality of their method. Positive and statistically significant results are far more likely to reach publication than null or negative ones, which stay in what Rosenthal (1979) called the file drawer: the file drawer problem, the invisible mass of unpublished studies that would make the full picture less encouraging. Dickersin (1990) gathered direct evidence of the phenomenon in medicine and identified its risk factors, showing that the decision to publish depends on the outcome. The gravest consequence appears in evidence synthesis: a meta-analysis that aggregates only what was published overestimates the true effect. Egger and colleagues (1997) provided the most-used instrument for detecting the problem, a test of the asymmetry of the funnel plot, which signals when small studies with null results are missing.
When it applies
The concept applies whenever a body of evidence is interpreted, not an isolated study. It applies to the critical reading of meta-analyses and systematic reviews, where publication bias is the central threat to the validity of the synthesis. It applies to the use of the funnel plot and Egger’s test, which diagnose the asymmetry compatible with missing studies. It applies to the design of safeguards: prospective trial registration, preregistration, registered reports, and databases such as PROSPERO exist in large part to neutralize the bias, ensuring a study is on record before its result is known. And it applies to the evaluation of an entire literature: a field with an excess of positive results and almost no null ones is suspect by construction.
When it does not apply
The concept does not apply to an individual study as an internal defect: publication bias is a property of the publication system, not of an isolated article. It does not apply as the sole explanation for a funnel’s asymmetry: Egger and colleagues (1997) warned that true heterogeneity between studies, quality differences, and chance also produce asymmetry, so the test suggests but does not prove publication bias. It does not apply reliably when there are few studies, a situation in which asymmetry tests have low power. It does not apply as a synonym for fraud: the omission of the null result usually comes from incentives and editorial decisions, not from individual bad faith. And it does not apply as a problem solved by statistics: no post hoc adjustment recovers the studies that were never written.
Applications by field
- Biomedicine: the field where the bias was first documented and where clinical-trial registration is the main countermeasure.
- Psychology and social sciences: the focus of the replication crisis, with strong pressure for positive results.
- Evidence synthesis: meta-analysis and systematic review, where detecting and discussing the bias is part of the protocol.
- Science policy: design of registries and results-publication mandates to reduce the file drawer.
Common pitfalls
The first pitfall is reading Egger’s test as proof of bias, when it only signals asymmetry that may have other causes. The second is applying funnel tests with few studies, where there is no power to detect anything. The third is treating the bias as a flaw of an article, rather than of the system. The fourth is trusting a meta-analysis without assessing the possibility of missing studies, accepting an inflated effect. The fifth is assuming statistical adjustments fix the problem: they estimate what might be missing, but the real fix is structural, through prospective registration and the publication of null results.