Extended definition
An umbrella review is a review of reviews: instead of synthesizing primary studies, it gathers, compares, and contrasts the findings of several systematic reviews and meta-analyses on the same topic or set of related questions. It is one of the highest tiers of evidence synthesis, sitting above the individual systematic review because it takes the latter as its unit of analysis. Aromataris and colleagues (2015) formalized the method, defining how to search for, select, and assess existing reviews and how to present in a structured way what each one found, including the methodological quality of each included review. Ioannidis (2009) had already pointed to the umbrella review as a way to integrate evidence from multiple meta-analyses and map an entire field at once. Fusar-Poli and Radua (2018) condensed the practice into operational rules, emphasizing that the robustness of an umbrella review depends on critically appraising the reviews that compose it, not merely stacking them.
When it applies
The umbrella review applies when several systematic reviews already exist on a topic and the question has shifted to another level: what the set of those reviews says, where they agree and where they diverge. It applies to the broad mapping of a mature field, where synthesizing primary studies would be redundant given the reviews already published. It applies to the comparison of multiple interventions and the identification of gaps between reviews. It applies to high-level decision-making, such as guidelines and policy, where a consolidated view and the relative strength of the evidence are needed. Fusar-Poli and Radua (2018) recommend it when the goal is a panoramic, critical snapshot of an already well-reviewed literature, with explicit appraisal of the quality of each included review.
When it does not apply
The umbrella review does not apply when there are few or no systematic reviews on the topic: without reviews to synthesize, it lacks its unit of analysis, and the correct route is a systematic review of primary studies. It does not apply to answering a new, specific question that the existing reviews did not cover. It does not apply as a shortcut that waives rigor: it inherits the biases of the included reviews, and an umbrella review built on weak reviews only amplifies their flaws. It does not apply without assessing the overlap of primary studies among the reviews, which can inflate the appearance of evidence by counting the same study several times. And it does not apply as a substitute for meta-analysis when the question requires reanalyzing the primary data, rather than summarizing secondhand conclusions.
Applications by field
- Medicine and health: the method’s field of origin, with umbrella reviews guiding guidelines from many meta-analyses.
- Mental health: a field that consolidated its own conduct rules, with strong appraisal of review quality.
- Public health and policy: high-level synthesis for decisions that integrate many bodies of evidence.
- Education and social sciences: mapping of mature fields with multiple reviews already published.
Common pitfalls
The first pitfall is conducting an umbrella review where there are almost no systematic reviews, lacking the method’s raw material. The second is stacking reviews without assessing the quality of each, inheriting and amplifying their biases. The third is ignoring the overlap of primary studies among reviews, counting the same evidence more than once. The fourth is treating it as a meta-analysis, when it summarizes conclusions and does not reanalyze primary data. The fifth is presenting it as the highest level of evidence by definition, forgetting that its strength depends entirely on the quality of the reviews it gathers.