Is the Simplicity of the Evidence Pyramid Actually Detrimental for Understanding Evidence?

Abstract

The evidence pyramid for assessing the efficacy of interventions under real world conditions has been used in various forms for many years, and to a lesser extent the pyramid has been used for assessing evidence for disease risk factors. While acknowledging minor differences, many pyramids list the following information sources for interventions in decreasing order of “validity”: systematic review and meta-analysis of randomised control trials, randomised controlled trials, cohort studies, case control studies, case series and case reports. The evidence pyramid is often used as a teaching aid to help clinicians and students visualise the concept that all studies might not have equal evidentiary value when evaluating real world efficacy.

The rationale for the hierarchy of the pyramid is partially based on the potential for bias in some designs and partially based on the external validity of the information source. Designs with greater potential for bias “on average” are placed lower on the pyramid. The greater risk of bias means a greater risk that effect size estimates from studies lower on the pyramid may be systematically incorrect (overestimated or underestimated). As such, the evidence pyramid makes very broad statements about the design “on average”. Of course, for any particular study or topic, the pyramid may not be correct. Regardless, the generalisations described in the evidence pyramid have over the years been considered useful and use of the tool continues likely because of its simplicity. However, the validity of most pyramids that we have seen (and used ourselves, and published ourselves) is predicated on two potentially false concepts. The first potentially false concept conveyed by most evidence pyramids, is that there is only one case-control design and it is of lesser evidentiary value than cohort studies. The second potentially false concept conveyed by evidence pyramids is that the terms cohort, case control, and case series can be used to “filter” out studies of lower evidentiary value.

In this presentation Annette discusses the validity of the evidence pyramid on the interpretation of evidence from primary research. She proposes a new way to think about evidence from primary studies using the framework for classifying epidemiologic studies proposed by Pearce (2012) based on incident and prevalent cases. This would also result in a rethinking of the current evidence pyramid.