Conclusiveness of Cochrane systematic reviews is low but accumulating across time in physical therapy: A meta-research study

Abstract

Background: Conclusiveness of a systematic review is the ability to reach a definitive conclusion about the effectiveness of one treatment compared to another. The large proportion of systematic reviews that do not reach a definitive conclusion might discourage clinicians from engaging with and interpreting systematic reviews. Objectives: To determine the percentage of conclusive Cochrane reviews in physical therapy and to investigate whether this percentage has increased over time. Methods: In this meta-research study, we performed a systematic search of the Physiotherapy Evidence Database (PEDro) for Cochrane reviews. We extracted a random sample of 200 published systematic reviews, with 50 reviews from each of the periods: 2001 to 2005, 2006 to 2010, 2011 to 2015, and 2016 to 2020. Two independent assessors extracted information. The Grading of Recommendations Assessment, Development and Evaluation (GRADE) data for the primary outcomes was used to assess conclusiveness. Reviews were considered conclusive when at least one primary outcome provided high certainty of evidence. Results: Outcomes with very low certainty of evidence represented 21 % of outcomes and increased from 14 % in 2001–2005 to 34 % in 2016–2020. Outcomes with low certainty of evidence comprised 55 % of outcomes and remained consistent over time. Moderate- and high-certainty outcomes remained consistent, composing 22 % and 2 % of outcomes, respectively. The proportion of outcomes with high certainty of evidence never exceeded 4 % per period. The percentage of conclusive reviews remained unchanged and consisted of 3 % of reviews in the sample; however, because the total number of reviews is increasing, there has been an accumulation in the number of conclusive reviews across the 20-year period. Conclusions: The percentage of Cochrane reviews deemed conclusive remains small, although conclusive reviews are accumulating over time.