The Methodological Quality Score of COVID-19 Systematic Reviews is Low, Except for Cochrane Reviews: A Meta-epidemiological Study

Abstract

BACKGROUND: The objective of this study was to investigate the methodological quality of coronavirus disease 2019 (COVID-19) systematic reviews (SRs) indexed in medRxiv and PubMed, compared with Cochrane COVID Reviews. METHODS: This is a cross-sectional meta-epidemiological study. We searched medRxiv, PubMed, and Cochrane Database of Systematic Reviews for SRs of COVID-19. We evaluated the methodological quality using A MeaSurement Tool to Assess systematic Reviews (AMSTAR) checklists. The maximum AMSTAR score is 11, and minimum is 0. Higher score means better quality. RESULTS: We included 9 Cochrane reviews as well as randomly selected 100 non-Cochrane reviews in medRxiv and PubMed. Compared with Cochrane reviews (mean 9.33, standard deviation 1.32), the mean AMSTAR scores of the articles in medRxiv were lower (mean difference (MD): −2.85, 98.3% confidence intervals (CI): −0.96 to −4.74), and those in PubMed were also lower (MD: −3.28, 98.3%CI: −1.40 to −5.15), with no difference between the latter two. CONCLUSIONS: Readers should pay attention to the potentially low methodological quality of SRs related to COVID-19 in both PubMed and medRxiv. Evidence users might be better to search the Cochrane Library rather than medRxiv or PubMed to search SRs related to COVID-19.