Data extraction errors affected many diagnostic meta-analyses

Researchers reviewed Cochrane Library systematic reviews with diagnostic meta-analyses published through May 2023. They compared data used in the original analyses with independently extracted data from primary studies, classified discrepancies, and reran analyses using corrected data and the original methods. The review included 46 systematic reviews, 879 diagnostic comparisons, and 6,173 primary studies.

Data extraction could not be reproduced for 338 studies (5.5%); errors affected 137 meta-analyses (33.1%) and 34 systematic reviews (73.9%). Of 412 identified errors, 40.3% were incorrect numerical values and 34.0% involved incorrect derivation from reported statistics. Large changes occurred in 0.7% of reproduced analyses for sensitivity, 3.6% for diagnostic odds ratio, and 2.9% for area under the curve, using the authors’ thresholds.

The findings highlight potential weaknesses in evidence synthesis and support rigorous extraction and careful selection of diagnostic performance measures. The analysis was limited to Cochrane diagnostic reviews and does not establish that the reported errors altered clinical decisions. Most corrected estimates did not cross the study’s thresholds for large change.