Bayesian reanalysis highlights differences between statistical significance and clinically meaningful effects

A methodological study in BMC Medicine examined how Bayesian probability statements could complement conventional reporting of clinical trials. Researchers reanalyzed trial-level data for 169 primary outcomes from 130 statistically negative trials across medical fields and 234 outcomes from 225 statistically positive oncology trials. They estimated probabilities of treatment benefit and of effects exceeding specified clinical thresholds, using uninformative priors for illustration and skeptical priors in sensitivity analyses.

Among statistically nonsignificant outcomes, 26 of 169 (15%) had a greater than 90% probability that the experimental treatment was better than control; 12 (7%) had a probability exceeding 95%. Conversely, 25 of 234 statistically significant oncology outcomes (11%) had less than a 50% probability of reducing hazards by at least 20%. The article also provided instructional case studies, including an additional trial concerning anticoagulation timing after stroke.

The findings illustrate why statistical significance alone does not establish or exclude clinically important benefit. Bayesian statements can express uncertainty about both any benefit and a specified magnitude of benefit. However, these probabilities depend on the model and prior assumptions, and the positive-trial dataset was restricted to oncology. This was a reanalysis of published trial results, not a prospective test of whether Bayesian communication improves clinical decisions or patient outcomes.