Bayesian reanalysis of null results reported in medicine: Strong yet variable evidence for the absence of treatment effects
Rink Hoekstra; Rei Monden; Don van Ravenzwaaij; Eric‐Jan Wagenmakers · 2018 · PLoS ONE
WASTE classifies this as Negative / Null Result Report · AI classification, approximate
The study found no significant effect — useful as a negative control or null benchmark for your own design.
The finding, in one line
“Unfortunately, standard statistical analyses are unable to quantify the degree to which these null results actually support the null hypothesis.”
Abstract
Efficient medical progress requires that we know when a treatment effect is absent. We considered all 207 Original Articles published in the 2015 volume of the New England Journal of Medicine and found that 45 (21.7%) reported a null result for at least one of the primary outcome measures. Unfortunately, standard statistical analyses are unable to quantify the degree to which these null results actually support the null hypothesis. Such quantification is possible, however, by conducting a Bayesian hypothesis test. Here we reanalyzed a subset of 43 null results from 36 articles using a default
Abstract by Rink Hoekstra; Rei Monden; Don van Ravenzwaaij; Eric‐Jan Wagenmakers, PLoS ONE (2018) — licensed CC BY 4.0.
About to run something similar?
Run an AI Precheck on your own design to catch failure modes like this one before you spend the time. Your first desk check is free.
Related failures
Channeling Fisher: Randomization Tests and the Statistical Insignificance of Seemingly Significant Experimental Results*
Negative / Null Result ReportThe harmonic mean p -value for combining dependent tests
Negative / Null Result ReportGeneralizability of heterogeneous treatment effect estimates across samples
Negative / Null Result ReportNumerical predictors of arithmetic success in grades 1–6
Negative / Null Result ReportMethods Matter: p-Hacking and Publication Bias in Causal Analysis in Economics
Negative / Null Result ReportShould multiple imputation be the method of choice for handling missing data in randomized trials?
WASTE indexes this work — it does not host or republish it. Failure-type classification is automated and approximate.
Metadata source: OpenAlex · DOI 10.1371/journal.pone.0195474