A searchable index of real negative results, null findings, and replication failures from the published literature — so you can learn what didn't work before repeating it.
WASTE indexes published research — it does not host or republish full papers. Each entry is a metadata record (title, authors, DOI) compiled from open scholarly databases, with the abstract shown in full only where the paper is openly licensed (e.g. Creative Commons); otherwise a short excerpt is shown for reference under fair use. WASTE classifies each work by failure type; classifications are automated and approximate.
46 results for "reproducibility" · page 1 of 2
Alexander A. Aarts · 2015 · Science
Reproducibility is a defining feature of science, but the extent to which it characterizes current research is unknown. We conducted replications of 100 experimental and correlational studies published in three psychology journals using…
Sayash Kapoor, Arvind Narayanan · 2023 · Patterns
Machine-learning (ML) methods have gained prominence in the quantitative sciences. However, there are many known methodological pitfalls, including data leakage, in ML-based science. We systematically investigate reproducibility issues in ML-based science. Through a survey of literature in fields that have adopted ML methods, we find 17 fields where leakage has been found, collectively affecting 294 papers and, in some cases, leading to wildly overoptimistic conclusions. Based on our survey, we introduce a detailed taxonomy of eight types of leakage, ranging from textbook errors to open resear
Jay J. Van Bavel, Peter Mende‐Siedlecki, William J. Brady et al. · 2016 · Proceedings of the National Academy of Sciences
In recent years, scientists have paid increasing attention to reproducibility. For example, the Reproducibility Project, a large-scale replication attempt of 100 studies published in top psychology journals found that only 39% could be…
Tom E Hardwicke, Maya B. Mathur, Kyle MacDonald et al. · 2018 · Royal Society Open Science
Access to data is a critical feature of an efficient, progressive and ultimately self-correcting scientific ecosystem. But the extent to which in-principle benefits of data sharing are realized in practice is unclear. Crucially, it is largely unknown whether published findings can be reproduced by repeating reported analyses upon shared data (‘analytic reproducibility’). To investigate this, we conducted an observational evaluation of a mandatory open data policy introduced at the journal Cognition . Interrupted time-series analyses indicated a substantial post-policy increase in data availabl
Marianna Fontana, Steve K White, Sanjay M Banypersad et al. · 2012 · Journal of Cardiovascular Magnetic Resonance
BACKGROUND: Myocardial extracellular volume (ECV) is elevated in fibrosis or infiltration and can be quantified by measuring the haematocrit with pre and post contrast T1 at sufficient contrast equilibrium. Equilibrium CMR (EQ-CMR), using a bolus-infusion protocol, has been shown to provide robust measurements of ECV using a multibreath-hold T1 pulse sequence. Newer, faster sequences for T1 mapping promise whole heart coverage and improved clinical utility, but have not been validated. METHODS: Multibreathhold T1 quantification with heart rate correction and single breath-hold T1 mapping using
Federico Innocenti, Gregory M. Cooper, Ian B. Stanaway et al. · 2011 · PLoS Genetics
The discovery of expression quantitative trait loci ("eQTLs") can help to unravel genetic contributions to complex traits. We identified genetic determinants of human liver gene expression variation using two independent collections of primary tissue profiled with Agilent (n = 206) and Illumina (n = 60) expression arrays and Illumina SNP genotyping (550K), and we also incorporated data from a published study (n = 266). We found that ∼30% of SNP-expression correlations in one study failed to replicate in either of the others, even at thresholds yielding high reproducibility in simulations, and
Pepijn Obels, Daniël Lakens, Nicholas A. Coles et al. · 2020 · Advances in Methods and Practices in Psychological Science
Ongoing technological developments have made it easier than ever before for scientists to share their data, materials, and analysis code. Sharing data and analysis code makes it easier for other researchers to reuse or check published research. However, these benefits will emerge only if researchers can reproduce the analyses reported in published articles and if data are annotated well enough so that it is clear what all variable and value labels mean. Because most researchers are not trained in computational reproducibility, it is important to evaluate current practices to identify those tha
Ahmed M. Elkhatat · 2023 · International Journal for Educational Integrity
Abstract Academic plagiarism is a pressing concern in educational institutions. With the emergence of artificial intelligence (AI) chatbots, like ChatGPT, potential risks related to cheating and plagiarism have increased. This study aims to investigate the authenticity capabilities of ChatGPT models 3.5 and 4 in generating novel, coherent, and accurate responses that evade detection by text-matching software. The repeatability and reproducibility of both models were analyzed, showing that the generation of responses remains consistent. However, a two-sample t-test revealed insufficient evidenc
Brit Mollenhauer, Frederick DuBois Bowman, Daniel Drake et al. · 2018 · Journal of Neurochemistry
Abstract α‐Synuclein is the major component of Lewy bodies and a candidate biomarker for neurodegenerative diseases in which Lewy bodies are common, including Parkinson's disease and dementia with Lewy bodies. A large body of literature suggests that these disorders are characterized by reduced concentrations of α‐synuclein in cerebrospinal fluid (CSF), with overlapping concentrations compared to healthy controls and variability across studies. Several reasons can account for this variability, including technical ones, such as inter‐assay and inter‐laboratory variation (reproducibility). We co
Kenia Martínez, Sarah K. Madsen, Anand A. Joshi et al. · 2015 · Human Brain Mapping
People differ in their cognitive functioning. This variability has been exhaustively examined at the behavioral, neural and genetic level to uncover the mechanisms by which some individuals are more cognitively efficient than others.…
Meryem A. Yücel, Robert Luke, Rickson C. Mesquita et al. · 2025 · Communications Biology
As data analysis pipelines grow more complex in brain imaging research, understanding how methodological choices affect results is essential for ensuring reproducibility and transparency. This is especially relevant for functional Near-Infrared Spectroscopy (fNIRS), a rapidly growing technique for assessing brain function in naturalistic settings and across the lifespan, yet one that still lacks standardized analysis approaches. In the fNIRS Reproducibility Study Hub (FRESH) initiative, we asked 38 research teams worldwide to independently analyze the same two fNIRS datasets. Despite using dif
Lawrence Grierson, Samantha Winemaker, Alan Taniguchi et al. · 2020 · Perspectives on Medical Education
INTRODUCTION: The medical education community has implemented writing exercises that foster critical analysis and nurture reflective capacity. The REFLECT rubric (Wald et al. 2012) was developed to address the challenge of assessing these written reflections. The objective of this replication work is to explore the reproducibility of the reliability characteristics presented by the REFLECT developers. METHODS: Five raters evaluated narratives written by medical students and experienced clinicians using the REFLECT rubric. Reliability across rubric domains was determined via intraclass correlat
Wei Li, Laura Germine, Samuel A. Mehr et al. · 2022 · Infant and Child Development
Widespread failures of replication and generalization are, ironically, a scientific triumph, in that they confirm the fundamental metascientific theory that underlies our field. Generalizable and replicable findings require testing large…
Joshua K. Hartshorne, Lauren Skorb, Sven L. Dietz et al. · 2019 · Collabra Psychology
We report the first set of results in a multi-year project to assess the robustness – and the factors promoting robustness – of the adult statistical word segmentation literature. This includes eight total experiments replicating six different experiments. The purpose of these replications is to assess the reproducibility of reported experiments, examine the replicability of their results, and provide more accurate effect size estimates. Reproducibility was mixed, with several papers either lacking crucial details or containing errors in the description of method, making it difficult to ascert
Waylon J. Hastings, Laura Etzel, Christine M. Heim et al. · 2022 · Aging
Various approaches exist to assess population differences in biological aging. Telomere length (TL) is one such measure, and is associated with disease, disability and early mortality. Yet, issues surrounding precision and reproducibility are a concern for TL measurement. An alternative method to estimate TL using DNA methylation (DNAmTL) was recently developed. Although DNAmTL has been characterized in adult and elderly cohorts, its utility in pediatric populations remains unknown. We examined the comparability of leukocyte TL measurements generated using qPCR (absolute TL; aTL) to those esti
Rasit Dinc, Nurittin Ardıç · 2026 · Cells
Exosomes and other extracellular vesicles (EVs) carry microRNAs, proteins, and lipids that reflect cardiovascular pathophysiology and can enable minimally invasive biomarker discovery. However, EV datasets are highly dimensional and heterogeneous, strongly influenced by pre-analytic variables and non-standardized isolation/characterization workflows, limiting reproducibility across studies. Artificial intelligence (AI), including machine learning (ML), deep learning (DL), and network-based approaches, can support EV biomarker development by integrating multi-omics profiles with clinical metada
Chitra Yadav, Alexander S. Evtushenko, Andrea Bistrović et al. · 2025 · npj Biomedical Innovations.
3D spheroid culture has emerged as a valuable tool for studying complex intratumoral processes and screening novel therapeutics in vitro. However, spheroids face reproducibility and data interpretation issues, which limit their utility. This work describes a simple and reproducible co-culture spheroid model compatible with high-throughput screening designed to study pancreatic ductal adenocarcinoma (PDAC), a highly therapy-resistant cancer. These spheroids, composed of both cancer and stromal cells, recapitulate key features of PDAC which are difficult to study in traditional 2D cell culture,
Nasser Lubega, Abigail Anderson, Nicole Nelson · 2023 · PLoS ONE
High rates of irreproducibility and of poor mental health in graduate students have been reported in the biomedical sciences in the past ten years, but to date, little research has investigated whether these two trends interact. In this study, we ask whether the experience of failing to replicate an expected finding impacts graduate students' mental health. Using an online survey paired with semi-structured qualitative interviews, we examined how often biomedical science doctoral students at a large American public university experienced events that could be interpreted as failures to replicat
Jack W. O’Sullivan, John P. A. Ioannidis · 2021 · Scientific Reports
Abstract With the establishment of large biobanks, discovery of single nucleotide variants (SNVs, also known as single nucleotide polymorphisms (SNVs)) associated with various phenotypes has accelerated. An open question is whether…
Tümay Capraz, Wolfgang Huber · 2024 · Bioinformatics
Abstract Motivation A fundamental step in many analyses of high-dimensional data is dimension reduction. Two basic approaches are introduction of new synthetic coordinates and selection of extant features. Advantages of the latter include…
Caetano Souto-Maior · 2025 · PeerJ
Reports of crises of reproducibility have abounded in the scientific and popular press, and are often attributed to questionable research practices, lack of rigor in protocols, or fraud. On the other hand, it is a known fact that—just like…
Ke Xiao, Rongxin Zhang, Jing Tu · 2025 · International Journal of Molecular Sciences
G-quadruplex (G4) ChIP-Seq data are critical for studying the roles of G4 structures in various biological processes, yet their reproducibility remains systematically uncharacterized. In this study, we evaluated the consistency of in vivo…
I. Pinilla, E. Garcia-Martin, M. Idoipe et al. · 2012 · Journal of Ophthalmology
Purpose. To compare the retinal nerve fiber layer (RNFL) measurements using two different ocular coherence tomography (OCT) devices: Cirrus Fourier domain OCT and Stratus time domain OCT. To analyze reproducibility of Fourier domain…
Ali Subhan, Ashir Raza · 2026 · arXiv
DragDiffusion is a diffusion-based method for interactive point-based image editing that enables users to manipulate images by directly dragging selected points. The method claims that accurate spatial control can be achieved by optimizing a single diffusion latent at an intermediate timestep, together with identity-preserving fine-tuning and spatial regularization. This work presents a reproducibility study of DragDiffusion using the authors' released implementation and the DragBench benchmark. We reproduce the main ablation studies on diffusion timestep selection, LoRA-based fine-tuning, mas
F. Trinkmann, F. Kraus, M. Borggrefe et al. · 2011 · European Respiratory Journal
Background: Cardiac output (CO) is an important hemodynamic parameter, however its determination is difficult in daily clinical routine. Non-invasive inert gas rebreathing (IGR) showed promising results in recent investigations with the…
Yoav Zeevi, Sofi Astashenko, Liad Mudrik et al. · 2020 · arXiv
The field of psychological sciences has been grappling with the replicability crisis. Various issues have been identified as potential sources of this problem. We bring to light a potential source that has largely been overlooked and demonstrate its significant contribution to the problem: the practice of multiple comparisons. We analyzed 88 papers from the Reproducibility Project in Psychology and found that multiple results are commonly reported in a single paper, ranging from 4 to 730 (M=77.7), without multiple comparison adjustments. We retroactively applied such an adjustment using a hier
Jiarui Xie, Mutahar Safdar, Andrei Mircea et al. · 2024 · arXiv
Machine learning (ML)-based cyber-physical systems (CPSs) have been extensively developed to improve the print quality of additive manufacturing (AM). However, the reproducibility of these systems, as presented in published research, has not been thoroughly investigated due to a lack of formal evaluation methods. Reproducibility, a critical component of trustworthy artificial intelligence, is achieved when an independent team can replicate the findings or artifacts of a study using a different experimental setup and achieve comparable performance. In many publications, critical information nec
Han-Kyung Seo, Do-Cheol Choi, Cheol-Min Shim et al. · 2023 · Korean Journal of Nuclear Medicine Technology
Purpose: The precision error of a bone density meter reflects the equipment and reproducibility of results by an examiner. Precision error values can be expressed as coefficient of variation (CV), CV%, and root mean square-SD (RMS-SD). The…
Armin D. Tavakkoli, BA, Megan A. Clark, BE, Alireza Kheirollah, PhD et al. · 2024 · Advances in Radiation Oncology
Purpose: Ultra High Dose-Rate (UHDR) radiation has been reported to spare normal tissue, compared with Conventional Dose-Rate (CDR) radiation. However, important work remains to be done to improve the reproducibility of the FLASH effect. A better understanding of the biologic factors that modulate the FLASH effect may shed light on the mechanism of FLASH sparing. Here, we evaluated whether sex and/or the use of 100% oxygen as a carrier gas during irradiation contribute to the variability of the FLASH effect. Methods and Materials: C57BL/6 mice (24 male, 24 female) were anesthetized using isofl
Alexandre Pereira Tognon, Murilo Foppa, Vivian Cristine Luft et al. · 2015 · Arquivos Brasileiros de Cardiologia
Background: Echocardiography, though non-invasive and having relatively low-cost, presents issues of variability which can limit its use in epidemiological studies. Objective: To evaluate left ventricular mass reproducibility when assessed at acquisition (online) compared to when assessed at a reading center after electronic transmission (offline) and also when assessed by different readers at the reading center. Methods: Echocardiographers from the 6 ELSA-Brasil study investigation centers measured the left ventricular mass online during the acquisition from 124 studies before transmitting to