Einzeltreffer — DigiBib

Publication selection bias undermines the systematic accumulation of evidence. To assess the extent of this problem, we survey over 68,000 meta-analyses containing over 700,000 effect size estimates from medicine (67,386/597,699), environmental sciences (199/12,707), psychology (605/23,563), and economics (327/91,421). Our results indicate that meta-analyses in economics are the most severely contaminated by publication selection bias, closely followed by meta-analyses in environmental sciences and psychology, whereas meta-analyses in medicine are contaminated the least. After adjusting for publication selection bias, the median probability of the presence of an effect decreased from 99.9% to 29.7% in economics, from 98.9% to 55.7% in psychology, from 99.8% to 70.7% in environmental sciences, and from 38.0% to 29.7% in medicine. The median absolute effect sizes (in terms of standardized mean differences) decreased from d = 0.20 to d = 0.07 in economics, from d = 0.37 to d = 0.26 in psychology, from d = 0.62 to d = 0.43 in environmental sciences, and from d = 0.24 to d = 0.13 in medicine.

Titel:	Footprint of publication selection bias on meta-analyses in medicine, environmental sciences, psychology, and economics.
Autor/in / Beteiligte Person:	Bartoš, F ; Maier, M ; Wagenmakers, EJ ; Nippold, F ; Doucouliagos, H ; Ioannidis, JPA ; Otte, WM ; Sladekova, M ; Deresssa, TK ; Bruns, SB ; Fanelli, D ; Stanley, TD
Link:	Full Text Finder (Volltext)
Zeitschrift:	Research synthesis methods, Jg. 15 (2024-05-01), Heft 3, S. 500-511
Veröffentlichung:	<Mar. 2014-> : Chichester : Wiley Blackwell ; <i>Original Publication</i>: Malden, MA : John Wiley & Sons, 2010-, 2024
Medientyp:	academicJournal
ISSN:	1759-2887 (electronic)
DOI:	10.1002/jrsm.1703
Schlagwort:	Humans Ecology Research Design Selection Bias Probability Medicine Publication Bias Meta-Analysis as Topic Psychology Economics
Sonstiges:	Nachgewiesen in: MEDLINE Sprachen: English Publication Type: Journal Article Language: English [Res Synth Methods] 2024 May; Vol. 15 (3), pp. 500-511. <i>Date of Electronic Publication: </i>2024 Feb 07. MeSH Terms: Publication Bias* ; Meta-Analysis as Topic* ; Psychology* ; Economics* ; Humans ; Ecology ; Research Design ; Selection Bias ; Probability ; Medicine References: Chavalarias D, Ioannidis JP. Science mapping analysis characterizes 235 biases in biomedical research. J Clin Epidemiol. 2010;63(11):1205‐1215. ; Dwan K, Altman DG, Arnaiz JA, et al. Systematic review of the empirical evidence of study publication bias and outcome reporting bias. PloS One. 2008;3(8):e3081. ; Rosenthal R, Gaito J. Further evidence for the cliff effect in interpretation of levels of significance. Psychol Rep. 1964;15(2):570. ; Wicherts JM. The weak spots in contemporary science (and how to fix them). Animals. 2017;7(12):90‐119. ; Otte WM, Vinkers CH, Habets PC, IJzendoorn vDG, Tijdink JK. Analysis of 567,758 randomized controlled trials published over 30 years reveals trends in phrases used to discuss results that do not reach statistical significance. PLoS Biol. 2022;20(2):e3001562. ; John LK, Loewenstein G, Prelec D. Measuring the prevalence of questionable research practices with incentives for truth telling. Psychol Sci. 2012;23(5):524‐532. ; Fiedler K, Schwarz N. Questionable research practices revisited. Soc Psychol Personal Sci. 2016;7(1):45‐52. ; De Winter JC, Dodou D. A surge of p‐values between 0.041 and 0.049 in recent decades (but negative results are increasing rapidly too). PeerJ. 2015;3:e733. ; Fanelli D, Costas R, Ioannidis JP. Meta‐assessment of bias in science. Proc Natl Acad Sci. 2017;114(14):3714‐3719. ; Ioannidis JP, Stanley T, Doucouliagos H. The power of bias in economics research. Econ J. 2017;127(605):F236‐F265. ; Mathur VW. Finding common ground in meta‐analysis “wars” on violent video games. Perspect Psychol Sci. 2019;14(4):705‐708. ; Stanley TD, Carter EC, Doucouliagos H. What meta‐analyses reveal about the replicability of psychological research. Psychol Bull. 2018;144(12):1325‐1346. ; Van Aert RC, Wicherts JM, Van Assen MA. Publication bias examined in meta‐analyses from psychology and medicine: a meta‐meta‐analysis. PloS One. 2019;14(4):e0215052. ; Schwab S, Kreiliger G, Held L. Assessing treatment effects and publication bias across different specialties in medicine: a meta‐epidemiological study. BMJ Open. 2021;11(9):e045942. ; Kühberger A, Fritz A, Scherndl T. Publication bias in psychology: a diagnosis based on the correlation between effect size and sample size. PloS One. 2014;9(9):e105825. ; Fanelli D. “Positive” results increase down the hierarchy of the sciences. PloS One. 2010;5(4):e10068. ; Ioannidis JP. Excess significance bias in the literature on brain volume abnormalities. Arch Gen Psychiatry. 2011;68(8):773‐780. ; Scheel AM, Schijen MR, Lakens D. An excess of positive results: comparing the standard psychology literature with registered reports. Adv Methods Pract Psychol Sci. 2021;4(2):1‐12. ; Sterling TD. Publication decisions and their possible effects on inferences drawn from tests of significance—or vice versa. J Am Stat Assoc. 1959;54(285):30‐34. ; Fanelli D. Negative results are disappearing from most disciplines and countries. Scientometrics. 2012;90(3):891‐904. ; Deressa TK, Stern DI, Vangronsveld J, et al. More than half of statistically significant research findings in the environmental sciences are actually not. Submitted for publication 2022. ; Sladekova M, Webb LE, Field AP. Estimating the change in meta‐analytic effect size estimates after the application of publication bias adjustment methods. Psychol Methods. 2022;28:664‐686. ; Viechtbauer W. Conducting meta‐analyses in R with the metafor package. J Stat Softw. 2010;36(3):1‐48. ; Borenstein M, Hedges LV, Higgins JP, Rothstein HR. Introduction to Meta‐Analysis. John Wiley & Sons; 2009. ; Maier M, Bartoš F, Wagenmakers EJ. Robust Bayesian meta‐analysis: addressing publication bias with model‐averaging. Psychol Methods. 2022;28:107‐122. ; Bartoš F, Maier M, Wagenmakers EJ, Doucouliagos H, Stanley TD. Robust Bayesian meta‐analysis: model‐averaging across complementary publication bias adjustment methods. Research Synthesis Methods. 2022;14(1):99‐116. ; Hoeting JA, Madigan D, Raftery AE, Volinsky CT. Bayesian model averaging: a tutorial. Stat Sci. 1999;14(4):382‐401. ; Fragoso TM, Bertoli W, Louzada F. Bayesian model averaging: a systematic review and conceptual classification. Int Stat Rev. 2018;86(1):1‐28. ; Vevea JL, Hedges LV. A general linear model for estimating effect size in the presence of publication bias. Psychometrika. 1995;60(3):419‐435. ; Stanley TD, Doucouliagos H, Ioannidis JP. Finding the power to reduce publication bias. Stat Med. 2017;36(10):1580‐1598. ; Erp VS, Verhagen J, Grasman RP, Wagenmakers EJ. Estimates of between‐study heterogeneity for 705 meta‐analyses reported in Psychological Bulletin from 1990–2013. J Open Psychol Data. 2017;5(1):4. ; Wrinch D, Jeffreys H. On certain fundamental principles of scientific inquiry. Phil Mag. 1921;42:369‐390. ; Kass RE, Raftery AE. Bayes factors. J Am Stat Assoc. 1995;90(430):773‐795. ; Jeffreys H. Theory of Probability. 1st ed. Oxford University Press; 1939. ; Lee MD, Wagenmakers EJ. Bayesian Cognitive Modeling: A Practical Course. Cambridge University Press; 2013. ; Wagenmakers EJ, Morey RD, Lee MD. Bayesian benefits for the pragmatic researcher. Curr Dir Psychol Sci. 2016;25(3):169‐176. ; Wasserstein RL, Lazar NA. The ASA statement on p‐values: context, process, and purpose. Am Stat. 2016;70(2):129‐133. ; Ioannidis JP, Cappelleri JC, Lau J. Issues in comparisons between meta‐analyses and large trials. Jama. 1998;279(14):1089‐1093. ; Laine C, Horton R, DeAngelis CD, et al. Clinical trial registration—looking back and moving ahead. N Engl J Med. 2007;356(26):2734‐2736. ; Chambers CD. Registered reports: a new publishing initiative at cortex. Cortex. 2013;49(3):609‐610. ; Chambers CD, Dienes Z, McIntosh RD, Rotshtein P, Willmes K. Registered reports: realigning incentives in scientific publishing. Cortex. 2015;66:A1‐A2. ; Akker v dOR, Weston S, Campbell L, et al. Preregistration of secondary data analysis: a template and tutorial. Meta‐Psychology. 2021;5:5. Grant Information: Nederlandse Organisatie voor Wetenschappelijk Onderzoek; BOF20OWB05 Special Research Fund (BOF) of Hasselt University; 405039391 German Research Foundation (DFG) Contributed Indexing: Keywords: Bayesian; RoBMA; effect sizes; evidence; meta‐analysis; model‐averaging; publication bias Entry Date(s): Date Created: 20240208 Date Completed: 20240425 Latest Revision: 20240425 Update Code: 20240425

Klicken Sie ein Format an und speichern Sie dann die Daten oder geben Sie eine Empfänger-Adresse ein und lassen Sie sich per Email zusenden.

BibTeX Citavi, JabRef, u.a.
(Literaturverwaltung)

PDF kein Volltext!
(Merkzettel, Notizen)

RIS Endnote, Citavi u.a.
(Literaturverwaltung)

MODS
(XML zur Weiterverarbeitung)

oder

Wählen Sie das für Sie passende Zitationsformat und kopieren Sie es dann in die Zwischenablage, lassen es sich per Mail zusenden oder speichern es als PDF-Datei.

Gewünschter Zitations-Stil:

oder

Bitte prüfen Sie, ob die Zitation formal korrekt ist, bevor Sie sie in einer Arbeit verwenden. Benutzen Sie gegebenenfalls den "Exportieren"-Dialog, wenn Sie ein Literaturverwaltungsprogramm verwenden und die Zitat-Angaben selbst formatieren wollen.

Footprint of publication selection bias on meta-analyses in medicine, environmental sciences, psychology, and economics.