Better Accuracy for Better Science.. Through Random Conclusions

Clintin P. Davis-Stober, Jason Dana, David Kellen, Sara D. McMullin, Wes Bonifay

Research output: Contribution to journalArticlepeer-review

Abstract

Conducting research with human subjects can be difficult because of limited sample sizes and small empirical effects. We demonstrate that this problem can yield patterns of results that are practically indistinguishable from flipping a coin to determine the direction of treatment effects. We use this idea of random conclusions to establish a baseline for interpreting effect-size estimates, in turn producing more stringent thresholds for hypothesis testing and for statistical-power calculations. An examination of recent meta-analyses in psychology, neuroscience, and medicine confirms that, even if all considered effects are real, results involving small effects are indeed indistinguishable from random conclusions.

Original languageEnglish (US)
Pages (from-to)223-243
Number of pages21
JournalPerspectives on Psychological Science
Volume19
Issue number1
DOIs
StatePublished - Jan 2024

Keywords

  • benchmarks
  • estimation
  • hypothesis testing
  • random conclusions
  • t tests

ASJC Scopus subject areas

  • General Psychology

Fingerprint

Dive into the research topics of 'Better Accuracy for Better Science.. Through Random Conclusions'. Together they form a unique fingerprint.

Cite this