March 2019

The Limits of p-Hacking: a Thought Experiment

Andrew Y. Chen

Abstract:

Suppose that asset pricing factors are just p-hacked noise. How much p-hacking is required to produce the 300 factors documented by academics? I show that, if 10,000 academics generate 1 factor every minute, it takes 15 million years of p-hacking. This absurd conclusion comes from applying the p-hacking theory to published data. To fit the fat right tail of published t-stats, the p-hacking theory requires that the probability of publishing t-stats < 6.0 is infinitesimal. Thus it takes a ridiculous amount of p-hacking to publish a single t-stat. These results show that p-hacking alone cannot explain the factor zoo.

Accessible materials (.zip)

Keywords: Stock return anomalies, multiple testing, p-hacking, publication bias

DOI: https://doi.org/10.17016/FEDS.2019.016

PDF: Full Paper

Back to Top
Last Update: January 09, 2020