Uploaded November 2018 | Updated September 2026, 6 hours ago
P-hacking involves the manipulation of data to find a statistically significant result. Many claim that p-hacking is a problem in science, especially in the medical and social sciences, while others deny this. The problem with p-hacking is usually articulated from a frequentist perspective. In this presentation we articulate the epistemic peril of p-hacking using Bayesian confirmation theory and model selection theory, which we then draw on to explain the arguments on both sides of the debate. This requires a novel understanding of Bayesianism, since a standard criticism of Bayesian confirmation theory is that it cannot accommodate the influence of biased methods. A methodological device widely used to mitigate the peril of p- hacking is a pre-analysis plan. Some say that following a pre-analysis plan is epistemically meritorious while others deny this, and in practice pre-analysis plans are often violated. We use the formal groundwork developed, to resolve this debate, offering a modest defence of the use of pre-analysis plans. In the longer run our ambition is to use this approach to make sense of scenarios in which scientists depart from pre-analysis plans.
Jacob Stegenga, University of Cambridge
Rotman 2018 Annual Conference: Understanding Replication Across the Sciences
October 13, 2018
Visit the Rotman website for more information on applications, events, project descriptions, and openings. rotman.uwo.ca
Follow The Rotman Institute on Twitter: twitter.com/rotmanphilo
Like The Rotman Institute on Facebook: facebook.com/rotmanphilosophy
Subscribe to our channel: youtube.com/user/rotmanphilosophy
P-hacking involves the manipulation of data to find a statistically significant result. Many claim that p-hacking is a problem in science, especially in the medical and social sciences, while others deny this. The problem with p-hacking is usually articulated from a frequentist perspective. In this presentation we articulate the epistemic peril of p-hacking using Bayesian confirmation theory and model selection theory, which we then draw on to explain the arguments on both sides of the debate. This requires a novel understanding of Bayesianism, since a standard criticism of Bayesian confirmation theory is that it cannot accommodate the influence of biased methods. A methodological device widely used to mitigate the peril of p- hacking is a pre-analysis plan. Some say that following a pre-analysis plan is epistemically meritorious while others deny this, and in practice pre-analysis plans are often violated. We use the formal groundwork developed, to resolve this debate, offering a modest defence of the use of pre-analysis plans. In the longer run our ambition is to use this approach to make sense of scenarios in which scientists depart from pre-analysis plans.
Jacob Stegenga, University of Cambridge
Rotman 2018 Annual Conference: Understanding Replication Across the Sciences
October 13, 2018
Visit the Rotman website for more information on applications, events, project descriptions, and openings. rotman.uwo.ca
Follow The Rotman Institute on Twitter: twitter.com/rotmanphilo
Like The Rotman Institute on Facebook: facebook.com/rotmanphilosophy
Subscribe to our channel: youtube.com/user/rotmanphilosophy
