Uploaded August 2025 | Updated September 2026, 1 week ago
Angelika Stefan is a Lecturer in Psychological Methods at the University of Liverpool, based in the Department of Psychology’s Institute of Population Health. Her research focuses on the intersection of psychological methods and cognitive psychology, exploring Bayesian statistics, expert-informed models, and how people update beliefs based on data. A committed open‑science advocate, she publishes her materials, code, data, and even teaching resources on OSF and GitHub.
liverpool.ac.uk
In this session, Angelika Stefan unpacks the subtle and often unintentional research decisions known as p-hacking, revealing how they can inflate false positives and weaken scientific credibility.
Drawing on her paper "Big Little Lies", she outlines 12 common p-hacking strategies (outlier exclusion, optional stopping, variable transformation, subgroup analysis, and more) and demonstrates through simulations how each one can distort results. She shows how these practices increase false positives, bias effect sizes, and undermine replicability, regardless of sample size.
Angelika further clarifies the distinction between p-hacking strategies (analytical choices) and p-value reporting strategies (which results are shown), illustrating how both shape the p-curve. She concludes by exploring “possible worlds” of research through simulation, demonstrating that many distinct mixes of true and false effects, along with varying practices, can create identical published patterns.
Key Takeaways:
- P-hacking isn’t outright fraud—but it harms scientific trust.
- A few unplanned decisions can triple false positive rates.
- Bigger sample sizes don’t protect against p-hacking.
- Simulations are powerful tools for testing methodological risks.
- P-curves alone can’t tell the full story.
Angelika Stefan is a Lecturer in Psychological Methods at the University of Liverpool, based in the Department of Psychology’s Institute of Population Health. Her research focuses on the intersection of psychological methods and cognitive psychology, exploring Bayesian statistics, expert-informed models, and how people update beliefs based on data. A committed open‑science advocate, she publishes her materials, code, data, and even teaching resources on OSF and GitHub.
liverpool.ac.uk
In this session, Angelika Stefan unpacks the subtle and often unintentional research decisions known as p-hacking, revealing how they can inflate false positives and weaken scientific credibility.
Drawing on her paper "Big Little Lies", she outlines 12 common p-hacking strategies (outlier exclusion, optional stopping, variable transformation, subgroup analysis, and more) and demonstrates through simulations how each one can distort results. She shows how these practices increase false positives, bias effect sizes, and undermine replicability, regardless of sample size.
Angelika further clarifies the distinction between p-hacking strategies (analytical choices) and p-value reporting strategies (which results are shown), illustrating how both shape the p-curve. She concludes by exploring “possible worlds” of research through simulation, demonstrating that many distinct mixes of true and false effects, along with varying practices, can create identical published patterns.
Key Takeaways:
- P-hacking isn’t outright fraud—but it harms scientific trust.
- A few unplanned decisions can triple false positive rates.
- Bigger sample sizes don’t protect against p-hacking.
- Simulations are powerful tools for testing methodological risks.
- P-curves alone can’t tell the full story.
