Uploaded April 2024 | Updated September 2026, 2 weeks ago

---

Most experimentations fail, Kristi Angel shares her expertise on scaling experimentation and avoiding common A/B testing pitfalls. Learn five things that can help boost test velocity, designing impactful experiments, and leveraging knowledge repos. Subscribe to Daliana's newsletter on dalianaliu.com for more on data science and career.

Daliana's Twitter: twitter.com/DalianaLiu
Daliana’s LinkedIn: linkedin.com/in/dalianaliu
Kristi Angel’s LinkedIn: linkedin.com/in/kristiangel

00:00:00 Introduction
00:01:21 Why do most experimentations fail?
00:06:60 Mistakes in choosing metrics
00:10:00 Is revenue a good metric?
00:13:13 Split metrics in three ways
00:15:05 Daliana's story with too many category breakdowns
00:16:54 What makes the best data science team?
00:19:19 Data scientist work in silo vs in a data science team
00:21:10 Building a knowledge center
00:23:35 Example of knowledge center; nuance of experimentations
00:26:04 How many metrics and variants?
00:30:51 How to reduce noise - CUPED
00:32:56 Future of A/B testing
00:38:28 Q&A: Low statistical power