About This Course

This course takes you from statistical fundamentals to advanced, industrial-scale experimentation practices, giving you both the technical foundation and practical judgment needed to design, evaluate, and communicate trustworthy A/B tests in real product environments.

You’ll build a strong understanding of probability, sampling, distributions, hypothesis testing, and the statistical methods commonly used in product experimentation. From there, you’ll learn how to design reliable experiments, select appropriate metrics, determine sample sizes, assess randomization assumptions, and identify common sources of bias and invalid conclusions.

The course then moves into advanced experimentation topics, including multiple testing, sequential testing, Sample Ratio Mismatch (SRM), AA test validation, CUPED and variance reduction, segmentation bias, and experiment debugging. You’ll also explore Bayesian experimentation, causal inference methods such as Difference-in-Differences, Regression Discontinuity, and Synthetic Controls, as well as simulation techniques for understanding experiment reliability, Type S and Type M errors, novelty effects, and concurrent experiments.

Throughout the roadmap, emphasis is placed not only on statistical correctness but also on decision-making under uncertainty. You’ll practice explaining technical findings to non-technical stakeholders, defending metric choices, discussing assumptions and tradeoffs, and communicating results clearly.

The course concludes with an end-to-end portfolio capstone in which you’ll design an experiment, simulate realistic data, perform frequentist and Bayesian analyses, and produce an executive recommendation memo. Optional advanced modules allow you to deepen your expertise in Bayesian methods, causal inference, and experiment reliability through simulation.