Statistics

Statistics gives us the language for reasoning about uncertainty, for example, what outcomes are possible, how likely each one is, and what a sample of data can tell us about the process that produced it. It underpins the other notes on this site. Machine learning models are probability distributions over targets, reinforcement learning agents maximize expected returns, and experiments are judged by how their results would vary from sample to sample.

These notes are a concise reference for the core ideas. The main path starts with probability, moves on to estimating quantities from samples and fitting distributions to data, and ends with the tools for judging whether a result is real, such as confidence intervals and hypothesis tests. A second track covers the design and analysis of experiments with many conditions or factors. Each page gives the definitions and key formulas and, where it helps, something to play with.

Start with chapter 1 →Try the central limit theorem simulator

Learning path

Chapters 1 to 4 cover the probability that the ML and RL notes assume, chapters 5 to 9 cover estimation, and chapters 10 to 13 cover statistical inference. Each chapter builds on the previous one, so reading them in order works best. Chapters marked Interactive have examples you can play with, and chapters marked Planned are not written yet.

Probability basics

The language for describing uncertain outcomes.

1. 1
   Chapter 1: Random VariablesInteractive

   Sets up sample spaces, events and the axioms of probability, then defines a random variable as a function that turns each outcome into a number.

   • Sample space
   • Probability axioms
   • Discrete vs continuous
   • CDF
2. 2
   Chapter 2: Joint and Conditional ProbabilityPlanned

   Describes how two random quantities relate, from joint and conditional probabilities to Bayes' rule and independence.

   • Marginals
   • Bayes' rule
   • Independence
   • i.i.d.
3. 3
   Chapter 3: Probability DistributionsInteractive

   Covers the distribution families that appear throughout the ML and RL notes, with sliders to see how each parameter changes the shape.

   • PMF & PDF
   • Binomial
   • Categorical
   • Normal
   • Beta
4. 4
   Chapter 4: Expectation, Variance, and Covariance

   Summarizes a distribution by its center and spread, and two variables by how they move together, ending with the conditional expectation behind value functions.

   • Linearity
   • Variance
   • Covariance
   • Conditional expectation

From samples to populations

What a sample can tell you about the process that produced it.

1. 5
   Chapter 5: Sampling Distributions and the CLTInteractive

   Treats a statistic computed from a random sample as a random variable, and shows why the sample mean concentrates around the true mean and looks normal.

   • Standard error
   • Law of large numbers
   • Central limit theorem
2. 6
   Chapter 6: Monte Carlo EstimationPlanned

   Estimates expectations by averaging samples, updates the average one sample at a time, and reweights samples drawn from the wrong distribution.

   • Incremental mean
   • Step sizes
   • Importance sampling

Fitting models to data

Choose the parameters of a distribution from observed data.

1. 7
   Chapter 7: Estimators and Maximum LikelihoodPlanned

   Judges estimators by their bias and variance, and picks parameters that make the observed data most probable.

   • Bias & variance
   • MSE decomposition
   • Likelihood
   • MLE
2. 8
   Chapter 8: Bayesian InferencePlanned

   Treats parameters as random variables, updating a prior belief into a posterior as data arrive.

   • Prior & posterior
   • Conjugacy
   • MAP
   • Thompson sampling
3. 9
   Chapter 9: Entropy, Cross-Entropy, and KL DivergencePlanned

   Measures the uncertainty of a distribution and the gap between two of them, which is where the cross-entropy loss comes from.

   • Entropy
   • Cross-entropy
   • KL divergence
   • MLE as KL

Statistical inference

Quantify how sure you can be about a conclusion drawn from data.

1. 10
   Chapter 10: Confidence IntervalsPlanned

   Builds ranges that contain the true parameter in a known fraction of repeated samples, and shows how to report results across random seeds.

   • Coverage
   • t intervals
   • Bootstrap
   • Learning curves
2. 11
   Chapter 11: Hypothesis TestingPlanned

   Decides whether data are surprising under a default assumption, and what the two kinds of error cost.

   • Null hypothesis
   • p-values
   • Type I & II errors
   • Power
3. 12
   Chapter 12: Comparing Two GroupsPlanned

   Tests whether two groups differ, such as two agents trained over several seeds or two versions of a web page.

   • Paired t-test
   • Welch's t-test
   • Proportions
   • Permutation tests
4. 13
   Chapter 13: Power, Sample Size, and Multiple ComparisonsPlanned

   Plans how much data an experiment needs, and keeps false discoveries in check when many comparisons are made at once.

   • Sample size
   • How many seeds
   • Bonferroni & Holm
   • False discovery rate

Design of experiments

This track covers experiments that compare more than two conditions or vary several factors at once, and the analysis of variance (ANOVA) used to analyze them. It follows Design and Analysis of Experiments by Douglas Montgomery, and each card shows the book chapter it covers. It assumes the inference chapters of the main path, especially chapters 11 to 13.

Design and analysis of experiments

Plan experiments with many conditions or factors, and analyze them with ANOVA.

1. 1
   Chapter 1: Principles of Experimental DesignPlannedMontgomery Ch. 1

   Sets up the vocabulary of factors, levels and experimental units, and explains why randomization, replication and blocking matter.

   • Experiments vs observation
   • Randomization
   • Replication
   • Blocking
2. 2
   Chapter 2: One-Way ANOVAPlannedMontgomery Ch. 3

   Compares the means of several groups with one test by splitting the total variation into parts between and within groups.

   • Sum of squares
   • F-test
   • Residual checks
   • ANOVA as regression
3. 3
   Chapter 3: Randomized Block DesignsPlannedMontgomery Ch. 4

   Removes the effect of a nuisance factor, such as the environment an agent is tested in, by comparing treatments within blocks.

   • Nuisance factors
   • RCBD
   • Latin squares
4. 4
   Chapter 4: Factorial DesignsPlannedMontgomery Ch. 5–6

   Varies several factors at once to estimate their main effects and how they interact, as in a sweep over two hyperparameters.

   • Main effects
   • Interactions
   • Two-way ANOVA
   • 2^k designs

Where this shows up

• Put it to useProbabilistic Interpretation of ML

  Maximum likelihood turns a distribution over targets into a loss function.

• NotesMachine Learning

  From linear regression to kernel methods, with probability running through the middle chapters.

• NotesReinforcement Learning

  Value functions are expected returns, so expectations show up from the second chapter on.

References

• Blitzstein, J. K., & Hwang, J. (2019)
  Introduction to Probability (2nd ed.). CRC Press.
• Ross, S. (2023)
  A First Course in Probability (10th ed.). Pearson.
• Wasserman, L. (2004)
  All of Statistics: A Concise Course in Statistical Inference. Springer.
• Cover, T. M., & Thomas, J. A. (2006)
  Elements of Information Theory (2nd ed.). Wiley.
• Montgomery, D. C. (2019)
  Design and Analysis of Experiments (10th ed.). Wiley.
• Patterson, A., Neumann, S., White, M., & White, A. (2024)
  Empirical Design in Reinforcement Learning. Journal of Machine Learning Research, 25.