Probability Distributions

A probability distribution describes how the values of a random variable are distributed. It gives the probability that a random variable will take on each of its possible values.

Types of Probability Distributions

Discrete Probability Distributions

Include probability mass functions (PMFs) which assign probabilities to discrete outcomes, such that:

$$\begin{gather} 0 \le \mathbb{P}(x) \le 1 \qquad \forall x \\ \mathbb{P}(x=y)=\mathbb{P}(y) \qquad \forall y \\ \sum_X \mathbb{P}(x) = 1 \end{gather}$$

For example:

• Bernoulli Distribution: The Bernoulli distribution describes experiments that have binary outcomes, e.g., result in either a success or a failure.

  • Example: Flipping a coin once, where heads is considered a success.

  • Formula:

    $$\mathbb{P}(x = k) = p^k (1-p)^{1-k} \quad \forall k \in \{0,1\}$$

    Where $p$ is the probability of success, $1-p$ is the probability of failure, and $k$ can be either 0 (failure) or 1 (success).

• Poisson Distribution: The Poisson distribution describes the number of events occurring within a fixed interval of time or space when these events happen independently of each other and at a constant rate.

  • Example: The number of emails you receive in an hour.

  • Formula:

    $$\mathbb{P}(x = k) = \frac{\lambda^k e^{-\lambda}}{k!}$$

    Where $\lambda$ is the average number of events in the interval, $k$ is the number of occurrences, and $e$ is the Euler's constant.

Continuous Probability Distributions

Include probability density functions (PDFs) which describe the likelihood of a continuous random variable falling within a particular range of values, such that:

$$\begin{gather} 0 \le f(x) \\ \mathbb{P}(a \le x \le b) = \int_a^b f(x) \,dx \\ \int_\infty^{-\infty} f(x) \,dx = 1 \end{gather}$$

For example:

• Uniform Distribution: The uniform distribution describes an equal probability for all values within a specified range.

  • Example: The probability of randomly selecting a number between 1 and 10.

  • Formula:

    $$f(x) = \begin{cases} \frac{1}{b-a} & \text{for } a \leq x \leq b \\ 0 & \text{otherwise} \end{cases}$$

    Where $a$ and $b$ are the minimum and maximum values of the range.

• Normal Distribution: The normal distribution, often referred to as the Gaussian distribution or bell curve, describes data that clusters around a mean.

  • Example: Heights of people in a population.

  • Formula:

    $$f(x) = \frac{1}{\sigma \sqrt{2\pi}} e^{-\frac{(x-\mu)^2}{2\sigma^2}}$$

    Where $\mu$ is the mean and $\sigma$ is the standard deviation.

• Student's t-Distribution: The Student's t-distribution is used to estimate population parameters when the sample size is small and the population standard deviation is unknown.

  • Example: The distribution of the sample mean for small sample sizes.

  • Formula:

    $$f(x) = \frac{\Gamma \left(\frac{\nu+1}{2}\right)}{\sqrt{\nu \pi} \Gamma \left(\frac{\nu}{2}\right)} \left(1 + \frac{x^2}{\nu}\right)^{-\frac{\nu+1}{2}}$$

    Where $\nu$ is the degrees of freedom, and $\Gamma$ the gamma function.