Distributions
Normal, Binomial, and Poisson distributions.
For a small example, two sets share mean 5 but have very different variability. Plot both, calculate variance, and notice why one number cannot describe shape.
Distributions
In practice
Most requests are fast but a few are extremely slow. Use quantiles and a right-skewed model instead of relying on average latency, then investigate the tail.
Initializing Interactive Playground...
Experiment notes
Record your experiment
Predict how one control changes the Distributions result.
Implementation
Repair a failing Distributions starter
The starter deliberately follows a tempting but flawed assumption: Assuming normality by default
- Run the starter once and inspect the first wrong value, failed check, or incomplete branch.
- Repair the smallest relevant part of the code. Do not rewrite the whole solution.
- Check the solution, then explain why this correction works: Inspect skew, tails, bounds, and generation mechanisms first.
Check: The browser checks pass and you can name the failure mechanism before opening the explanation.
Failing starter lab
Simulate a sampling distribution
Use seeded Bernoulli samples to see how the standard deviation of sample means changes with sample size.
Run the incomplete starter before opening hints. Use the first wrong output or failed check to form a repair hypothesis.
Knowledge check
10 questions
Explanations after each answer · 80% to pass · Distributions
Chapter 1 of 6 · Core
Discrete Probability Distributions: Foundations and Convergence
In probability theory, discrete random variables model phenomena where outcomes are distinct, countable, and typically integer-valued. Formally, let be a probability space. A random variable is discrete if its image is countable. The probability distribution of is completely characterized by its probability mass function (PMF) , which satisfies .
To analyze the moments and behaviors of discrete distributions, we define the Probability Generating Function (PGF) of a non-negative integer-valued random variable as: The PGF is analytic for all complex numbers in the closed unit disk . By differentiating and evaluating at , we can systematically recover the factorial moments of :
Key Discrete Distributions
- Bernoulli Distribution (): Models a single binary trial with success probability .
- PMF: for .
- PGF: .
- Moments: , and .
- Binomial Distribution (): Models the total number of successes in independent and identically distributed (i.i.d.) Bernoulli trials. Let where are i.i.d.
- PMF: for .
- PGF: Due to independence, the PGF of the sum is the product of individual PGFs:
- Moment Derivations via PGF:
- Poisson Distribution (): Models the number of events occurring in a fixed interval of time or space, assuming these events occur with a constant average rate and independently of the time since the last event.
- PMF: for .
- PGF:
- Moment Derivations via PGF:
Thus, a key characteristic of the Poisson distribution is equidispersion, where its variance equals its mean.
Rigorous Derivation of the Poisson Limit Theorem
The Poisson distribution is also known as the "law of rare events," arising as the limiting distribution of a Binomial distribution when the number of trials goes to infinity while the probability of success approaches zero, such that the expected value converges to a constant .
Let where and . We compute the pointwise limit of the PMF for any fixed integer : Expanding the binomial coefficient: Regrouping the terms: Now, we evaluate the limit of each component as :
- (since ).
- For any fixed , . The bracketed term is a product of terms (where is fixed), each of which converges to . Therefore, the entire bracketed product converges to .
- Recall that . Thus, .
- Since is fixed, .
Combining these limits, we obtain: This completes the analytical proof that the Binomial PMF converges pointwise to the Poisson PMF.
Machine Learning and Practical Implications: Overdispersion
In machine learning, modeling discrete counts (e.g., website clicks, word frequencies, or genomic counts) using a Poisson likelihood is standard. However, real-world data often exhibits overdispersion () due to latent, unobserved heterogeneity. To model overdispersed count data, we use the Negative Binomial distribution, which can be mathematically formulated as a continuous mixture of Poissons: Integrating out the latent variable yields a Negative Binomial marginal distribution for , which accommodates arbitrary variance scaling.
Key equations
Video explanation
The Normal Distribution, Clearly Explained!
StatQuest
Connect a probability distribution to location, spread, density, and the kinds of claims it supports.
References
Primary sources and free technical reading for Distributions.