Back to Curriculum
Basic•Statistics
Distributions
Normal, Binomial, and Poisson distributions.
Interactive Playground
Initializing Interactive Playground...
Research-Level Deep Dive & Equations
In probability theory, discrete random variables model phenomena where outcomes are distinct, countable, and typically integer-valued. Formally, let be a probability space. A random variable is discrete if its image is countable. The probability distribution of is completely characterized by its probability mass function (PMF) , which satisfies .
To analyze the moments and behaviors of discrete distributions, we define the **Probability Generating Function (PGF)** of a non-negative integer-valued random variable as:
The PGF is analytic for all complex numbers in the closed unit disk . By differentiating and evaluating at , we can systematically recover the factorial moments of :
### Key Discrete Distributions
1. **Bernoulli Distribution ()**: Models a single binary trial with success probability .
* **PMF**: for .
* **PGF**: .
* **Moments**: , and .
2. **Binomial Distribution ()**: Models the total number of successes in independent and identically distributed (i.i.d.) Bernoulli trials. Let where are i.i.d.
* **PMF**: for .
* **PGF**: Due to independence, the PGF of the sum is the product of individual PGFs:
* **Moment Derivations via PGF**:
3. **Poisson Distribution ()**: Models the number of events occurring in a fixed interval of time or space, assuming these events occur with a constant average rate and independently of the time since the last event.
* **PMF**: for .
* **PGF**:
* **Moment Derivations via PGF**:
Thus, a key characteristic of the Poisson distribution is **equidispersion**, where its variance equals its mean.
### Rigorous Derivation of the Poisson Limit Theorem
The Poisson distribution is also known as the "law of rare events," arising as the limiting distribution of a Binomial distribution when the number of trials goes to infinity while the probability of success approaches zero, such that the expected value converges to a constant .
Let where and . We compute the pointwise limit of the PMF for any fixed integer :
Expanding the binomial coefficient:
Regrouping the terms:
Now, we evaluate the limit of each component as :
1. (since ).
2. For any fixed , . The bracketed term is a product of terms (where is fixed), each of which converges to . Therefore, the entire bracketed product converges to .
3. Recall that . Thus, .
4. Since is fixed, .
Combining these limits, we obtain:
This completes the analytical proof that the Binomial PMF converges pointwise to the Poisson PMF.
### Machine Learning and Practical Implications: Overdispersion
In machine learning, modeling discrete counts (e.g., website clicks, word frequencies, or genomic counts) using a Poisson likelihood is standard. However, real-world data often exhibits **overdispersion** () due to latent, unobserved heterogeneity. To model overdispersed count data, we use the **Negative Binomial distribution**, which can be mathematically formulated as a continuous mixture of Poissons:
Integrating out the latent variable yields a Negative Binomial marginal distribution for , which accommodates arbitrary variance scaling.
Key Equations
Test Your Knowledge
Check whether you have mastered this concept with a quick quiz.
Was this lesson helpful?
Your feedback helps us continuously improve the curriculum and interactive visualizations.