The Law of Large Numbers
“The law of large numbers: one trial is luck, but the average of many is law.”
The formula
x̄ₙ = (X₁ + X₂ + ⋯ + Xₙ) / n → μ (n → ∞)How to read it: the average of values from repeating the same experiment many times approaches the theoretical expected value μ as the number of trials grows
- Xᵢ
- — the outcome of the i-th trial
- x̄ₙ
- — the sample mean of n trials
- n
- — the number of trials—the larger, the steadier
- μ
- — the theoretical expected value (the true mean)
The hook
Ten coin flips can be wild, but flip ten thousand times and the fraction of heads homes in on 0.5—chance tamed into statistics.
In plain words
The law of large numbers says that if you repeat the same random experiment many times, the average of the results gets closer and closer to the theoretical expected value μ. Individual outcomes stay unpredictable, but the average becomes predictable.
The intuition
A single random event fluctuates. But stack thousands of them and average, and the ups cancel the downs, leaving only the true value. As the sample grows, the influence of any one lucky event fades like 1/n—so the average 'forgets luck' and settles onto the expected value.
How it's built
Two pieces—the sample mean x̄ₙ and the true mean μ. The law says their gap goes to 0 as n grows. Why? Because the wobble (standard deviation) of the sample mean shrinks like σ/√n. Multiply n by 100 and the wobble shrinks by 10. Flip forever and the wobble vanishes, pinning the average to μ.
Example
A fair coin over 10 flips might land 7 heads (ratio 0.7)—not unusual. But over 10,000 flips heads cluster near 5,000 (ratio ≈ 0.500). Same for a die—the average of many rolls converges to (1+2+3+4+5+6)/6 = 3.5.
Common mistake
Beware the gambler's fallacy. Five heads in a row does not make tails 'due'. Each flip is still 50:50. The ratio settles to 0.5 not because the past is 'compensated', but because the vast number of future trials 'dilutes' the early imbalance.
Where it's used
Insurance (loss rates over many policies are predictable), a casino's house edge, opinion polls, Monte Carlo simulation, quality control, every statistical estimate that leans on large samples—used wherever the crowd is predictable even when the individual is not.
Where it came from
Jacob Bernoulli proved it in his posthumous 1713 Ars Conjectandi and called it his 'golden theorem'. It was the first time mathematics proved the order hidden inside chance.
Prerequisites
Quick check
A fair coin has just landed heads five times in a row. What is the probability the next flip is heads?
- much less than 0.5
- 0.5✓
- much greater than 0.5
- 0
Practice
As a fair die is rolled many times, what value does the average roll converge to?
Answer: 3.5
- expected value = (1+2+3+4+5+6)/6
- = 21/6 = 3.5
Takeaway: The sample mean converges to the expected value μ.
Over very many flips of a fair coin, what value does the fraction of heads converge to?
Answer: 0.5
- the probability of heads is 1/2
- so the fraction converges to 0.5
Takeaway: The long-run fraction equals the event's probability.
After four heads in a row, find the probability the next flip of a fair coin is heads.
Answer: 0.5
- each flip is independent—unaffected by the past
- the probability is still 0.5
Takeaway: The gambler's fallacy—a coin has no memory.
Explain with the law of large numbers why a casino profits in the long run even though individual big winners appear.
Answer: undefined
- each game has a small expected value in the house's favor (the house edge)
- one round fluctuates—a player may win big
- over millions of rounds the average converges to the house edge, so the casino wins
Takeaway: A small edge times large numbers equals sure profit.