Bayes' Theorem
“Don't judge by the evidence's accuracy alone—how rare it was to begin with can flip the answer.”
The formula
P(A|B) = P(B|A)·P(A) / P(B)How to read it: the probability of A after seeing evidence B is the prior of A times how likely B is under A, divided by the total probability of B
- P(A|B)
- — the probability of A after seeing B (the posterior)
- P(A)
- — the probability of A before the evidence (the prior)
- P(B|A)
- — the likelihood of seeing evidence B when A holds
- P(B)
- — the total probability of seeing evidence B
The hook
A 99%-accurate test comes back positive—yet your chance of actually having the disease is only about 17%. Bayes' theorem explains why.
In plain words
Bayes' theorem is the rule for updating a belief with new evidence. Take the probability of A before seeing B (the prior), multiply by how likely B is if A holds, divide by the total chance of B, and you get the probability of A after seeing B (the posterior).
The intuition
The key intuition is the 'base rate'. If a disease is very rare (1%), then even a 5% false-alarm rate among the 99% who are healthy produces far more positives than the true patients. So most positives are 'healthy but wrongly flagged'. Don't look only at the test's accuracy—look also at how rare the thing was to begin with.
How it's built
Three pieces build the posterior: the prior P(A) is your starting belief, the likelihood P(B|A) is how convincing the evidence is, and the denominator P(B) is the total chance of the evidence across all paths. The denominator usually expands as P(B|A)P(A) + P(B|¬A)P(¬A), counting both 'true positives' and 'false positives'.
Example
Disease prevalence P(D)=1%, test sensitivity P(+|D)=99%, false-positive rate P(+|healthy)=5%. Given a positive, the chance of really being sick? P(+) = 0.99·0.01 + 0.05·0.99 = 0.0099 + 0.0495 = 0.0594. So P(D|+) = 0.0099/0.0594 ≈ 0.167, about 17%.
Common mistake
The biggest trap is confusing P(A|B) with P(B|A) (the 'prosecutor's fallacy'). 'Chance of testing positive given the disease' and 'chance of the disease given a positive' are wholly different. The bridge between them is Bayes' theorem—and that bridge always involves the prior.
Where it's used
Spam filters, medical diagnosis, weighing evidence in court, machine learning (naive Bayes classifiers), search and recommendation, testing hypotheses in science—it works wherever beliefs are updated from uncertain evidence.
Where it came from
The idea, left posthumously by the 18th-century Reverend Thomas Bayes, was developed by Laplace. Long controversial, it is today a central framework of statistics and artificial intelligence.
Prerequisites
Quick check
Why is the chance of actually having a very rare disease low even after a highly accurate test is positive?
- because the test is broken
- because the large healthy majority produces many false positives✓
- because the test's sensitivity is low
- because the disease is common
Practice
Given P(A)=0.5, P(B|A)=0.8, P(B|¬A)=0.4, find P(A|B).
Answer: 0.667
- P(B) = 0.5·0.8 + 0.5·0.4 = 0.4 + 0.2 = 0.6
- P(A|B) = 0.4 / 0.6 ≈ 0.667
Takeaway: The denominator sums the evidence probability over all paths.
Prevalence P(D)=0.01, sensitivity P(+|D)=0.99, false positive P(+|healthy)=0.05. Find P(D|+) given a positive.
Answer: 0.167
- P(+) = 0.99·0.01 + 0.05·0.99 = 0.0099 + 0.0495 = 0.0594
- P(D|+) = 0.0099 / 0.0594 ≈ 0.167
Takeaway: For a rare disease even a positive stays low—the power of the base rate.
Factory 1 makes 60% of bulbs (2% defective), Factory 2 makes 40% (5% defective). Find the probability a defective bulb came from Factory 1.
Answer: 0.375
- P(defective) = 0.6·0.02 + 0.4·0.05 = 0.012 + 0.02 = 0.032
- P(Factory 1 | defective) = 0.012 / 0.032 = 0.375
Takeaway: A reverse probability just compares each path's contribution.
A friend says 'the test is 95% accurate, so a positive means 95% chance of disease'. Explain in your notebook why this is wrong.
Answer: undefined
- they confuse P(positive|disease) with P(disease|positive)
- they ignore the prior (prevalence)—if the disease is rare the answer drops sharply
- Bayes' theorem is the bridge between the two conditionals, and it needs the prior
Takeaway: Accuracy is not the posterior—you can't answer without the prior.