Conditional Probability & Bayes

Medium4–5 hrsBayesConditionalIndependence

Overview

Bayes' theorem is the machine for updating beliefs. The key insight: P(H|E) is not P(E|H). A test that is 99% accurate (sensitivity) does NOT mean a positive result means 99% chance of disease — you must account for the base rate (prior). The 2×2 table approach makes this visual and nearly mechanical.

How to Recognize

→'Given that X happened, what is the probability of Y?'
→Disease testing, false positives, Monty Hall–style problems
→Information is revealed and you must update a prior probability

Step-by-Step Approach

1.Step 1: Draw a probability tree or 2×2 table — always visualize first
2.Step 2: Write out P(A|B) = P(A∩B)/P(B) explicitly
3.Step 3: Use Bayes' theorem: P(H|E) = P(E|H)·P(H) / P(E)
4.Step 4: P(E) via law of total probability: P(E) = Σ P(E|Hᵢ)·P(Hᵢ)
5.Step 5: Check independence: P(A∩B) = P(A)·P(B) only if independent

Key Formulas

P(H|E) = \frac{P(E|H)\,P(H)}{P(E|H)\,P(H)+P(E|\bar{H})\,P(\bar{H})}

02Total Probability:

P(B) = \sum_i P(B|A_i)\,P(A_i)

03Independence:

P(A \cap B) = P(A)\cdot P(B)

04Monty Hall: switching wins with probability

\frac{2}{3}

Worked Examples

0Quick Example

Test with 1% false-positive rate, 99% sensitivity, 0.1% disease prevalence. P(disease|positive) = (0.99·0.001)/(0.99·0.001+0.01·0.999) ≈ 9%.

Problem

A disease affects 1 in 1000 people. A test has 99% sensitivity (true positive rate) and 1% false positive rate. You test positive. What is the probability you have the disease?

Solution

Set up 2×2 table with 100,000 people: 100 have disease (0.1%), 99,900 don't.

Of 100 with disease: 99 test positive (99% sensitivity), 1 tests negative.

Of 99,900 without disease: 999 test positive (1% false positive rate), 98,901 test negative.

Total positives = 99 + 999 = 1,098. True positives = 99.

P(disease | positive) = 99/1098 ≈ 9.0%.

Answer

$P(\text{disease}|\text{positive}) \approx 9\%$ . Despite 99% accuracy, rare diseases mean most positives are false. The base rate dominates.

Common Mistakes

!
Confusing P(E|H) with P(H|E). The test is 99% accurate = P(positive|disease) = 0.99. But P(disease|positive) also depends on P(disease).
!
Ignoring the base rate. Even an extremely accurate test gives poor positive predictive value for rare diseases.
!
Assuming Monty's door choice is random — it isn't. He always reveals a goat, making his action informative.
!
For conditional probability, always check: are these events actually independent? P(A|B) = P(A) only if A and B are independent.

Practice Problems

Click "Show Answer" to reveal

A factory has two machines. Machine A makes 60% of products, Machine B makes 40%. Machine A has 3% defect rate; Machine B has 5% defect rate. A product is defective. What is P(it came from Machine B)?

Hint: P(B|defective) = P(defective|B)·P(B) / P(defective).

Flip a fair coin twice. Given the first flip is heads, what is P(both heads)?

Hint: Simple — conditional probability on known first flip.

I roll a fair die and tell you the result is even. What is P(the roll was a 4)?

Hint: Update sample space to only even outcomes.

Back to Study Plan

Card & Combinatorics Problems Birthday & Coupon Collector