Probability Foundations - Probability Theory (MATH 3200)

1Experiments, Sample Spaces, and Events¶

📍 Where We’re Going

What we’ll learn: The three foundational objects of probability — the experiment, the sample space, and the event — and how to express them precisely using set notation.

Why it matters: Before you can compute the probability of anything, you need to define what the possible outcomes are. A cybersecurity team monitoring network packets, a clinical trial testing a drug, a quality inspector checking parts — each is an experiment with a well-defined set of possible results. Getting the sample space right is the essential first step.

The real-life question: A cybersecurity team classifies each packet as normal (N), suspicious (S), or malicious (M). If they monitor three consecutive packets, how many different outcome sequences are possible? Which sequences represent “at least one malicious packet”?

Probability theory begins with a simple setup: something happens, and we observe the result.

Example 1 (EXAMPLE 2.1: Network Packet Monitoring)

The Problem.

A cybersecurity system classifies each incoming packet as Normal (N), Suspicious (S), or Malicious (M). Three consecutive packets are monitored.

(a) List the sample space. (b) List the event $A$ = “at least one malicious packet.” (c) List the event $B$ = “all three packets have the same classification.” (d) How many simple events are in $S$ ?

Solution.

(a) Each packet has 3 classifications, and there are 3 packets:

S = \{NNN, NNS, NNM, NSN, NSS, NSM, NMN, NMS, NMM,

SNN, SNS, SNM, SSN, SSS, SSM, SMN, SMS, SMM,

MNN, MNS, MNM, MSN, MSS, MSM, MMN, MMS, MMM\}

(b) $A$ contains all outcomes with at least one M. It’s easier to find the complement: outcomes with no M are from $\{N, S\}^3$ , giving $2^3 = 8$ outcomes. So $|A| = 27 - 8 = 19$ .

(c) $B = \{NNN, SSS, MMM\}$ — 3 outcomes.

(d) $|S| = 3^3 = 27$ simple events.

In R.

classifications <- c("N", "S", "M")
S <- expand.grid(P1 = classifications, P2 = classifications, P3 = classifications)
nrow(S)  # 27

# Event A: at least one M
A <- S[S$P1 == "M" | S$P2 == "M" | S$P3 == "M", ]
nrow(A)  # 19

1.1Set Operations¶

Events are sets, and we manipulate them using three operations:

Example 2 (EXAMPLE 2.2: Combining Defect Events)

The Problem.

A gasket can be defective in thickness ( $T$ ) or diameter ( $D$ ). From historical data: $P(T) = 0.08$ , $P(D) = 0.05$ , $P(T \cap D) = 0.02$ .

Find: (a) $P(T \cup D)$ , (b) $P(\overline{T \cup D})$ , (c) $P(T \cap \bar{D})$ .

Solution.

(a) $P(T \cup D) = P(T) + P(D) - P(T \cap D) = 0.08 + 0.05 - 0.02 = 0.11$

(b) $P(\overline{T \cup D}) = 1 - 0.11 = 0.89$ (probability of no defect)

(c) $P(T \cap \bar{D}) = P(T) - P(T \cap D) = 0.08 - 0.02 = 0.06$ (thickness defect only)

✏️ Section 2.1 Homework

2.1.1. A traffic light can be Red (R), Yellow (Y), or Green (G). Two consecutive lights are observed. (a) List $S$ . (b) List $A$ = “at least one red.” (c) List $B$ = “both same color.” (d) Find $A \cap B$ and $A \cup B$ . (e) Are $A$ and $B$ mutually exclusive?

2.1.2. Let $S = \{1,2,...,10\}$ , $A = \{2,4,6,8,10\}$ , $B = \{1,2,3,4,5\}$ . Find $A \cup B$ , $A \cap B$ , $\bar{A}$ , and verify De Morgan’s Laws for both $\overline{A \cup B}$ and $\overline{A \cap B}$ .

2.1.3. A circuit board can fail in soldering ( $S$ ) or wiring ( $W$ ). Given $P(S) = 0.12$ , $P(W) = 0.07$ , $P(S \cap W) = 0.03$ , find $P(S \cup W)$ , $P(\overline{S \cup W})$ , $P(S \cap \bar{W})$ , and $P(\bar{S} \cap W)$ .

2The Axioms of Probability¶

From these three axioms, we derive every other rule:

Proof 3

PROOF of (d): General Addition Rule

Strategy: Decompose A ∪ B into three mutually exclusive “slices” — think of the three non-overlapping regions in a Venn diagram — then apply Axiom 3.

Any outcome in $A \cup B$ falls into exactly one of three categories: (i) in $A$ but not $B$ , (ii) in both $A$ and $B$ , or (iii) in $B$ but not $A$ . In set notation:

A \cup B = (A \cap \bar{B}) \cup (A \cap B) \cup (\bar{A} \cap B)

These three pieces are pairwise disjoint (an outcome can’t be in two of them simultaneously), so by Axiom 3:

P(A \cup B) = P(A \cap \bar{B}) + P(A \cap B) + P(\bar{A} \cap B) \tag{*}

Now we express the first and third terms using quantities we know. Since $A = (A \cap B) \cup (A \cap \bar{B})$ and these two pieces are disjoint:

P(A) = P(A \cap B) + P(A \cap \bar{B}) \quad \Longrightarrow \quad P(A \cap \bar{B}) = P(A) - P(A \cap B)

Similarly, $P(\bar{A} \cap B) = P(B) - P(A \cap B)$ .

Substituting into ( $*$ ):

P(A \cup B) = [P(A) - P(A \cap B)] + P(A \cap B) + [P(B) - P(A \cap B)] = P(A) + P(B) - P(A \cap B)

🔑 What We Learned — Section 2.2

The math: Three axioms (non-negativity, normalization, additivity for disjoint events) generate all of probability theory. The complement rule and general addition rule are the two most-used consequences.

In real life: The weather app question is now decidable. The axioms give you a consistency checker: any probability assignment that violates $0 \leq P(A) \leq 1$ , or $P(A) + P(\bar{A}) \neq 1$ , or double-counts overlapping events, is wrong. A student who claims $P(A) = 0.7$ , $P(B) = 0.6$ , $P(A \cap B) = 0.1$ has made an error — because $P(A \cup B) = 1.2 > 1$ , violating Theorem 2.2(c).

Coming up: The axioms tell us what rules probabilities obey. The sample-point method tells us how to compute them.

✏️ Section 2.2 Homework

2.2.1. Prove $P(\emptyset) = 0$ using $S = S \cup \emptyset$ and the axioms.

2.2.2. Given $P(A) = 0.6$ , $P(B) = 0.4$ , $P(A \cap B) = 0.2$ : find $P(A \cup B)$ , $P(\bar{A})$ , $P(\bar{A} \cap \bar{B})$ , $P(A \cap \bar{B})$ .

2.2.3. Events $C$ and $D$ are mutually exclusive with $P(C) = 0.35$ , $P(D) = 0.25$ . Find $P(C \cup D)$ and $P(\overline{C \cup D})$ .

2.2.4. Show that $P(A) = 0.7$ , $P(B) = 0.6$ , $P(A \cap B) = 0.1$ is inconsistent.

2.2.5. Prove: if $A \subset B$ , then $P(A) \leq P(B)$ . (Hint: $B = A \cup (B \cap \bar{A})$ .)

3The Sample-Point Method¶

Example 3 (EXAMPLE 2.3: Sum of Two Dice)

Solution.

$S = \{(i,j) : i,j \in \{1,...,6\}\}$ , so $|S| = 36$ . All equally likely.

$A = \text{"sum = 7"} = \{(1,6),(2,5),(3,4),(4,3),(5,2),(6,1)\}$ , so $|A| = 6$ .

P(A) = \frac{6}{36} = \frac{1}{6} \approx 0.167

set.seed(123)
die1 <- sample(1:6, 100000, replace = TRUE)
die2 <- sample(1:6, 100000, replace = TRUE)
mean(die1 + die2 == 7)  # ≈ 0.1667

4Counting Tools: The mn Rule¶

Example 4 (EXAMPLE 2.4: License Plates)

Solution.

(a) With repetition: We identify each character position as a stage:

Stage	Description	Choices
1	First letter	26
2	Second letter	26
3	Third letter	26
4	First digit	10
5	Second digit	10
6	Third digit	10
7	Fourth digit	10

By the mn rule: $26 \times 26 \times 26 \times 10 \times 10 \times 10 \times 10 = 26^3 \times 10^4 = 175{,}760{,}000$

(b) Without repetition: Each letter/digit used is no longer available for the next stage:

Letters: $26 \times 25 \times 24 = 15{,}600$
Digits: $10 \times 9 \times 8 \times 7 = 5{,}040$
Total: $15{,}600 \times 5{,}040 = 78{,}624{,}000$

5Permutations¶

Why it works: Position 1 has $n$ choices, position 2 has $n-1$ , ..., position $r$ has $n-r+1$ . By the mn rule: $n(n-1)\cdots(n-r+1)$ .

Example 5 (EXAMPLE 2.5: DJ Set List)

Solution.

_{20}P_5 = 20 \times 19 \times 18 \times 17 \times 16 = 1{,}860{,}480

Nearly 1.9 million set lists from just 20 songs.

prod(20:16)  # 1860480

Example 6 (EXAMPLE 2.6: Arranging Race Finishers)

The Problem.

A track meet has 12 runners in a race. In how many ways can the gold, silver, and bronze medals be awarded (no ties)?

Solution.

We are selecting 3 runners from 12 and the order matters — gold is different from silver. This is a permutation:

_{12}P_3 = \frac{12!}{9!} = 12 \times 11 \times 10 = 1{,}320

Note: if we only cared about which 3 runners medal (not who gets gold vs. silver vs. bronze), we’d use the combination $\binom{12}{3} = 220$ , which is exactly $_{12}P_3 / 3! = 1320/6 = 220$ . The factor of $3! = 6$ is the number of ways to arrange 3 people in order — the permutation counts each of these arrangements separately, the combination doesn’t.

5.1When to Use Permutations vs. Combinations¶

This is the single most common source of counting errors. Use this checklist:

6Combinations and Partitions¶

Why it works: $_nP_r$ counts ordered arrangements. Each group of $r$ objects appears in $r!$ different orders. Dividing removes the redundancy: $\binom{n}{r} = \frac{_nP_r}{r!}$ .

Example 7 (EXAMPLE 2.7: Quality Control Inspection)

The Problem.

An inspector selects 4 items from a shipment of 20 (5 defective, 15 good). What’s the probability exactly 2 are defective?

Solution.

$|S| = \binom{20}{4} = 4{,}845$ . Choose 2 defective from 5: $\binom{5}{2} = 10$ . Choose 2 good from 15: $\binom{15}{2} = 105$ .

P(\text{exactly 2 defective}) = \frac{\binom{5}{2}\binom{15}{2}}{\binom{20}{4}} = \frac{10 \times 105}{4{,}845} = \frac{1{,}050}{4{,}845} \approx 0.2167

choose(5, 2) * choose(15, 2) / choose(20, 4)  # 0.2167

Example 8 (EXAMPLE 2.8: Clinical Trial Assignment)

Solution.

Assign 18 patients to 3 groups of 6:

\frac{18!}{6! \cdot 6! \cdot 6!} = 17{,}153{,}136

factorial(18) / (factorial(6))^3  # 17153136

7Putting It All Together: The Birthday Problem¶

The complement strategy: Computing “at least one match” directly is a nightmare. Instead:

P(\text{at least one match}) = 1 - P(\text{all birthdays different})

Assumptions: 365 days, all equally likely, independence.

Computing P(\text{all n birthdays different}):

Person 1: any day ( $365/365$ ). Person 2 must avoid Person 1’s birthday: $364/365$ . Person 3: $363/365$ . Person $k$ : $(365-k+1)/365$ .

We can multiply these fractions because we assumed birthdays are independent — knowing Person 1’s birthday doesn’t affect the probability of Person 2’s birthday. (This is the mn rule in action: each person’s birthday is a “stage” with outcomes that don’t depend on previous stages.)

P(\text{no match}) = \frac{365}{365} \cdot \frac{364}{365} \cdots \frac{365-n+1}{365} = \frac{_{365}P_n}{365^n}

\boxed{P(\text{at least one match among } n \text{ people}) = 1 - \frac{_{365}P_n}{365^n}}

$n$	$P(\text{match})$
10	0.117
23	0.507
30	0.706
50	0.970
57	0.990

birthday_prob <- function(n) {
  1 - prod((365:(365 - n + 1)) / 365)
}
n_vals <- 1:80
plot(n_vals, sapply(n_vals, birthday_prob), type = "l", lwd = 2,
     col = "steelblue", xlab = "People", ylab = "P(match)",
     main = "The Birthday Problem")
abline(h = 0.5, v = 23, lty = 2, col = "red")

Why so low? With $n$ people, there are $\binom{n}{2}$ pairs. At $n = 23$ : $\binom{23}{2} = 253$ pairs — each a potential match. The quadratic growth of pairs compensates for the low probability of any single match.

Chapter 2 Summary

Experiment / Sample Space / Event: An experiment produces outcomes in $S$ . Events are subsets of $S$ .
Set operations: $A \cup B$ (or), $A \cap B$ (and), $\bar{A}$ (not). De Morgan’s Laws: $\overline{A \cup B} = \bar{A} \cap \bar{B}$ .
Three axioms: $P(A) \geq 0$ ; $P(S) = 1$ ; disjoint events add.
Complement rule: $P(\bar{A}) = 1 - P(A)$ . Addition rule: $P(A \cup B) = P(A) + P(B) - P(A \cap B)$ .
Sample-point method: $P(A) = |A|/|S|$ when equally likely.
mn rule: stages multiply: $n_1 \times n_2 \times \cdots \times n_k$ .
Permutations: $_nP_r = n!/(n-r)!$ (order matters).
Combinations: $\binom{n}{r} = n!/[r!(n-r)!]$ (order doesn’t matter).
Partitions: $n!/(n_1! \cdots n_k!)$ for dividing into groups.
Birthday Problem: $1 - _{365}P_n / 365^n$ . Only 23 people for 50%.

Think About It

Every probability calculation in this chapter assumed we knew the sample space and the probabilities of each outcome. But here’s the uncomfortable truth: in real life, you almost never know the true probabilities.

When we said the die was “fair,” we assumed $P(i) = 1/6$ . When we said birthdays were “equally likely,” we assumed a uniform distribution over 365 days. These are models — simplifications of reality. Real dice are slightly unbalanced. Real birthdays cluster in September.

How sensitive are our answers to these assumptions? If birthdays are not equally likely, does the Birthday Problem answer change? (Yes — it actually gets more likely to have a match.) If the die is slightly loaded, does $P(\text{sum} = 7)$ change? (Yes, but by how much?) Learning to question your model is as important as learning to compute within it.

8Chapter 2 Refresh Homework¶

These problems integrate material across all sections of Chapter 2.

R2.1. A company selects a team of 3 from 10 engineers, then designates one as leader. (a) How many teams? (b) How many leaders per team? (c) Total (team + leader) outcomes? (d) Verify by computing: pick leader first, then 2 others.

R2.2. An alarm system has 3 sensors, each detecting an intruder (D) 90% of the time or failing (F) 10%. (a) List $S$ . (b) Are outcomes equally likely? (c) List $A$ = “at least 2 detect.” (d) Compute $P(A)$ by assigning probabilities ( $P(DDF) = 0.9^2 \times 0.1$ ). (e) Verify using the complement.

R2.3. A club has 9 seniors and 6 juniors. A committee of 4 is formed at random. Find: (a) total committees, (b) committees with exactly 2 seniors and 2 juniors, (c) $P(\text{exactly 2 seniors, 2 juniors})$ , (d) $P(\text{at least 1 junior})$ using the complement.

R2.4. Passwords are 8 characters (26 lowercase + 26 uppercase + 10 digits). (a) Total passwords? (b) Lowercase-only passwords? (c) $P(\text{random password is all lowercase})$ ? (d) Why does this matter for security?

R2.5. Use R to compute the exact birthday probability for $n = 40$ , then simulate: generate 40 random birthdays 100,000 times and count matches. Compare.

R2.6. Three events satisfy: $P(A) = 0.4$ , $P(B) = 0.3$ , $P(C) = 0.2$ , $P(A \cap B) = 0.15$ , $P(A \cap C) = 0.08$ , $P(B \cap C) = 0.06$ , $P(A \cap B \cap C) = 0.02$ . Find $P(A \cup B \cup C)$ using inclusion-exclusion, and $P(\overline{A \cup B \cup C})$ .

9Chapter 2 Quiz¶

Test your understanding. You need 80% (12/15) to earn the badge.