The Big Picture — Why Probability? - Probability Theory (MATH 3200)

Learning Objectives

After completing this chapter, you will be able to:

Explain in plain language what probability theory is and why it matters
Identify at least five real-world domains where probability theory is essential
Distinguish between probability theory and introductory statistics
Define and identify: individual, variable, population, parameter, sample, statistic
Classify variables as quantitative (discrete or continuous) or categorical (nominal or ordinal)
Compute the sample mean, sample variance, and sample standard deviation by hand
Construct a relative frequency histogram and explain its connection to probability
Describe the roadmap of this book — what each chapter covers and why it matters

Prerequisites: Calculus I and II (integration and series). No prior statistics course needed.

Content current as of April 2026.

The Opening Problem

The Monty Hall Problem. You’re on a game show. There are three doors. Behind one is a brand-new car; behind the other two are goats. You pick Door 1. The host, Monty Hall, who knows what’s behind every door, opens Door 3 to reveal a goat. He then asks: “Would you like to switch to Door 2?”

Most people say it doesn’t matter — there are two doors left, so it’s 50/50. But that’s wrong. Switching wins the car two-thirds of the time. Staying wins only one-third of the time.

When the puzzle was published in Parade magazine in 1990, roughly 10,000 readers wrote in to say the answer was wrong — including nearly 1,000 people with PhDs. They were all mistaken.

Why does our intuition fail so badly? Because our brains are not wired to reason carefully about uncertainty. Probability theory gives us a set of precise tools to do what intuition cannot. That’s what this entire book is about.

1What Is This Book About?¶

Let’s start with the simplest possible answer:

This book teaches you the mathematical rules that govern randomness.

That sounds abstract, so here’s what it means in practice. By the end of this book, you’ll be able to answer questions like these:

If you flip a coin 100 times, what’s the probability of getting exactly 60 heads?
If a certain gene mutation occurs in 1 out of every 500 people, and a test detects it 98% of the time but also has a 5% false positive rate — and you test positive — what’s the probability you actually have the mutation?
If the time between customer arrivals at a coffee shop follows an exponential distribution with a mean of 3 minutes, what’s the probability that no one arrives for 10 minutes?
If you measure the heights and weights of 200 people, how do you describe the joint behavior of both measurements simultaneously?
If $X$ has a known distribution and you define $Y = X^2$ , what distribution does $Y$ follow?

These aren’t toy questions. They are the foundational calculations behind medical diagnostics, insurance pricing, quality control in manufacturing, machine learning algorithms, financial risk models, sports analytics, and criminal justice.

This book gives you the mathematical machinery to solve all of them.

2Probability Is Everywhere¶

Before we dive into mathematics, let’s look at where probability theory actually shows up in the real world. The point of this section is to convince you that what you’re about to learn is not just a math course requirement — it’s a set of tools that professional adults use every single day.

2.1Medicine¶

A doctor orders a blood test for a rare disease. The test is 99% accurate, meaning it correctly identifies sick patients 99% of the time. But here’s the problem: among healthy patients, it incorrectly returns a positive result 2% of the time (a false positive). The disease affects 1 in 10,000 people. If a patient tests positive, what’s the probability they actually have the disease?

Your gut says “about 99%.” The real answer? About 0.5%. Yes — less than 1%. Understanding why requires Bayes’ Rule, which we’ll derive in Chapter 3.

2.2Sports¶

A basketball player shoots free throws at 80%. In a game, she takes 15 free throws. What’s the probability she makes at least 13? This is a direct application of the binomial distribution (Chapter 4). Sports analytics teams use these exact calculations to evaluate player performance, build lineup models, and set game strategy.

2.3Finance¶

An options trader needs to estimate the probability that a stock price moves more than 5% in either direction over the next month. The price changes on any given day are approximately normally distributed. The probability comes from the normal distribution (Chapter 5), and the joint behavior of multiple assets involves multivariate distributions (Chapter 6).

2.4Weather Forecasting¶

When a meteorologist says “there is a 70% chance of rain tomorrow,” that number comes from probability models. The models combine information from atmospheric measurements, historical patterns, and satellite data using conditional probability and Bayesian updating (Chapter 3).

2.5Manufacturing and Quality Control¶

A factory produces microchips. On average, 3 defects occur per 1,000 chips. If a batch of 500 chips is inspected, what’s the probability of finding more than 5 defects? This is a textbook application of the Poisson distribution (Chapter 4). Quality engineers use this every day to decide whether to ship or hold production lots.

2.6Criminal Justice¶

DNA evidence is presented in court. “The probability of a random match is 1 in 10 million.” The prosecutor argues that the defendant is therefore almost certainly guilty. But this confuses P(evidence | innocent) with P(innocent | evidence) — a mistake so common it has a name: the prosecutor’s fallacy. Correcting this error requires Bayes’ Rule (Chapter 3).

2.7Insurance¶

An insurance company needs to set premiums for car insurance. They model the number of claims a policyholder will file per year using a Poisson distribution, and the dollar amount of each claim using a Gamma distribution (Chapters 4 and 5). The premiums are computed using expected values — a concept we’ll define precisely in Chapter 4.

2.8Machine Learning¶

Every machine learning algorithm — from logistic regression to neural networks — is built on probability. The training process maximizes the probability of the observed data given the model parameters. This is called maximum likelihood estimation, and the probability distributions it relies on are exactly the ones you’ll learn in Chapters 4 and 5.

2.9Gaming and Gambling¶

What’s the probability of being dealt a full house in poker? What’s the expected value of a lottery ticket? These are counting and expected value problems (Chapters 2 and 4). Casinos set their rules specifically so that the expected value favors the house — always.

2.10Epidemiology¶

During a disease outbreak, epidemiologists model the number of daily hospital admissions using count distributions. The number of new cases per day might follow a Poisson or negative binomial distribution (Chapter 4), and the time between cases follows a Gamma or exponential distribution (Chapter 5).

3How This Book Works¶

This book follows a three-step learning pipeline for every major concept:

3.1Step 1: See It¶

Before we define anything formally, we start with a simulation or a concrete example. We’ll run R code that generates random data and lets you observe the pattern before we prove it exists. This is the “ground up” philosophy — you see the phenomenon first, then we build the theory underneath it.

3.2Step 2: Prove It¶

After you’ve seen the pattern, we prove it rigorously. Every theorem in this book comes with a complete proof. We never write “it can be shown that” or “the proof is left as an exercise.” If a result is important enough to state, it’s important enough to prove. And every proof begins with a strategy note — one or two sentences explaining the approach before we dive into the details.

3.3Step 3: Simulate It¶

After the proof, we return to R. We write code that confirms the theoretical result empirically. You’ll see the numbers match. This closes the loop: observation → theory → verification.

4A Quick Review of the Basics¶

📍 Where We’re Going

What we’ll learn: The six building blocks of any statistical investigation and how to classify the types of data we encounter.

Why it matters: A hospital wants to test whether a new surgical technique reduces recovery time. Before they can design the study, collect data, or compute a single probability, they need to answer: Who are we studying? What are we measuring? What’s the bigger group we care about? Every probability model you’ll build in this book rests on these foundational distinctions.

The real-life question: A news headline says “a study of 2,000 adults found that coffee drinkers live 3 years longer.” What’s the population? What’s the sample? What’s the variable? Is the “3 years longer” a parameter or a statistic? By the end of this section, you’ll be able to dissect any study like this in 30 seconds.

If you’ve taken an introductory statistics course, this section will be a refresher. If you haven’t, this section teaches you everything you need to know before Chapter 2. Either way, read it — the notation we establish here is used throughout the book.

4.1Individuals, Variables, Populations, and Samples¶

Every statistical investigation begins with the same six components. We use the acronym IVPPSS to remember them:

Example 1 (EXAMPLE 1.1: Identifying the IVPPSS)

The Problem.

A pharmaceutical company is testing a new blood pressure medication. They recruit 150 adults with high blood pressure and measure the change in systolic blood pressure after 8 weeks of treatment. They want to estimate the average blood pressure reduction for all adults who might take this drug.

Identify the individual, variable, population, parameter, sample, and statistic.

Solution.

Component	Answer
Individual	One adult with high blood pressure
Variable	Change in systolic blood pressure after 8 weeks (mmHg)
Population	All adults with high blood pressure who might take this drug
Parameter	The true average blood pressure reduction for the entire population (unknown)
Sample	The 150 adults recruited for the study
Statistic	The average blood pressure reduction in the sample of 150 (known, computable)

4.2Variable Types¶

Variables come in different flavors, and the type determines which probability models apply:

Why does this matter for probability? Discrete random variables get probability mass functions (Chapter 4). Continuous random variables get probability density functions (Chapter 5). The distinction is fundamental and runs through the entire book.

🔑 What We Learned — Section 1.4

The math: Every statistical investigation has six components (IVPPSS): Individual, Variable, Population, Parameter, Sample, Statistic. Variables are either quantitative (discrete or continuous) or categorical (ordinal or nominal).

In real life: When you read that headline — “a study of 2,000 adults found that coffee drinkers live 3 years longer” — you now know the individual is one adult, the variable is lifespan (continuous quantitative), the sample is 2,000 adults, the population is all adults, and the “3 years longer” is a statistic (computed from the sample), not a parameter (the true difference for all adults). That distinction — statistic vs. parameter — is the entire reason probability theory exists: we need it to quantify how much we should trust a statistic as an estimate of the parameter.

Coming up: Now that we know what we’re measuring, we need tools to summarize measurements — the mean, variance, and standard deviation.

✏️ Section 1.4 Homework

1.4.1. An automotive manufacturer tests 50 randomly selected batteries from a production run to estimate the average lifespan (in months) of all batteries produced. Identify the individual, variable, population, parameter, sample, and statistic.

1.4.2. Classify each variable as discrete quantitative, continuous quantitative, ordinal categorical, or nominal categorical:

(a) The number of siblings a person has (b) A patient’s blood pressure (mmHg) (c) Military rank (Private, Corporal, Sergeant, …) (d) Social Security Number (e) The number of goals scored in a soccer match (f) The pH level of a water sample

1.4.3. A news report states: “Researchers surveyed 1,500 registered voters and found that 62% support the new transit proposal.” Identify each IVPPSS component. Is 62% a parameter or a statistic? What would the corresponding parameter represent?

1.4.4. Explain why ZIP codes (e.g., 93301) are nominal categorical variables even though they consist entirely of numbers.

1.4.5. Give your own example of a statistical investigation. Clearly identify all six IVPPSS components and classify the variable type.

5Summarizing Data: Center and Spread¶

📍 Where We’re Going

What we’ll learn: Three formulas — the sample mean, sample variance, and sample standard deviation — that compress an entire dataset into two numbers: where the center is and how spread out the values are.

Why it matters: An engineer testing the tensile strength of steel cables needs more than a list of 200 measurements. She needs to know: “On average, how strong are these cables?” (center) and “How much do individual cables vary from that average?” (spread). If the average is high but the spread is huge, some cables might be dangerously weak. The mean and standard deviation together tell her whether the product is reliable.

The real-life question: Two pizza delivery services both advertise “30-minute average delivery.” Service A delivers in exactly 30 minutes every time. Service B delivers in 10 minutes half the time and 50 minutes the other half. Same mean — completely different experiences. How do we capture that difference mathematically?

When we have a collection of numerical data, the two most important questions are: Where is the center? and How spread out are the values?

5.1The Sample Mean¶

Before we define the mean, let’s make sure the notation is solid. The formulas in this section — and throughout the book — use summation notation ( $\Sigma$ ), which is the discrete cousin of integration ( $\int$ ). If you’re rusty on either, this box will save you time.

Calculus Flashback: Summation Properties

The symbol $\sum_{i=1}^{n} y_i$ means “add up $y_1 + y_2 + \cdots + y_n$ .” Here are the properties you’ll use repeatedly — notice each one has an integration analog you’ll need starting in Chapter 5:

Summation Property	Integration Analog
$\sum_{i=1}^{n} c \cdot y_i = c \sum_{i=1}^{n} y_i$ (constant out)	$\int c \cdot f(x)\,dx = c\int f(x)\,dx$
$\sum_{i=1}^{n} (y_i + z_i) = \sum y_i + \sum z_i$ (split sums)	$\int [f(x)+g(x)]\,dx = \int f(x)\,dx + \int g(x)\,dx$
$\sum_{i=1}^{n} c = nc$ (sum of a constant)	$\int_a^b c\,dx = c(b-a)$

These are not deep results — they say “adding is commutative and associative.” But being fluent with them prevents errors in every derivation from here through Chapter 7.

Example 2 (EXAMPLE 1.2: Computing the Sample Mean)

The Problem.

A wildlife biologist captures 8 adult male largemouth bass from a lake and records their weights (in pounds):

4.2, \quad 5.1, \quad 3.8, \quad 6.0, \quad 4.7, \quad 5.5, \quad 3.6, \quad 5.3

Compute the sample mean weight.

Solution.

\bar{y} = \frac{4.2 + 5.1 + 3.8 + 6.0 + 4.7 + 5.5 + 3.6 + 5.3}{8} = \frac{38.2}{8} = 4.775 \text{ pounds}

In R.

weights <- c(4.2, 5.1, 3.8, 6.0, 4.7, 5.5, 3.6, 5.3)
mean(weights)
# [1] 4.775

5.2The Sample Median¶

For the bass weights, we first sort the data:

3.6, \quad 3.8, \quad 4.2, \quad 4.7, \quad 5.1, \quad 5.3, \quad 5.5, \quad 6.0

With $n = 8$ (even), the median is the average of the 4th and 5th values:

\tilde{y} = \frac{4.7 + 5.1}{2} = 4.9 \text{ pounds}

5.3The Sample Variance and Standard Deviation¶

The mean tells us where the center is. But two datasets can have the same mean and look completely different if one is tightly clustered and the other is spread out. We need a measure of spread.

Definition 1.5: Sample Variance and Standard Deviation

Given $n$ observations $y_1, y_2, \ldots, y_n$ with sample mean $\bar{y}$ , the sample variance is:

s^2 = \frac{1}{n-1}\sum_{i=1}^{n}(y_i - \bar{y})^2

The sample standard deviation is:

s = \sqrt{s^2}

We divide by $n - 1$ (not $n$ ) because this makes $s^2$ an unbiased estimator of the population variance $\sigma^2$ . The intuition: once you know $\bar{y}$ , only $n - 1$ of the deviations $(y_i - \bar{y})$ are free to vary, because they are constrained to sum to zero (you’ll prove this in the homework). We say the data have $n - 1$ degrees of freedom for estimating variance.

Example 3 (EXAMPLE 1.3: Computing the Sample Variance and Standard Deviation)

The Problem.

Using the bass weight data from Example 1.2, compute the sample variance and standard deviation.

Solution.

We already computed $\bar{y} = 4.775$ . Now we compute each squared deviation:

$y_i$	$y_i - \bar{y}$	$(y_i - \bar{y})^2$
4.2	$4.2 - 4.775 = -0.575$	0.330625
5.1	$5.1 - 4.775 = 0.325$	0.105625
3.8	$3.8 - 4.775 = -0.975$	0.950625
6.0	$6.0 - 4.775 = 1.225$	1.500625
4.7	$4.7 - 4.775 = -0.075$	0.005625
5.5	$5.5 - 4.775 = 0.725$	0.525625
3.6	$3.6 - 4.775 = -1.175$	1.380625
5.3	$5.3 - 4.775 = 0.525$	0.275625

\sum_{i=1}^{8}(y_i - \bar{y})^2 = 0.330625 + 0.105625 + 0.950625 + 1.500625 + 0.005625 + 0.525625 + 1.380625 + 0.275625 = 5.075

s^2 = \frac{5.075}{8 - 1} = \frac{5.075}{7} = 0.725

s = \sqrt{0.725} \approx 0.8515 \text{ pounds}

In R.

weights <- c(4.2, 5.1, 3.8, 6.0, 4.7, 5.5, 3.6, 5.3)
var(weights)   # sample variance
# [1] 0.725
sd(weights)    # sample standard deviation
# [1] 0.8514693

Interpretation.

The average weight is about 4.78 pounds. The standard deviation of about 0.85 pounds tells us that a “typical” bass weight is roughly 0.85 pounds away from the mean. Most of the weights fall in the interval $\bar{y} \pm 2s = (3.07, 6.48)$ pounds.

5.4The Shortcut Formula for Variance¶

Computing variance with the definition is tedious. There’s a shortcut that’s faster for hand calculations:

Theorem 1.1: Shortcut Formula for Sample Variance

s^2 = \frac{1}{n-1}\left[\sum_{i=1}^{n} y_i^2 - n\bar{y}^2\right] = \frac{1}{n-1}\left[\sum_{i=1}^{n} y_i^2 - \frac{\left(\sum_{i=1}^{n} y_i\right)^2}{n}\right]

Why this works: Expanding $(y_i - \bar{y})^2 = y_i^2 - 2y_i\bar{y} + \bar{y}^2$ and summing over all $i$ :

\sum_{i=1}^{n}(y_i - \bar{y})^2 = \sum_{i=1}^{n} y_i^2 - 2\bar{y}\sum_{i=1}^{n} y_i + \sum_{i=1}^{n}\bar{y}^2

The last sum has $n$ identical terms, so $\sum \bar{y}^2 = n\bar{y}^2$ . For the middle term, recall that $\sum_{i=1}^{n} y_i = n\bar{y}$ (from the definition of $\bar{y}$ ), so $-2\bar{y}\sum y_i = -2\bar{y}(n\bar{y}) = -2n\bar{y}^2$ . Combining:

= \sum y_i^2 - 2n\bar{y}^2 + n\bar{y}^2 = \sum y_i^2 - n\bar{y}^2

5.5The Empirical Rule¶

How do we interpret the standard deviation? The Empirical Rule gives a powerful answer for data that are approximately bell-shaped (symmetric, single-peaked):

Why does this matter for probability? The Empirical Rule is an informal preview of the normal distribution (Chapter 5). In Chapter 5, you’ll prove that for a normal random variable $X$ with mean $\mu$ and standard deviation $\sigma$ , $P(\mu - 2\sigma < X < \mu + 2\sigma) = 0.9545$ — which is where the “95%” comes from. The Empirical Rule is the data-based version of a theoretical probability statement.

Guided Practice 1.2: Compute the mean, variance, and standard deviation

The number of customer complaints received by an online retailer over 10 consecutive days was:

3, \quad 7, \quad 2, \quad 5, \quad 8, \quad 4, \quad 6, \quad 1, \quad 5, \quad 9

Mean:

\bar{y} = \frac{3 + 7 + 2 + 5 + 8 + 4 + 6 + 1 + 5 + 9}{10} = \frac{50}{10} = 5.0

Variance (shortcut):

\sum y_i^2 = 9 + 49 + 4 + 25 + 64 + 16 + 36 + 1 + 25 + 81 = 310

s^2 = \frac{310 - 10(5.0)^2}{9} = \frac{310 - 250}{9} = \frac{60}{9} = 6.667

Standard deviation:

s = \sqrt{6.667} \approx 2.582

In R:

complaints <- c(3, 7, 2, 5, 8, 4, 6, 1, 5, 9)
mean(complaints)  # 5
var(complaints)   # 6.666667
sd(complaints)    # 2.581989

⚠️ Common Mistakes

Dividing by $n$ instead of $n - 1$ when computing the sample variance. The formula uses $n - 1$ in the denominator. Dividing by $n$ gives you the population variance formula, which underestimates the true variance when applied to a sample.
Forgetting to square the deviations. The variance uses $(y_i - \bar{y})^2$ , not $|y_i - \bar{y}|$ . Squaring ensures the variance is always non-negative and gives more weight to large deviations.
Confusing the standard deviation with the variance. The variance ( $s^2$ ) is in squared units. The standard deviation ( $s$ ) is in the original units. If you measured weight in pounds, $s^2$ is in “pounds squared” (which makes no physical sense), but $s$ is in pounds.
Computing the mean of the squared values instead of the sum of squared deviations. The variance is about deviations from the mean, not about the raw squared values.

🔑 What We Learned — Section 1.5

The math: The sample mean $\bar{y} = \frac{1}{n}\sum y_i$ locates the center. The sample variance $s^2 = \frac{1}{n-1}\sum(y_i - \bar{y})^2$ and its square root, the standard deviation $s$ , measure spread. The shortcut formula $s^2 = \frac{\sum y_i^2 - n\bar{y}^2}{n-1}$ is faster for hand computation.

In real life: Remember the two pizza delivery services? Service A (always 30 min) has $s = 0$ — no variability at all. Service B (10 or 50 min) has $\bar{y} = 30$ but $s = 20$ minutes — massive variability. The standard deviation captures exactly why Service A is reliable and Service B is maddening. In probability theory, we’ll soon define the expected value $E(X)$ and variance $V(X)$ as the theoretical versions of $\bar{y}$ and $s^2$ . The formulas you just learned are the data-based estimates of those theoretical quantities — the sample versions of the population truth.

Coming up: Numbers alone don’t show the shape of data. We need a picture — the histogram — which will turn out to be the visual bridge from data to probability.

✏️ Section 1.5 Homework

1.5.1. The following are test scores (out of 100) for 10 students on a probability quiz:

72, \quad 85, \quad 91, \quad 68, \quad 77, \quad 83, \quad 95, \quad 74, \quad 88, \quad 67

(a) Compute the sample mean. (b) Compute the sample variance using the shortcut formula. (c) Compute the sample standard deviation. (d) Interpret $s$ in context: what does the standard deviation tell you about quiz performance?

1.5.2. A dataset of 8 values has $\sum y_i = 56$ and $\sum y_i^2 = 448$ . Compute $\bar{y}$ , $s^2$ , and $s$ .

1.5.3. Two factories produce bolts. Factory A’s bolt lengths (mm): 10.1, 10.0, 9.9, 10.0, 10.1. Factory B’s bolt lengths (mm): 9.5, 10.5, 9.7, 10.3, 10.0. Compute $\bar{y}$ and $s$ for each factory. Which factory produces more consistent bolts? How does the standard deviation quantify this?

1.5.4. Show algebraically that $\sum_{i=1}^{n}(y_i - \bar{y}) = 0$ for any dataset. Why does this mean we can’t use $\frac{1}{n}\sum(y_i - \bar{y})$ as a measure of spread?

1.5.5. If every observation in a dataset is multiplied by 3, what happens to (a) the sample mean and (b) the sample variance? Prove your answers using the formulas.

6Visualizing Data: Histograms and Relative Frequency¶

📍 Where We’re Going

What we’ll learn: How to build a histogram that shows the shape of data, and how switching from raw counts to relative frequencies produces something that behaves exactly like a probability distribution.

Why it matters: A hospital emergency department tracks patient wait times. The mean wait is 45 minutes, but that single number hides critical information: Are most patients seen quickly with a few extreme waits? Or are waits spread uniformly? A histogram reveals the shape — and shape is what probability distributions are built to describe. The histogram is the bridge between data you can see and mathematical models you can prove things about.

The real-life question: A coffee shop manager notices that customers sometimes cluster in bursts and sometimes arrive in long gaps. She records inter-arrival times for 40 customers. What does the data look like? Is there a recognizable pattern? The histogram will show us.

A histogram is a graphical display of the distribution of quantitative data. It groups the data into intervals (called bins or classes) and displays the frequency (or relative frequency) of observations in each interval.

6.1From Data to Histogram¶

Here’s the key idea: instead of plotting the raw count in each bin, we can plot the relative frequency — the proportion of observations in each bin:

\text{Relative frequency of bin } j = \frac{\text{count in bin } j}{n}

When we do this, something remarkable happens: the relative frequencies sum to 1, and the histogram starts to look like a probability distribution.

Example 6 (EXAMPLE 1.6: Building a Relative Frequency Histogram)

The Problem.

A coffee shop records the time (in minutes) between consecutive customer arrivals during the morning rush for 40 observations:

0.3  1.2  0.8  2.5  0.1  3.7  1.5  0.6  4.1  1.8
0.2  0.9  2.1  0.4  1.3  0.7  3.2  1.1  0.5  2.8
1.6  0.3  2.4  0.8  1.9  0.1  1.4  3.5  0.6  2.0
0.4  1.0  0.2  2.7  1.7  0.9  0.5  3.0  1.3  0.7

Construct a relative frequency histogram with bins of width 1 minute starting at 0.

Solution.

Bin (minutes)	Count	Relative Frequency
[0, 1)	18	18/40 = 0.450
[1, 2)	11	11/40 = 0.275
[2, 3)	6	6/40 = 0.150
[3, 4)	4	4/40 = 0.100
[4, 5)	1	1/40 = 0.025
Total	40	1.000

Notice that the relative frequencies sum to 1 — just like probabilities.

In R.

times <- c(0.3, 1.2, 0.8, 2.5, 0.1, 3.7, 1.5, 0.6, 4.1, 1.8,
           0.2, 0.9, 2.1, 0.4, 1.3, 0.7, 3.2, 1.1, 0.5, 2.8,
           1.6, 0.3, 2.4, 0.8, 1.9, 0.1, 1.4, 3.5, 0.6, 2.0,
           0.4, 1.0, 0.2, 2.7, 1.7, 0.9, 0.5, 3.0, 1.3, 0.7)
hist(times, breaks = 0:5, freq = FALSE,
     main = "Relative Frequency Histogram: Time Between Arrivals",
     xlab = "Time (minutes)", ylab = "Relative Frequency",
     col = "steelblue", border = "white")

The freq = FALSE argument tells R to plot relative frequency density rather than raw counts.

🔑 What We Learned — Section 1.6

The math: A histogram groups data into bins and displays counts or relative frequencies. The relative frequency of a bin is $\frac{\text{count in bin}}{n}$ , and relative frequencies always sum to 1. This is not a coincidence — it’s the same property that probabilities must satisfy.

In real life: The coffee shop manager now sees her inter-arrival data is right-skewed — most gaps are short (under 1 minute), with increasingly rare longer gaps. This shape has a name in probability theory: it looks like an exponential distribution. In Chapter 5, you’ll learn that the exponential distribution models “waiting times between random events” — which is exactly what inter-arrival times are. The histogram just showed us the shape; the theory will explain why it has that shape and let us make predictions from it.

Coming up: The most important conceptual leap in this chapter — what happens when we collect more and more data? The histogram gets smoother, and the relative frequencies stabilize into something remarkable: probabilities.

✏️ Section 1.6 Homework

1.6.1. A professor records the scores of 30 students on a midterm exam:

55 62 71 74 78 79 80 81 82 83
84 85 85 86 87 88 89 90 91 92
92 93 94 95 96 97 98 99 45 73

(a) Construct a relative frequency table with bins [40, 50), [50, 60), [60, 70), [70, 80), [80, 90), [90, 100]. (b) What proportion of students scored 80 or above? (c) Verify that the relative frequencies sum to 1. (d) Describe the shape of the distribution (symmetric, left-skewed, or right-skewed).

1.6.2. Why must relative frequencies always sum to 1? Prove this algebraically using the definition.

1.6.3. A relative frequency histogram has 5 bins with relative frequencies 0.10, 0.25, 0.35, 0.20, and 0.10. If the dataset has 200 observations, how many observations are in each bin?

1.6.4. Using R, generate 500 random values from a uniform distribution on [0, 10] using runif(500, 0, 10) and plot a relative frequency histogram with hist(..., freq = FALSE). Describe the shape. Why does this shape make sense for a uniform distribution?

7From Relative Frequency to Probability¶

📍 Where We’re Going

What we’ll learn: The single most important idea in this chapter — what happens when we collect more and more data. The relative frequency histogram gets smoother, the relative frequencies settle down to fixed values, and those fixed values are probabilities. The histogram converges to a smooth curve called the probability density function.

Why it matters: An insurance company can’t compute the “true” probability that a 25-year-old driver has an accident this year — they’d need infinite data. But they can look at millions of past policies and compute relative frequencies that are close enough for pricing. This section explains why that works: because relative frequency converges to probability. This is the bridge between the world of data (what you can observe) and the world of probability models (what you can prove things about).

The real-life question: If we flip a coin 10 times and get 7 heads (70%), does that mean the coin is unfair? What if we flip it 10,000 times and get 5,023 heads (50.23%)? How many flips do we need before we can trust the relative frequency? The answer lies in the Law of Large Numbers.

This is the conceptual bridge that connects introductory statistics to probability theory. Pay close attention — this idea underlies the entire book.

When we look at the relative frequency histogram in Example 1.6, we see that 45.0% of the observed inter-arrival times fell in the [0, 1) interval. If we collected more data — say 400 observations, or 4,000, or 40,000 — the relative frequency histogram would get smoother and smoother, and the relative frequencies would settle down to stable values.

Those stable values are probabilities.

And here’s where the magic happens: as $n \to \infty$ , the relative frequency histogram doesn’t just get smoother — it approaches a smooth curve. That smooth curve is the probability density function (PDF), which is the central object of Chapter 5.

Calculus Flashback: Area Under a Curve = Integration

In a relative frequency histogram, the area of a bar represents the proportion of data in that bin. When the histogram converges to a smooth curve $f(x)$ , the area of a bar becomes the area under the curve — and area under a curve is computed by integration:

P(a \leq X \leq b) = \int_a^b f(x)\,dx

This is the single most important equation in Chapters 5–7. It says: probability = integral of the density function. Every probability calculation for continuous random variables reduces to evaluating an integral.

If your integration is rusty, here are the three integrals you’ll use most:

\int x^n\,dx = \frac{x^{n+1}}{n+1} + C \quad (n \neq -1), \qquad \int e^{ax}\,dx = \frac{e^{ax}}{a} + C, \qquad \int_0^{\infty} e^{-x}\,dx = 1

The last one is especially important — it’s why the exponential PDF $f(x) = e^{-x}$ integrates to 1 over $[0, \infty)$ , satisfying the requirement that total probability equals 1.

This is the journey of this entire book in miniature: we start with data (histograms), discover patterns (relative frequencies), and build mathematical models (probability distributions) that capture those patterns exactly.

Let’s see this in action. The R function rexp(n, rate) generates $n$ random values from an exponential distribution (you’ll derive this distribution formally in Chapter 5 — for now, just think of it as a model for waiting times):

# Simulate 100, 1000, and 10000 exponential random variables
par(mfrow = c(1, 3))

for (n_obs in c(100, 1000, 10000)) {
  sim_data <- rexp(n_obs, rate = 1)
  hist(sim_data, breaks = 30, freq = FALSE,
       main = paste("n =", n_obs),
       xlab = "x", ylab = "Density",
       col = "steelblue", border = "white")
  curve(dexp(x, rate = 1), add = TRUE, col = "red", lwd = 2)
}

Run this code. Watch what happens as $n$ increases: the histogram converges to the red curve. The red curve is the exponential PDF $f(x) = e^{-x}$ for $x \geq 0$ . You’ll derive this function from scratch in Chapter 5.

🔑 What We Learned — Section 1.7

The math: As $n \to \infty$ , the relative frequency of an event converges to its true probability: $\frac{\text{count of } A}{n} \to P(A)$ . This is the informal Law of Large Numbers. Simultaneously, the relative frequency histogram converges to a smooth curve — the probability density function (PDF).

In real life: This is why insurance works. An insurance company can’t know whether you will have a car accident this year. But by analyzing millions of policyholders, they can compute a relative frequency — say, 4.2% of 25-year-old drivers file a claim per year — and trust that this is close to the true probability $P(\text{claim})$ . The larger their dataset, the closer their estimate. They set your premium based on this number, and the Law of Large Numbers guarantees that across millions of customers, their revenue will be predictable. This is also why casinos are always profitable: any individual gambler might win, but across millions of bets, relative frequencies lock onto the true probabilities, and the house edge is real.

The journey of this book in one sentence: We start with data (histograms), discover patterns (relative frequencies), and build mathematical models (probability distributions) that capture those patterns exactly — then prove why the models work.

✏️ Section 1.7 Homework

1.7.1. A fair six-sided die is rolled repeatedly. After 10 rolls, the relative frequency of rolling a 3 is 0.30. After 100 rolls, it’s 0.19. After 10,000 rolls, it’s 0.1672.

(a) What value should the relative frequency converge to as the number of rolls approaches infinity? (b) Does the fact that the relative frequency was 0.30 after 10 rolls mean the die is unfair? Explain. (c) The convergence from 0.30 → 0.19 → 0.1672 illustrates what principle?

1.7.2. Explain in your own words why $P(\text{event}) = 0$ does not necessarily mean the event is impossible when dealing with continuous random variables. (Hint: Think about what happens to individual bars in a histogram as the bins get narrower and narrower.)

1.7.3. Run the following R code and describe what you observe. Explain your observations using the Law of Large Numbers.

set.seed(42)
n_values <- c(50, 500, 5000, 50000)
par(mfrow = c(2, 2))
for (n in n_values) {
  x <- rnorm(n, mean = 100, sd = 15)
  hist(x, breaks = 30, freq = FALSE, main = paste("n =", n),
       xlab = "x", col = "steelblue", border = "white")
  curve(dnorm(x, 100, 15), add = TRUE, col = "red", lwd = 2)
}

1.7.4. A coin is flipped 1,000 times and heads appears 532 times. Based on this data, estimate $P(\text{heads})$ . Would you conclude the coin is fair? What additional information would you want before deciding?

8This Book vs. Intro Stats¶

If you’ve taken an introductory statistics course (the one with confidence intervals, hypothesis tests, and p-values), you might be wondering: how is this different?

Here’s the honest answer:

	Introductory Statistics	Probability Theory (This Book)
Focus	Using formulas to analyze data	Deriving and proving where those formulas come from
Main activity	Plug numbers into formulas, interpret output	Prove theorems, derive distributions, do calculus
Math level	Algebra	Calculus (integrals, series, multivariable)
The question	“What does the data tell us?”	“Why does the math behind the data work?”
Random variables	Used as black boxes	Defined, constructed, and analyzed from scratch
Distributions	Looked up in tables	Derived from first principles
Expected value	Computed with a calculator	Proved to exist and derived as an integral/sum

In short: Intro stats teaches you to drive the car. This course teaches you to build the engine.

If you haven’t taken intro stats, that’s fine. Chapter 1 (this chapter) and Chapter 2 will give you everything you need. You might actually have an advantage — you won’t have to unlearn the cookbook approach.

9The Road Ahead¶

Here is what each chapter of this book covers, explained in plain language:

Chapter 2: Probability Foundations. We define the mathematical framework for measuring uncertainty. You’ll learn about sample spaces, events, the axioms that all probabilities must obey, and the counting tools (permutations and combinations) that make complex probability calculations possible. By the end, you’ll solve the Birthday Problem.

Chapter 3: Conditional Probability and Bayes’ Rule. We ask: how does new information change what we believe? You’ll learn to update probabilities when you receive evidence, understand why P(A given B) is not the same as P(B given A), and apply Bayes’ Rule — arguably the most important formula in all of statistics. By the end, you’ll understand why the medical test in Section 1.2 fools everyone.

Chapter 4: Discrete Random Variables. We introduce the concept of a random variable — a function that assigns numbers to random outcomes — and study discrete versions (those that take countable values). You’ll derive and apply the Binomial, Geometric, Negative Binomial, Hypergeometric, and Poisson distributions. You’ll also learn about expected value, variance, and moment generating functions. By the end, you’ll compute the exact probability of any number of heads in any number of coin flips.

Chapter 5: Continuous Random Variables. We extend everything from Chapter 4 to continuous random variables — those that can take any value in an interval. You’ll learn about probability density functions and study the Uniform, Normal, Gamma, Exponential, Chi-Square, and Beta distributions. By the end, you’ll understand why the bell curve appears everywhere in nature.

Chapter 6: Multivariate Distributions. Real life rarely involves just one random variable. We study the joint behavior of two or more random variables simultaneously: joint distributions, marginal distributions, conditional distributions, independence, covariance, and correlation. By the end, you’ll understand Simpson’s Paradox — when combining data reverses the conclusion.

Chapter 7: Functions of Random Variables. The grand finale. If $X$ has a known distribution and we define $Y = g(X)$ , what is the distribution of $Y$ ? We develop three powerful methods — the CDF method, the transformation method, and the MGF method — plus order statistics. By the end, you’ll have the tools to derive any sampling distribution you need in mathematical statistics.

And after this book? The sequel — Mathematical Statistics from the Ground Up — picks up where we left off, covering sampling distributions, estimation, hypothesis testing, and regression. The probability theory in this book is the engine that powers all of it.

9.1The Calculus You’ll Need¶

This book assumes Calculus I and II. You don’t need to be an expert — but you need to be functional with specific tools. Here’s exactly what you’ll need, chapter by chapter, so you can refresh before it comes up:

Calculus Toolkit — Chapter by Chapter

Chapters 2–3 (Probability, Bayes): No calculus. Pure algebra and counting.

Chapter 4 (Discrete RVs): Summation — manipulating $\sum$ , geometric series $\sum_{k=0}^{\infty} r^k = \frac{1}{1-r}$ for $|r| < 1$ , and the exponential series $e^x = \sum_{k=0}^{\infty} \frac{x^k}{k!}$ .

Chapter 5 (Continuous RVs): Integration — this is where calculus becomes essential.

Power rule: $\int x^n\,dx = \frac{x^{n+1}}{n+1} + C$
Exponential: $\int e^{ax}\,dx = \frac{e^{ax}}{a} + C$
Improper integrals: $\int_0^{\infty} x^{n} e^{-x}\,dx$ (the Gamma function)
Integration by parts: $\int u\,dv = uv - \int v\,du$
Substitution (u-sub): $\int f(g(x))g'(x)\,dx$

Chapter 6 (Multivariate): Double integrals — $\int\int f(x,y)\,dy\,dx$ with iterated integration. You’ll set up limits and integrate in one variable while treating the other as a constant.

Chapter 7 (Transformations): Change of variables and the Jacobian $\left|\frac{dx}{dy}\right|$ for one-variable transformations. Multivariable Jacobians (determinants) for two-variable transformations.

If any of these feel unfamiliar, review them before you reach that chapter. The R appendix won’t save you here — these derivations must be done by hand.

Chapter 1 Summary

Probability theory is the mathematical study of randomness and uncertainty. It provides precise tools for situations where our intuition fails — as demonstrated by the Monty Hall Problem.
Probability appears everywhere: medicine (diagnostic tests), sports (player performance), finance (risk models), weather (forecasting), manufacturing (quality control), criminal justice (DNA evidence), insurance (premium pricing), and machine learning (model training).
This book follows a three-step pipeline: See it (simulation) → Prove it (rigorous proof) → Simulate it (R verification).
The IVPPSS framework identifies the six components of any statistical investigation: Individual, Variable, Population, Parameter, Sample, Statistic.
Variables are classified as quantitative (discrete or continuous) or categorical (ordinal or nominal). This classification determines which probability models apply.
The sample mean $\bar{y} = \frac{1}{n}\sum y_i$ measures the center of a dataset. The sample variance $s^2 = \frac{1}{n-1}\sum(y_i - \bar{y})^2$ and standard deviation $s = \sqrt{s^2}$ measure the spread. The divisor $n - 1$ reflects $n - 1$ degrees of freedom.
The Empirical Rule: For bell-shaped data, about 68% of observations fall within $\bar{y} \pm 1s$ , about 95% within $\bar{y} \pm 2s$ , and about 99.7% within $\bar{y} \pm 3s$ . This previews the normal distribution (Chapter 5).
Relative frequency histograms approximate probability distributions. As $n \to \infty$ , the histogram converges to a smooth curve — the probability density function. This is the bridge from data to theory.
This book differs from intro stats in that we derive formulas rather than just use them, and we prove theorems rather than just state them. The prerequisite is calculus, not algebra.

10Chapter 1 Refresh Homework¶

These problems integrate material from all sections of Chapter 1. Unlike the section homework (which targets one topic), refresh problems require you to combine multiple concepts — identifying variables, computing summaries, building histograms, and connecting everything to probability.

R1.1. A city public health department randomly selects 500 restaurant inspections from the past year and records the number of health code violations at each restaurant. The department wants to estimate the average number of violations per restaurant citywide.

(a) Identify all six IVPPSS components. (b) Is the variable discrete or continuous? What type of probability distribution might model this variable? (Think about what you saw in Section 1.2.) (c) If the sample mean is $\bar{y} = 3.2$ violations and $s = 1.8$ , interpret both values in context.

R1.2. Consider the Monty Hall Problem described at the beginning of this chapter.

(a) There are three possible scenarios when you initially pick Door 1. List them and their probabilities. (b) In each scenario, explain what Monty does and what happens if you switch. (c) Show that switching wins the car 2/3 of the time. (d) Write R code to simulate the Monty Hall Problem 10,000 times and estimate the probability of winning by switching. Compute the relative frequency of winning by switching and compare it to 2/3. Does the Law of Large Numbers apply here?

R1.3. Two ride-sharing companies in the same city both claim “average pickup time under 8 minutes.” Company A’s pickup times (minutes) for 6 rides: 7.5, 8.2, 7.8, 8.0, 7.6, 7.9. Company B’s pickup times: 3.1, 14.2, 5.8, 11.4, 4.0, 8.5.

(a) Identify the individual, variable, and variable type. (b) Compute $\bar{y}$ and $s$ for each company. (c) Both means are close to 8 minutes. Explain, using the standard deviation, why a customer would strongly prefer Company A. (d) If you constructed a relative frequency histogram for each company (with many more observations), predict what each histogram would look like. Which would be more “peaked” near the center?

R1.4. A dataset of 6 values has $\sum y_i = 42$ and $\sum y_i^2 = 350$ .

(a) Compute $\bar{y}$ , $s^2$ (shortcut formula), and $s$ . (b) Is $\bar{y}$ a parameter or a statistic? Explain. (c) If these 6 values came from a population with true mean $\mu = 8$ , does $\bar{y}$ equal $\mu$ ? Should it? Why or why not?

R1.5. In Example 1.6 (coffee shop inter-arrival times), we observed that 45.0% of the 40 inter-arrival times fell in the [0, 1) interval. The theoretical probability for an exponential distribution with rate $\lambda = 1$ is $P(0 \leq X < 1) = 1 - e^{-1} \approx 0.6321$ .

(a) Why don’t the observed relative frequency (0.450) and the theoretical probability (0.632) match exactly? (b) What would you expect to happen if we observed 40,000 inter-arrival times instead of 40? (c) Does this difference mean our data does not follow an exponential distribution? Explain using the Law of Large Numbers.

R1.6. A student claims: “The standard deviation tells you the average distance of data points from the mean.” Is this exactly correct? If not, explain what the standard deviation actually measures and why the student’s statement is close but technically wrong. (Hint: Think about squaring.)

R1.7. Write R code to:

(a) Generate 1000 random values from a normal distribution with mean 50 and standard deviation 10. (b) Compute the sample mean and sample standard deviation. Are they exactly 50 and 10? Why not? (c) Plot a relative frequency histogram with the theoretical normal curve overlaid. (d) Explain what you observe, connecting histograms (Section 1.6), relative frequency (Section 1.7), the Law of Large Numbers, and the probability density function.

R1.8. This problem ties together every section of Chapter 1.

A public university wants to understand how long students spend on homework per week. They survey 200 students (the sample) to estimate the true average for all students at the university (the population).

(a) Identify all IVPPSS components and classify the variable type. (Section 1.4) (b) The sample mean is 14.3 hours with $s = 5.2$ hours. Interpret both. (Section 1.5) (c) A relative frequency histogram of the data shows a right-skewed shape with most students between 8–18 hours, a peak around 12 hours, and a long tail extending to 35 hours. What does this shape tell us that the mean alone does not? (Section 1.6) (d) If the university surveyed 20,000 students instead of 200, how would the histogram change? What would the relative frequencies converge to? (Section 1.7) (e) An administrator says: “14.3 hours — that’s the true average for our students.” Is this statement correct? What’s the correct way to state the relationship between 14.3 hours and the true average? (Section 1.4, parameter vs. statistic)

11Chapter 1 Quiz¶

Test your understanding with this self-assessment quiz. You need 80% (12/15) to earn the badge.

Chapter 1 Practice Quiz

What is the sample mean of the values 2, 4, 6, 8, 10?
- 5
- 6 (correct)
- 7
- 30
In the IVPPSS framework, what is a ‘parameter’?
- A numerical summary of the sample
- A numerical summary of the entire population (correct)
- A variable measured on each individual
- The size of the sample
A variable that records whether a patient’s blood type is A, B, AB, or O is classified as:
- Discrete quantitative
- Continuous quantitative
- Ordinal categorical
- Nominal categorical (correct)
The sample variance uses n − 1 in the denominator instead of n because:
- It makes the calculation easier
- It makes the variance an unbiased estimator of the population variance (correct)
- The textbook says so
- It prevents division by zero when n = 1
In the Monty Hall Problem, switching doors wins the car with probability:
- 1/2
- 1/3
- 2/3 (correct)
- 3/4
As the number of observations n approaches infinity, a relative frequency histogram converges to:
- A bar chart
- The sample mean
- The probability density function (correct)
- A normal distribution always
Which of the following is a STATISTIC (not a parameter)?
- The true average height of all adults in the country
- The average GPA of 200 randomly selected students (correct)
- The proportion of all defective items in a factory
- The standard deviation of all SAT scores nationwide
The number of cars passing through a toll booth in an hour is what type of variable?
- Continuous quantitative
- Discrete quantitative (correct)
- Nominal categorical
- Ordinal categorical
If a dataset has values 3, 5, 7, 9, 11, the sum of deviations from the mean Σ(yᵢ − ȳ) equals:
- 0 (correct)
- 7
- 35
- 5
The shortcut formula for sample variance is:

s² = Σ(yᵢ − ȳ) / n
s² = [Σyᵢ² − n·ȳ²] / (n − 1) (correct)
s² = Σyᵢ² / (n − 1)
s² = [Σyᵢ − n·ȳ]² / (n − 1)

In a relative frequency histogram, the relative frequencies must sum to:

0
1 (correct)
n
100

Temperature measured in degrees Fahrenheit is what type of variable?

Discrete quantitative
Continuous quantitative (correct)
Ordinal categorical
Nominal categorical

The standard deviation is measured in:

Squared units of the data
The same units as the original data (correct)
No units (dimensionless)
Percentage units

Which best describes the difference between this book and an intro stats course?

This book has easier math
This book derives and proves formulas; intro stats applies them (correct)
This book has no R code
This book skips hypothesis testing entirely

The sample median of the values 1, 3, 5, 7, 9, 11 is:

5
6 (correct)
5.5
7

A researcher wants to estimate the average rainfall in a region. She collects data from 30 weather stations. The ‘population’ in this study is:

The 30 weather stations
The researcher
All possible rainfall measurements in the region (correct)
The average of the 30 measurements

Probability theory gives us tools to reason about uncertainty because:

Computers need probability to function
Human intuition about uncertainty is systematically unreliable (correct)
It is a graduation requirement
It was invented by gamblers

If Σyᵢ = 60 and Σyᵢ² = 400 for n = 10 observations, the sample variance is:

(400 − 60)/10 = 34
(400 − 10·36)/9 = 4.44 (correct)
400/9 = 44.44
(400 − 360)/10 = 4

The ‘See it → Prove it → Simulate it’ pipeline means:

Read the textbook, do the homework, take the exam
Observe a pattern via simulation, prove it mathematically, then verify with R code (correct)
Watch a video, take notes, do practice problems
See the formula, plug in numbers, check the answer

If every data point in a dataset is the same value (e.g., all 5’s), the sample variance is:

0 (correct)
1
Undefined
5

Which R function computes the sample standard deviation?

std()
var()
sd() (correct)
stdev()

A survey asks respondents to rate a product as ‘poor’, ‘fair’, ‘good’, or ‘excellent’. This variable is:

Discrete quantitative
Continuous quantitative
Ordinal categorical (correct)
Nominal categorical

The difference between discrete and continuous random variables determines:

Whether we use mean or median
Whether we use probability mass functions or probability density functions (correct)
Whether the variable is categorical or not
Whether we need calculus

In the medical testing example (Section 1.2), the true probability of having the disease given a positive test is about:

99%
50%
0.5% (correct)
2%

Which is NOT a component of the IVPPSS framework?

Individual
Variable
Hypothesis (correct)
Parameter

Why does the textbook recommend starting with R’s base functions rather than external packages?

External packages are more expensive
Base R is faster
Base R requires no additional installation and works on any system (correct)
External packages are too complex for probability

The population mean is typically denoted by:

ȳ
μ (correct)
s
x̄

If you add a constant c to every data value, the sample mean:

Stays the same
Increases by c, but the variance stays the same (correct)
Both mean and variance increase by c
Increases by c, and variance increases by c²

ZIP codes (like 93301) are classified as:

Discrete quantitative
Continuous quantitative
Ordinal categorical
Nominal categorical (correct)

The phrase ‘long-run relative frequency’ informally describes:

The sample mean
Probability (correct)
The standard deviation
The median

In the Monty Hall Problem, Monty always opens a door that:

Is randomly selected
Has the car behind it
Has a goat behind it and is not your chosen door (correct)
You already chose

The formula s² = Σ(yᵢ − ȳ)²/(n−1) is called the:

Population variance
Sample variance (definitional formula) (correct)
Shortcut formula
Standard deviation

Which of these is a continuous quantitative variable?

The number of siblings a person has
A person’s favorite color
The weight of a newborn baby in grams (correct)
A student’s letter grade

When we say the sample mean ȳ ‘estimates’ the population mean μ, we mean:

ȳ equals μ exactly
ȳ is computed from data and is our best guess at the unknown μ (correct)
ȳ is always larger than μ
μ is computed from the sample

A relative frequency of 0.25 for the bin [10, 20) means:

There were 25 observations in that bin
25% of the observations fell in that interval (correct)
The probability is exactly 0.25
The bin width is 0.25

The median is preferred over the mean when the data:

Is perfectly symmetric
Contains extreme outliers or is heavily skewed (correct)
Has a small sample size
Contains only integers

In R, the function var() computes the sample variance with denominator:

n
n − 1 (correct)
n + 1
n − 2

What branch of mathematics is the primary prerequisite for this book?

Linear algebra
Calculus (integration and series) (correct)
Differential equations
Number theory

Which of the following is FALSE about this book’s approach?

Every theorem is fully proved
R code verifies theoretical results
Simulations come before formal definitions
Proofs are left as exercises for the reader (correct)

An insurance company uses the Poisson distribution to model:

The color of a policyholder’s car
The number of claims filed per year by a policyholder (correct)
The name of the insurance agent
Whether a policyholder is male or female

If ȳ = 10 and s = 3, the interval ȳ ± 2s is:

(7, 13)
(4, 16) (correct)
(1, 19)
(10, 16)

The key insight connecting histograms to probability is:

All histograms are bell-shaped
Relative frequencies converge to probabilities as n → ∞ (correct)
Histograms always have 10 bins
The area under any histogram is n

Bayes’ Rule is used to:

Compute the sample mean
Update the probability of an event after receiving new evidence (correct)
Construct histograms
Determine if a variable is discrete or continuous

A dataset has n = 5 observations: 10, 20, 30, 40, 50. The value Σyᵢ² is:

150
5500 (correct)
22500
30

The statement ‘probability is the long-run relative frequency’ is an informal version of:

The Central Limit Theorem
The Law of Large Numbers (correct)
Bayes’ Rule
Tchebysheff’s Theorem

Which of the following is the correct R code to compute the mean of c(1,2,3,4,5)?

avg(c(1,2,3,4,5))
mean(c(1,2,3,4,5)) (correct)
sum(c(1,2,3,4,5))
average(1,2,3,4,5)

Machine learning algorithms rely on probability because:

They are programmed in R
Training maximizes the probability of observed data given model parameters (correct)
They only work with categorical data
Probability makes code run faster

The difference between a parameter and a statistic is:

Parameters are bigger than statistics
A parameter describes the population; a statistic describes the sample (correct)
Statistics are always more accurate
Parameters come from samples

If every relative frequency in a histogram is 0.10 and there are 10 bins, then each bin contains:

10 observations
10% of the data (correct)
Exactly 1 observation
100% of the data

The Empirical Rule states that about 95% of data falls within:

ȳ ± s
ȳ ± 2s (correct)
ȳ ± 3s
ȳ ± 0.5s

For bell-shaped data, approximately what percentage falls within one standard deviation of the mean?

50%
68% (correct)
95%
99.7%

The Empirical Rule applies to data that are approximately:

Skewed right
Uniform
Bell-shaped (normal) (correct)
Any shape

If ȳ = 100 and s = 15, the interval ȳ ± 3s is:

(85, 115)
(70, 130)
(55, 145) (correct)
(100, 145)

The summation property Σ(c·yᵢ) = c·Σyᵢ is analogous to which integration rule?

Integration by parts
Constant multiple rule: ∫cf(x)dx = c∫f(x)dx (correct)
Chain rule
Product rule

Σᵢ₌₁ⁿ c = ?

c
nc (correct)
c/n
cn²

In probability theory, P(a ≤ X ≤ b) for continuous X equals:

f(b) − f(a)
∫ₐᵇ f(x)dx (correct)
Σ f(xᵢ)
F(a) − F(b)

The degrees of freedom for sample variance with n observations is:

n
n − 1 (correct)
n + 1
n/2

Why do we have n−1 degrees of freedom for variance?

It makes the formula easier
The deviations from ȳ must sum to 0, constraining one value (correct)
n−1 is always better than n
Tradition

If s = 0, then all observations are:

Zero
Equal to ȳ (identical values) (correct)
Negative
Missing

The integral ∫₀^∞ e^(−x)dx equals:

0
1 (correct)
∞
e

Which calculus skill is NOT needed for Chapters 2-3?

Integration (correct)
Algebra
Set operations
Counting

The Gamma function integral ∫₀^∞ xⁿe^(−x)dx appears in which chapter?

Chapter 2
Chapter 3
Chapter 4
Chapter 5 (Continuous RVs) (correct)

Double integrals are needed starting in:

Chapter 3
Chapter 4
Chapter 5
Chapter 6 (Multivariate) (correct)

A dataset with ȳ = 50, s = 10 has about 99.7% of observations between:

40 and 60
30 and 70
20 and 80 (correct)
0 and 100

If you multiply every observation by 2, the standard deviation:

Stays the same
Also doubles (correct)
Quadruples
Halves

If you add 10 to every observation, the standard deviation:

Increases by 10
Stays the same (correct)
Doubles
Decreases by 10

The range of a dataset is:

Mean − Median
Maximum − Minimum (correct)
Q3 − Q1
Standard deviation × 2

A histogram with a long right tail is called:

Left-skewed
Right-skewed (positively skewed) (correct)
Symmetric
Uniform

For right-skewed data, typically:

Mean = Median
Mean > Median (correct)
Mean < Median
Mean = Mode

The total area under a relative frequency histogram equals:

n
100
1 (correct)
The range

Which is NOT a measure of center?

Mean
Median
Variance (correct)
Mode

Which is NOT a measure of spread?

Variance
Standard deviation
Range
Median (correct)

If Σyᵢ = 100 and n = 20, then ȳ =

100
20
5 (correct)
2000

If Σyᵢ = 80, Σyᵢ² = 720, n = 10, then s² =

72
(720 − 10·64)/9 = 80/9 ≈ 8.89 (correct)
720/9
80

The sample variance is always:

Positive
≥ 0 (correct)
Between 0 and 1
Equal to s

A population parameter is typically:

Known exactly
Unknown and estimated by a statistic (correct)
Always larger than the statistic
Computed from the sample

The Law of Large Numbers says relative frequency converges to probability as:

n → 0
n → ∞ (correct)
P(A) → 0
s → 0

Which R function computes the sample mean?

avg()
mean() (correct)
average()
xbar()

In R, var() divides by:

n
n − 1 (correct)
n + 1
2

hist(x, freq=FALSE) in R plots:

Raw counts
Relative frequency density (area = 1) (correct)
Cumulative frequency
Box plot

The PDF f(x) = e^(−x) for x ≥ 0 integrates to:

0
1 (valid density) (correct)
∞
e

Probability theory uses calculus because:

It is a math prerequisite
Continuous distributions require integration for probability computation (correct)
All statistics needs calculus
Counting requires derivatives

A continuous random variable differs from discrete in that:

It is always positive
It can take any value in an interval, not just countable values (correct)
It has larger variance
It requires more data

The shortcut formula uses Σyᵢ² and (Σyᵢ)². These are:

The same thing
Different quantities: sum of squares vs square of sum (correct)
Both equal to nȳ²
Both equal to s²

If n = 1, the sample variance formula gives:

0
1
0/0 — undefined (you need at least 2 observations) (correct)
The observation itself

The Monty Hall Problem demonstrates:

Probability is always 50/50
Human intuition about probability is unreliable (correct)
Doors have equal probability
The host is cheating

In the Monty Hall Problem, the key fact is that Monty:

Chooses randomly
Knows what is behind each door and always reveals a goat (correct)
Always reveals the car
Lets you pick again

Machine learning relies on probability because:

All code needs probability
Machine learning is just statistics
Model training maximizes P(data | parameters) (correct)
GPUs require probability

The ȳ ± 2s interval captures about 95% of data. This is useful because:

It is exact for all distributions
It gives a quick range containing nearly all observations (correct)
It replaces the need for histograms
It works without computing ȳ

Integration by parts ∫u dv = uv − ∫v du is needed in:

Chapter 1
Chapter 2
Chapter 3
Chapter 5 (deriving Gamma distribution moments) (correct)

The Jacobian |dx/dy| is needed in:

Chapter 4
Chapter 5
Chapter 6
Chapter 7 (transformations of random variables) (correct)

The geometric series Σᵢ₌₀^∞ rⁱ = 1/(1−r) converges when:

|r| > 1
|r| < 1 (correct)
r > 0
r = 1

The exponential series e^x = Σ xᵏ/k! is used in Chapter 4 for:

Computing means
Moment generating functions and the Poisson distribution (correct)
Variance only
Histograms

In a relative frequency table, the relative frequency 0.25 for bin [10,20) means:

Exactly 25 observations
25% of observations fell between 10 and 20 (correct)
The probability is exactly 0.25
The bin width is 25

When bin widths in a histogram are unequal, you should plot:

Raw counts
Density (relative frequency / bin width) not raw relative frequency (correct)
Only the tallest bin
Nothing — use equal widths

The u-substitution technique ∫f(g(x))g′(x)dx is needed for:

Chapter 1 only
Chapter 2 only
Computing CDFs of transformed random variables (correct)
Never in probability

A statistic ȳ = 14.3 estimates a parameter μ. The best statement is:

ȳ = μ exactly
ȳ is our best estimate of μ but likely differs from it (correct)
ȳ is always larger than μ
μ is computed from the sample

Which variable type gets a probability DENSITY function in Chapter 5?

Discrete quantitative
Continuous quantitative (correct)
Ordinal categorical
Nominal categorical

The Empirical Rule gives APPROXIMATE percentages. For EXACT probabilities from a normal distribution, you need:

A bigger sample
The integral ∫ of the normal PDF (Chapter 5) (correct)
A different formula
The shortcut formula

The sample standard deviation s has the same units as:

The variance
The original data values (correct)
No units
Squared units