Continuous Random Variables and Their Distributions

Learning Objectives

After completing this chapter, you will be able to:

Explain in plain language why continuous random variables require a fundamentally different approach than discrete ones
Define probability density functions and cumulative distribution functions for continuous random variables, and compute probabilities using integration
Calculate expected values, variances, and standard deviations for continuous distributions
Derive and apply the Uniform distribution to model equally-likely outcomes over an interval
Derive and apply the Normal distribution, standardize variables, and compute probabilities using tables and R
Derive and apply the Gamma distribution family — including the Exponential and Chi-Square distributions as special cases
Derive and apply the Beta distribution to model proportions and percentages
Compute and use moment generating functions for continuous distributions
Apply Tchebysheff’s theorem to bound probabilities for any continuous distribution

Prerequisites: Chapters 2–4. Calculus II (integration techniques, improper integrals, the Gamma function). Builds on: Discrete random variables, expected values, MGFs (Chapter 4).

Content current as of April 2026.

The Opening Problem

Buffon’s Needle. In 1777, Georges-Louis Leclerc, Comte de Buffon, posed one of the strangest questions in the history of mathematics: drop a needle of length $\ell$ onto a floor ruled with parallel lines spaced $d$ apart. What is the probability that the needle crosses a line?

The answer is breathtaking: $P = \frac{2\ell}{\pi d}$ . Yes — $\pi$ , the ratio of a circle’s circumference to its diameter, appears in the answer to a problem about straight needles and straight lines. No circles anywhere in sight.

This means you can estimate $\pi$ by dropping needles. In 1901, the Italian mathematician Mario Lazzarini claimed to have tossed a needle 3,408 times and obtained $\pi \approx 3.1415929$ — accurate to six decimal places. (His result was almost certainly rigged, but the mathematics behind it is flawless.)

The derivation requires two things you don’t yet have: a continuous random variable for the needle’s angle, and integration over a probability density function. By the end of this chapter, you’ll derive Buffon’s formula yourself — and understand why $\pi$ was hiding in the floorboards all along.

1From Discrete to Continuous: The Conceptual Leap¶

🔭 What Are We About to Learn?

In Chapter 4, we studied discrete random variables — things you can count, like the number of defective items in a batch or the number of heads in 10 coin flips. Every possible value got its own probability: $p(0) = 0.10$ , $p(1) = 0.25$ , and so on, and all the probabilities summed to 1.

But what if you’re measuring something that can take any value in an interval? The exact weight of a newborn baby. The precise amount of rainfall tomorrow. The time until the next earthquake. These measurements don’t come in neat, countable packages — they live on a continuum.

Here’s the fundamental problem: you cannot assign a positive probability to every single point on a line and still have everything sum to 1. There are uncountably many points. If each one got even the tiniest positive probability, the sum would be infinite.

So how do we handle probability for continuous quantities? The answer changes everything: we stop asking “What is the probability of this exact value?” and start asking “What is the probability of falling in this interval?” Instead of summing probabilities, we integrate them. This section makes that transition precise.

Real-life motivation: A weather station measures daily rainfall to the nearest hundredth of an inch. Could the rainfall tomorrow be exactly 1.73000000... inches? In theory, sure — but you’d never observe that exact value even if you measured for a million years. What matters is the probability of rainfall between 1.5 and 2.0 inches. That’s a question continuous probability can answer.

1.1Why the Discrete Approach Breaks Down¶

In Chapter 4, we defined a discrete random variable $Y$ by assigning probabilities $p(y)$ to each possible value, subject to two rules: $p(y) \geq 0$ for all $y$ , and $\sum_{\text{all } y} p(y) = 1$ . This works beautifully when $Y$ can take only finitely many or countably many values.

Now imagine trying to do the same for the daily rainfall $Y$ at a weather station, where $Y$ can take any value between 0 and, say, 6 inches. Suppose we try to assign a positive probability $p(y) = c > 0$ to every point in the interval $[0, 6]$ . How many points are in that interval? Uncountably many — more than can be listed, more than the integers, more than all the rational numbers combined. And if each point gets probability $c > 0$ , the “sum” over uncountably many terms would be $\infty$ , not 1. The axioms of probability are violated.

The inescapable conclusion:

This might feel strange. If $P(Y = 1.73) = 0$ , does that mean a rainfall of 1.73 inches is impossible? No. It means that 1.73 is possible but has probability zero — just as a single point has length zero, yet a collection of points makes up the entire real line.

The practical consequence is profound: for continuous random variables, only intervals have positive probability. We ask $P(1.5 \leq Y \leq 2.0)$ , not $P(Y = 1.73)$ .

1.2The Distribution Function: Our Starting Point¶

The key to defining continuous random variables is a function you’ve already met briefly in Chapter 4.

Every random variable — discrete or continuous — has a CDF. What distinguishes discrete from continuous is the shape of $F(y)$ :

For a discrete random variable, $F(y)$ is a step function — it jumps at each value with positive probability and is flat in between.
For a continuous random variable, $F(y)$ is a smooth, continuous curve with no jumps.

This leads us to the formal definition of a continuous random variable:

Because $F(y)$ is continuous (no jumps), there are no single points with positive probability — confirming the zero-probability fact above.

1.3See It First: Histograms Converging to a Curve¶

Before we go further into the theory, let’s watch the discrete-to-continuous transition happen. In Chapter 1, we saw that as the number of observations grows, a relative frequency histogram gets smoother. Let’s see this with exponential data:

# Watch the discrete-to-continuous transition
par(mfrow = c(2, 2))

for (n_obs in c(50, 500, 5000, 50000)) {
  sim_data <- rexp(n_obs, rate = 0.5)
  hist(sim_data, breaks = 40, freq = FALSE,
       main = paste("n =", format(n_obs, big.mark = ",")),
       xlab = "y", ylab = "Density",
       col = "steelblue", border = "white",
       xlim = c(0, 15))
  curve(dexp(x, rate = 0.5), add = TRUE, col = "red", lwd = 2)
}

Run this code. As $n$ increases from 50 to 50,000, the histogram bars melt into the smooth red curve. That red curve — $f(y) = 0.5 e^{-0.5y}$ for $y \geq 0$ — is the probability density function (PDF). The histogram is its discrete approximation; the PDF is the continuous truth.

This is the “See It” step of our pipeline. Now we formalize.

🔬 What Did We Learn?

We learned that continuous random variables require a fundamentally different approach than discrete ones. You can’t assign a positive probability to every point on a line — there are too many points. Instead, probabilities live in intervals, and we compute them using areas under curves rather than sums.

The real-life interpretation: When a weather app says there’s a “30% chance of 1–2 inches of rain,” that 30% is the area under a probability density function between 1 and 2. The probability of exactly 1.3742 inches is zero — but the probability of an interval around it is positive and meaningful. Every continuous measurement you’ve ever encountered — blood pressure, reaction time, the weight of a package — works this way.

We also saw that the cumulative distribution function $F(y) = P(Y \leq y)$ is the foundation: smooth and continuous for continuous random variables, with step-function jumps for discrete ones.

Looking ahead: In the next section, we’ll define the probability density function — the smooth curve that the histogram converges to — and develop the tools to compute probabilities, expected values, and everything else we need.

2Probability Density Functions and CDFs¶

🔭 What Are We About to Learn?

We just established that continuous random variables need areas, not sums. But areas under what? In this section, we define the probability density function (PDF) — the smooth curve that a histogram converges to — and show how it connects to the CDF from Section 5.1.

Here’s the key relationship: the PDF is the derivative of the CDF, and the CDF is the integral of the PDF. If you know calculus, this is the Fundamental Theorem of Calculus applied to probability.

We’ll learn to verify that a function qualifies as a valid PDF, compute probabilities by integrating the PDF over intervals, and find specific quantiles (like the median) by solving equations involving the CDF.

Real-life motivation: A hospital measures the time (in hours) that patients spend in the emergency room. The distribution of wait times isn’t uniform — most patients are seen quickly, but a few wait a very long time. The PDF captures the shape of this distribution: tall near zero (many short waits), tapering off to the right (fewer long waits). The probability that you’ll wait between 2 and 4 hours is the area under this curve between 2 and 4.

2.1The Probability Density Function¶

The CDF $F(y)$ tells us accumulated probability: “How much probability sits to the left of $y$ ?” The PDF tells us the rate at which probability accumulates at the point $y$ .

Because $F(y)$ is the integral of $f(y)$ , we can write:

F(y) = \int_{-\infty}^{y} f(t) \, dt

This is just the Fundamental Theorem of Calculus. The CDF at any point $y$ is the total area under the PDF from $-\infty$ to $y$ .

2.2Properties of a PDF¶

Since the CDF has properties (Theorem 5.1), the PDF inherits corresponding properties:

Theorem 2 (Theorem 5.2: Properties of a Density Function)

If $f(y)$ is a density function for a continuous random variable, then:

$f(y) \geq 0$ for all $y$ , $-\infty < y < \infty$
$\displaystyle\int_{-\infty}^{\infty} f(y) \, dy = 1$

Why this makes sense: Property 1 says the density is never negative — you can’t have a negative rate of probability accumulation. Property 2 says the total area under the entire PDF is 1 — all the probability is accounted for.

Proof.

Strategy: Both properties follow directly from the CDF properties in Theorem 5.1.

Property 1: Because $F(y)$ is nondecreasing, its derivative $f(y) = F'(y)$ must be nonnegative wherever it exists. $\checkmark$

Property 2: Because $F(-\infty) = 0$ and $F(\infty) = 1$ :

\int_{-\infty}^{\infty} f(y) \, dy = F(\infty) - F(-\infty) = 1 - 0 = 1 \qquad \blacksquare

2.3Computing Probabilities: Integration¶

The reason we care about PDFs is that they let us compute probabilities:

This probability is the area under the PDF between $a$ and $b$ . Notice a beautiful consequence: because $P(Y = a) = 0$ and $P(Y = b) = 0$ for continuous random variables, it doesn’t matter whether we use strict or non-strict inequalities:

P(a < Y < b) = P(a \leq Y < b) = P(a < Y \leq b) = P(a \leq Y \leq b) = \int_a^b f(y) \, dy

This is not true for discrete random variables — and it simplifies our life enormously.

2.4Worked Examples¶

Example 1 (EXAMPLE 5.1: Verifying a PDF and Finding Probabilities)

The Problem.

A random variable $Y$ has the density function:

f(y) = \begin{cases} cy^2, & 0 \leq y \leq 2 \\ 0, & \text{elsewhere} \end{cases}

(a) Find the value of $c$ that makes $f(y)$ a valid density function.

(b) Find $P(1 \leq Y \leq 2)$ .

Solution.

(a) For $f(y)$ to be a valid PDF, we need $\int_{-\infty}^{\infty} f(y) \, dy = 1$ :

\int_0^2 cy^2 \, dy = c \cdot \frac{y^3}{3}\bigg|_0^2 = c \cdot \frac{8}{3} = 1

c = \frac{3}{8}

So $f(y) = \frac{3}{8}y^2$ on $[0, 2]$ .

(b) Using the PDF:

P(1 \leq Y \leq 2) = \int_1^2 \frac{3}{8}y^2 \, dy = \frac{3}{8} \cdot \frac{y^3}{3}\bigg|_1^2 = \frac{1}{8}(y^3)\bigg|_1^2 = \frac{1}{8}(8 - 1) = \frac{7}{8}

(c) The CDF is $F(y) = \int_{-\infty}^{y} f(t) \, dt$ . We need to consider three regions:

For $y < 0$ : $F(y) = \int_{-\infty}^{y} 0 \, dt = 0$
For $0 \leq y \leq 2$ : $F(y) = \int_0^y \frac{3}{8}t^2 \, dt = \frac{3}{8} \cdot \frac{t^3}{3}\bigg|_0^y = \frac{y^3}{8}$
For $y > 2$ : $F(y) = \int_0^2 \frac{3}{8}t^2 \, dt + \int_2^y 0 \, dt = 1$

So:

F(y) = \begin{cases} 0, & y < 0 \\ y^3/8, & 0 \leq y \leq 2 \\ 1, & y > 2 \end{cases}

Check: $F(0) = 0$ ✓, $F(2) = 8/8 = 1$ ✓, and $F'(y) = 3y^2/8 = f(y)$ on $(0,2)$ ✓.

In R.

# Define the PDF
f <- function(y) ifelse(y >= 0 & y <= 2, (3/8) * y^2, 0)

# Verify total area = 1
integrate(f, -Inf, Inf)
# 1 with absolute error < 2e-14

# P(1 <= Y <= 2)
integrate(f, 1, 2)
# 0.875 with absolute error < 9.7e-15
# That's 7/8 ✓

Interpretation.

The density $f(y) = \frac{3}{8}y^2$ is zero at $y = 0$ and increases to $f(2) = \frac{3}{2}$ at $y = 2$ . Notice that $f(2) = 1.5 > 1$ — that’s fine, because $f(y)$ is a density, not a probability. The total area under the curve is what must equal 1, and it does. Most of the probability ( $\frac{7}{8} = 87.5\%$ ) is concentrated in the upper half of the interval $[1, 2]$ because the density is much higher there.

Example 2 (EXAMPLE 5.2: From CDF to PDF)

The Problem.

The time $Y$ (in hours) that a library book on 3-hour reserve is actually checked out has the CDF:

F(y) = \begin{cases} 0, & y < 0 \\ \frac{y^2}{9}, & 0 \leq y < 3 \\ 1, & y \geq 3 \end{cases}

(a) Find the PDF $f(y)$ .

(b) Find $P(1 \leq Y \leq 2)$ .

Solution.

(a) Differentiate $F(y)$ :

f(y) = F'(y) = \begin{cases} \frac{2y}{9}, & 0 < y < 3 \\ 0, & \text{elsewhere} \end{cases}

Check: $f(y) \geq 0$ for all $y$ ✓, and $\int_0^3 \frac{2y}{9} \, dy = \frac{2}{9} \cdot \frac{y^2}{2}\big|_0^3 = \frac{1}{9}(9) = 1$ ✓.

(b) Using the CDF directly:

P(1 \leq Y \leq 2) = F(2) - F(1) = \frac{4}{9} - \frac{1}{9} = \frac{3}{9} = \frac{1}{3}

Or equivalently by integrating: $\int_1^2 \frac{2y}{9} \, dy = \frac{1}{9}(y^2)\big|_1^2 = \frac{4 - 1}{9} = \frac{1}{3}$ ✓.

(c) The median $\phi_{0.5}$ satisfies $F(\phi_{0.5}) = 0.5$ :

\frac{\phi_{0.5}^2}{9} = 0.5 \implies \phi_{0.5}^2 = 4.5 \implies \phi_{0.5} = \sqrt{4.5} \approx 2.12 \text{ hours}

In R.

# Define PDF and CDF
f <- function(y) ifelse(y > 0 & y < 3, 2*y/9, 0)
Fy <- function(y) ifelse(y < 0, 0, ifelse(y < 3, y^2/9, 1))

# P(1 <= Y <= 2)
Fy(2) - Fy(1)  # 0.3333333

# Median
sqrt(4.5)  # 2.12132

Interpretation.

The density $f(y) = \frac{2y}{9}$ starts at zero and increases linearly, meaning longer checkout times are more likely than shorter ones. The median is about 2.12 hours — half of all checkouts are returned before 2.12 hours, and half after. Notice that the median ( $\approx 2.12$ ) is greater than the midpoint of the interval (1.5), which makes sense because the density puts more weight on higher values.

2.5Quantiles and Percentiles¶

Guided Practice 5.1: Verify a PDF, find probabilities, and compute quantiles

Solution.

A continuous random variable $Y$ has the density function:

f(y) = \begin{cases} k(3 - y), & 0 \leq y \leq 3 \\ 0, & \text{elsewhere} \end{cases}

(a) Find $k$ .

\int_0^3 k(3 - y) \, dy = k\left[3y - \frac{y^2}{2}\right]_0^3 = k\left(9 - \frac{9}{2}\right) = \frac{9k}{2} = 1 \implies k = \frac{2}{9}

(b) Find $F(y)$ .

For $0 \leq y \leq 3$ :

F(y) = \int_0^y \frac{2}{9}(3 - t) \, dt = \frac{2}{9}\left[3t - \frac{t^2}{2}\right]_0^y = \frac{2}{9}\left(3y - \frac{y^2}{2}\right) = \frac{6y - y^2}{9}

F(y) = \begin{cases} 0, & y < 0 \\ \frac{6y - y^2}{9}, & 0 \leq y \leq 3 \\ 1, & y > 3 \end{cases}

(c) Find $P(0.5 \leq Y \leq 2)$ .

P(0.5 \leq Y \leq 2) = F(2) - F(0.5) = \frac{12 - 4}{9} - \frac{3 - 0.25}{9} = \frac{8}{9} - \frac{2.75}{9} = \frac{5.25}{9} \approx 0.5833

(d) Find the median.

\frac{6\phi_{0.5} - \phi_{0.5}^2}{9} = 0.5 \implies \phi_{0.5}^2 - 6\phi_{0.5} + 4.5 = 0

Using the quadratic formula: $\phi_{0.5} = \frac{6 \pm \sqrt{36 - 18}}{2} = \frac{6 \pm \sqrt{18}}{2} = 3 \pm \frac{3\sqrt{2}}{2}$

Since $0 \leq \phi_{0.5} \leq 3$ : $\phi_{0.5} = 3 - \frac{3\sqrt{2}}{2} \approx 3 - 2.121 = 0.879$

In R:

f <- function(y) ifelse(y >= 0 & y <= 3, (2/9) * (3 - y), 0)
integrate(f, 0.5, 2)  # 0.5833333
# Median: solve the quadratic
3 - 3*sqrt(2)/2  # 0.8786797

2.6Conditional Probability with Continuous Random Variables¶

Conditional probabilities work the same way as in the discrete case, but with integrals instead of sums:

P(Y > 1.5 \mid Y < 3) = \frac{P(Y > 1.5 \text{ and } Y < 3)}{P(Y < 3)} = \frac{P(1.5 < Y < 3)}{P(Y < 3)}

Example 3 (EXAMPLE 5.3: Conditional Probability)

The Problem.

Using the checkout time distribution from Example 5.2, where $f(y) = \frac{2y}{9}$ for $0 < y < 3$ , find the probability that a book is checked out for more than 2 hours, given that it has already been out for at least 1 hour.

Solution.

P(Y > 2 \mid Y > 1) = \frac{P(Y > 2 \text{ and } Y > 1)}{P(Y > 1)} = \frac{P(Y > 2)}{P(Y > 1)}

P(Y > 2) = 1 - F(2) = 1 - \frac{4}{9} = \frac{5}{9}

P(Y > 1) = 1 - F(1) = 1 - \frac{1}{9} = \frac{8}{9}

P(Y > 2 \mid Y > 1) = \frac{5/9}{8/9} = \frac{5}{8} = 0.625

Interpretation.

Given that the book has already been out for an hour, there’s a 62.5% chance it will be out for more than 2 hours. Notice this is higher than the unconditional probability $P(Y > 2) = 5/9 \approx 55.6\%$ . This makes intuitive sense: the density increases with $y$ , so a book that’s already been out for a while is more likely to stay out even longer.

⚠️ Common Mistakes — PDFs and CDFs

Interpreting $f(y)$ as a probability. The density $f(y)$ is NOT a probability. It is a rate — the rate at which probability accumulates at the point $y$ . Only the integral of $f(y)$ over an interval gives a probability. The density can exceed 1.
Forgetting that $P(Y = a) = 0$ . For continuous random variables, the probability of any single exact value is zero. This means $P(Y < 3) = P(Y \leq 3)$ . If you’re computing $P(1 \leq Y \leq 2)$ and you accidentally exclude the endpoints, you get the same answer — no harm done.
Confusing PDF and CDF. The PDF $f(y)$ is the derivative of the CDF $F(y)$ . The CDF goes from 0 to 1 and is always nondecreasing. The PDF can be any nonnegative function whose total area is 1 — it does NOT need to be between 0 and 1.
Leaving $f(y)$ undefined outside the support. Always define $f(y) = 0$ outside the interval where the density is positive. If $f(y) = 2y$ for $0 \leq y \leq 1$ , write the complete definition with “0 elsewhere.” If you forget, your integrals will have wrong limits.
Not checking your CDF. After computing $F(y)$ , always verify: does $F(y) = 0$ at the left boundary? Does $F(y) = 1$ at the right boundary? Is $F(y)$ continuous (no jumps)? If any check fails, retrace your integration.

2.7Section 5.2 Exercises¶

5.2.1. A continuous random variable $Y$ has the density function:

f(y) = \begin{cases} cy(2 - y), & 0 \leq y \leq 2 \\ 0, & \text{elsewhere} \end{cases}

(a) Find the value of $c$ . (b) Find $F(y)$ . (c) Find $P(0.5 \leq Y \leq 1.5)$ . (d) Find $P(Y > 1 \mid Y < 1.5)$ .

5.2.2. The distribution function of a continuous random variable $Y$ is:

F(y) = \begin{cases} 0, & y < 1 \\ 1 - \frac{1}{y^2}, & y \geq 1 \end{cases}

(a) Show that $F(y)$ has all the properties of a valid CDF. (b) Find the PDF $f(y)$ . (c) Find $P(Y > 3)$ . (d) Find the median of $Y$ . (e) Find $P(Y > 4 \mid Y > 2)$ .

5.2.3. A continuous random variable $Y$ has the density function:

f(y) = \begin{cases} 0.3, & -1 < y \leq 0 \\ 0.3 + ky, & 0 < y \leq 1 \\ 0, & \text{elsewhere} \end{cases}

(a) Find $k$ . (b) Find $F(y)$ . (c) Find $P(Y > 0.5)$ .

5.2.4. A supermarket receives a delivery of a perishable product. The proportion of the delivery that is still usable after $y$ days follows the density function:

f(y) = \begin{cases} \frac{3}{8}(2 - y)^2, & 0 \leq y \leq 2 \\ 0, & \text{elsewhere} \end{cases}

(a) Verify that this is a valid PDF. (b) What fraction of the delivery is still usable after 1 day? (That is, find $P(Y > 1)$ .) (c) Find the median shelf life.

5.2.5. (Proof) Let $f(y)$ be a PDF that is symmetric about 0, meaning $f(y) = f(-y)$ for all $y$ . Show that $P(Y > 0) = 1/2$ . (Hint: Use the substitution $u = -y$ in the integral $\int_{0}^{\infty} f(y) \, dy$ .)

🔬 What Did We Learn?

We now have the two main tools for working with continuous random variables: the PDF $f(y) = F'(y)$ and the CDF $F(y) = \int_{-\infty}^y f(t) \, dt$ . They are two sides of the same coin — the CDF accumulates probability, and the PDF describes the rate of accumulation.

The real-life interpretation: The PDF is the “shape” of uncertainty. When you see a weather forecast showing a graph of possible rainfall amounts, with a peak around 1 inch and a long tail to the right — that graph is a PDF. The height of the curve at any point tells you how “concentrated” the probability is near that value. To find the probability of any range of outcomes, you compute the area under the curve between those values.

We also learned practical techniques: given a PDF, find $c$ to make it valid (set the total area to 1); given a CDF, differentiate to find the PDF; compute conditional probabilities by dividing integrals. And we got our first Common Mistakes — especially the crucial fact that $f(y)$ is a density, not a probability, and can exceed 1.

Looking ahead: Now that we know how to compute probabilities with PDFs, we need measures of center and spread: the expected value and variance. That’s Section 5.3.

3Expected Values for Continuous Random Variables¶

🔭 What Are We About to Learn?

In Chapter 4, we defined the expected value of a discrete random variable as $E(Y) = \sum_y y \cdot p(y)$ — a weighted average where each value is weighted by its probability. We also defined the variance $V(Y) = E[(Y - \mu)^2]$ as a measure of spread.

Now we need the continuous versions. The conceptual idea is identical — a “weighted average” — but the mechanism changes: sums become integrals, and $p(y)$ becomes $f(y) \, dy$ . Everything else carries over: the variance shortcut formula, the linearity of expectation, the rules for constants. If you understood expected values in Chapter 4, this section is the calculus upgrade.

Real-life motivation: An insurance company models the dollar amount of individual claims using a continuous distribution. The expected value $E(Y)$ tells them the average claim size — the number they use to set premiums. The variance $V(Y)$ tells them how wildly claims vary — the number they use to set reserves. Both require integrating the claim distribution. Without expected values, there are no premiums, no reserves, and no insurance industry.

3.1The Expected Value¶

Compare this to the discrete case: $E(Y) = \sum y \cdot p(y)$ . The integral replaces the sum, $f(y) \, dy$ replaces $p(y)$ , and everything else is the same.

📐 Calculus Flashback: Integration Rules You’ll Need

Throughout this chapter, we integrate expressions of the form $\int y^n f(y) \, dy$ . Here are the rules that appear repeatedly:

Power Rule: $\displaystyle\int y^n \, dy = \frac{y^{n+1}}{n+1} + C$ for $n \neq -1$

Exponential Rule: $\displaystyle\int e^{ay} \, dy = \frac{1}{a}e^{ay} + C$

Definite Integral (Fundamental Theorem of Calculus): $\displaystyle\int_a^b g(y) \, dy = G(b) - G(a)$ , where $G'(y) = g(y)$

Constant Multiple Rule: $\displaystyle\int c \cdot g(y) \, dy = c \int g(y) \, dy$ — constants pull out of integrals

Integration by Parts: $\displaystyle\int u \, dv = uv - \int v \, du$ — used for products like $y \cdot e^{-y}$

$u$ -Substitution: If $u = h(y)$ , then $du = h'(y) \, dy$ , and the integral is rewritten in terms of $u$

If you’re rusty on any of these, revisit them now — every section of this chapter uses them.

3.2Expected Value of a Function of $Y$ ¶

Just as in the discrete case, we often need the expected value of some function $g(Y)$ rather than $Y$ itself:

3.3Variance and Standard Deviation¶

The shortcut formula from Chapter 4 still works:

Theorem 5 (Theorem 5.5: Shortcut Formula for Variance)

V(Y) = E(Y^2) - [E(Y)]^2 = E(Y^2) - \mu^2

Proof.

Strategy: Expand $(Y - \mu)^2$ inside the expectation, then use the fact that the expected value of a sum is the sum of the expected values (linearity), and that $\mu$ is a constant.

V(Y) = E[(Y - \mu)^2]

First, expand the square algebraically — this is just $(a - b)^2 = a^2 - 2ab + b^2$ :

= E[Y^2 - 2\mu Y + \mu^2]

Now apply linearity of expectation (Theorem 5.6, property 3) — the expectation of a sum is the sum of expectations:

= E(Y^2) - E(2\mu Y) + E(\mu^2)

Since $\mu$ is a constant, we can pull it out (property 2): $E(2\mu Y) = 2\mu \cdot E(Y) = 2\mu \cdot \mu = 2\mu^2$ . And $E(\mu^2) = \mu^2$ (property 1: the expectation of a constant is itself). So:

= E(Y^2) - 2\mu^2 + \mu^2 = E(Y^2) - \mu^2 \qquad \blacksquare

In practice: To compute $V(Y)$ , you compute two integrals — $E(Y) = \int y \, f(y) \, dy$ and $E(Y^2) = \int y^2 \, f(y) \, dy$ — then subtract the square of the first from the second. This is almost always easier than computing $\int (y - \mu)^2 f(y) \, dy$ directly.

3.4Properties of Expected Value¶

The linearity results from Chapter 4 carry over unchanged:

3.5Worked Examples¶

Example 4 (EXAMPLE 5.4: Finding the Mean and Variance)

The Problem.

The density function from Example 5.1 was $f(y) = \frac{3}{8}y^2$ for $0 \leq y \leq 2$ . Find $\mu = E(Y)$ and $\sigma^2 = V(Y)$ .

Solution.

Step 1: Find $E(Y)$ .

E(Y) = \int_0^2 y \cdot \frac{3}{8}y^2 \, dy = \frac{3}{8}\int_0^2 y^3 \, dy = \frac{3}{8} \cdot \frac{y^4}{4}\bigg|_0^2 = \frac{3}{8} \cdot \frac{16}{4} = \frac{3}{8} \cdot 4 = \frac{3}{2} = 1.5

Step 2: Find $E(Y^2)$ .

E(Y^2) = \int_0^2 y^2 \cdot \frac{3}{8}y^2 \, dy = \frac{3}{8}\int_0^2 y^4 \, dy = \frac{3}{8} \cdot \frac{y^5}{5}\bigg|_0^2 = \frac{3}{8} \cdot \frac{32}{5} = \frac{12}{5} = 2.4

Step 3: Apply the shortcut formula.

V(Y) = E(Y^2) - [E(Y)]^2 = 2.4 - (1.5)^2 = 2.4 - 2.25 = 0.15

\sigma = \sqrt{0.15} \approx 0.3873

In R.

f <- function(y) ifelse(y >= 0 & y <= 2, (3/8) * y^2, 0)

# E(Y)
EY <- integrate(function(y) y * f(y), 0, 2)$value
EY  # 1.5

# E(Y^2)
EY2 <- integrate(function(y) y^2 * f(y), 0, 2)$value
EY2  # 2.4

# Variance and SD
VY <- EY2 - EY^2
VY    # 0.15
sqrt(VY)  # 0.3872983

Interpretation.

The mean of 1.5 sits at the center of the interval $[0, 2]$ , but the distribution is right-skewed (the density increases as $y$ increases), so values near 2 are more common than values near 0. The standard deviation of about 0.39 tells us that a “typical” observation is roughly 0.39 units from the mean. The interval $\mu \pm 2\sigma = (0.73, 2.27)$ captures most of the probability.

Example 5 (EXAMPLE 5.5: Expected Value of a Cost Function)

The Problem.

The proportion of time $Y$ that a manufacturing robot is in operation during a 40-hour work week has the density function $f(y) = 2y$ for $0 \leq y \leq 1$ . The weekly profit (in hundreds of dollars) from the robot is $X = 200Y - 60$ .

(a) Find $E(Y)$ and $V(Y)$ .

(b) Find the expected weekly profit $E(X)$ and $V(X)$ .

Solution.

(a)

E(Y) = \int_0^1 y \cdot 2y \, dy = 2\int_0^1 y^2 \, dy = 2 \cdot \frac{y^3}{3}\bigg|_0^1 = \frac{2}{3}

E(Y^2) = \int_0^1 y^2 \cdot 2y \, dy = 2\int_0^1 y^3 \, dy = 2 \cdot \frac{y^4}{4}\bigg|_0^1 = \frac{1}{2}

V(Y) = \frac{1}{2} - \left(\frac{2}{3}\right)^2 = \frac{1}{2} - \frac{4}{9} = \frac{9 - 8}{18} = \frac{1}{18}

(b) Since $X = 200Y - 60$ (a linear function of $Y$ ):

E(X) = 200 \cdot E(Y) - 60 = 200 \cdot \frac{2}{3} - 60 = \frac{400}{3} - 60 = \frac{400 - 180}{3} = \frac{220}{3} \approx \$73.33 \text{ (hundreds)}

V(X) = 200^2 \cdot V(Y) = 40000 \cdot \frac{1}{18} = \frac{40000}{18} \approx 2222.22

The expected weekly profit is about $7,333, with a standard deviation of $\sqrt{2222.22} \approx \$4,714$ (in original dollar units, multiply hundreds by 100).

Guided Practice 5.2: Mean, variance, and a linear transformation

Solution.

A thermostatically controlled switch turns on at a temperature $Y$ (in °F) with density:

f(y) = \begin{cases} 1/4, & 58 \leq y \leq 62 \\ 0, & \text{elsewhere} \end{cases}

(a) Find $E(Y)$ and $V(Y)$ .

E(Y) = \int_{58}^{62} y \cdot \frac{1}{4} \, dy = \frac{1}{4} \cdot \frac{y^2}{2}\bigg|_{58}^{62} = \frac{1}{8}(62^2 - 58^2) = \frac{1}{8}(3844 - 3364) = \frac{480}{8} = 60

E(Y^2) = \int_{58}^{62} y^2 \cdot \frac{1}{4} \, dy = \frac{1}{4} \cdot \frac{y^3}{3}\bigg|_{58}^{62} = \frac{1}{12}(238328 - 195112) = \frac{43216}{12} = 3601.333

V(Y) = 3601.333 - 60^2 = 3601.333 - 3600 = \frac{4}{3} \approx 1.333

(b) The cost to recalibrate is $C = 5(Y - 60)^2 + 10$ . Find $E(C)$ .

E(C) = 5E[(Y - 60)^2] + 10 = 5V(Y) + 10 = 5 \cdot \frac{4}{3} + 10 = \frac{20}{3} + 10 = \frac{50}{3} \approx \$16.67

Note: $E[(Y - 60)^2] = V(Y)$ because $\mu = 60$ , so $(Y - 60)^2 = (Y - \mu)^2$ .

3.6Section 5.3 Exercises¶

5.3.1. If $Y$ has the density function from Guided Practice 5.1 — $f(y) = \frac{2}{9}(3 - y)$ for $0 \leq y \leq 3$ — find $E(Y)$ and $V(Y)$ .

5.3.2. A continuous random variable $Y$ has the density function:

f(y) = \begin{cases} \frac{3}{2}y^2 + y, & 0 \leq y \leq 1 \\ 0, & \text{elsewhere} \end{cases}

(a) Verify that this is a valid PDF. (b) Find $E(Y)$ and $V(Y)$ .

5.3.3. The weekly repair cost $Y$ for a machine (in hundreds of dollars) has density function $f(y) = 4y^3$ for $0 \leq y \leq 1$ , and 0 elsewhere.

(a) Find $E(Y)$ and $V(Y)$ . (b) If the machine generates a weekly revenue of $R = 3 - 2Y$ (in hundreds), find $E(R)$ and $V(R)$ . (c) What is the probability that the repair cost exceeds the expected repair cost?

5.3.4. (Proof) Prove Theorem 5.6, property 2: $E[cg(Y)] = cE[g(Y)]$ for a continuous random variable $Y$ with PDF $f(y)$ . (Hint: Pull the constant $c$ out of the integral.)

5.3.5. If $Y$ has the CDF from Exercise 5.2.2 — $F(y) = 1 - 1/y^2$ for $y \geq 1$ — find $E(Y)$ . Does $E(Y^2)$ exist? (Hint: Check whether the improper integral converges.)

🔬 What Did We Learn?

We extended the expected value and variance from discrete to continuous random variables. The conceptual meaning is unchanged — $E(Y)$ is the “balance point” of the density, and $V(Y)$ measures how spread out the distribution is — but the computation now uses integrals instead of sums.

The real-life interpretation: When an insurance company says “the average claim is $5,200,” they computed $E(Y) = \int y \, f(y) \, dy$ where $f(y)$ is the density of claim amounts. When they say “claims are highly variable,” they computed $V(Y)$ or $\sigma$ . The linear transformation rule $E(aY + b) = aE(Y) + b$ lets them instantly convert between units — if claims are in thousands, multiply $E(Y)$ by 1000 to get dollars.

The shortcut formula $V(Y) = E(Y^2) - \mu^2$ saves enormous effort. Compute two integrals ( $E(Y)$ and $E(Y^2)$ ) instead of one ugly integral with $(y - \mu)^2$ .

Exercise 5.3.5 hints at something subtle: some distributions have infinite variance, or even infinite mean. These “heavy-tailed” distributions arise in finance and insurance — and they’re part of why extreme events are more common than people expect.

Looking ahead: We now have the general tools — PDF, CDF, $E(Y)$ , $V(Y)$ . Starting in Section 5.4, we study specific named distributions that appear everywhere in practice, beginning with the simplest: the Uniform distribution.

4The Uniform Distribution¶

🔭 What Are We About to Learn?

The Uniform distribution is the simplest continuous distribution: every value in an interval is equally likely. The density is a flat, horizontal line — a rectangle whose area is 1.

Despite its simplicity, the Uniform distribution is surprisingly important. It models measurement rounding errors, the arrival time of an event within a known window, and the output of random number generators. It is also the building block for simulating any other continuous distribution — a fact exploited by every statistical software package on the planet.

Real-life motivation: A city bus is scheduled to arrive at your stop sometime between 8:00 and 8:12 AM. You arrive at exactly 8:00. If the bus is equally likely to arrive at any moment in that 12-minute window, how long should you expect to wait? What’s the probability you wait more than 8 minutes? The Uniform distribution answers both questions immediately.

4.1Definition and Density Function¶

The density is a constant $\frac{1}{\theta_2 - \theta_1}$ — a flat line. The height is chosen so the area of the rectangle equals 1:

\text{Area} = \text{base} \times \text{height} = (\theta_2 - \theta_1) \times \frac{1}{\theta_2 - \theta_1} = 1 \checkmark

This is the “equally likely” distribution: any subinterval of a given length has the same probability, regardless of where it sits within $(\theta_1, \theta_2)$ .

4.2CDF of the Uniform Distribution¶

For $\theta_1 \leq y \leq \theta_2$ :

F(y) = \int_{\theta_1}^{y} \frac{1}{\theta_2 - \theta_1} \, dt = \frac{y - \theta_1}{\theta_2 - \theta_1}

The complete CDF:

F(y) = \begin{cases} 0, & y < \theta_1 \\ \dfrac{y - \theta_1}{\theta_2 - \theta_1}, & \theta_1 \leq y \leq \theta_2 \\[6pt] 1, & y > \theta_2 \end{cases}

4.3Mean and Variance¶

Theorem 7 (Theorem 5.7: Mean and Variance of the Uniform Distribution)

If $Y \sim \text{Uniform}(\theta_1, \theta_2)$ , then:

\mu = E(Y) = \frac{\theta_1 + \theta_2}{2} \qquad \text{and} \qquad \sigma^2 = V(Y) = \frac{(\theta_2 - \theta_1)^2}{12}

Proof.

Strategy: Integrate directly using the constant density $\frac{1}{\theta_2 - \theta_1}$ .

Mean:

E(Y) = \int_{\theta_1}^{\theta_2} y \cdot \frac{1}{\theta_2 - \theta_1} \, dy = \frac{1}{\theta_2 - \theta_1} \cdot \frac{y^2}{2}\bigg|_{\theta_1}^{\theta_2} = \frac{\theta_2^2 - \theta_1^2}{2(\theta_2 - \theta_1)} = \frac{(\theta_2 + \theta_1)(\theta_2 - \theta_1)}{2(\theta_2 - \theta_1)} = \frac{\theta_1 + \theta_2}{2}

The mean is simply the midpoint of the interval — exactly what “equally likely” suggests.

Variance:

E(Y^2) = \int_{\theta_1}^{\theta_2} y^2 \cdot \frac{1}{\theta_2 - \theta_1} \, dy = \frac{1}{\theta_2 - \theta_1} \cdot \frac{y^3}{3}\bigg|_{\theta_1}^{\theta_2} = \frac{\theta_2^3 - \theta_1^3}{3(\theta_2 - \theta_1)} = \frac{\theta_2^2 + \theta_1\theta_2 + \theta_1^2}{3}

Now apply the shortcut:

V(Y) = E(Y^2) - \mu^2 = \frac{\theta_2^2 + \theta_1\theta_2 + \theta_1^2}{3} - \left(\frac{\theta_1 + \theta_2}{2}\right)^2

= \frac{\theta_2^2 + \theta_1\theta_2 + \theta_1^2}{3} - \frac{\theta_1^2 + 2\theta_1\theta_2 + \theta_2^2}{4}

= \frac{4(\theta_2^2 + \theta_1\theta_2 + \theta_1^2) - 3(\theta_1^2 + 2\theta_1\theta_2 + \theta_2^2)}{12}

= \frac{4\theta_2^2 + 4\theta_1\theta_2 + 4\theta_1^2 - 3\theta_1^2 - 6\theta_1\theta_2 - 3\theta_2^2}{12} = \frac{\theta_2^2 - 2\theta_1\theta_2 + \theta_1^2}{12} = \frac{(\theta_2 - \theta_1)^2}{12} \qquad \blacksquare

4.4Worked Examples¶

Example 6 (EXAMPLE 5.6: The Bus Stop Problem)

The Problem.

A bus arrives at a stop uniformly at random between 8:00 and 8:12 AM. You arrive at exactly 8:00.

(a) What is the probability that you wait more than 8 minutes?

(b) What is your expected waiting time?

Solution.

Let $Y$ = your waiting time (in minutes). Then $Y \sim \text{Uniform}(0, 12)$ .

(a) $P(Y > 8) = \int_8^{12} \frac{1}{12} \, dy = \frac{12 - 8}{12} = \frac{4}{12} = \frac{1}{3}$

Or using the CDF: $P(Y > 8) = 1 - F(8) = 1 - \frac{8 - 0}{12 - 0} = 1 - \frac{2}{3} = \frac{1}{3}$

(b) $E(Y) = \frac{0 + 12}{2} = 6$ minutes.

You should expect to wait 6 minutes — the midpoint of the interval.

(c) $P(3 \leq Y \leq 7) = \frac{7 - 3}{12} = \frac{4}{12} = \frac{1}{3}$

In R.

# Y ~ Uniform(0, 12)
# P(Y > 8)
punif(8, min = 0, max = 12, lower.tail = FALSE)  # 0.3333333

# E(Y) and V(Y)
(0 + 12) / 2      # 6  (mean)
(12 - 0)^2 / 12   # 12 (variance)
sqrt(12)           # 3.464  (SD)

# P(3 <= Y <= 7)
punif(7, 0, 12) - punif(3, 0, 12)  # 0.3333333

# Simulate to verify
set.seed(42)
sim <- runif(100000, min = 0, max = 12)
mean(sim > 8)       # ≈ 0.333
mean(sim)           # ≈ 6.0
mean(sim >= 3 & sim <= 7)  # ≈ 0.333

Example 7 (EXAMPLE 5.7: Poisson Arrivals and the Uniform Connection)

The Problem.

Customer arrivals at a checkout counter follow a Poisson process. During a given 20-minute period, exactly one customer arrived. Find the probability that the customer arrived in the last 4 minutes.

Solution.

This is a classic result: given that exactly one event occurs in an interval, the actual time of occurrence is uniformly distributed over that interval. (This is a property of the Poisson process that we state without proof here.)

So the arrival time $Y \sim \text{Uniform}(0, 20)$ .

P(Y > 16) = \frac{20 - 16}{20} = \frac{4}{20} = \frac{1}{5} = 0.20

The probability is $1/5$ — the same as the ratio of the subinterval length (4 minutes) to the full interval (20 minutes). For a Uniform distribution, probability equals proportion of length. This is true for any 4-minute window within the 20-minute period.

Example 8 (EXAMPLE 5.8: Cost with a Uniform Component)

The Problem.

A circuit board failure interrupts computing work until a replacement arrives. The delivery time $Y$ (in days) is uniformly distributed between 1 and 5 days. The total cost of the interruption is $C = c_0 + c_1 Y^2$ , where $c_0$ is the fixed replacement cost and $c_1$ is the proportional disruption cost.

(a) Find the expected delivery time.

(b) Find $E(C)$ in terms of $c_0$ and $c_1$ .

Solution.

$Y \sim \text{Uniform}(1, 5)$ .

(a) $E(Y) = \frac{1 + 5}{2} = 3$ days.

(b) We need $E(Y^2)$ :

E(Y^2) = \int_1^5 y^2 \cdot \frac{1}{4} \, dy = \frac{1}{4} \cdot \frac{y^3}{3}\bigg|_1^5 = \frac{1}{12}(125 - 1) = \frac{124}{12} = \frac{31}{3}

E(C) = c_0 + c_1 E(Y^2) = c_0 + \frac{31c_1}{3}

Check with variance formula: $V(Y) = (5-1)^2/12 = 16/12 = 4/3$ . Then $E(Y^2) = V(Y) + [E(Y)]^2 = 4/3 + 9 = 31/3$ ✓.

Guided Practice 5.3: The Uniform Distribution

Solution.

A precision instrument has a measurement error $Y$ (in millimeters) that is uniformly distributed between -0.04 and +0.04.

(a) Find the probability that the measurement error is within $\pm 0.01$ mm.

$Y \sim \text{Uniform}(-0.04, 0.04)$

P(-0.01 \leq Y \leq 0.01) = \frac{0.01 - (-0.01)}{0.04 - (-0.04)} = \frac{0.02}{0.08} = 0.25

(b) Find $E(Y)$ and $V(Y)$ .

E(Y) = \frac{-0.04 + 0.04}{2} = 0 \qquad V(Y) = \frac{(0.08)^2}{12} = \frac{0.0064}{12} \approx 0.000533

The mean error is 0 (errors are equally likely to be positive or negative), and the SD is $\sqrt{0.000533} \approx 0.0231$ mm.

(c) Find the median and the first quartile.

Median: $\phi_{0.5} = \frac{-0.04 + 0.04}{2} = 0$ (same as the mean, since Uniform is symmetric).

First quartile: $\phi_{0.25} = -0.04 + 0.25(0.08) = -0.04 + 0.02 = -0.02$ mm.

4.5The Uniform Distribution and Random Number Generation¶

Every computer’s random number generator fundamentally produces values from a $\text{Uniform}(0, 1)$ distribution. In R, the function runif(n) generates $n$ such values. To transform these into any other continuous distribution, we use a technique called the probability integral transform (which you’ll explore in Chapter 7):

If $U \sim \text{Uniform}(0, 1)$ and $F$ is any continuous CDF, then $Y = F^{-1}(U)$ has CDF $F$ .

This is how R internally generates exponential, normal, gamma, and all other continuous random variables — starting from the humble Uniform.

# Generate Uniform(0,1) and transform to Exponential(rate = 2)
set.seed(123)
U <- runif(10000)          # Uniform(0, 1)
Y <- -log(1 - U) / 2      # Transform: F_inv for Exponential(2)

# Verify: compare to direct exponential generation
par(mfrow = c(1, 2))
hist(Y, breaks = 50, freq = FALSE, main = "Transformed from Uniform",
     col = "steelblue", border = "white")
curve(dexp(x, rate = 2), add = TRUE, col = "red", lwd = 2)

hist(rexp(10000, rate = 2), breaks = 50, freq = FALSE,
     main = "Direct rexp()",
     col = "steelblue", border = "white")
curve(dexp(x, rate = 2), add = TRUE, col = "red", lwd = 2)

4.6Section 5.4 Exercises¶

5.4.1. A parachutist lands at a random point on a line between markers A and B that are 100 meters apart. The landing point $Y$ (measured from A) is uniformly distributed on $(0, 100)$ .

(a) Find the probability that the parachutist lands within 15 meters of marker A. (b) Find the probability that the parachutist is closer to A than to B. (c) Find the probability that the distance from A is more than three times the distance from B.

5.4.2. The amount of daily rainfall (in inches) at a monitoring station during the rainy season can be modeled as $\text{Uniform}(0, 0.5)$ on days when it rains.

(a) Find the mean and standard deviation of daily rainfall on rainy days. (b) Find the probability that rainfall exceeds 0.3 inches on a rainy day. (c) Given that rainfall exceeded 0.2 inches, find the conditional probability it exceeded 0.4 inches.

5.4.3. A circular wheel of circumference 2 feet is spun. The distance $Y$ (in feet) traveled by a mark on the wheel’s rim before stopping is $\text{Uniform}(0, 2)$ . The area $A$ of the circular sector swept out is $A = Y^2 / (4\pi)$ .

(a) Find $E(A)$ and $V(A)$ . (b) Find the probability that $A > 0.1$ square feet.

5.4.4. (Proof) Show that the median of a $\text{Uniform}(\theta_1, \theta_2)$ distribution equals its mean. Then show that for any Uniform distribution, $P(Y \leq \mu) = P(Y \geq \mu) = 0.5$ . Is this property true for all continuous distributions? Explain.

5.4.5. Three independent measurements are made by a precision instrument whose error is $\text{Uniform}(-0.05, 0.05)$ . What is the probability that exactly one of the three errors exceeds 0.03 in absolute value? (Hint: First find $p = P(|Y| > 0.03)$ , then use a result from Chapter 4.)

⚠️ Common Mistakes — The Uniform Distribution

Confusing the density height with a probability. For $\text{Uniform}(0, 2)$ , the density is $f(y) = 0.5$ everywhere on $[0, 2]$ . The number 0.5 is the density, not the probability of anything. The probability of the interval $[0, 1]$ is $0.5 \times 1 = 0.5$ — area, not height.
Computing variance as $(\theta_2 - \theta_1)^2$ instead of $(\theta_2 - \theta_1)^2/12$ . The interval width appears squared in the variance formula, but divided by 12. Don’t forget the 12.
Assuming the conditional distribution is still Uniform. It is! If $Y \sim \text{Uniform}(a, b)$ and you condition on $Y$ being in a subinterval $(c, d) \subset (a, b)$ , then $Y \mid c < Y < d \sim \text{Uniform}(c, d)$ . This makes conditional probability calculations easy but is easy to overlook.

🔬 What Did We Learn?

The Uniform distribution is the “flat” distribution — every value in the interval $(\theta_1, \theta_2)$ is equally likely. Its density is a constant $1/(\theta_2 - \theta_1)$ , its mean is the midpoint $(\theta_1 + \theta_2)/2$ , and its variance is $(\theta_2 - \theta_1)^2/12$ .

The real-life interpretation: The Uniform distribution models situations of perfect ignorance within a known range. If a bus arrives “sometime between 8:00 and 8:12” and you have no further information, the Uniform is the appropriate model. It also models measurement rounding error (equally likely to round up or down within the rounding precision) and the arrival time of a single Poisson event within a known time window.

Perhaps most importantly, the Uniform $(0, 1)$ distribution is the engine of all simulation. Every random number generator produces Uniform values, which are then transformed into whatever distribution you need. When you call rnorm() or rexp() in R, the software starts with runif() and applies a mathematical transformation. The Uniform distribution is simple on the surface, but it powers all of computational statistics.

Looking ahead: The next distribution is the most famous and most important in all of statistics — the Normal (bell-shaped) distribution. It appears everywhere in nature, and Section 5.5 will explain why.

5The Normal Distribution¶

🔭 What Are We About to Learn?

The Normal distribution — the famous “bell curve” — is the single most important distribution in all of statistics. SAT scores, blood pressure readings, measurement errors in physics experiments, heights of adults, IQ scores — they all follow approximately Normal distributions.

But why? What is it about the Normal distribution that makes it appear everywhere in nature? The answer is the Central Limit Theorem (Chapter 7): whenever you average many independent random effects, the result is approximately Normal, regardless of the individual distributions. Height is the sum of thousands of genetic and environmental factors. Measurement error is the sum of many tiny instrument imperfections. The bell curve emerges whenever many small contributions add up.

In this section, we’ll define the Normal distribution, learn to compute probabilities by standardizing (converting any Normal to the standard Normal), and practice reading the $Z$ -table and using R. We will also prove that the parameters $\mu$ and $\sigma$ truly are the mean and variance — but that proof uses moment generating functions, so we’ll defer it to Section 5.8.

Real-life motivation: A pharmaceutical company manufactures tablets that should contain 500 mg of active ingredient. Due to natural variation in the production process, the actual amount follows a Normal distribution with mean $\mu = 500$ mg and standard deviation $\sigma = 5$ mg. A tablet is considered defective if it contains less than 490 mg or more than 510 mg. What proportion of tablets are defective? The answer — about 4.56% — comes directly from the Normal distribution.

5.1Definition and Shape¶

This formula looks intimidating, but each piece has a clear purpose:

The exponent $-(y - \mu)^2/(2\sigma^2)$ creates the bell shape — it’s largest when $y = \mu$ (the peak) and falls off symmetrically on both sides.
The term $\sigma$ in the exponent controls the width: larger $\sigma$ means a wider, flatter bell; smaller $\sigma$ means a taller, narrower bell.
The constant $\frac{1}{\sigma\sqrt{2\pi}}$ in front is a normalizing constant that forces the total area to equal 1. You never need to memorize this — just know it’s there to satisfy Theorem 5.2.

5.2Key Properties of the Normal Distribution¶

Symmetric about $\mu$ . The density satisfies $f(\mu + a) = f(\mu - a)$ for all $a$ . The bell curve is a mirror image around the vertical line $y = \mu$ .
The mean, median, and mode are all equal to $\mu$ . This is a consequence of symmetry.
The inflection points are at $y = \mu \pm \sigma$ . The curve changes from “curving downward” to “curving upward” exactly one standard deviation from the mean.
The total area under the curve is 1. This is guaranteed by the normalizing constant. Proving $\int_{-\infty}^{\infty} e^{-x^2/2} \, dx = \sqrt{2\pi}$ requires a clever double-integral trick (switching to polar coordinates). We accept this result.
The empirical rule (68-95-99.7):
- $P(\mu - \sigma < Y < \mu + \sigma) \approx 0.6827$ — about 68% of values within 1 SD
- $P(\mu - 2\sigma < Y < \mu + 2\sigma) \approx 0.9545$ — about 95% within 2 SDs
- $P(\mu - 3\sigma < Y < \mu + 3\sigma) \approx 0.9973$ — about 99.7% within 3 SDs

# See the empirical rule in action
par(mfrow = c(1, 1))
curve(dnorm(x, 0, 1), from = -4, to = 4, lwd = 2,
      main = "Standard Normal: The 68-95-99.7 Rule",
      xlab = "z (standard deviations from the mean)", ylab = "f(z)")

# Shade the 68% region
x_fill <- seq(-1, 1, length = 200)
polygon(c(-1, x_fill, 1), c(0, dnorm(x_fill), 0),
        col = rgb(0, 0.5, 1, 0.3), border = NA)
text(0, 0.15, "68.27%", col = "blue", font = 2)

# Mark ±1, ±2, ±3
abline(v = c(-3, -2, -1, 1, 2, 3), lty = 2, col = "gray50")

5.3Standardization: The $Z$ -Transformation¶

There are infinitely many Normal distributions — one for every pair $(\mu, \sigma)$ . Yet we need only one table. The trick is standardization:

The key result:

Theorem 9 (Theorem 5.9: Standardization)

If $Y \sim N(\mu, \sigma^2)$ , then:

Z = \frac{Y - \mu}{\sigma} \sim N(0, 1)

Why this works: The transformation $Z = (Y - \mu)/\sigma$ shifts the distribution to center it at 0 (subtracting $\mu$ ) and rescales it to have standard deviation 1 (dividing by $\sigma$ ). The proof that $Z$ is Normal with mean 0 and variance 1 will be given in Chapter 7 using the transformation method.

The practical consequence: To find $P(a \leq Y \leq b)$ for any Normal distribution:

P(a \leq Y \leq b) = P\left(\frac{a - \mu}{\sigma} \leq Z \leq \frac{b - \mu}{\sigma}\right) = \Phi\left(\frac{b - \mu}{\sigma}\right) - \Phi\left(\frac{a - \mu}{\sigma}\right)

The $Z$ -value tells you how many standard deviations a value is from the mean. A $Z$ -value of 2 means “2 standard deviations above the mean.” A $Z$ -value of -1.5 means “1.5 standard deviations below the mean.”

5.4Using the Standard Normal Table¶

Table 4 in the Appendix gives $P(Z > z)$ — the right-tail area — for nonnegative values of $z$ . We denote this area $A(z) = P(Z > z) = 1 - \Phi(z)$ .

To handle all cases:

You Want	How to Get It
$P(Z > z)$ for $z > 0$	Read $A(z)$ directly from the table
$P(Z < z)$ for $z > 0$	$1 - A(z)$
$P(Z > z)$ for $z < 0$	$1 - A(
$P(Z < z)$ for $z < 0$	$A(
$P(a < Z < b)$	$\Phi(b) - \Phi(a)$ , or combine $A$ -values

5.5Worked Examples¶

Example 9 (EXAMPLE 5.9: Standard Normal Probabilities)

The Problem.

Let $Z \sim N(0, 1)$ . Find:

(a) $P(Z > 1.65)$

(b) $P(-2 \leq Z \leq 2)$

(d) Find $c$ such that $P(Z > c) = 0.10$ .

Solution.

(a) From the $Z$ -table: $A(1.65) = 0.0495$ .

P(Z > 1.65) = 0.0495

(b) By symmetry:

P(-2 \leq Z \leq 2) = 1 - 2 \cdot A(2.0) = 1 - 2(0.0228) = 1 - 0.0456 = 0.9544

About 95.44% of the probability lies within 2 standard deviations of the mean — the “95%” in the empirical rule.

(c) The area from 0 to 1.73:

P(0 \leq Z \leq 1.73) = P(Z > 0) - P(Z > 1.73) = 0.5 - A(1.73) = 0.5 - 0.0418 = 0.4582

(d) We need $A(c) = 0.10$ . Looking in the table for the area closest to 0.10:

$A(1.28) = 0.1003$ and $A(1.29) = 0.0985$ , so $c \approx 1.28$ .

In R.

# (a) P(Z > 1.65)
pnorm(1.65, lower.tail = FALSE)  # 0.04947

# (b) P(-2 <= Z <= 2)
pnorm(2) - pnorm(-2)  # 0.9545

# (c) P(0 <= Z <= 1.73)
pnorm(1.73) - pnorm(0)  # 0.4582

# (d) Find c such that P(Z > c) = 0.10
qnorm(0.10, lower.tail = FALSE)  # 1.2816

Example 10 (EXAMPLE 5.10: Pharmaceutical Tablet Quality Control)

The Problem.

A pharmaceutical company manufactures tablets that should contain 500 mg of active ingredient. Due to production variability, the actual content $Y$ follows $N(500, 25)$ — that is, $\mu = 500$ mg and $\sigma = 5$ mg. A tablet is defective if it contains less than 490 mg or more than 510 mg.

(a) What proportion of tablets are defective?

(b) The company wants to reduce the defective rate to at most 1%. What standard deviation $\sigma$ would achieve this?

Solution.

(a) Standardize:

P(Y < 490 \text{ or } Y > 510) = P\left(Z < \frac{490 - 500}{5}\right) + P\left(Z > \frac{510 - 500}{5}\right)

= P(Z < -2) + P(Z > 2) = 2 \cdot A(2.0) = 2(0.0228) = 0.0456

About 4.56% of tablets are defective.

(b) We need $P(Y < 490 \text{ or } Y > 510) \leq 0.01$ . By symmetry:

2 \cdot P\left(Z > \frac{10}{\sigma}\right) = 0.01 \implies P\left(Z > \frac{10}{\sigma}\right) = 0.005

From the table: $A(2.576) \approx 0.005$ , so:

\frac{10}{\sigma} = 2.576 \implies \sigma = \frac{10}{2.576} \approx 3.88 \text{ mg}

The company needs to tighten the standard deviation from 5 mg to about 3.88 mg — a 22% reduction in variability.

In R.

# (a) Defective proportion
pnorm(490, 500, 5) + pnorm(510, 500, 5, lower.tail = FALSE)  # 0.04550

# (b) Required sigma
10 / qnorm(0.005, lower.tail = FALSE)  # 3.882

Interpretation.

This is exactly how quality control works in manufacturing. The specification limits (490, 510) are fixed by regulation. The mean $\mu = 500$ is set by the production target. The only thing the company can control is $\sigma$ — the precision of the process. Reducing $\sigma$ requires better equipment, tighter process control, or both. The Normal distribution turns “how precise must we be?” into a concrete number: $\sigma \leq 3.88$ mg.

Example 11 (EXAMPLE 5.11: Achievement Test Scores)

The Problem.

Scores on a college entrance exam are Normally distributed with mean 72 and standard deviation 8.

(a) What fraction of scores falls between 78 and 90?

(b) What score marks the 90th percentile?

Solution.

$Y \sim N(72, 64)$ , so $\mu = 72$ and $\sigma = 8$ .

(a) Standardize both endpoints:

z_1 = \frac{78 - 72}{8} = 0.75 \qquad z_2 = \frac{90 - 72}{8} = 2.25

P(78 \leq Y \leq 90) = P(0.75 \leq Z \leq 2.25) = A(0.75) - A(2.25) = 0.2266 - 0.0122 = 0.2144

About 21.4% of scores fall between 78 and 90.

(b) The 90th percentile: $P(Y \leq \phi_{0.9}) = 0.90$ , so $P(Z \leq z_{0.9}) = 0.90$ , which means $P(Z > z_{0.9}) = 0.10$ .

From the table: $z_{0.9} \approx 1.28$ . Unstandardize:

\phi_{0.9} = \mu + z_{0.9} \cdot \sigma = 72 + 1.28(8) = 72 + 10.24 = 82.24

(c) The top 5% means $P(Y > c) = 0.05$ , so $P(Z > z) = 0.05$ . From the table: $z \approx 1.645$ .

c = 72 + 1.645(8) = 72 + 13.16 = 85.16

The minimum scholarship score is about 85.2.

In R.

# (a) P(78 <= Y <= 90)
pnorm(90, 72, 8) - pnorm(78, 72, 8)  # 0.2144

# (b) 90th percentile
qnorm(0.90, 72, 8)  # 82.25

# (c) Top 5% cutoff
qnorm(0.95, 72, 8)  # 85.16

Guided Practice 5.4: Normal Distribution Calculations

Solution.

The tensile strength of an aluminum alloy is Normally distributed with mean 12 GPa and standard deviation 1.6 GPa.

(a) Find the probability that a specimen has strength greater than 14.5 GPa.

z = \frac{14.5 - 12}{1.6} = \frac{2.5}{1.6} = 1.5625

P(Y > 14.5) = P(Z > 1.5625) \approx A(1.56) = 0.0594

In R: pnorm(14.5, 12, 1.6, lower.tail = FALSE) → 0.0593

(b) Find the first quartile of the strength distribution.

The first quartile satisfies $P(Y \leq \phi_{0.25}) = 0.25$ , so $P(Z \leq z_{0.25}) = 0.25$ .

By symmetry: $z_{0.25} = -z_{0.75}$ . From the table: $P(Z > 0.674) \approx 0.25$ , so $z_{0.25} = -0.674$ .

\phi_{0.25} = 12 + (-0.674)(1.6) = 12 - 1.078 = 10.92 \text{ GPa}

In R: qnorm(0.25, 12, 1.6) → 10.92

(c) The manufacturer rejects specimens below 9 GPa. What proportion is rejected?

z = \frac{9 - 12}{1.6} = -1.875

P(Y < 9) = P(Z < -1.875) = A(1.875) \approx A(1.88) = 0.0301

About 3% are rejected.

In R: pnorm(9, 12, 1.6) → 0.03013

5.6The Standard Normal as a Reference¶

Notice the pattern in every example:

Standardize: Convert $Y$ -values to $Z$ -values using $z = (y - \mu)/\sigma$ .
Look up: Find the probability using the $Z$ -table or R’s pnorm().
Unstandardize (if needed): Convert $Z$ -values back to $Y$ -values using $y = \mu + z\sigma$ .

This three-step pattern works for every Normal probability problem. The key R functions:

Function	What It Computes
`pnorm(y, mu, sigma)`	$P(Y \leq y)$ — CDF
`pnorm(y, mu, sigma, lower.tail = FALSE)`	$P(Y > y)$ — right tail
`qnorm(p, mu, sigma)`	$\phi_p$ — the $p$ th quantile
`dnorm(y, mu, sigma)`	$f(y)$ — the density (not a probability!)
`rnorm(n, mu, sigma)`	Generate $n$ random Normal values

# Simulate: verify the 68-95-99.7 rule
set.seed(2026)
Y <- rnorm(1000000, mean = 50, sd = 10)
mean(abs(Y - 50) < 10)   # ≈ 0.6827 (within 1 SD)
mean(abs(Y - 50) < 20)   # ≈ 0.9545 (within 2 SDs)
mean(abs(Y - 50) < 30)   # ≈ 0.9973 (within 3 SDs)

⚠️ Common Mistakes — The Normal Distribution

Writing $N(\mu, \sigma)$ instead of $N(\mu, \sigma^2)$ . The second parameter of $N(\mu, \sigma^2)$ is the variance, not the standard deviation. If $Y \sim N(100, 25)$ , then $\sigma^2 = 25$ and $\sigma = 5$ . Some textbooks use the convention $N(\mu, \sigma)$ — this book uses $N(\mu, \sigma^2)$ , and so does the reference textbook.
Forgetting to standardize. You cannot look up $P(Y > 78)$ directly in the $Z$ -table if $Y \sim N(72, 64)$ . You must first compute $z = (78 - 72)/8 = 0.75$ and then look up $P(Z > 0.75)$ .
Using the wrong tail. If the table gives $A(z) = P(Z > z)$ , then $P(Z < z) = 1 - A(z)$ , and $P(Z < -z) = A(z)$ by symmetry. Draw a picture every time until the pattern becomes automatic.
Confusing $f(y)$ with $P(Y = y)$ . The Normal density $f(y)$ can be computed with dnorm(), but this is the height of the bell curve, not a probability. For continuous distributions, $P(Y = y) = 0$ always. Use pnorm() for probabilities.
Thinking the Normal distribution is bounded. The Normal distribution extends from $-\infty$ to $+\infty$ . A $N(500, 25)$ model technically assigns positive probability to negative values — but $P(Y < 0) = P(Z < -100) \approx 0$ , which is effectively zero. In practice, this is fine.

5.7Section 5.5 Exercises¶

5.5.1. Let $Z \sim N(0, 1)$ . Find:

(a) $P(Z > 1.96)$ (b) $P(-1.5 < Z < 0.5)$ (c) $P(|Z| > 2.33)$ (d) The value $c$ such that $P(-c < Z < c) = 0.90$

5.5.2. The diastolic blood pressure of adult males is Normally distributed with mean 82 mmHg and standard deviation 10 mmHg.

(a) What proportion of men have blood pressure above 100 mmHg? (b) A man is classified as having “high blood pressure” if his reading exceeds 90 mmHg. What proportion of men fall in this category? (c) A health study recruits men with blood pressure between 70 and 90 mmHg. What proportion of the male population is eligible? (d) What blood pressure value marks the 95th percentile?

5.5.3. A machine fills cereal boxes. The weight of cereal per box is $N(368, 16)$ grams. The label says 360 grams.

(a) What proportion of boxes are underweight (less than 360 g)? (b) The company wants at most 2% of boxes to be underweight. If $\mu$ cannot be changed, what is the maximum allowable $\sigma$ ? (c) Alternatively, if $\sigma = 4$ is fixed, what mean $\mu$ would ensure at most 2% underweight?

5.5.4. Two lightbulbs are installed in a fixture. Bulb A has a lifetime that is $N(900, 10000)$ hours and Bulb B has a lifetime that is $N(1100, 22500)$ hours. The lifetimes are independent.

(a) Find the probability that Bulb A lasts more than 1000 hours. (b) Let $D = B - A$ be the difference in lifetimes. What is the distribution of $D$ ? (Hint: A linear combination of independent Normals is Normal. What are $E(D)$ and $V(D)$ ?) (c) Find the probability that Bulb B outlasts Bulb A by more than 300 hours.

5.5.5. (Proof) Show that if $Y \sim N(\mu, \sigma^2)$ , then $P(Y \leq \mu) = 0.5$ . (Hint: Use the substitution $u = 2\mu - y$ to show that $\int_{-\infty}^{\mu} f(y) \, dy = \int_{\mu}^{\infty} f(y) \, dy$ .)

🔬 What Did We Learn?

The Normal distribution $N(\mu, \sigma^2)$ is the bell-shaped curve defined by two parameters: the mean $\mu$ (the center) and the variance $\sigma^2$ (the spread). Its density is symmetric about $\mu$ , and 68/95/99.7% of the probability lies within 1/2/3 standard deviations of the mean.

The key technique is standardization: convert any Normal variable $Y$ to the standard Normal $Z = (Y - \mu)/\sigma$ , look up the probability, and convert back if needed. The $Z$ -value tells you “how many standard deviations from the mean.”

The real-life interpretation: When a doctor says your cholesterol is “1.5 standard deviations above average,” that’s a $Z$ -value. When a student asks “is a score of 82 good on this test?”, the answer depends on $\mu$ and $\sigma$ — a score of 82 on a $N(75, 25)$ exam ( $z = 1.4$ , top 8%) is very different from 82 on a $N(85, 9)$ exam ( $z = -1.0$ , bottom 16%). The $Z$ -score is the universal yardstick for comparing values across different Normal distributions.

The Normal distribution also has a deep connection to other distributions we’ll study. The Chi-Square distribution (Section 5.6) arises from squaring standard Normals. The $t$ -distribution and $F$ -distribution (which you’ll meet in mathematical statistics) are built from ratios involving Normal and Chi-Square variables. The Normal is not just one distribution — it’s the foundation of an entire family.

Looking ahead: The next section introduces the Gamma distribution family — a versatile family of right-skewed distributions that includes the Exponential (for modeling wait times) and the Chi-Square (for statistical inference) as special cases.

6The Gamma Distribution Family¶

🔭 What Are We About to Learn?

The Normal distribution is symmetric — it works beautifully when data clusters around a central value with equal spread on both sides. But many real-world quantities are right-skewed: they have a natural lower bound (often zero), a peak near the left, and a long tail stretching to the right. Wait times, lifetimes of equipment, insurance claims, rainfall amounts — all share this shape.

The Gamma distribution is a flexible family designed for exactly these situations. It has two parameters — shape ( $\alpha$ ) and scale ( $\beta$ ) — that let it take on a wide range of right-skewed shapes, from sharply peaked to nearly flat.

What makes the Gamma family powerful is that it contains two famous distributions as special cases:

The Exponential distribution ( $\alpha = 1$ ): models the waiting time until the next event in a Poisson process. It has the remarkable memoryless property — the probability of waiting $t$ more minutes is the same whether you’ve been waiting 5 minutes or 5 hours.
The Chi-Square distribution ( $\alpha = \nu/2$ , $\beta = 2$ ): arises from sums of squared standard Normals. It is the foundation of hypothesis testing in mathematical statistics.

By learning the Gamma distribution first, you get all three distributions at once.

Real-life motivation: A nuclear reactor contains components whose lifetimes follow Gamma distributions. A technician needs to know: what’s the probability a component fails within the first 1,000 hours? What’s the expected replacement cost per year? The Gamma distribution answers both. Meanwhile, the call center down the road uses the Exponential distribution to model the time between incoming calls — and the quality engineer upstairs uses the Chi-Square distribution to test whether the production line meets specifications. All three are the same family.

6.1The Gamma Function¶

Before defining the Gamma distribution, we need a mathematical tool: the Gamma function.

The Gamma function is an extension of the factorial to all positive real numbers (not just integers). Its key properties:

Theorem 10 (Theorem 5.10: Properties of the Gamma Function)

Recursion: $\Gamma(\alpha) = (\alpha - 1) \cdot \Gamma(\alpha - 1)$ for $\alpha > 1$
Factorial connection: $\Gamma(n) = (n - 1)!$ for any positive integer $n$
Special value: $\Gamma(1) = 1$
Half-integer value: $\Gamma(1/2) = \sqrt{\pi}$

Proof of Property 1 (Recursion).

Strategy: We’ll use integration by parts on $\Gamma(\alpha) = \int_0^{\infty} t^{\alpha-1} e^{-t} \, dt$ .

Set $u = t^{\alpha-1}$ and $dv = e^{-t} \, dt$ . Then:

du = (\alpha - 1)t^{\alpha - 2} \, dt \qquad \text{and} \qquad v = \int e^{-t} \, dt = -e^{-t}

Applying integration by parts:

\Gamma(\alpha) = \underbrace{\left[t^{\alpha-1} \cdot (-e^{-t})\right]_0^{\infty}}_{\text{boundary term}} + (\alpha - 1)\int_0^{\infty} t^{\alpha-2} e^{-t} \, dt

Evaluating the boundary term: We need $\lim_{t \to \infty} t^{\alpha-1}e^{-t}$ and the value at $t = 0$ .

As $t \to \infty$ : the exponential $e^{-t}$ decays to zero faster than any polynomial $t^{\alpha-1}$ grows (this is a standard calculus fact: exponentials always beat polynomials). So $\lim_{t \to \infty} t^{\alpha-1}e^{-t} = 0$ .
At $t = 0$ : $0^{\alpha-1} \cdot e^{0} = 0 \cdot 1 = 0$ (since $\alpha > 1$ means $\alpha - 1 > 0$ ).

The boundary term is $0 - 0 = 0$ . Therefore:

\Gamma(\alpha) = 0 + (\alpha - 1) \int_0^{\infty} t^{\alpha-2} e^{-t} \, dt = (\alpha - 1) \cdot \Gamma(\alpha - 1) \qquad \blacksquare

Proof of Property 3: $\Gamma(1) = \int_0^{\infty} t^{1-1}e^{-t} \, dt = \int_0^{\infty} e^{-t} \, dt = \left[-e^{-t}\right]_0^{\infty} = 0 - (-1) = 1$ .

Proof of Property 2 (by induction): From Properties 1 and 3:

\Gamma(2) = 1 \cdot \Gamma(1) = 1 = 1!

\Gamma(3) = 2 \cdot \Gamma(2) = 2 \cdot 1 = 2!

\Gamma(4) = 3 \cdot \Gamma(3) = 3 \cdot 2 = 6 = 3!

In general: $\Gamma(n) = (n-1) \cdot (n-2) \cdots 2 \cdot 1 = (n-1)!$ for any positive integer $n$ .

# Verify Gamma function properties
gamma(1)      # 1  (= 0!)
gamma(2)      # 1  (= 1!)
gamma(5)      # 24 (= 4!)
gamma(0.5)    # 1.7724539 (= sqrt(pi))
sqrt(pi)      # 1.7724539 ✓
gamma(10)     # 362880 (= 9!)
factorial(9)  # 362880 ✓

6.2The Gamma Distribution¶

The density is zero for $y \leq 0$ (the Gamma distribution lives on the positive real line), and the denominator $\beta^{\alpha}\Gamma(\alpha)$ is the normalizing constant that makes the total area 1.

Verification that $\int_0^{\infty} f(y) \, dy = 1$ :

We need to show $\int_0^{\infty} \frac{y^{\alpha-1}e^{-y/\beta}}{\beta^{\alpha}\Gamma(\alpha)} \, dy = 1$ . Substitute $u = y/\beta$ :

If $u = y/\beta$ , then $y = u\beta$ and $dy = \beta \, du$
When $y = 0$ : $u = 0$ . When $y \to \infty$ : $u \to \infty$ . (Limits don’t change their form.)

Substituting piece by piece:

\int_0^{\infty} \frac{y^{\alpha-1} e^{-y/\beta}}{\beta^{\alpha}\Gamma(\alpha)} \, dy = \frac{1}{\beta^{\alpha}\Gamma(\alpha)} \int_0^{\infty} \underbrace{(u\beta)^{\alpha-1}}_{y^{\alpha-1}} \cdot \underbrace{e^{-u}}_{e^{-y/\beta}} \cdot \underbrace{\beta \, du}_{dy}

Now expand $(u\beta)^{\alpha-1} = u^{\alpha-1} \cdot \beta^{\alpha-1}$ , and combine with the extra $\beta$ from $dy$ :

= \frac{1}{\beta^{\alpha}\Gamma(\alpha)} \cdot \beta^{\alpha-1} \cdot \beta \int_0^{\infty} u^{\alpha-1}e^{-u} \, du = \frac{\beta^{\alpha}}{\beta^{\alpha}\Gamma(\alpha)} \int_0^{\infty} u^{\alpha-1}e^{-u} \, du

The remaining integral is exactly $\Gamma(\alpha)$ by definition:

= \frac{\cancel{\beta^{\alpha}}}{\cancel{\beta^{\alpha}}} \cdot \frac{\cancel{\Gamma(\alpha)}}{\cancel{\Gamma(\alpha)}} = 1 \qquad \checkmark

This substitution trick — replacing $y/\beta$ with $u$ to convert a Gamma integral into the definition of $\Gamma(\alpha)$ — will appear repeatedly throughout this chapter and in Section 5.8.

# See how the Gamma shape changes with alpha
par(mfrow = c(2, 2))
for (a in c(1, 2, 5, 10)) {
  curve(dgamma(x, shape = a, scale = 2), from = 0, to = 40, lwd = 2,
        main = bquote(alpha == .(a) ~ ", " ~ beta == 2),
        xlab = "y", ylab = "f(y)", col = "steelblue")
  abline(v = a * 2, lty = 2, col = "red")  # mean
}

Run this code and notice: when $\alpha = 1$ , the density is a decreasing exponential. As $\alpha$ increases, the density becomes more bell-shaped and shifts to the right. The red dashed line marks the mean $\alpha\beta$ . The Gamma distribution is remarkably flexible.

6.3Special Case 1: The Exponential Distribution¶

The CDF has a closed form (unusual for the Gamma family). Let’s derive it step by step:

F(y) = \int_0^y \frac{1}{\beta}e^{-t/\beta} \, dt

F(y) = \frac{1}{\beta} \cdot \left(-\beta \, e^{-t/\beta}\right)\bigg|_0^y = -e^{-t/\beta}\bigg|_0^y = -e^{-y/\beta} - (-e^{0}) = 1 - e^{-y/\beta}

So the Exponential CDF is:

F(y) = 1 - e^{-y/\beta}, \qquad y > 0

This is one of the few distributions where the CDF has a clean closed form — no tables or software needed.

6.3.1The Memoryless Property¶

The Exponential distribution is the only continuous distribution with this remarkable property:

Theorem 12 (Theorem 5.12: Memoryless Property of the Exponential)

If $Y \sim \text{Exponential}(\beta)$ , then for any $s, t > 0$ :

P(Y > s + t \mid Y > s) = P(Y > t)

In words: Given that you’ve already waited $s$ units of time, the probability of waiting at least $t$ more is the same as if you’d just started waiting. The distribution “forgets” how long you’ve been waiting.

Proof.

Strategy: Use the definition of conditional probability and the exponential survival function $P(Y > y) = e^{-y/\beta}$ .

P(Y > s + t \mid Y > s) = \frac{P(Y > s + t \text{ and } Y > s)}{P(Y > s)} = \frac{P(Y > s + t)}{P(Y > s)}

= \frac{e^{-(s+t)/\beta}}{e^{-s/\beta}} = e^{-t/\beta} = P(Y > t) \qquad \blacksquare

Why this matters: The memoryless property is why the Exponential distribution models the waiting time between Poisson events. If calls arrive at a switchboard according to a Poisson process, the time until the next call is Exponential — and no matter how long you’ve been waiting, the distribution of your remaining wait is the same.

6.3.2The Poisson–Exponential Connection¶

There is a deep link between two distributions from different chapters:

Example 12 (EXAMPLE 5.12: Exponential Distribution — Transistor Lifetimes)

The Problem.

Under harsh operating conditions, the lifetime (in months) of a transistor is Exponentially distributed with parameter $\beta = 5$ (mean lifetime = 5 months).

(a) Find the probability that the transistor lasts longer than 8 months.

(b) Find the probability that the lifetime is between 3 and 7 months.

(d) Given that a transistor has survived 6 months, find the probability it survives at least 2 more months.

Solution.

$Y \sim \text{Exponential}(\beta = 5)$ , so $f(y) = \frac{1}{5}e^{-y/5}$ and $F(y) = 1 - e^{-y/5}$ .

(a) $P(Y > 8) = e^{-8/5} = e^{-1.6} \approx 0.2019$

(b) $P(3 < Y < 7) = F(7) - F(3) = (1 - e^{-7/5}) - (1 - e^{-3/5}) = e^{-0.6} - e^{-1.4} = 0.5488 - 0.2466 = 0.3022$

(c) Solve $F(\phi_{0.5}) = 0.5$ : $1 - e^{-\phi_{0.5}/5} = 0.5 \implies e^{-\phi_{0.5}/5} = 0.5 \implies \phi_{0.5} = -5\ln(0.5) = 5\ln(2) \approx 3.466$ months.

The median (3.47 months) is less than the mean (5 months) — the Exponential distribution is heavily right-skewed.

(d) By the memoryless property:

P(Y > 8 \mid Y > 6) = P(Y > 2) = e^{-2/5} = e^{-0.4} \approx 0.6703

No calculation involving the 6 months of past survival is needed — the distribution doesn’t “remember.”

In R.

# Y ~ Exponential(beta = 5), or equivalently rate = 1/5
# (a) P(Y > 8)
pexp(8, rate = 1/5, lower.tail = FALSE)  # 0.2019

# (b) P(3 < Y < 7)
pexp(7, rate = 1/5) - pexp(3, rate = 1/5)  # 0.3022

# (c) Median
qexp(0.5, rate = 1/5)  # 3.466

# (d) Memoryless: P(Y > 2)
pexp(2, rate = 1/5, lower.tail = FALSE)  # 0.6703

Example 13 (EXAMPLE 5.13: Poisson–Exponential Connection)

The Problem.

A radioactive source emits particles according to a Poisson process at a mean rate of 3 particles per second. Let $T$ be the waiting time (in seconds) between consecutive emissions.

(a) What is the distribution of $T$ ? Find its mean and standard deviation.

(b) Find $P(T > 1)$ .

(d) If 2 seconds have elapsed since the last emission, what is the probability that the next emission occurs within 0.5 seconds?

Solution.

The Poisson rate is $\lambda = 3$ per second, so $T \sim \text{Exponential}(\beta = 1/3)$ , or equivalently rate $= 3$ .

(a) $E(T) = 1/\lambda = 1/3$ seconds, $\sigma = 1/\lambda = 1/3$ seconds. (For the Exponential, the mean and SD are always equal.)

(b) $P(T > 1) = e^{-3(1)} = e^{-3} \approx 0.0498$

(c) $P(0.2 < T < 0.8) = e^{-3(0.2)} - e^{-3(0.8)} = e^{-0.6} - e^{-2.4} = 0.5488 - 0.0907 = 0.4581$

(d) Memoryless property: $P(T \leq 2.5 \mid T > 2) = P(T \leq 0.5) = 1 - e^{-3(0.5)} = 1 - e^{-1.5} \approx 0.7769$ .

The 2 seconds of waiting are irrelevant.

6.4Special Case 2: The Chi-Square Distribution¶

Why does the Chi-Square matter? If $Z_1, Z_2, \ldots, Z_\nu$ are independent standard Normal random variables, then:

Z_1^2 + Z_2^2 + \cdots + Z_\nu^2 \sim \chi^2(\nu)

This is the bridge between the Normal distribution and statistical inference. In mathematical statistics, the Chi-Square distribution is used for testing hypotheses about variances, goodness-of-fit tests, and constructing confidence intervals.

# Visualize the Gamma family tree
par(mfrow = c(1, 3))

# Exponential: Gamma(1, 2)
curve(dgamma(x, 1, scale = 2), 0, 15, lwd = 2, col = "steelblue",
      main = "Exponential(β=2)\n= Gamma(1, 2)", ylab = "f(y)")

# Chi-Square(6): Gamma(3, 2)
curve(dgamma(x, 3, scale = 2), 0, 25, lwd = 2, col = "darkorange",
      main = "Chi-Square(ν=6)\n= Gamma(3, 2)", ylab = "f(y)")

# General Gamma(5, 3)
curve(dgamma(x, 5, scale = 3), 0, 40, lwd = 2, col = "purple",
      main = "Gamma(α=5, β=3)", ylab = "f(y)")

6.5The Gamma Family Tree¶

The relationships are worth seeing in one place:

The Gamma Family at a Glance

Distribution	Parameters	As Gamma	Mean	Variance
$\text{Gamma}(\alpha, \beta)$	shape $\alpha$ , scale $\beta$	—	$\alpha\beta$	$\alpha\beta^2$
$\text{Exponential}(\beta)$	scale $\beta$ (or rate $\lambda = 1/\beta$ )	$\text{Gamma}(1, \beta)$	$\beta$	$\beta^2$
$\chi^2(\nu)$	degrees of freedom $\nu$	$\text{Gamma}(\nu/2, 2)$	$\nu$	$2\nu$

One family, three names, one set of formulas.

Example 14 (EXAMPLE 5.14: General Gamma Distribution)

The Problem.

The time (in minutes) a randomly selected student spends using a computer terminal has a Gamma distribution with mean 20 minutes and variance 80 min $^2$ .

(a) Find the parameters $\alpha$ and $\beta$ .

(b) Find the probability that a student uses the terminal for at most 24 minutes.

Solution.

(a) From the mean and variance formulas:

\alpha\beta = 20 \qquad \text{and} \qquad \alpha\beta^2 = 80

Dividing the second by the first: $\beta = 80/20 = 4$ . Then $\alpha = 20/4 = 5$ .

So $Y \sim \text{Gamma}(5, 4)$ .

(b) There is no closed-form CDF for the general Gamma distribution (except when $\alpha$ is a positive integer, via integration by parts). We use R:

# Y ~ Gamma(shape = 5, scale = 4)
pgamma(24, shape = 5, scale = 4)  # 0.7149

$P(Y \leq 24) \approx 0.715$ .

(c) $P(15 \leq Y \leq 30) = F(30) - F(15)$ :

pgamma(30, shape = 5, scale = 4) - pgamma(15, shape = 5, scale = 4)
# 0.5455

About 55% of students spend between 15 and 30 minutes.

Interpretation.

To find $\alpha$ and $\beta$ from the mean and variance, always use the same trick: divide $V(Y) = \alpha\beta^2$ by $E(Y) = \alpha\beta$ to get $\beta$ , then solve for $\alpha$ . This works because the $\alpha$ cancels cleanly.

Guided Practice 5.5: Identifying and using Gamma family distributions

Solution.

A random variable $Y$ has the density function:

f(y) = \begin{cases} k \, y^3 \, e^{-y/2}, & y > 0 \\ 0, & \text{elsewhere} \end{cases}

(a) Identify the distribution by matching to the Gamma density.

Compare with the Gamma density: $f(y) = \frac{y^{\alpha-1}e^{-y/\beta}}{\beta^{\alpha}\Gamma(\alpha)}$ .

Exponent of $y$ : $3 = \alpha - 1 \implies \alpha = 4$
Exponent of $e$ : $-y/2 \implies \beta = 2$

So $Y \sim \text{Gamma}(4, 2)$ , which is also $\chi^2(8)$ (since $\alpha = \nu/2 = 4 \implies \nu = 8$ , and $\beta = 2$ ).

(b) Find $k$ .

k = \frac{1}{\beta^{\alpha}\Gamma(\alpha)} = \frac{1}{2^4 \cdot \Gamma(4)} = \frac{1}{16 \cdot 6} = \frac{1}{96}

(c) Find $E(Y)$ and $V(Y)$ .

E(Y) = \alpha\beta = 4(2) = 8 \qquad V(Y) = \alpha\beta^2 = 4(4) = 16

Or, since $Y \sim \chi^2(8)$ : $E(Y) = \nu = 8$ and $V(Y) = 2\nu = 16$ ✓.

(d) Find $P(Y > 12)$ .

pgamma(12, shape = 4, scale = 2, lower.tail = FALSE)  # 0.1512
# Equivalently:
pchisq(12, df = 8, lower.tail = FALSE)  # 0.1512  ✓

6.6Section 5.6 Exercises¶

5.6.1. The time (in hours) until a server crashes follows an Exponential distribution with mean 200 hours.

(a) Find the probability that the server runs for at least 300 hours without crashing. (b) Given that it has run for 150 hours, find the probability it runs at least 100 more hours. (c) What is the median time to failure? (d) Find the probability that the server crashes within the first 50 hours.

5.6.2. Calls arrive at a helpline at a Poisson rate of 4 per hour. Let $T$ be the time (in hours) between consecutive calls.

(a) What is the distribution of $T$ ? State its mean and variance. (b) Find $P(T > 0.5)$ . (c) Find $P(T < 0.1)$ . (d) A new operator is told “on average, you’ll wait 15 minutes between calls.” Verify this claim.

5.6.3. A random variable $Y$ has the density:

f(y) = \begin{cases} \frac{1}{6}y \, e^{-y/3}, & y > 0 \\ 0, & \text{elsewhere} \end{cases}

(a) Identify the Gamma parameters $\alpha$ and $\beta$ . (b) Find $E(Y)$ and $\sigma$ . (c) Find $P(Y > 10)$ using R.

5.6.4. A Chi-Square random variable has 12 degrees of freedom.

(a) State the mean and variance. (b) Express this as a Gamma distribution. (c) Use R to find $P(\chi^2_{12} > 21.026)$ . What significance does this value have?

5.6.5. (Proof) Prove the memoryless property of the Exponential distribution: if $Y \sim \text{Exp}(\beta)$ , then $P(Y > s + t \mid Y > s) = P(Y > t)$ for all $s, t > 0$ . Then show that if $Y$ has a non-Exponential continuous distribution supported on $(0, \infty)$ , the memoryless property fails. (Hint: For the second part, consider a specific counterexample such as the Gamma(2, 1) distribution.)

5.6.6. Show that $\Gamma(1/2) = \sqrt{\pi}$ . (Hint: Write $\Gamma(1/2) = \int_0^{\infty} t^{-1/2}e^{-t} \, dt$ , make the substitution $t = u^2/2$ , and recognize the resulting integral as related to the standard Normal density.)

⚠️ Common Mistakes — The Gamma Family

Confusing scale $\beta$ with rate $\lambda = 1/\beta$ . Some textbooks (and R’s dexp function) parameterize the Exponential by the rate $\lambda$ . This book (and the reference textbook) use the scale $\beta = 1/\lambda$ . In R: dexp(y, rate = lambda) = dgamma(y, shape = 1, scale = 1/lambda). Always check which convention your source uses.
Forgetting that $\Gamma(n) = (n-1)!$ , not $n!$ . The Gamma function is “shifted” by one compared to the factorial: $\Gamma(5) = 4! = 24$ , not $5! = 120$ .
Trying to find a closed-form CDF for the general Gamma. There is no closed-form CDF when $\alpha$ is not a positive integer. Use R’s pgamma() or tables. The Exponential ( $\alpha = 1$ ) is the exception — its CDF is $1 - e^{-y/\beta}$ .
Applying the memoryless property to non-Exponential distributions. Only the Exponential distribution (among continuous distributions) is memoryless. If a component’s lifetime is Gamma(2, 5) instead of Exponential(5), the conditional probability $P(Y > s+t \mid Y > s) \neq P(Y > t)$ . Don’t assume memorylessness without checking that $\alpha = 1$ .
Mixing up Chi-Square parameters. The $\chi^2(\nu)$ distribution has $E(Y) = \nu$ and $V(Y) = 2\nu$ . Students sometimes write $V(Y) = \nu$ or $E(Y) = 2\nu$ . Remember: the variance is twice the mean.

🔬 What Did We Learn?

The Gamma distribution is a flexible, right-skewed family defined by shape $\alpha$ and scale $\beta$ . It lives on $(0, \infty)$ and has mean $\alpha\beta$ and variance $\alpha\beta^2$ . Two critically important distributions are special cases:

Exponential ( $\alpha = 1$ ): models wait times between Poisson events, has the memoryless property, and has a closed-form CDF $F(y) = 1 - e^{-y/\beta}$ .
Chi-Square ( $\alpha = \nu/2$ , $\beta = 2$ ): the sum of $\nu$ squared standard Normals, with mean $\nu$ and variance $2\nu$ .

The real-life interpretation: The Exponential distribution governs “waiting for the next event” — the next call, the next failure, the next emission. Its memoryless property is both mathematically elegant and practically useful: if a component’s lifetime is truly Exponential, then a 5-year-old component is just as likely to fail tomorrow as a brand-new one. This is why the Exponential models random failure (like a light bulb burning out) but not wear-out failure (like brake pads wearing thin). For wear-out, you need a Gamma with $\alpha > 1$ — a distribution that “remembers” how long the component has been in use.

The Gamma function $\Gamma(\alpha)$ is the mathematical glue holding the family together, extending the factorial to non-integer values and serving as the normalizing constant for the Gamma density.

Looking ahead: The Beta distribution (Section 5.7) is the next named distribution — it models proportions and percentages, living on the interval $[0, 1]$ rather than on $(0, \infty)$ .

7The Beta Distribution¶

🔭 What Are We About to Learn?

All the distributions so far live on infinite or semi-infinite intervals: the Normal on $(-\infty, \infty)$ , the Gamma on $(0, \infty)$ , the Uniform on any finite interval. But what if your random variable is a proportion — a number that must lie between 0 and 1?

The proportion of defective items in a shipment. The fraction of the day a machine is running. A baseball player’s batting average. A website’s click-through rate. All are numbers between 0 and 1, and the Beta distribution is designed specifically to model them.

The Beta distribution has two shape parameters, $\alpha$ and $\beta$ , that control whether the density is U-shaped, bell-shaped, left-skewed, right-skewed, or flat. When $\alpha = \beta = 1$ , it reduces to the Uniform(0, 1) — so the Uniform is actually a special case of the Beta.

Real-life motivation: A factory’s weekly repair cost, expressed as a fraction of the total budget, follows a Beta distribution. If $\alpha$ and $\beta$ are known, management can answer: “What is the probability that repairs consume more than 40% of our budget this week?” That’s a Beta probability.

7.1Definition¶

Definition 14 (Definition 5.14: The Beta Distribution)

A random variable $Y$ has a Beta distribution with parameters $\alpha > 0$ and $\beta > 0$ , written $Y \sim \text{Beta}(\alpha, \beta)$ , if and only if:

f(y) = \begin{cases} \dfrac{y^{\alpha - 1}(1 - y)^{\beta - 1}}{B(\alpha, \beta)}, & 0 \leq y \leq 1 \\[8pt] 0, & \text{elsewhere} \end{cases}

where $B(\alpha, \beta)$ is the Beta function:

B(\alpha, \beta) = \int_0^1 t^{\alpha-1}(1-t)^{\beta-1} \, dt = \frac{\Gamma(\alpha)\Gamma(\beta)}{\Gamma(\alpha + \beta)}

The Beta function $B(\alpha, \beta)$ serves as the normalizing constant — it ensures $\int_0^1 f(y) \, dy = 1$ . The relationship $B(\alpha, \beta) = \Gamma(\alpha)\Gamma(\beta)/\Gamma(\alpha + \beta)$ connects the Beta function to the Gamma function, and allows us to evaluate it using factorials when $\alpha$ and $\beta$ are integers.

7.2Mean and Variance¶

Theorem 13 (Theorem 5.13: Mean and Variance of the Beta Distribution)

If $Y \sim \text{Beta}(\alpha, \beta)$ , then:

E(Y) = \frac{\alpha}{\alpha + \beta} \qquad \text{and} \qquad V(Y) = \frac{\alpha\beta}{(\alpha + \beta)^2(\alpha + \beta + 1)}

Proof of $E(Y)$ .

Strategy: Integrate $y \cdot f(y)$ over $[0, 1]$ , then recognize the result as a ratio of Beta functions.

E(Y) = \int_0^1 y \cdot \frac{y^{\alpha-1}(1-y)^{\beta-1}}{B(\alpha,\beta)} \, dy = \frac{1}{B(\alpha,\beta)} \int_0^1 y^{\alpha}(1-y)^{\beta-1} \, dy

The integral $\int_0^1 y^{\alpha}(1-y)^{\beta-1} \, dy$ has the form $\int_0^1 y^{(\alpha+1)-1}(1-y)^{\beta-1} \, dy = B(\alpha+1, \beta)$ . So:

E(Y) = \frac{B(\alpha+1, \beta)}{B(\alpha, \beta)} = \frac{\Gamma(\alpha+1)\Gamma(\beta)/\Gamma(\alpha+\beta+1)}{\Gamma(\alpha)\Gamma(\beta)/\Gamma(\alpha+\beta)}

The $\Gamma(\beta)$ terms cancel. Using the recursion $\Gamma(\alpha+1) = \alpha \cdot \Gamma(\alpha)$ and $\Gamma(\alpha+\beta+1) = (\alpha+\beta)\Gamma(\alpha+\beta)$ :

= \frac{\alpha \cdot \cancel{\Gamma(\alpha)} \cdot \cancel{\Gamma(\alpha+\beta)}}{\cancel{\Gamma(\alpha)} \cdot (\alpha+\beta) \cdot \cancel{\Gamma(\alpha+\beta)}} = \frac{\alpha}{\alpha + \beta} \qquad \blacksquare

The variance proof follows the same pattern (computing $E(Y^2)$ via $B(\alpha+2, \beta)/B(\alpha, \beta)$ and applying the shortcut formula). We leave the detailed algebra as Exercise 5.7.5.

7.3Special Cases and Shape¶

The Beta distribution is a shape-shifter:

$\alpha = \beta = 1$ : $f(y) = 1$ — the $\text{Uniform}(0, 1)$
$\alpha = \beta > 1$ : symmetric, bell-shaped, centered at $1/2$
$\alpha > \beta$ : left-skewed (piles up near 1)
$\alpha < \beta$ : right-skewed (piles up near 0)
$\alpha < 1$ and $\beta < 1$ : U-shaped (concentrated near 0 and 1)

# Visualize the Beta family's flexibility
par(mfrow = c(2, 2))
curve(dbeta(x, 1, 1), 0, 1, lwd = 2, col = "steelblue", ylim = c(0, 3),
      main = "Beta(1,1) = Uniform", ylab = "f(y)")
curve(dbeta(x, 5, 5), 0, 1, lwd = 2, col = "darkorange",
      main = "Beta(5,5): Symmetric bell", ylab = "f(y)")
curve(dbeta(x, 2, 8), 0, 1, lwd = 2, col = "purple",
      main = "Beta(2,8): Right-skewed", ylab = "f(y)")
curve(dbeta(x, 0.5, 0.5), 0, 1, lwd = 2, col = "red",
      main = "Beta(0.5,0.5): U-shaped", ylab = "f(y)")

Example 15 (EXAMPLE 5.15: Weekly Repair Costs)

The Problem.

The weekly repair cost $Y$ for a machine, measured as a proportion of the weekly budget, has the density:

f(y) = \begin{cases} 6y(1 - y), & 0 \leq y \leq 1 \\ 0, & \text{elsewhere} \end{cases}

(a) Identify the distribution.

(b) Find $E(Y)$ and $V(Y)$ .

(d) How much should be budgeted so actual costs exceed the budget only 10% of the time?

Solution.

(a) Compare $f(y) = 6y(1 - y) = 6 y^{2-1}(1-y)^{2-1}$ with the Beta density. We identify $\alpha = 2$ , $\beta = 2$ , and the constant is $6 = 1/B(2,2) = \Gamma(4)/[\Gamma(2)\Gamma(2)] = 6/(1 \cdot 1) = 6$ ✓.

So $Y \sim \text{Beta}(2, 2)$ .

(b) Using the formulas:

E(Y) = \frac{2}{2+2} = \frac{1}{2} = 0.50

V(Y) = \frac{2 \cdot 2}{(2+2)^2(2+2+1)} = \frac{4}{16 \cdot 5} = \frac{4}{80} = \frac{1}{20} = 0.05

\sigma = \sqrt{0.05} \approx 0.2236

(c) $P(Y > 0.6) = \int_{0.6}^{1} 6y(1-y) \, dy$

Using the power rule: $6y(1-y) = 6y - 6y^2$

= \int_{0.6}^{1} (6y - 6y^2) \, dy = \left[3y^2 - 2y^3\right]_{0.6}^{1} = (3 - 2) - (3(0.36) - 2(0.216)) = 1 - (1.08 - 0.432) = 1 - 0.648 = 0.352

About 35.2% of weeks, costs exceed 60% of the budget.

(d) Find $c$ such that $P(Y > c) = 0.10$ , i.e., $P(Y \leq c) = 0.90$ :

qbeta(0.90, 2, 2)  # 0.8042

Budget about 80.4% of the maximum to keep overruns below 10%.

Guided Practice 5.6: Recognizing and using the Beta distribution

Solution.

During an eight-hour shift, the proportion of time $Y$ a stamping machine is down for maintenance has a $\text{Beta}(1, 2)$ distribution.

(a) Write down the PDF.

With $\alpha = 1$ , $\beta = 2$ : $B(1,2) = \Gamma(1)\Gamma(2)/\Gamma(3) = (1)(1)/(2) = 1/2$ .

f(y) = \frac{y^{0}(1-y)^{1}}{1/2} = 2(1-y), \qquad 0 \leq y \leq 1

(b) Find $E(Y)$ and $V(Y)$ .

E(Y) = \frac{1}{1+2} = \frac{1}{3} \qquad V(Y) = \frac{1 \cdot 2}{(3)^2(4)} = \frac{2}{36} = \frac{1}{18}

(c) The cost of downtime is $C = 10 + 20Y + 4Y^2$ . Find $E(C)$ .

We need $E(Y^2) = V(Y) + [E(Y)]^2 = 1/18 + 1/9 = 1/18 + 2/18 = 3/18 = 1/6$ .

E(C) = 10 + 20 \cdot \frac{1}{3} + 4 \cdot \frac{1}{6} = 10 + \frac{20}{3} + \frac{2}{3} = 10 + \frac{22}{3} = \frac{52}{3} \approx \$17.33

7.4Section 5.7 Exercises¶

5.7.1. A random variable $Y$ has density $f(y) = 12y^2(1 - y)$ for $0 \leq y \leq 1$ .

(a) Identify the Beta parameters $\alpha$ and $\beta$ . (b) Find $E(Y)$ and $V(Y)$ using the Beta formulas. (c) Verify $E(Y)$ by direct integration.

5.7.2. The proportion of impurities in ore samples follows a $\text{Beta}(3, 5)$ distribution. Each sample is worth $W = 5 - 0.5Y$ dollars.

(a) Find $E(Y)$ , $V(Y)$ . (b) Find $E(W)$ and $V(W)$ . (c) Find $P(Y < 0.5)$ using R.

5.7.3. Show that the $\text{Beta}(1, 1)$ distribution is the $\text{Uniform}(0, 1)$ distribution by verifying that the Beta density with $\alpha = \beta = 1$ reduces to $f(y) = 1$ for $0 \leq y \leq 1$ .

5.7.4. A random variable $Y \sim \text{Beta}(\alpha, \beta)$ with $\alpha = \beta$ has mean $1/2$ . Show this algebraically. Then argue from symmetry that the PDF satisfies $f(y) = f(1-y)$ , and explain why this implies the mean must be $1/2$ .

5.7.5. (Proof) Derive $V(Y)$ for $Y \sim \text{Beta}(\alpha, \beta)$ . (Hint: Find $E(Y^2)$ using the same Beta-function ratio technique from the $E(Y)$ proof: $E(Y^2) = B(\alpha+2, \beta)/B(\alpha, \beta)$ . Then apply the shortcut formula.)

🔬 What Did We Learn?

The Beta distribution models random proportions on $[0, 1]$ , with density $f(y) \propto y^{\alpha-1}(1-y)^{\beta-1}$ . Its two parameters give it extraordinary flexibility: symmetric or skewed, bell-shaped or U-shaped, concentrated or spread out.

The real-life interpretation: Any time you have a random variable that is naturally a fraction or percentage — market share, completion rate, defect rate, the proportion of a budget consumed — the Beta distribution is a natural candidate. Its mean $\alpha/(\alpha + \beta)$ is a “weighted average” of 0 and 1, where $\alpha$ represents “successes” and $\beta$ represents “failures.” In Bayesian statistics (which you’ll meet in mathematical statistics), the Beta distribution is the standard prior for unknown probabilities.

Looking ahead: Section 5.8 introduces moment generating functions for continuous distributions — the tool that lets us prove the mean and variance formulas for the Normal and Gamma distributions, and provides a powerful method for identifying distributions.

8Moments and Moment-Generating Functions¶

🔭 What Are We About to Learn?

In Chapter 4, we introduced moment generating functions (MGFs) for discrete random variables. The idea was simple but powerful: if you can find $M(t) = E(e^{tY})$ , then the $k$ th moment $E(Y^k)$ is the $k$ th derivative $M^{(k)}(0)$ .

Now we extend MGFs to continuous distributions. The definition is the same — only the computation changes from a sum to an integral. We’ll use MGFs to prove the mean and variance formulas for the Normal and Gamma distributions that we stated without proof in earlier sections.

MGFs also have a uniqueness property: if two distributions have the same MGF, they are the same distribution. This means MGFs serve as “fingerprints” — you can identify a distribution by computing its MGF and matching it to a known form.

Real-life motivation: Suppose you know a random variable has MGF $M(t) = (1 - 2t)^{-3}$ . Without any other information, you can conclude it’s a $\text{Gamma}(3, 2)$ distribution (or equivalently, a $\chi^2(6)$ ). The MGF uniquely identifies the distribution.

8.1Definition and Basic Properties¶

Before the MGF, let us formally define moments — a concept also covered in the reference textbook (Definition 4.13).

8.2The Gamma MGF — Full Derivation¶

This is the most important MGF derivation in this chapter, because it also gives us the Exponential and Chi-Square MGFs as special cases.

Theorem 15 (Theorem 5.15: MGF of the Gamma Distribution)

If $Y \sim \text{Gamma}(\alpha, \beta)$ , then:

M(t) = (1 - \beta t)^{-\alpha}, \qquad t < 1/\beta

Proof.

Strategy: Compute $E(e^{tY})$ by integrating, then use the same substitution trick from the Gamma normalization proof to convert the integral into a recognizable $\Gamma(\alpha)$ form.

M(t) = E(e^{tY}) = \int_0^{\infty} e^{ty} \cdot \frac{y^{\alpha-1}e^{-y/\beta}}{\beta^{\alpha}\Gamma(\alpha)} \, dy

Combine the two exponentials: $e^{ty} \cdot e^{-y/\beta} = e^{y(t - 1/\beta)} = e^{-y(1/\beta - t)}$ .

For this integral to converge, we need $1/\beta - t > 0$ , i.e., $t < 1/\beta$ . Assuming this:

M(t) = \frac{1}{\beta^{\alpha}\Gamma(\alpha)} \int_0^{\infty} y^{\alpha-1} e^{-y(1/\beta - t)} \, dy

Substitute $u = y \cdot \frac{1 - \beta t}{\beta}$ , so $y = \frac{u\beta}{1 - \beta t}$ and $dy = \frac{\beta}{1 - \beta t} \, du$ :

M(t) = \frac{1}{\beta^{\alpha}\Gamma(\alpha)} \int_0^{\infty} \left(\frac{u\beta}{1-\beta t}\right)^{\alpha-1} e^{-u} \cdot \frac{\beta}{1-\beta t} \, du

Collect the powers of $\beta/(1-\beta t)$ : the exponent $\alpha - 1$ from the $y^{\alpha-1}$ term plus 1 from the $dy$ term gives $\alpha$ total:

= \frac{1}{\beta^{\alpha}\Gamma(\alpha)} \cdot \frac{\beta^{\alpha}}{(1-\beta t)^{\alpha}} \int_0^{\infty} u^{\alpha-1}e^{-u} \, du

The $\beta^{\alpha}$ terms cancel. The integral is $\Gamma(\alpha)$ :

= \frac{\cancel{\beta^{\alpha}}}{\cancel{\beta^{\alpha}} \cdot \cancel{\Gamma(\alpha)}} \cdot \frac{\cancel{\Gamma(\alpha)}}{(1-\beta t)^{\alpha}} = (1 - \beta t)^{-\alpha} \qquad \blacksquare

8.3Deriving $E(Y)$ and $V(Y)$ for the Gamma from the MGF¶

Now we can finally prove Theorem 5.11:

M(t) = (1 - \beta t)^{-\alpha}

First derivative (using the chain rule: $\frac{d}{dt}(1 - \beta t)^{-\alpha} = -\alpha(1 - \beta t)^{-\alpha - 1} \cdot (-\beta) = \alpha\beta(1 - \beta t)^{-\alpha - 1}$ ):

M'(t) = \alpha\beta(1 - \beta t)^{-\alpha-1}

E(Y) = M'(0) = \alpha\beta(1)^{-\alpha-1} = \alpha\beta \qquad \checkmark

Second derivative (chain rule again):

M''(t) = \alpha\beta \cdot (-\alpha-1)(1-\beta t)^{-\alpha-2} \cdot (-\beta) = \alpha(\alpha+1)\beta^2(1-\beta t)^{-\alpha-2}

E(Y^2) = M''(0) = \alpha(\alpha+1)\beta^2

Variance:

V(Y) = E(Y^2) - [E(Y)]^2 = \alpha(\alpha+1)\beta^2 - (\alpha\beta)^2 = \alpha^2\beta^2 + \alpha\beta^2 - \alpha^2\beta^2 = \alpha\beta^2 \qquad \checkmark

8.4MGFs for Special Cases¶

Setting $\alpha = 1$ (Exponential): $M(t) = (1 - \beta t)^{-1}$ for $t < 1/\beta$ .

Setting $\alpha = \nu/2$ , $\beta = 2$ (Chi-Square): $M(t) = (1 - 2t)^{-\nu/2}$ for $t < 1/2$ .

8.5The Normal MGF¶

Theorem 16 (Theorem 5.16: MGF of the Normal Distribution)

If $Y \sim N(\mu, \sigma^2)$ , then:

M(t) = e^{\mu t + \sigma^2 t^2 / 2}

Proof.

Strategy: Start with the standard Normal ( $\mu = 0$ , $\sigma = 1$ ), then generalize using $Y = \mu + \sigma Z$ .

Step 1: Standard Normal MGF. For $Z \sim N(0, 1)$ :

M_Z(t) = \int_{-\infty}^{\infty} e^{tz} \cdot \frac{1}{\sqrt{2\pi}}e^{-z^2/2} \, dz = \frac{1}{\sqrt{2\pi}}\int_{-\infty}^{\infty} e^{tz - z^2/2} \, dz

The key trick is completing the square in the exponent:

tz - \frac{z^2}{2} = -\frac{1}{2}(z^2 - 2tz) = -\frac{1}{2}(z^2 - 2tz + t^2) + \frac{t^2}{2} = -\frac{(z-t)^2}{2} + \frac{t^2}{2}

So:

M_Z(t) = \frac{1}{\sqrt{2\pi}} e^{t^2/2} \int_{-\infty}^{\infty} e^{-(z-t)^2/2} \, dz

The remaining integral is the total area under a $N(t, 1)$ density (shifted Normal), which equals $\sqrt{2\pi}$ . Therefore:

M_Z(t) = \frac{1}{\cancel{\sqrt{2\pi}}} \cdot e^{t^2/2} \cdot \cancel{\sqrt{2\pi}} = e^{t^2/2}

Step 2: General Normal. If $Y = \mu + \sigma Z$ , then $E(e^{tY}) = E(e^{t(\mu + \sigma Z)}) = e^{t\mu} \cdot E(e^{(t\sigma)Z}) = e^{t\mu} \cdot e^{(t\sigma)^2/2} = e^{\mu t + \sigma^2 t^2/2}$ $\qquad \blacksquare$

Deriving $E(Y)$ and $V(Y)$ from the Normal MGF:

M'(t) = (\mu + \sigma^2 t) \cdot e^{\mu t + \sigma^2 t^2/2} \implies E(Y) = M'(0) = \mu \cdot 1 = \mu \qquad \checkmark

M''(t) = [\sigma^2 + (\mu + \sigma^2 t)^2] \cdot e^{\mu t + \sigma^2 t^2/2} \implies E(Y^2) = M''(0) = \sigma^2 + \mu^2

V(Y) = (\sigma^2 + \mu^2) - \mu^2 = \sigma^2 \qquad \checkmark

8.6Section 5.8 Exercises¶

5.8.1. Find the MGF of the $\text{Uniform}(\theta_1, \theta_2)$ distribution. (Hint: Integrate $e^{ty}/(\theta_2 - \theta_1)$ from $\theta_1$ to $\theta_2$ . The result is $M(t) = \frac{e^{\theta_2 t} - e^{\theta_1 t}}{(\theta_2 - \theta_1)t}$ for $t \neq 0$ .) Verify that $M'(0) = (\theta_1 + \theta_2)/2$ .

5.8.2. A random variable has MGF $M(t) = (1 - 4t)^{-3}$ . Identify the distribution, and state its mean and variance.

5.8.3. A random variable has MGF $M(t) = e^{5t + 8t^2}$ . Identify the distribution, and state its mean and variance.

5.8.4. (Proof) Using the Gamma MGF, prove that the sum of $\nu$ independent Exponential $(\beta)$ random variables has a $\text{Gamma}(\nu, \beta)$ distribution. (Hint: The MGF of a sum of independent RVs is the product of their individual MGFs. What is $[(1-\beta t)^{-1}]^{\nu}$ ?)

🔬 What Did We Learn?

The moment generating function $M(t) = E(e^{tY})$ is a powerful tool with three uses:

Finding moments: $E(Y) = M'(0)$ , $E(Y^2) = M''(0)$ , then $V(Y) = M''(0) - [M'(0)]^2$ .
Proving mean/variance formulas: We used the Gamma MGF $(1-\beta t)^{-\alpha}$ to prove $E(Y) = \alpha\beta$ and $V(Y) = \alpha\beta^2$ , and the Normal MGF $e^{\mu t + \sigma^2 t^2/2}$ to prove $E(Y) = \mu$ and $V(Y) = \sigma^2$ .
Identifying distributions: If you compute an MGF and it matches a known form, you’ve identified the distribution — because MGFs are unique.

The real-life interpretation: MGFs are a theoretical tool, not something you’d compute on the job. But they’re the engine behind many results you’ll use constantly: the fact that a sum of independent Normals is Normal, the fact that a sum of independent Gammas (with the same $\beta$ ) is Gamma, and the Central Limit Theorem itself. These are proved using MGFs.

Looking ahead: Our final section gives a universal bound on probabilities — Tchebysheff’s theorem works for any distribution, with no assumptions about its shape.

9Tchebysheff’s Theorem¶

🔭 What Are We About to Learn?

Every distribution we’ve studied — Uniform, Normal, Gamma, Beta — has its own specific formula for computing $P(|Y - \mu| \geq k\sigma)$ . But what if you don’t know the distribution? What if all you know is the mean $\mu$ and variance $\sigma^2$ ?

Tchebysheff’s theorem gives a universal bound: for any distribution (continuous or discrete), at least $1 - 1/k^2$ of the probability lies within $k$ standard deviations of the mean. This bound is conservative — it’s rarely tight — but it works with no assumptions about the shape of the distribution.

Real-life motivation: A quality engineer knows a part has mean diameter $\mu = 10$ mm and standard deviation $\sigma = 0.3$ mm, but doesn’t know the exact distribution. Tchebysheff guarantees that at least 75% of parts are within $\pm 0.6$ mm of the mean ( $k = 2$ ), and at least 89% are within $\pm 0.9$ mm ( $k = 3$ ) — regardless of the distribution’s shape.

Theorem 17 (Theorem 5.17: Tchebysheff’s Theorem (Continuous Version))

Let $Y$ be a continuous random variable with mean $\mu$ and finite variance $\sigma^2$ . Then for any $k > 0$ :

P(|Y - \mu| \geq k\sigma) \leq \frac{1}{k^2}

Equivalently: $P(|Y - \mu| < k\sigma) \geq 1 - \frac{1}{k^2}$

Proof.

Strategy: Start from the definition of variance and cleverly restrict the domain of integration to show that the variance is at least as large as $k^2\sigma^2$ times the probability we want to bound.

\sigma^2 = \int_{-\infty}^{\infty} (y - \mu)^2 f(y) \, dy

This integral is over all $y$ . Since the integrand $(y-\mu)^2 f(y)$ is nonnegative everywhere, the integral over all of $\mathbb{R}$ is at least as large as the integral over any subset. In particular, restricting to the set where $|y - \mu| \geq k\sigma$ :

\sigma^2 \geq \int_{|y - \mu| \geq k\sigma} (y - \mu)^2 f(y) \, dy

On this restricted domain, $|y - \mu| \geq k\sigma$ , which means $(y - \mu)^2 \geq k^2\sigma^2$ . Replace the integrand with this lower bound:

\geq \int_{|y - \mu| \geq k\sigma} k^2\sigma^2 \, f(y) \, dy = k^2\sigma^2 \int_{|y - \mu| \geq k\sigma} f(y) \, dy = k^2\sigma^2 \cdot P(|Y - \mu| \geq k\sigma)

So $\sigma^2 \geq k^2\sigma^2 \cdot P(|Y - \mu| \geq k\sigma)$ . Divide both sides by $k^2\sigma^2$ :

P(|Y - \mu| \geq k\sigma) \leq \frac{1}{k^2} \qquad \blacksquare

Standard values:

$k$	At most $1/k^2$ outside $\mu \pm k\sigma$	At least $1 - 1/k^2$ inside
2	$\leq 25\%$	$\geq 75\%$
3	$\leq 11.1\%$	$\geq 88.9\%$
4	$\leq 6.25\%$	$\geq 93.75\%$

Compare these to the Normal: for $k=2$ , the Normal has 4.56% outside (not 25%), and for $k=3$ , only 0.27% (not 11.1%). Tchebysheff is conservative — but it works for every distribution, not just the Normal.

Example 16 (EXAMPLE 5.16: Tchebysheff Applied to an Unknown Distribution)

The Problem.

Experience shows that the time (in minutes) for a periodic maintenance check on a machine follows a Gamma distribution with $\alpha = 3.1$ and $\beta = 2$ . A new maintenance worker takes 22.5 minutes. Does this time disagree with prior experience?

Solution.

$\mu = \alpha\beta = 3.1(2) = 6.2$ minutes. $\sigma^2 = \alpha\beta^2 = 3.1(4) = 12.4$ , so $\sigma = \sqrt{12.4} \approx 3.52$ .

How far is 22.5 from the mean? $|22.5 - 6.2| = 16.3$ , and $k = 16.3/3.52 \approx 4.63$ .

By Tchebysheff: $P(|Y - 6.2| \geq 16.3) \leq 1/4.63^2 \approx 0.047$ .

There is at most a 4.7% chance of a maintenance time this far from the mean. This suggests the new worker’s time is unusually long — possibly due to inexperience.

We can also compute the exact probability using R:

# Exact: P(Y > 22.5) for Gamma(3.1, scale = 2)
pgamma(22.5, shape = 3.1, scale = 2, lower.tail = FALSE)
# 0.001150 — much smaller than Tchebysheff's bound of 0.047

The exact probability is about 0.12% — Tchebysheff’s bound of 4.7% is very conservative (as expected), but both tell the same story: this time is unusually long.

9.1Section 5.9 Exercises¶

5.9.1. A random variable $Y$ has $E(Y) = 50$ and $V(Y) = 25$ . Using Tchebysheff’s theorem, find a lower bound on $P(40 < Y < 60)$ .

5.9.2. A distribution has mean 100 and standard deviation 15. What is the minimum probability that a randomly selected value lies between 55 and 145? What does the bound become if the distribution is known to be Normal? Compare.

5.9.3. (Proof) Use Tchebysheff’s theorem to prove the following: if $V(Y) = 0$ , then $P(Y = \mu) = 1$ . (Hint: For any $\epsilon > 0$ , set $k\sigma = \epsilon$ and take $\sigma \to 0$ .)

🔬 What Did We Learn?

Tchebysheff’s theorem gives a distribution-free bound: for any distribution with finite mean and variance, at least $1 - 1/k^2$ of the probability lies within $k$ standard deviations of the mean. It requires no assumptions about symmetry, shape, or tail behavior.

The real-life interpretation: When you don’t know the distribution but you know $\mu$ and $\sigma$ , Tchebysheff gives you a guaranteed minimum. An engineer who knows a component’s average lifetime is 1000 hours with SD 100 hours can guarantee — without knowing the distribution — that at least 75% of components last between 800 and 1200 hours. If the distribution is actually Normal, the true percentage is 95.4%, but 75% is the guaranteed floor.

The bound is conservative because it must hold for all distributions, including the worst-case ones. But conservative guarantees are still guarantees — and in engineering and quality control, guarantees matter.

10Which Distribution Should I Use? (Optional — Can Be Skipped)¶

Choosing a distribution is a modeling decision, not a mathematical derivation. No real-world data follows any theoretical distribution exactly — the question is whether the model is useful enough for the inferences you need to make.

Here is a practical guide:

10.1Step 1: What is the support?¶

Support	Candidate Distributions
$(-\infty, \infty)$ — any real number	Normal
$(0, \infty)$ — positive values only	Exponential, Gamma, Chi-Square
$[0, 1]$ — proportions/fractions	Beta, Uniform(0,1)
$(a, b)$ — bounded interval	Uniform(a,b), Beta (rescaled)

10.2Step 2: What is the shape?¶

Shape	Distribution
Symmetric, bell-shaped	Normal
Right-skewed, one mode	Gamma (with $\alpha > 1$ )
Decreasing from zero	Exponential (or Gamma with $\alpha \leq 1$ )
Flat (all values equally likely)	Uniform
Flexible on $[0, 1]$	Beta

10.3Step 3: What is the context?¶

Some distributions are linked to specific physical processes:

Poisson events in time? The count is Poisson, the wait time between events is Exponential, and the wait for the $\alpha$ th event is Gamma.
Sum of many small independent effects? The result is approximately Normal (Central Limit Theorem).
Sum of squared standard Normals? The result is Chi-Square.
Measuring a proportion or percentage? Use Beta.
Complete ignorance within a known range? Use Uniform.

10.4The Key Insight¶

As the reference textbook emphasizes: a good model is not one that perfectly matches reality — it is one that yields good inferences about the population of interest. A Normal approximation that’s slightly off in the tails might still produce excellent confidence intervals and hypothesis tests.

11The Weibull Distribution (Optional — Can Be Skipped)¶

🔭 What Are We About to Learn?

The Exponential distribution models lifetimes with a constant failure rate — the item is just as likely to fail in the next hour whether it’s brand new or ten years old. But many real-world items wear out: their failure rate increases with age (brake pads, tires, bearings). Others have decreasing failure rates due to early manufacturing defects being weeded out (“infant mortality” in electronics).

The Weibull distribution handles all three cases with a single family. It has a shape parameter $\alpha$ that controls whether the failure rate is decreasing ( $\alpha < 1$ ), constant ( $\alpha = 1$ , which gives the Exponential), or increasing ( $\alpha > 1$ ). This makes it the most widely used distribution in reliability engineering.

Real-life motivation: An automotive manufacturer needs to set warranty periods for brake pads. The pads wear out faster as they age — a classic increasing failure rate. The Weibull distribution models this perfectly, letting engineers compute “what fraction of brake pads will fail before 50,000 miles?” The reference textbook includes Weibull exercises in its supplementary problems.

11.1Definition¶

Connection to the Exponential: When $\alpha = 1$ , the Weibull density becomes $f(y) = \frac{1}{\beta}e^{-y/\beta}$ — exactly the Exponential $(\beta)$ density. So the Exponential is a special case of both the Gamma and the Weibull families.

The CDF has a closed form (just like the Exponential):

F(y) = 1 - e^{-(y/\beta)^{\alpha}}, \qquad y > 0

11.2Mean and Variance¶

Theorem 18 (Theorem 5.18: Mean and Variance of the Weibull Distribution)

If $Y \sim \text{Weibull}(\alpha, \beta)$ , then:

E(Y) = \beta \, \Gamma\!\left(1 + \frac{1}{\alpha}\right) \qquad \text{and} \qquad V(Y) = \beta^2 \left[\Gamma\!\left(1 + \frac{2}{\alpha}\right) - \left(\Gamma\!\left(1 + \frac{1}{\alpha}\right)\right)^2\right]

Proof of $E(Y)$ (sketch).

E(Y) = \int_0^{\infty} y \cdot \frac{\alpha}{\beta^{\alpha}}y^{\alpha-1}e^{-(y/\beta)^{\alpha}} \, dy

Substitute $u = (y/\beta)^{\alpha}$ , so $y = \beta u^{1/\alpha}$ and $dy = \frac{\beta}{\alpha}u^{1/\alpha - 1} \, du$ :

= \int_0^{\infty} \beta u^{1/\alpha} \cdot e^{-u} \, du = \beta \int_0^{\infty} u^{(1 + 1/\alpha) - 1} e^{-u} \, du = \beta \, \Gamma\!\left(1 + \frac{1}{\alpha}\right) \qquad \blacksquare

Check: When $\alpha = 1$ (Exponential): $E(Y) = \beta \, \Gamma(2) = \beta \cdot 1 = \beta$ ✓

11.3The Hazard Function: Why Weibull Matters¶

The hazard function (or failure rate) of a distribution is $h(y) = f(y) / [1 - F(y)]$ — the instantaneous risk of failure at time $y$ , given survival to $y$ .

For the Weibull:

h(y) = \frac{\alpha}{\beta^{\alpha}} y^{\alpha - 1}

$\alpha < 1$ : $h(y)$ decreases — “infant mortality” (early failures dominate)
$\alpha = 1$ : $h(y) = 1/\beta$ = constant — memoryless (Exponential)
$\alpha > 1$ : $h(y)$ increases — wear-out (aging causes failure)

This is precisely why the Weibull is the workhorse of reliability engineering.

# Weibull shapes and hazard functions
par(mfrow = c(1, 3))
for (a in c(0.5, 1, 3)) {
  curve(dweibull(x, shape = a, scale = 2), 0, 5, lwd = 2,
        main = bquote(alpha == .(a)),
        xlab = "y", ylab = "f(y)", col = "steelblue")
}

Example 17 (EXAMPLE 5.17: Weibull Reliability Analysis)

The Problem.

The lifetime (in thousands of hours) of a bearing follows a Weibull distribution with $\alpha = 2$ and $\beta = 5$ .

(a) Find the probability the bearing lasts at least 4,000 hours.

(b) Find the mean lifetime.

Solution.

(a) Using the closed-form CDF:

P(Y > 4) = 1 - F(4) = e^{-(4/5)^2} = e^{-0.64} \approx 0.5273

About 52.7% of bearings last at least 4,000 hours.

(b) $E(Y) = 5 \cdot \Gamma(1 + 1/2) = 5 \cdot \Gamma(3/2) = 5 \cdot \frac{\sqrt{\pi}}{2} \approx 5(0.8862) = 4.431$ thousand hours.

(c) Median: solve $F(\phi_{0.5}) = 0.5$ :

1 - e^{-(\phi_{0.5}/5)^2} = 0.5 \implies e^{-(\phi_{0.5}/5)^2} = 0.5 \implies (\phi_{0.5}/5)^2 = \ln 2

\phi_{0.5} = 5\sqrt{\ln 2} \approx 5(0.8326) = 4.163 \text{ thousand hours}

In R.

# Y ~ Weibull(shape = 2, scale = 5)
# (a) P(Y > 4)
pweibull(4, shape = 2, scale = 5, lower.tail = FALSE)  # 0.5273

# (b) Mean
5 * gamma(1 + 1/2)  # 4.4312

# (c) Median
qweibull(0.5, shape = 2, scale = 5)  # 4.1628

11.4Section 5.11 Exercises¶

5.11.1. The time to failure (in years) of a semiconductor device follows a Weibull distribution with $\alpha = 0.5$ and $\beta = 3$ .

(a) Is the failure rate increasing or decreasing? What does this mean physically? (b) Find $P(Y > 2)$ . (c) Find the mean time to failure.

5.11.2. Show that when $\alpha = 1$ , the Weibull CDF $F(y) = 1 - e^{-(y/\beta)^{\alpha}}$ reduces to the Exponential CDF $F(y) = 1 - e^{-y/\beta}$ .

12Normal Approximation to the Binomial (Optional — Can Be Skipped)¶

🔭 What Are We About to Learn?

In Chapter 4, we learned the Binomial distribution for counting successes. But computing Binomial probabilities for large $n$ is painful — try evaluating $\binom{500}{237}$ by hand! The Normal approximation to the Binomial saves us: when $n$ is large enough, a $\text{Binomial}(n, p)$ random variable is approximately $N(np, np(1-p))$ .

This is actually a preview of the Central Limit Theorem from Chapter 7, applied to the special case where we’re summing independent Bernoulli trials. The approximation includes a continuity correction to account for the fact that we’re approximating a discrete distribution (integer values) with a continuous one (all real values).

Real-life motivation: A polling firm surveys 1,200 voters. If the true proportion supporting Candidate A is $p = 0.52$ , what’s the probability the poll shows Candidate A with at least 55% support? This is a Binomial problem with $n = 1200$ , but using the Normal approximation makes it a simple $Z$ -score calculation.

12.1The Approximation¶

12.2The Continuity Correction¶

Because the Binomial takes integer values but the Normal is continuous, we adjust by $\pm 0.5$ :

$P(Y \leq k) \approx P\left(Z \leq \frac{k + 0.5 - np}{\sqrt{np(1-p)}}\right)$ — add 0.5 because the Normal must cover the full integer $k$
$P(Y \geq k) \approx P\left(Z \geq \frac{k - 0.5 - np}{\sqrt{np(1-p)}}\right)$ — subtract 0.5
$P(Y = k) \approx P\left(\frac{k - 0.5 - np}{\sqrt{npq}} \leq Z \leq \frac{k + 0.5 - np}{\sqrt{npq}}\right)$

The continuity correction improves accuracy, especially when $n$ is moderate (20–100).

Example 18 (EXAMPLE 5.18: Normal Approximation to the Binomial)

The Problem.

A fair coin is flipped 100 times. Find the probability of getting at least 60 heads.

Solution.

$Y \sim \text{Binomial}(100, 0.5)$ . We have $\mu = np = 50$ and $\sigma = \sqrt{np(1-p)} = \sqrt{25} = 5$ .

Check: $np = 50 \geq 5$ ✓ and $n(1-p) = 50 \geq 5$ ✓. The approximation is valid.

With continuity correction: $P(Y \geq 60) \approx P\left(Z \geq \frac{59.5 - 50}{5}\right) = P(Z \geq 1.90) = 0.0287$

Without continuity correction: $P(Y \geq 60) \approx P\left(Z \geq \frac{60 - 50}{5}\right) = P(Z \geq 2.00) = 0.0228$

Exact (Binomial): $P(Y \geq 60) = \sum_{k=60}^{100}\binom{100}{k}(0.5)^{100} = 0.0284$

The continuity-corrected answer (0.0287) is very close to the exact value (0.0284). The uncorrected version (0.0228) is noticeably off.

In R.

# Exact Binomial
pbinom(59, 100, 0.5, lower.tail = FALSE)  # 0.02844

# Normal approximation WITH continuity correction
pnorm(59.5, 50, 5, lower.tail = FALSE)    # 0.02872

# Normal approximation WITHOUT continuity correction
pnorm(60, 50, 5, lower.tail = FALSE)      # 0.02275

Interpretation.

With 100 fair coin flips, there’s about a 2.8% chance of getting 60 or more heads. The Normal approximation with continuity correction nails this to within 0.03 percentage points. Without the correction, the error is about 0.6 percentage points — small but avoidable.

12.3Section 5.12 Exercises¶

5.12.1. A manufacturer knows that 8% of parts are defective. In a batch of 200 parts, use the Normal approximation (with continuity correction) to find $P(Y \geq 20)$ , where $Y$ is the number of defective parts. Verify with R’s pbinom.

5.12.2. A polling firm surveys 1,500 voters. If the true proportion is $p = 0.48$ , find the probability the poll shows a majority ( $Y/n > 0.50$ , i.e., $Y > 750$ ) using the Normal approximation.

13The Distribution Family Tree (Optional — Encouraged Reading)¶

13.1The Connections¶

                    BERNOULLI(p)
                         │
                    sum of n trials
                         │
                    BINOMIAL(n, p)
                       /    \
              n→∞, p→0     n large
              np = λ         │
                 /     NORMAL(np, npq)  ←── Central Limit Theorem
            POISSON(λ)       │
                │        Z = (Y-μ)/σ
           count events      │
           in time      STANDARD NORMAL
                │            │
           time between    Z² summed
           events            │
                │        CHI-SQUARE(ν)
          EXPONENTIAL(β)     │
                │        = Gamma(ν/2, 2)
           = Gamma(1, β)     │
                │            │
           wait for α    ────┘
           events            
                │            
           GAMMA(α, β)       
                             
           UNIFORM(a, b)     
                │            
           = Beta(1,1)       
           when a=0, b=1     
                │            
           BETA(α, β)        

           WEIBULL(α, β)
                │
           = Exponential(β)
           when α = 1

13.2Master Reference Table¶

All Continuous Distributions at a Glance

Distribution	PDF	Support	$E(Y)$	$V(Y)$	MGF $M(t)$	R functions
$\text{Uniform}(\theta_1, \theta_2)$	$\frac{1}{\theta_2 - \theta_1}$	$[\theta_1, \theta_2]$	$\frac{\theta_1+\theta_2}{2}$	$\frac{(\theta_2-\theta_1)^2}{12}$	$\frac{e^{\theta_2 t}-e^{\theta_1 t}}{(\theta_2-\theta_1)t}$	`dunif, punif, qunif, runif`
$N(\mu, \sigma^2)$	$\frac{1}{\sigma\sqrt{2\pi}}e^{-(y-\mu)^2/(2\sigma^2)}$	$(-\infty,\infty)$	$\mu$	$\sigma^2$	$e^{\mu t + \sigma^2 t^2/2}$	`dnorm, pnorm, qnorm, rnorm`
$\text{Gamma}(\alpha, \beta)$	$\frac{y^{\alpha-1}e^{-y/\beta}}{\beta^{\alpha}\Gamma(\alpha)}$	$(0, \infty)$	$\alpha\beta$	$\alpha\beta^2$	$(1-\beta t)^{-\alpha}$	`dgamma, pgamma, qgamma, rgamma`
$\text{Exponential}(\beta)$	$\frac{1}{\beta}e^{-y/\beta}$	$(0, \infty)$	$\beta$	$\beta^2$	$(1-\beta t)^{-1}$	`dexp, pexp, qexp, rexp`
$\chi^2(\nu)$	$\frac{y^{\nu/2-1}e^{-y/2}}{2^{\nu/2}\Gamma(\nu/2)}$	$(0, \infty)$	$\nu$	$2\nu$	$(1-2t)^{-\nu/2}$	`dchisq, pchisq, qchisq, rchisq`
$\text{Beta}(\alpha, \beta)$	$\frac{y^{\alpha-1}(1-y)^{\beta-1}}{B(\alpha,\beta)}$	$[0, 1]$	$\frac{\alpha}{\alpha+\beta}$	$\frac{\alpha\beta}{(\alpha+\beta)^2(\alpha+\beta+1)}$	—	`dbeta, pbeta, qbeta, rbeta`
$\text{Weibull}(\alpha, \beta)$	$\frac{\alpha}{\beta^{\alpha}}y^{\alpha-1}e^{-(y/\beta)^{\alpha}}$	$(0, \infty)$	$\beta\Gamma(1+1/\alpha)$	(see §5.11)	—	`dweibull, pweibull, qweibull, rweibull`

13.3Key Relationships Summary¶

Exponential = Gamma( $1, \beta$ ) = Weibull( $1, \beta$ )
Chi-Square( $\nu$ ) = Gamma( $\nu/2, 2$ )
Uniform(0,1) = Beta( $1, 1$ )
Poisson count ↔ Exponential gap (same process, different views)
Binomial → Normal (when $n$ large) and Binomial → Poisson (when $n$ large, $p$ small)
Sum of $\nu$ independent Exponential $(\beta)$ = Gamma $(\nu, \beta)$
Sum of $\nu$ independent $\chi^2(1)$ = $\chi^2(\nu)$
$Z^2 \sim \chi^2(1)$ where $Z \sim N(0,1)$

🔬 What Did We Learn?

The distributions we’ve studied are not isolated — they form a connected family tree. The Exponential sits at the intersection of the Gamma and Weibull families. The Chi-Square bridges the Normal and Gamma worlds. The Uniform is a special case of the Beta. The Poisson process generates both Poisson counts and Exponential gaps.

Why this matters: When you encounter a new problem, you don’t pick a distribution at random. You reason from the physical process: “Events arrive at a constant rate → Poisson count → Exponential gap → Gamma for the $k$ th event.” Or: “I’m modeling a proportion → Beta.” Or: “Wear-out failure → Weibull with $\alpha > 1$ .” The family tree is your navigation chart.

Keep this reference table handy — you’ll use it throughout mathematical statistics and beyond.

14Solving Buffon’s Needle¶

We now have all the tools to solve the problem from the chapter opening.

Setup: A needle of length $\ell$ is dropped onto a floor with parallel lines spaced $d$ apart ( $\ell \leq d$ ). Two random variables describe the needle’s position:

$\Theta$ = the acute angle between the needle and the lines, with $\Theta \sim \text{Uniform}(0, \pi/2)$
$D$ = the distance from the needle’s center to the nearest line, with $D \sim \text{Uniform}(0, d/2)$

We assume $\Theta$ and $D$ are independent (the angle and position of a randomly dropped needle are unrelated).

When does the needle cross a line? The needle crosses if and only if the vertical projection of its half-length exceeds $D$ :

D \leq \frac{\ell}{2}\sin(\Theta)

The probability: Since $\Theta$ and $D$ are independent Uniforms, the joint density is constant over the rectangle $[0, \pi/2] \times [0, d/2]$ . The probability of crossing is the ratio of the area where $D \leq (\ell/2)\sin\theta$ to the total rectangle area:

P(\text{cross}) = \frac{\int_0^{\pi/2} \frac{\ell}{2}\sin\theta \, d\theta}{\frac{\pi}{2} \cdot \frac{d}{2}}

P(\text{cross}) = \frac{(\ell/2) \cdot 1}{\pi d / 4} = \frac{\ell/2}{\pi d/4} = \frac{4\ell}{2\pi d} = \frac{2\ell}{\pi d}

And there it is — $\pi$ emerges from the integral of $\sin\theta$ over $[0, \pi/2]$ . The circle was hiding in the rotation of the needle.

# Buffon's Needle simulation
set.seed(42)
n_drops <- 100000
ell <- 1; d <- 2  # needle length 1, line spacing 2

theta <- runif(n_drops, 0, pi/2)
D <- runif(n_drops, 0, d/2)
crosses <- sum(D <= (ell/2) * sin(theta))

pi_estimate <- (2 * ell * n_drops) / (crosses * d)
cat("Estimated pi:", pi_estimate, "\n")
cat("Actual pi:   ", pi, "\n")
# Estimated pi: 3.143... (close!)

Chapter 5 Summary

Continuous random variables assign probability zero to every individual point. Probabilities exist only for intervals and are computed as areas under the probability density function via integration.
The PDF $f(y) = F'(y)$ is the derivative of the CDF. It satisfies $f(y) \geq 0$ everywhere and $\int_{-\infty}^{\infty} f(y) \, dy = 1$ . The PDF is a density, not a probability — it can exceed 1.
Expected values for continuous RVs use integrals: $E(Y) = \int y \, f(y) \, dy$ and $V(Y) = E(Y^2) - \mu^2$ . The linearity rules $E(aY+b) = a\mu + b$ and $V(aY+b) = a^2\sigma^2$ carry over unchanged from Chapter 4.
The Uniform distribution on $(\theta_1, \theta_2)$ has constant density $1/(\theta_2 - \theta_1)$ , mean $(\theta_1 + \theta_2)/2$ , and variance $(\theta_2 - \theta_1)^2/12$ . It is the engine of all random number generation.
The Normal distribution $N(\mu, \sigma^2)$ is the bell curve. Standardize via $Z = (Y - \mu)/\sigma$ to use one table for all Normal distributions. The 68-95-99.7 rule gives quick probability estimates.
The Gamma distribution $\text{Gamma}(\alpha, \beta)$ models right-skewed positive quantities. The Exponential ( $\alpha = 1$ ) has the memoryless property and connects to the Poisson process. The Chi-Square ( $\alpha = \nu/2$ , $\beta = 2$ ) connects to squared standard Normals.
The Beta distribution $\text{Beta}(\alpha, \beta)$ models proportions on $[0, 1]$ . Its two parameters give it extraordinary shape flexibility.
Moment generating functions $M(t) = E(e^{tY})$ provide moments via derivatives ( $E(Y^k) = M^{(k)}(0)$ ) and uniquely identify distributions.
Tchebysheff’s theorem guarantees at least $1 - 1/k^2$ of probability within $k$ standard deviations of the mean — for any distribution.
Buffon’s Needle connects continuous probability to geometry: the probability of a needle crossing a line involves the Uniform distribution, integration of $\sin\theta$ , and $\pi$ .

Think About It

You’ve now learned six named continuous distributions — Uniform, Normal, Gamma, Exponential, Chi-Square, Beta — each designed to model a different type of randomness. But here’s something to sit with: why are there so many?

The real world doesn’t come labeled “this is a Gamma distribution.” A data scientist looking at hospital wait times doesn’t see a mathematical formula — she sees a histogram. The act of choosing a distribution is a modeling decision: “I believe this data behaves approximately like a Gamma(3, 2).” The model is never the truth. It’s a useful approximation — and the utility depends on whether the approximation is close enough for the decision at hand.

So when you see a real dataset, how do you decide which distribution fits? That question — the bridge between theoretical distributions and messy data — is the central challenge of statistical inference, and it’s what the sequel to this book (Mathematical Statistics from the Ground Up) is all about.

15Chapter Refresh Exercises¶

5.R.1. A continuous random variable $Y$ has the density function:

f(y) = \begin{cases} c(4y - 2y^2), & 0 \leq y \leq 2 \\ 0, & \text{elsewhere} \end{cases}

(a) Find $c$ . (b) Find $F(y)$ . (c) Find $E(Y)$ and $V(Y)$ . (d) Find the median.

5.R.2. The lifetime (in thousands of hours) of a type of electronic component follows an Exponential distribution with mean 4.

(a) Find $P(Y > 5)$ . (b) Find $P(Y > 8 \mid Y > 3)$ . (c) Find the 90th percentile of the lifetime distribution. (d) What is the probability the component lasts between 2 and 6 thousand hours?

5.R.3. Scores on a standardized test follow $N(520, 10000)$ .

(a) What proportion of scores are above 700? (b) What score separates the bottom 15% from the top 85%? (c) Two students take the test independently. What is the probability that both score above 600?

5.R.4. A random variable $Y$ has the density $f(y) = \frac{1}{4}y e^{-y/2}$ for $y > 0$ .

(a) Identify the Gamma parameters and verify the density is valid. (b) Find $E(Y)$ and $V(Y)$ . (c) Is this a Chi-Square distribution? If so, what are the degrees of freedom?

5.R.5. The fraction of a company’s daily output that meets quality standards follows a $\text{Beta}(4, 2)$ distribution.

(a) Find the average fraction meeting standards. (b) Find the probability that more than 80% of the output meets standards on a given day. (c) If the revenue is $R = 1000Y - 200$ , find $E(R)$ .

5.R.6. A random variable $Y$ has $E(Y) = 30$ and $\sigma = 4$ . The distribution is unknown.

(a) Use Tchebysheff to bound $P(22 < Y < 38)$ . (b) If $Y$ is actually $N(30, 16)$ , find the exact probability and compare.

5.R.7. (Proof) Show that if $f(y)$ is a valid PDF on $[a, b]$ , then $E(Y) \in [a, b]$ . (Hint: Use the fact that $a \leq y \leq b$ implies $a \cdot f(y) \leq y \cdot f(y) \leq b \cdot f(y)$ , and integrate.)

5.R.8. Using the result from Buffon’s Needle ( $P = 2\ell/(\pi d)$ ), suppose you drop a needle with $\ell = 1$ cm on a floor with lines $d = 2$ cm apart. You drop it $n = 2000$ times, and it crosses a line $c = 637$ times.

(a) Estimate $\pi$ using the formula $\hat{\pi} = 2\ell n / (cd)$ . (b) How close is your estimate to the true value? (c) How many drops would you need for an estimate accurate to 2 decimal places (in expectation)?

5.R.9. A random variable has MGF $M(t) = e^{3t + 2t^2}$ .

(a) Identify the distribution. (b) Find $P(Y > 7)$ . (c) Find the interquartile range (IQR).

5.R.10. (Comprehensive) A hospital models emergency room wait times (in hours) with the density:

f(y) = \begin{cases} \frac{1}{4}ye^{-y/2}, & y > 0 \\ 0, & y \leq 0 \end{cases}

(a) Identify the distribution family and parameters. (b) Find $E(Y)$ , $V(Y)$ , and $\sigma$ . (c) Find the probability a patient waits more than 6 hours. (d) Find the median wait time using R. (e) A patient has already waited 3 hours. Is the memoryless property applicable here? Why or why not? (f) Use Tchebysheff to find a guaranteed upper bound on the probability of waiting more than 12 hours, and compare to the exact value.

16Chapter 5 Quiz¶

Test your understanding with this self-assessment quiz. 15 questions are randomly selected from a bank of 100. You need 80% (12/15) to earn the badge.

Continuous Random Variables and Their Distributions

1From Discrete to Continuous: The Conceptual Leap¶

1.1Why the Discrete Approach Breaks Down¶

1.2The Distribution Function: Our Starting Point¶

1.3See It First: Histograms Converging to a Curve¶

2Probability Density Functions and CDFs¶

2.1The Probability Density Function¶

2.2Properties of a PDF¶

2.3Computing Probabilities: Integration¶

2.4Worked Examples¶

2.5Quantiles and Percentiles¶

2.6Conditional Probability with Continuous Random Variables¶

2.7Section 5.2 Exercises¶

3Expected Values for Continuous Random Variables¶

3.1The Expected Value¶

3.2Expected Value of a Function of YYY¶

3.3Variance and Standard Deviation¶

3.4Properties of Expected Value¶

3.5Worked Examples¶

3.6Section 5.3 Exercises¶

4The Uniform Distribution¶

4.1Definition and Density Function¶

4.2CDF of the Uniform Distribution¶

4.3Mean and Variance¶

4.4Worked Examples¶

4.5The Uniform Distribution and Random Number Generation¶

4.6Section 5.4 Exercises¶

5The Normal Distribution¶

5.1Definition and Shape¶

5.2Key Properties of the Normal Distribution¶

5.3Standardization: The ZZZ-Transformation¶

5.4Using the Standard Normal Table¶

5.5Worked Examples¶

5.6The Standard Normal as a Reference¶

5.7Section 5.5 Exercises¶

6The Gamma Distribution Family¶

6.1The Gamma Function¶

6.2The Gamma Distribution¶

6.3Special Case 1: The Exponential Distribution¶

6.3.1The Memoryless Property¶

6.3.2The Poisson–Exponential Connection¶

6.4Special Case 2: The Chi-Square Distribution¶

6.5The Gamma Family Tree¶

6.6Section 5.6 Exercises¶

7The Beta Distribution¶

7.1Definition¶

7.2Mean and Variance¶

7.3Special Cases and Shape¶

7.4Section 5.7 Exercises¶

8Moments and Moment-Generating Functions¶

8.1Definition and Basic Properties¶

8.2The Gamma MGF — Full Derivation¶

8.3Deriving E(Y)E(Y)E(Y) and V(Y)V(Y)V(Y) for the Gamma from the MGF¶

8.4MGFs for Special Cases¶

8.5The Normal MGF¶

8.6Section 5.8 Exercises¶

9Tchebysheff’s Theorem¶

9.1Section 5.9 Exercises¶

10Which Distribution Should I Use? (Optional — Can Be Skipped)¶

10.1Step 1: What is the support?¶

10.2Step 2: What is the shape?¶

10.3Step 3: What is the context?¶

10.4The Key Insight¶

11The Weibull Distribution (Optional — Can Be Skipped)¶

11.1Definition¶

11.2Mean and Variance¶

11.3The Hazard Function: Why Weibull Matters¶

11.4Section 5.11 Exercises¶

12Normal Approximation to the Binomial (Optional — Can Be Skipped)¶

12.1The Approximation¶

12.2The Continuity Correction¶

12.3Section 5.12 Exercises¶

13The Distribution Family Tree (Optional — Encouraged Reading)¶

13.1The Connections¶

13.2Master Reference Table¶

13.3Key Relationships Summary¶

14Solving Buffon’s Needle¶

15Chapter Refresh Exercises¶

16Chapter 5 Quiz¶

3.2Expected Value of a Function of $Y$ ¶

5.3Standardization: The $Z$ -Transformation¶

8.3Deriving $E(Y)$ and $V(Y)$ for the Gamma from the MGF¶