Functions of Random Variables

Here’s the fundamental question of this chapter:

If we know the distribution of $X$, and we define $Y = g(X)$, what is the distribution of $Y$?

This question seems simple, but it turns out to be one of the most powerful tools in all of probability theory. Let’s see why.

0.1Transformations Are Everywhere¶

Every time you apply a formula to a random variable, you create a new random variable. Consider these examples:

• Temperature conversion. If $X$ is temperature in Celsius, then $Y = \frac{9}{5}X + 32$ is temperature in Fahrenheit. If you know the distribution of $X$, what’s the distribution of $Y$?

• Profit from revenue. If $X$ is revenue and costs are fixed at $c$, then profit is $Y = X - c$. What’s the distribution of profit?

• Signal strength. In telecommunications, signal power is often measured in decibels: $Y = 10 \log_{10}(X)$, where $X$ is the raw power. If $X$ follows a Gamma distribution, what distribution does $Y$ follow?

• Portfolio returns. If $X_1, X_2, \ldots, X_n$ are the returns on $n$ stocks, the portfolio return is $Y = w_1 X_1 + w_2 X_2 + \cdots + w_n X_n$, where $w_i$ are portfolio weights. What’s the distribution of $Y$?

• Sample mean. The most important statistic in all of statistics is $\bar{Y} = \frac{1}{n}\sum_{i=1}^n Y_i$. It’s a function of $n$ random variables. What’s its distribution?

In every case, we know the distribution of the input(s), and we need the distribution of the output. This chapter gives you the tools to solve all of these.

0.2Three Methods¶

We develop three distinct techniques, each with its own strengths:

There is no single “best” method. The CDF method always works but can involve tricky integration. The transformation method is elegant but requires a one-to-one function. The MGF method is the fastest when it works but only applies to sums and requires the MGF to match a known distribution.

The strategy: Try the simplest method first. If the function is a sum of independent RVs, try the MGF method. If the function is one-to-one and differentiable, try the transformation method. If neither works, fall back on the CDF method — it always works.

1The CDF Method (Method of Distribution Functions)¶

1.1The Core Idea¶

Suppose $X$ is a random variable with a known PDF $f_X(x)$ and CDF $F_X(x)$, and let $U = g(X)$ for some function $g$. The CDF method has exactly three steps:

That’s it. The method is universal — it works for any transformation, whether it’s linear, quadratic, logarithmic, or involves multiple variables. Let’s see it in action.

1.2Worked Examples: One Variable¶

1.3Discrete Transformations¶

Before diving into continuous examples, note that the CDF method also works for discrete random variables — just use summation instead of integration.

Example 6.0 (Discrete Transformation). A fair die is rolled: $X \in \{1, 2, 3, 4, 5, 6\}$, each with probability $1/6$. Let $U = (X - 3)^2$ — the squared deviation from 3.

The values of $U$ are: $X = 1 \to U = 4$, $X = 2 \to U = 1$, $X = 3 \to U = 0$, $X = 4 \to U = 1$, $X = 5 \to U = 4$, $X = 6 \to U = 9$.

Multiple $X$-values map to the same $U$: both $X = 2$ and $X = 4$ give $U = 1$; both $X = 1$ and $X = 5$ give $U = 4$. So we add their probabilities:

| $u$ | $x$-values that produce it | $p_U(u)$ | |:

1.4Worked Examples: Continuous Transformations¶

1.5Worked Examples: Two Variables¶

The CDF method extends naturally to functions of two (or more) random variables.

1.6The Convolution Formula¶

The sum $U = X + Y$ of two independent continuous random variables appears so frequently that its PDF formula has a special name:

This is derived from the CDF method by differentiating $P(X + Y \leq u) = \int\int_{x+y \leq u} f_X(x)f_Y(y)\,dy\,dx$ under the integral sign (substituting $y = u - x$). We used it implicitly in Examples 7.4 and 7.5 — now it has a name.

Why “convolution”? The operation is the continuous analog of multiplying two polynomials: the coefficients of the product are convolutions of the individual coefficients. In signal processing, convolution appears when a signal passes through a filter. In probability, it appears when independent random quantities add.

1.7Section 7.2 Exercises¶

1. Let $X$ have PDF $f_X(x) = 3x^2$ for $0 \leq x \leq 1$. Use the CDF method to find the PDF of $U = 1 - X$.

2. Let $X \sim \text{Exponential}(\beta = 2)$, i.e., $f_X(x) = \frac{1}{2}e^{-x/2}$ for $x > 0$. Find the PDF of $U = \sqrt{X}$ using the CDF method.

3. Let $X$ have PDF $f_X(x) = \frac{1}{2}(x+1)$ for $-1 \leq x \leq 1$. Find the PDF of $U = |X|$ using the CDF method.

4. Let $X$ and $Y$ be independent Exponential(1) random variables. Use the CDF method to find the PDF of $U = X + Y$. (Hint: the region of integration for $P(X + Y \leq u)$ is a triangle.)

5. Let $X \sim \text{Uniform}(0, 1)$. Find the PDF of $U = X(1-X)$. (This is challenging — think carefully about the range and whether the function is one-to-one.)

2The Transformation Method (One Variable)¶

2.1The Theorem¶

PROOF

Strategy: We derive this directly from the CDF method. We compute $F_U(u)$ in each case, then differentiate using the chain rule. The absolute value naturally emerges from handling both increasing and decreasing functions.

Case 1: $h$ is strictly increasing.

When $h$ is increasing, $h(X) \leq u$ if and only if $X \leq h^{-1}(u)$. (Why? Because an increasing function preserves the direction of inequalities: if $a \leq b$ then $h(a) \leq h(b)$, so $h(X) \leq u \iff X \leq h^{-1}(u)$.)

Therefore:

Differentiating with respect to $u$ using the chain rule:

Since $h$ is increasing, $h^{-1}$ is also increasing, so $\frac{d}{du}h^{-1}(u) > 0$. The expression is already positive — no absolute value needed.

Case 2: $h$ is strictly decreasing.

When $h$ is decreasing, $h(X) \leq u$ if and only if $X \geq h^{-1}(u)$. (Why? Because a decreasing function reverses the direction of inequalities: if $a \leq b$ then $h(a) \geq h(b)$. So $h(X) \leq u$ means $X$ is on the side where $h$ gives smaller values, which for a decreasing function is the right side — i.e., $X \geq h^{-1}(u)$.)

Therefore:

Differentiating with respect to $u$ using the chain rule:

Now, since $h$ is decreasing, $h^{-1}$ is also decreasing, so $\frac{d}{du}h^{-1}(u) < 0$. The product of the negative sign out front and the negative derivative gives a positive result. This is exactly $f_X(h^{-1}(u)) \cdot \left|\frac{d}{du}h^{-1}(u)\right|$.

Combining both cases: In Case 1, the derivative is positive and equals its own absolute value. In Case 2, the negative signs cancel and again produce the absolute value. Therefore:

∎

WHY THIS MATTERS: The absolute value of the derivative $|dx/du|$ acts as a “stretching factor.” It tells you how much the transformation compresses or expands probability density at each point. When $h$ stretches the $x$-axis (large $|dx/du|$), probability density gets diluted; when it compresses (small $|dx/du|$), density gets concentrated.

2.2When the Transformation Is Not One-to-One¶

Theorem 7.1 requires $h$ to be one-to-one. But what if it isn’t? For example, $U = X^2$ when $X$ ranges over both positive and negative values — both $x = 0.5$ and $x = -0.5$ give $U = 0.25$. We handled this with the CDF method in Example 7.2, but there’s also a direct formula:

2.3Worked Examples¶

2.4Section 7.3 Exercises¶

1. Let $X \sim \text{Uniform}(0, 1)$. Use the transformation method to find the PDF of $U = e^X$.

2. Let $X$ have PDF $f_X(x) = 4x^3$ for $0 \leq x \leq 1$. Use the transformation method to find the PDF of $U = -\ln(X)$.

3. Let $X \sim \text{Exponential}(\beta = 1)$. Use the transformation method to find the PDF of $U = e^{-X}$. What well-known distribution is this?

4. If $Y \sim N(\mu, \sigma^2)$, show that $Z = \frac{Y - \mu}{\sigma} \sim N(0, 1)$ using the transformation method.

5. Let $X \sim \text{Gamma}(\alpha, \beta)$. Use the transformation method to find the PDF of $U = cX$ for a positive constant $c$. Verify that $U \sim \text{Gamma}(\alpha, c\beta)$.

3The Method of Moment-Generating Functions¶

3.1The Key Principle¶

The MGF method rests on one powerful theorem from Chapter 4:

This means: if we compute $M_U(t)$ and it matches the MGF of a known distribution, then $U$ has that distribution. No integration needed.

3.2The Recipe for Sums¶

The MGF method is especially powerful for sums of independent random variables, because of the multiplication property:

PROOF

Strategy: Use the definition of the MGF and the fact that independence lets us factor a joint expectation into a product of individual expectations.

By definition, the MGF of $U$ is:

(Calculus flashback: The exponent rule $e^{a+b} = e^a \cdot e^b$ extends to sums: $e^{t(X_1 + X_2 + \cdots + X_n)} = e^{tX_1} \cdot e^{tX_2} \cdots e^{tX_n}$. This is the key step — it converts a sum inside an exponent into a product of exponentials.)

Using this rule:

Now we use independence. Recall from Chapter 6 that if $X_1, \ldots, X_n$ are independent, then for any functions $g_1, \ldots, g_n$:

Applying this with $g_i(X_i) = e^{tX_i}$:

∎

WHY THIS IS SO POWERFUL: Without this theorem, finding the PDF of a sum $X_1 + X_2 + \cdots + X_n$ would require computing an $(n-1)$-fold convolution integral — a nightmare for $n > 2$. This theorem reduces the problem to multiplying $n$ simple functions and recognizing the result. Multiplication is always easier than convolution.

3.3Worked Examples¶

3.4Section 7.4 Exercises¶

1. Let $X_1 \sim N(5, 4)$ and $X_2 \sim N(2, 9)$ be independent. Find the distribution of $U = X_1 - X_2$. (Hint: $X_1 - X_2 = X_1 + (-1)X_2$. Use the result from Example 7.10.)

2. Let $X_1, X_2, X_3$ be iid $\text{Poisson}(4)$. Find the distribution of $U = X_1 + X_2 + X_3$.

3. Let $X_i \sim \chi^2(3)$ for $i = 1, 2, 3, 4$ (independent). Find the distribution of $U = X_1 + X_2 + X_3 + X_4$.

4. If $Z_1, Z_2, \ldots, Z_n$ are iid $N(0, 1)$, show that $\sum_{i=1}^n Z_i^2 \sim \chi^2(n)$. (Hint: Use Examples 7.14 and 7.11.)

5. Let $X$ and $Y$ be independent, $X \sim \text{Binomial}(10, 0.3)$ and $Y \sim \text{Binomial}(15, 0.3)$. Without computing any convolutions, find $P(X + Y = 8)$.

4Multivariable Transformations¶

4.1The Jacobian¶

This is the direct generalization of Theorem 7.1. In one variable, the “Jacobian” was just $|dx/du|$. In two variables, it’s the absolute value of the determinant of a $2 \times 2$ matrix of partial derivatives.

PROOF

Strategy: We generalize the one-variable proof. The key idea is the same — we express probabilities as integrals, then apply the multivariable change-of-variables theorem from calculus.

For any region $B$ in the $(u_1, u_2)$-space, the probability that $(U_1, U_2) \in B$ equals the probability that $(X_1, X_2) \in A$, where $A$ is the corresponding region in $(x_1, x_2)$-space. That is:

By the multivariable change-of-variables theorem from calculus, when we substitute $x_1 = g_1(u_1, u_2)$ and $x_2 = g_2(u_1, u_2)$, the differential element transforms as:

where $J$ is the Jacobian determinant of the inverse transformation. (This is the same change-of-variables rule you used for double integrals in Calculus III — the Jacobian accounts for how the transformation distorts area elements.) Substituting:

Since this holds for every region $B$, the integrand must be the joint PDF of $(U_1, U_2)$:

∎

WHY THE JACOBIAN APPEARS: The Jacobian determinant $|J|$ measures how the transformation distorts area. If a small rectangle in $(x_1, x_2)$-space gets stretched into a larger region in $(u_1, u_2)$-space, the probability density must decrease proportionally to keep the total probability equal to 1. The Jacobian captures exactly this stretching/compression factor — it is the ratio of infinitesimal areas: $|J| = dx_1 \, dx_2 / (du_1 \, du_2)$.

4.2Section 7.5 Exercises¶

1. Let $X_1, X_2$ be iid $\text{Exponential}(1)$. Use a multivariable transformation with $U = X_1 + X_2$, $V = X_1/(X_1 + X_2)$ to show that $U$ and $V$ are independent, and find their marginal distributions.

2. Let $X_1$ and $X_2$ be independent with $X_i \sim N(0, 1)$. Define $U = X_1^2 + X_2^2$ and $V = X_1/X_2$. Find the marginal distributions of $U$ and $V$. (This is the derivation of the $\chi^2(2)$ and Cauchy distributions from standard normals.)

3. Let $X_1, X_2$ be independent Uniform(0, 1). Find the joint PDF of $U_1 = \min(X_1, X_2)$ and $U_2 = \max(X_1, X_2)$ using the Jacobian method. (Hint: first condition on $X_1 < X_2$ and $X_1 > X_2$ separately.)

5Order Statistics¶

5.1The Distribution of the Maximum and Minimum¶

We start with the two most important order statistics: the maximum and minimum.

PROOF

Strategy: The maximum is $\leq y$ if and only if ALL values are $\leq y$. Use independence.

By independence:

Differentiating to get the PDF:

(Calculus flashback: To differentiate $[F(y)]^n$ with respect to $y$, use the Chain Rule. Let $w = F(y)$, so $\frac{d}{dy}w^n = nw^{n-1} \cdot \frac{dw}{dy} = n[F(y)]^{n-1} \cdot f(y)$, since $F'(y) = f(y)$ by the Fundamental Theorem of Calculus.)

∎

PROOF

Strategy: It’s easier to compute $P(Y_{(1)} > y)$ — the minimum exceeds $y$ if and only if ALL values exceed $y$.

So $F_{Y_{(1)}}(y) = 1 - [1 - F(y)]^n$. Differentiating gives the PDF. ∎

5.2The General $k$th Order Statistic¶

PROOF

Strategy: We derive this rigorously using the CDF and the multinomial counting argument. For $Y_{(k)}$ to fall in an infinitesimal interval $[y, y + dy)$, the $n$ observations must partition into three groups: exactly $k - 1$ below $y$, exactly one in $[y, y+dy)$, and exactly $n - k$ above $y + dy$.

Step 1: Compute the CDF of $Y_{(k)}$.

Since the $Y_i$ are iid, each observation is $\leq y$ with probability $p = F(y)$, independently. The number of observations $\leq y$ follows a Binomial$(n, F(y))$ distribution:

Step 2: Differentiate to get the PDF.

Rather than differentiating this sum directly (which is possible but tedious), we use a more elegant argument. The probability that $Y_{(k)}$ falls in the infinitesimal interval $[y, y + dy)$ is:

For this to happen, we need exactly three things simultaneously:

• Exactly $k - 1$ of the $n$ observations fall below $y$. Each does so with probability $F(y)$.

• Exactly 1 observation falls in $[y, y + dy)$. Each does so with probability $f(y)\,dy$.

• Exactly $n - k$ observations fall above $y + dy \approx y$. Each does so with probability $1 - F(y)$.

The number of ways to assign $n$ observations into these three groups is the multinomial coefficient:

This counts the number of ways to choose which $k-1$ observations are below, which 1 is “at” $y$, and which $n-k$ are above. Multiplying:

Dividing both sides by $dy$:

∎

Sanity check: Setting $k = 1$: $f_{Y_{(1)}}(y) = \frac{n!}{0!(n-1)!}[F(y)]^0[1-F(y)]^{n-1}f(y) = n[1-F(y)]^{n-1}f(y)$ ✓ (matches Theorem 7.5).

Setting $k = n$: $f_{Y_{(n)}}(y) = \frac{n!}{(n-1)!0!}[F(y)]^{n-1}[1-F(y)]^0 f(y) = n[F(y)]^{n-1}f(y)$ ✓ (matches Theorem 7.4).

5.3Joint Distribution of Order Statistics¶

PROOF

Strategy: The joint PDF of the original (unsorted) sample $(Y_1, \ldots, Y_n)$ is $\prod f(y_i)$ by independence. The order statistics are a specific sorted arrangement. We count how many original orderings produce the same sorted result.

Step 1: The joint PDF of the original iid sample is:

Step 2: Consider a specific sorted sequence $y_1 < y_2 < \cdots < y_n$. How many orderings of the original sample produce this exact sorted sequence as the order statistics? The answer is $n!$ — any of the $n!$ permutations of the indices maps to the same sorted order. (For example, if $n = 3$ and the sorted values are $0.2 < 0.5 < 0.9$, then $(Y_1, Y_2, Y_3)$ could be any of the 6 permutations $(0.2, 0.5, 0.9)$, $(0.2, 0.9, 0.5)$, $(0.5, 0.2, 0.9)$, etc., and all six produce the same order statistics $(Y_{(1)}, Y_{(2)}, Y_{(3)}) = (0.2, 0.5, 0.9)$.)

Step 3: Since each of these $n!$ permutations contributes probability density $\prod f(y_i)$ to the sorted sequence, and they are mutually exclusive (continuous random variables have probability 0 of ties), the total density at the sorted point is:

The restriction $y_1 < y_2 < \cdots < y_n$ is essential: order statistics are defined to be sorted, so the density is zero outside this region.

∎

The factor $n!$ accounts for the $n!$ ways the original random sample could have produced this sorted order. This result is the foundation for deriving the joint distribution of any subset of order statistics (e.g., the joint distribution of $Y_{(1)}$ and $Y_{(n)}$ used in range problems).

5.4Section 7.6 Exercises¶

1. Let $Y_1, Y_2, Y_3$ be iid $\text{Uniform}(0, \theta)$. Find the PDF of $Y_{(3)} = \max(Y_1, Y_2, Y_3)$. Show that $E(Y_{(3)}) = \frac{3\theta}{4}$.

2. Let $Y_1, \ldots, Y_n$ be iid $\text{Exponential}(\beta)$. Show that $Y_{(1)} \sim \text{Exponential}(\beta/n)$.

3. Let $Y_1, Y_2, Y_3, Y_4$ be iid $\text{Uniform}(0, 1)$. Find the PDF of the median, $Y_{(2)}$, using the formula for the $k$th order statistic.

4. Five sensors independently measure a temperature. Each measurement follows a $\text{Uniform}(98, 102)$ distribution. Find the PDF of the range $R = Y_{(5)} - Y_{(1)}$. (Hint: use the joint distribution of $Y_{(1)}$ and $Y_{(5)}$ from Theorem 7.7 applied to the first and last order statistics, then transform.)

5. Let $Y_1, \ldots, Y_{10}$ be iid with CDF $F(y) = 1 - e^{-y}$ for $y > 0$. Find $P(Y_{(10)} > 3)$.

6⭐ Bonus: The Probability Integral Transform¶

PROOF

Strategy: Apply the CDF method. Since $F_X$ is a CDF, it maps from $(-\infty, \infty)$ to $(0, 1)$, so $U$ takes values in $(0, 1)$. We compute $F_U(u)$ directly.

Step 1: Since $F_X$ is strictly increasing, it has an inverse $F_X^{-1}$. So:

(Why? Because $F_X$ is increasing, $F_X(X) \leq u$ if and only if $X \leq F_X^{-1}(u)$.)

Step 2: By the definition of CDF:

Step 3: So $F_U(u) = u$ for $0 < u < 1$, which is the CDF of Uniform(0, 1).

∎

WHY THIS MATTERS: The PIT says that every continuous distribution, when passed through its own CDF, becomes Uniform(0, 1). This is remarkable — it means that behind every distribution lurks a uniform random variable. Running the process in reverse gives us a simulation method:

7⭐ Bonus: The Box-Muller Transform¶

PROOF

Strategy: Work backwards. Start with two independent $N(0,1)$ variables, convert to polar coordinates using a Jacobian, and show the polar components are functions of Uniform random variables.

Step 1: Joint density of two independent standard normals.

If $Z_1, Z_2 \stackrel{\text{iid}}{\sim} N(0,1)$, their joint PDF is:

Notice: this depends on $z_1$ and $z_2$ only through $z_1^2 + z_2^2 = r^2$ — it’s rotationally symmetric. This is the key insight.

Step 2: Convert to polar coordinates.

Define $R \geq 0$ and $0 \leq \Theta < 2\pi$ by $Z_1 = R\cos\Theta$, $Z_2 = R\sin\Theta$.

(Calculus flashback: The Jacobian of polar-to-Cartesian is $|J| = r$. This comes from the area element $dA = r\,dr\,d\theta$. Specifically: $\frac{\partial z_1}{\partial r} = \cos\theta$, $\frac{\partial z_1}{\partial \theta} = -r\sin\theta$, $\frac{\partial z_2}{\partial r} = \sin\theta$, $\frac{\partial z_2}{\partial \theta} = r\cos\theta$. Determinant: $r\cos^2\theta + r\sin^2\theta = r$.)

Applying Theorem 7.3 with $|J| = r$:

for $r > 0$, $0 \leq \theta < 2\pi$.

Step 3: Factor into independent components.

The joint PDF factors as $g(r) \cdot h(\theta)$, so $R$ and $\Theta$ are independent. $\Theta \sim \text{Uniform}(0, 2\pi)$ and $R$ has the Rayleigh PDF $f_R(r) = re^{-r^2/2}$.

Step 4: Express $R$ and $\Theta$ via Uniforms using the Inverse CDF Method.

For $\Theta$: Since $\Theta \sim \text{Unif}(0, 2\pi)$, set $\Theta = 2\pi U_2$ where $U_2 \sim \text{Unif}(0,1)$.

For $R$: The CDF is $F_R(r) = \int_0^r t\,e^{-t^2/2}\,dt$. Using $u$-substitution with $w = t^2/2$, $dw = t\,dt$:

Set $U_1 = F_R(R) = 1 - e^{-R^2/2}$ and solve for $R$:

Since $1 - U_1 \sim \text{Unif}(0,1)$ when $U_1 \sim \text{Unif}(0,1)$, we simplify: $R = \sqrt{-2\ln U_1}$.

Step 5: Substitute back into $Z_1 = R\cos\Theta$, $Z_2 = R\sin\Theta$:

∎

WHY THIS IS BEAUTIFUL: One result that uses every major idea from this chapter — the Jacobian (polar coordinates), the PIT (generating $R$ from $U_1$), independence via factorization (Step 3), and the rotational symmetry of the Normal distribution. This is how GPU-based scientific computing generates billions of normal random numbers.

# Box-Muller in action
set.seed(42)
n <- 10000
u1 <- runif(n); u2 <- runif(n)
z1 <- sqrt(-2 * log(u1)) * cos(2 * pi * u2)
z2 <- sqrt(-2 * log(u1)) * sin(2 * pi * u2)
cat("Z1: mean =", round(mean(z1), 3), " sd =", round(sd(z1), 3), "\n")
cat("Z2: mean =", round(mean(z2), 3), " sd =", round(sd(z2), 3), "\n")
cat("Cor(Z1,Z2) =", round(cor(z1, z2), 4), "\n")

87.8* The Delta Method: Approximating Transformations (Optional Enrichment)¶

8.1The Key Idea¶

If $Y$ is a random variable with mean $\mu$ and variance $\sigma^2$, and $g$ is a differentiable function, then we can approximate $g(Y)$ near $\mu$ using a Taylor expansion:

This is a linear approximation. Taking the expectation of both sides:

Taking the variance:

8.2Why Is This Useful?¶

The exact distribution of $g(Y)$ can be very hard to find (try finding the density of $g(Y) = \ln(Y)$ when $Y$ is Gamma!). But the Delta Method gives you the mean and variance immediately, without any of the CDF/transformation machinery.

8.3Worked Examples¶

Example D.1 (Variance of a Reciprocal). Suppose $Y$ has mean $\mu = 10$ and variance $\sigma^2 = 4$. You need the variance of $U = 1/Y$.

Here $g(y) = 1/y$ and $g'(y) = -1/y^2$, so $g'(\mu) = -1/100$.

No transformation theorem needed — just one line of calculus.

Example D.2 (Variance of a Log-Return). Stock returns are often analyzed as $U = \ln(Y)$ where $Y$ is the gross return. If $Y$ has mean $\mu = 1.08$ and $\sigma^2 = 0.04$:

$g(y) = \ln(y)$, $g'(y) = 1/y$, $g'(\mu) = 1/1.08 \approx 0.926$.

Compare with the exact answer (if $Y$ were lognormal): $\ln(1 + 0.04/1.08^2) = \ln(1.0343) = 0.0337$. The approximation is very close!

Example D.3 (Variance Stabilizing Transformation for Poisson). If $X \sim \text{Poisson}(\lambda)$ with large $\lambda$, then $E[X] = \lambda$ and $\text{Var}(X) = \lambda$. The variance depends on the mean, which is inconvenient for many statistical procedures.

Let $U = \sqrt{X}$. By the Delta Method with $g(x) = \sqrt{x}$ and $g'(x) = \frac{1}{2\sqrt{x}}$:

The variance of $\sqrt{X}$ is approximately $1/4$, regardless of $\lambda$! This is why the square root transformation is used to “stabilize” the variance of Poisson data — a result you will use extensively in regression analysis and ANOVA.

97.R The R Verification Lab¶

# ================================================================
# CHAPTER 6 GRAND VERIFICATION: Functions of Random Variables
# ================================================================

set.seed(2026)
n <- 200000  # simulation size

cat("========================================\n")
cat("  CHAPTER 6: ALL METHODS VERIFIED IN R  \n")
cat("========================================\n\n")

# ------ METHOD 1: CDF Method (Max of two uniforms) ------
Y1 <- runif(n); Y2 <- runif(n)
U_max <- pmax(Y1, Y2)
cat("CDF Method — Max of 2 Uniform(0,1):\n")
cat("  Theoretical E[max] = 2/3 =", round(2/3, 4), "\n")
cat("  Simulated E[max]   =", round(mean(U_max), 4), "\n\n")

# ------ METHOD 2: Transformation (Lognormal) ------
X <- rnorm(n, mean = 1, sd = 0.5)
S <- exp(X)
cat("Transformation — exp(N(1, 0.25)) = Lognormal:\n")
cat("  Theoretical mean = exp(mu + sigma^2/2) =", round(exp(1 + 0.25/2), 4), "\n")
cat("  Simulated mean   =", round(mean(S), 4), "\n\n")

# ------ METHOD 3: MGF (Sum of independent normals) ------
Y_sum <- rnorm(n, 3, 1) + rnorm(n, -1, 2) + rnorm(n, 5, 0.5)
cat("MGF Method — Sum of N(3,1) + N(-1,4) + N(5,0.25):\n")
cat("  Theoretical: N(7, 5.25), sd =", round(sqrt(5.25), 4), "\n")
cat("  Simulated mean =", round(mean(Y_sum), 4), "  sd =", round(sd(Y_sum), 4), "\n\n")

# ------ METHOD 4: Jacobian (Beta from Gamma ratio) ------
alpha1 <- 3; alpha2 <- 5; beta_param <- 2
G1 <- rgamma(n, shape = alpha1, scale = beta_param)
G2 <- rgamma(n, shape = alpha2, scale = beta_param)
U_ratio <- G1 / (G1 + G2)
cat("Jacobian — Gamma(3,2)/(Gamma(3,2)+Gamma(5,2)) = Beta(3,5):\n")
cat("  Theoretical mean = alpha1/(alpha1+alpha2) =", round(alpha1/(alpha1+alpha2), 4), "\n")
cat("  Simulated mean   =", round(mean(U_ratio), 4), "\n\n")

# ------ ORDER STATISTICS ------
# 7 draws from Exponential(100), find min
Y_mat <- matrix(rexp(7 * n, rate = 1/100), nrow = 7)
Y_min <- apply(Y_mat, 2, min)
Y_max <- apply(Y_mat, 2, max)
cat("Order Statistics — 7 draws from Exp(100):\n")
cat("  Min: theoretical mean = 100/7 =", round(100/7, 2),
    "  simulated =", round(mean(Y_min), 2), "\n")
cat("  Max: simulated mean =", round(mean(Y_max), 2), "\n\n")

# ------ PROBABILITY INTEGRAL TRANSFORM ------
Y_gamma <- rgamma(n, shape = 4, rate = 1/3)
U_pit <- pgamma(Y_gamma, shape = 4, rate = 1/3)
ks_result <- ks.test(U_pit, "punif")
cat("Probability Integral Transform (Gamma → Uniform):\n")
cat("  KS test p-value =", round(ks_result$p.value, 4),
    "(large p = uniform confirmed)\n\n")

cat("All results match theory. The math works!\n")

10Chapter 7 Refresh Exercises¶

These exercises integrate concepts from across the entire chapter.

1. Let $X \sim \text{Uniform}(0, 2)$. Use the CDF method to find the PDF of $U = X^3$. Then verify your answer using the transformation method.

2. Let $X \sim N(0, 1)$. Find the PDF of $U = |X|$ using the CDF method. (This is the “half-normal” distribution — used in Bayesian statistics as a prior for standard deviations.)

3. Let $X_1, X_2, \ldots, X_{10}$ be iid $\text{Exponential}(\beta = 5)$. Find the distribution of $U = 2\sum_{i=1}^{10} X_i / \beta$. (Hint: first find the distribution of the sum using MGFs, then apply the scaling property of the Gamma family.)

4. Let $X$ and $Y$ be independent $\text{Uniform}(0, 1)$. Define $U = \max(X, Y)$ and $V = \min(X, Y)$. Find the joint PDF of $(U, V)$ and the PDF of $W = U - V$ (the range).

5. Let $X_1 \sim \text{Gamma}(3, 2)$ and $X_2 \sim \text{Gamma}(5, 2)$ be independent. Find the distribution of $R = X_1 / (X_1 + X_2)$.

6. A reliability engineer tests 20 identical light bulbs. Each bulb’s lifetime is iid $\text{Exponential}(\beta = 1000)$ hours. (a) Find the distribution of the time until the first bulb fails. (b) What is the expected time until the first failure? (c) Find $P($all 20 bulbs last at least 50 hours$)$.

7. Let $Z_1, Z_2, \ldots, Z_8$ be iid $N(0,1)$. Find the distribution of $W = Z_1^2 + Z_2^2 + \cdots + Z_8^2$ and compute $P(W > 15.507)$.

8. Let $Y_1, Y_2, Y_3, Y_4, Y_5$ be iid from a distribution with CDF $F(y) = y^3$ for $0 \leq y \leq 1$. Find the PDF of the median $Y_{(3)}$.

9. A factory makes resistors whose resistance (in ohms) follows a $N(100, 4)$ distribution. Power dissipated is $P = V^2/R$, where voltage is a constant $V = 12$ volts. Find the approximate distribution of $P$ using a first-order Taylor expansion: $P \approx V^2/\mu_R + (-V^2/\mu_R^2)(R - \mu_R)$. (This is the “Delta Method” preview.)

10. (Challenge) Let $X$ and $Y$ be iid $\text{Exponential}(1)$. Use the transformation $U = X/(X+Y)$, $V = X + Y$ to show that $U \sim \text{Uniform}(0, 1)$ and that $U$ and $V$ are independent. (This is a special case of Example 7.15 with $\alpha_1 = \alpha_2 = 1$, and it explains why $\text{Beta}(1, 1) = \text{Uniform}(0,1)$.)