Correlation and Linear Regression

# The MATH 2200 teaching stack: mosaic (formula-based summaries, plots, and the
# lm() regression workflow) and BSDA. mosaic auto-loads ggplot2/ggformula.
library(mosaic)
library(BSDA)

# Build-portability shim (hidden): make the curated data reachable by a plain
# relative path "data/processed/..." no matter which folder the book builds from.
if (!dir.exists("data/processed")) {
  .d <- getwd()
  for (.i in 1:6) {
    .d <- dirname(.d)
    if (dir.exists(file.path(.d, "data", "processed"))) { setwd(.d); break }
  }
}

# Okabe-Ito colorblind-safe palette for this chapter's plots.
okabe_ito <- c("#0072B2", "#E69F00", "#009E73", "#CC79A7",
               "#56B4E9", "#D55E00", "#F0E442", "#999999")

# The Kern energy/employment panel (real federal + state public data).
energy <- read.csv("data/processed/kern_energy_employment.csv")

# A teaching-scaled predictor: crude in MILLIONS of barrels instead of
# thousands, so the slope is a clean "jobs per million barrels" number.
energy$crude_mmbbl <- energy$ca_crude_oil_prod_thsd_bbl / 1000

1Why this chapter: an oilfield in numbers¶

Drive north out of Bakersfield on Highway 65 and you pass the nodding pumpjacks of the Kern River and Cymric oil fields — the visible edge of the industry that has paid Kern County paychecks for a century. Over the last decade that industry shrank. California crude-oil production fell from 201,283 thousand barrels in 2015 to 109,828 thousand barrels in 2024, and over the same years the Bakersfield-area oilfield workforce (the “Mining & Logging” payroll line) fell from 11.4 thousand jobs to 7.2 thousand jobs.

Two questions follow naturally, and they are the two questions of this whole chapter:

1. Do the two numbers move together? When production is high, are jobs high? That is a question about correlation — the strength and direction of a straight-line relationship.

2. Can we put a number on the trade-off? “Each extra million barrels of production is worth about how many oilfield jobs?” That is a question about regression — fitting a line and reading its slope.

Here is the picture. Each point is one year, 2015–2024.

gf_point(ces_mining_logging_thsd ~ crude_mmbbl, data = energy,
         color = okabe_ito[1], size = 2.4) %>%
  gf_lm(color = okabe_ito[6]) %>%
  gf_labs(title = "More oil produced, more oilfield jobs?",
          x = "California crude production (million barrels/year)",
          y = "Bakersfield-area oilfield jobs (thousands)") %>%
  gf_theme(theme_minimal(base_size = 13))

The cloud of points slopes upward. By the end of the chapter you will be able to say how strongly (“the correlation is Unexecuted inline expression for: round(cor(energy$crude_mmbbl, energy$ces_mining_logging_thsd), 3)” — a number you will learn to read), how steep (“about 37 jobs per million barrels”), and whether the pattern could be a fluke (“no — the slope is statistically significant”). Every one of those numbers comes from the same real Kern data.

2Learning objectives¶

By the end of this chapter you will be able to:

1. Compute and interpret the correlation coefficient $r$, and recognize its limits — it measures linear association only, it is sensitive to outliers, and it never proves causation. (Apply)

2. Fit a least-squares regression line in R and interpret the slope and intercept in the units of the problem. (Apply)

3. Use the model for prediction and interpret $R^2$ and a residual plot. (Apply)

4. Diagnose outliers, leverage, and influential points and describe their effect on the fitted line. (Analyze)

5. Conduct inference on the slope (a hypothesis test and a confidence interval) and state the conditions a regression requires. (Apply)

This chapter expands ISRS sections 5.1–5.4 (line fitting and correlation, least squares, outliers, and inference for regression). It assumes you have met numerical summaries (Chapter 2) and inference for means (Chapter 10) — the slope test reuses the same $t$-distribution logic.

31. Correlation: measuring straight-line association¶

3.1Intuition¶

Two numerical variables are positively associated when large values of one tend to come with large values of the other (oil up, jobs up). They are negatively associated when large values of one come with small values of the other (as you will see: oil up, total county payroll down, because the economy diversified away from oil). The correlation coefficient puts a single number on two things at once:

• Direction — its sign (+ for upward, − for downward).

• Strength — its magnitude, from 0 (no straight-line pattern) up to 1 (the points fall exactly on a line).

Think of $r$ as a strength-and-direction dial pinned between -1 and +1. A value near $\pm 1$ means a tight line; a value near 0 means a shapeless blob (or a curved pattern that a straight-line measure cannot see).

3.2Formula¶

For $n$ data pairs $(x_i, y_i)$, the Pearson correlation coefficient is

Defining every symbol the first time it appears:

• $n$ — the number of paired observations (here, $n = 10$ years).

• $x_i$ — the $i$-th value of the explanatory variable (the predictor on the horizontal axis; here, crude production).

• $y_i$ — the $i$-th value of the response variable (the outcome on the vertical axis; here, oilfield jobs).

• $\bar{x}$ — the mean of the $x$ values; $\bar{y}$ — the mean of the $y$ values.

• The numerator $\sum (x_i-\bar{x})(y_i-\bar{y})$ is large and positive when $x$ and $y$ are both above (or both below) their means together.

• The two square roots in the denominator rescale that quantity so the result is always between -1 and +1, with no units — $r$ is the same whether oil is in barrels or millions of barrels.

3.3Properties to remember¶

• $-1 \le r \le 1$ always.

• $r$ has no units and does not change if you rescale or shift either variable.

• $r$ measures linear association only. A perfect U-shape can have $r \approx 0$.

• $r$ is not resistant: a single far-out point can swing it dramatically.

• Correlation is symmetric: $r$ of $x$ with $y$ equals $r$ of $y$ with $x$.

3.4R¶

In R, cor() computes $r$ directly. With mosaic you can write it in the same y ~ x formula grammar you use everywhere else — read it as “jobs versus crude.”

# Correlation between oilfield jobs and crude production:
cor(ces_mining_logging_thsd ~ crude_mmbbl, data = energy)

This returns Unexecuted inline expression for: round(cor(energy$crude_mmbbl, energy$ces_mining_logging_thsd), 4) (a value computed from kern_energy_employment). A correlation of about 0.86 is strong and positive: the upward line in Figure 1 is real and tight.

42. The least-squares regression line¶

4.1Intuition¶

A correlation says the points trend upward; a regression line draws the one straight line that best summarizes that trend. “Best” has a precise meaning. For any candidate line, each data point sits some vertical distance above or below it — that gap is the residual. The least-squares line is the unique line that makes the sum of the squared residuals as small as possible. Squaring keeps positive and negative gaps from cancelling and punishes big misses hardest, so the line is pulled to run through the “middle” of the cloud.

4.2Formula¶

We write the fitted line as

where:

• $\hat{y}$ (read “y-hat”) — the predicted value of the response for a given $x$. The hat always means “estimated/predicted,” not observed.

• $b_0$ — the intercept: the predicted $\hat{y}$ when $x = 0$.

• $b_1$ — the slope: the change in $\hat{y}$ for a one-unit increase in $x$. This is the number that answers “jobs per million barrels.”

• $x$ — a value of the explanatory variable you plug in.

The least-squares estimates are

with the building blocks

where $S_{xx}$ and $S_{yy}$ measure how much $x$ and $y$ each spread out, and $S_{xy}$ measures how they vary together; $s_x, s_y$ are the sample standard deviations of $x$ and $y$ (Chapter 2). Two facts worth memorizing fall straight out of these formulas:

• The line always passes through the point of averages $(\bar{x}, \bar{y})$ — substitute $x=\bar{x}$ into $\hat{y}=b_0+b_1x$ and you get $\bar{y}$.

• The slope shares the sign of the correlation, because $s_x$ and $s_y$ are positive: a positive $r$ gives a positive slope.

4.3R¶

Fit the line with base R’s lm(y ~ x, data = D) — the same formula grammar again — and read it with mosaic’s msummary(), which prints the coefficient table (each estimate with its standard error, $t$-statistic, and $p$-value), plus $R^2$ and the residual standard error.

fit <- lm(ces_mining_logging_thsd ~ crude_mmbbl, data = energy)
msummary(fit)   # coefficient table (estimate, SE, t, p), R^2, residual SE

The intercept is the (Intercept) row and the slope is the crude_mmbbl row of that table. Pull the key components out as plain numbers:

b0 <- coef(fit)[["(Intercept)"]]   # b0, the intercept
b1 <- coef(fit)[["crude_mmbbl"]]   # b1, the slope
R2 <- rsquared(fit)                # R^2 (mosaic helper)
c(intercept = b0, slope = b1, R_squared = R2)

From the real Kern data these are $b_0 =$ Unexecuted inline expression for: round(b0, 4), $b_1 =$ Unexecuted inline expression for: round(b1, 4), and $R^2 =$ Unexecuted inline expression for: round(R2, 4) (all derived from kern_energy_employment). The fitted line is therefore

The slope is shown rounded to four decimals. R carries more digits behind the scenes, so a prediction you compute from the printed equation may differ from R’s answer in the last digit — that is rounding, not an error.

Reading the slope. A slope of 0.0367 thousand jobs per million barrels means each additional million barrels of annual production is associated with about $0.0367 \times 1000 \approx 37$ more oilfield jobs. Reading the intercept. $b_0 = 3.04$ would be the predicted payroll at zero production — but zero is far outside the observed range (production never dipped below ~110 million barrels), so the intercept here is a mathematical anchor, not a real-world forecast.

gf_point(ces_mining_logging_thsd ~ crude_mmbbl, data = energy,
         color = okabe_ito[1], size = 2.4) %>%
  gf_lm(color = okabe_ito[6]) %>%
  gf_labs(title = "Least-squares fit: Kern oilfield jobs vs crude production",
          x = "California crude production (million barrels/year)",
          y = "Bakersfield-area oilfield jobs (thousands)") %>%
  gf_theme(theme_minimal(base_size = 13))

The least-squares line through the Kern energy data (its equation is $\widehat{\text{jobs}} = 3.0411 + 0.0367\,x$, with $R^2 = 0.73$). The line minimizes the total squared vertical distance from the points.

The fit above is the answer; the simulator below lets you discover it. On a small simulated scatter, drag the slope slider and watch the residual segments and the running SSE total. Because the intercept is always set to the best value for whatever slope you pick, every line pivots through the centroid $(\bar{x}, \bar{y})$ — and the SSE drops to its single lowest point exactly at the least-squares slope, then climbs again. That lowest point is what “least squares” means.

53. Prediction, R², and residuals¶

5.1Intuition¶

Once you have a line you can predict: plug an $x$ into $\hat{y}=b_0+b_1x$. But a prediction is only as trustworthy as the line is tight, and two summaries tell you how tight it is.

• $R^2$ — the coefficient of determination — is the fraction of the variation in $y$ that the line explains. It runs from 0 (the line explains nothing) to 1 (the line explains everything). For one predictor, $R^2 = r^2$.

• The residual standard error $s_e$ — the typical size of a residual — tells you, in the units of $y$, how far off a prediction is likely to be.

A residual plot (residuals on the vertical axis against $x$ or against $\hat{y}$) is the single most useful diagnostic: if the line fits well, the residuals should scatter randomly around zero with no curve and no fanning.

5.2Formula¶

• $\hat{y}_i = b_0 + b_1 x_i$ — the predicted value at the $i$-th observation.

• $\text{SSE}$ — the sum of squared errors (the quantity least squares minimizes).

• $n-2$ — the degrees of freedom: we used two pieces of the data to estimate $b_0$ and $b_1$, so two are “spent.”

5.3R¶

rsquared(fit)                    # fraction of job-variation explained
se_resid <- summary(fit)$sigma   # residual standard error, in thousands of jobs
se_resid

# Predict oilfield jobs in a year with 140 million barrels of production
# (b0 and b1 were pulled out above):
b0 + b1 * 140

For the Kern data, $R^2 =$ Unexecuted inline expression for: round(R2, 4) — about 73% of the year-to-year variation in oilfield jobs is explained by crude production (kern_energy_employment). The residual standard error is $s_e =$ Unexecuted inline expression for: round(se_resid, 4) thousand jobs, so a typical prediction is off by roughly 700 jobs. Predicting a 140-million-barrel year gives about Unexecuted inline expression for: round(b0 + b1*140, 2) thousand jobs.

Now the residual plot:

resid_df <- data.frame(
  crude = energy$crude_mmbbl,
  residual = resid(fit)
)
ggplot(resid_df, aes(x = crude, y = residual)) +
  geom_hline(yintercept = 0, linetype = "dashed", colour = okabe_ito[8]) +
  geom_point(colour = okabe_ito[1], size = 2.4) +
  labs(
    title = "Residual plot: no obvious pattern is good news",
    x = "California crude production (million barrels/year)",
    y = "Residual (thousands of jobs)"
  ) +
  theme_minimal(base_size = 13)

Residuals (observed minus predicted oilfield jobs) plotted against crude production. Points scatter around the dashed zero line with no strong curve, supporting the straight-line model; the highest point is the 2015 over-prediction.

64. Outliers, leverage, and influence¶

6.1Intuition¶

Because least squares minimizes squared distances, unusual points can pull the line toward themselves. Three labels keep them straight:

• An outlier is a point with a large residual — it sits far from the line.

• A high-leverage point has an extreme $x$ value (far left or far right). It has the potential to move the line a lot, just by where it sits.

• An influential point is one that actually does change the slope or intercept noticeably when removed. A point is influential when it is high leverage and off the trend.

The practical test is honest and simple: refit without the suspect point and see if your conclusions move.

6.2R¶

In the Kern data, 2015 is the extreme-$x$ year (highest production on record) and has the largest residual (+Unexecuted inline expression for: round(max(resid(fit)), 2) thousand jobs): the oilfield employed more people in 2015 than the line predicts. The COVID year 2020 is a natural suspect too. Let us refit without 2020 and compare the correlation.

# Correlation with all 10 years:
cor(ces_mining_logging_thsd ~ crude_mmbbl, data = energy)

# Correlation after dropping the 2020 (COVID) year:
no2020 <- energy[energy$year != 2020, ]
cor(ces_mining_logging_thsd ~ crude_mmbbl, data = no2020)

The correlation barely moves — from Unexecuted inline expression for: round(cor(energy$crude_mmbbl, energy$ces_mining_logging_thsd), 4) to Unexecuted inline expression for: round(cor(no2020$crude_mmbbl, no2020$ces_mining_logging_thsd), 4) (kern_energy_employment). So 2020, despite being a dramatic year, is not influential here: the relationship is robust to it. That is exactly the kind of check a careful analyst runs before trusting a line.

75. Inference for the slope¶

7.1Intuition¶

We have a slope from ten years of data. If we had a different ten years, we would get a slightly different slope. So the slope $b_1$ we computed is an estimate of an unknown true slope $\beta_1$ (the Greek letter “beta”) that describes the underlying relationship. The central question of regression inference is: could the true slope actually be zero? A true slope of zero would mean no linear relationship — the apparent tilt is just sampling noise.

We answer it the same way we tested a mean in Chapter 10: form a $t$-statistic and read a p-value.

7.2Formula¶

A $100(1-\alpha)\%$ confidence interval for the true slope is

where:

• $\beta_1$ — the unknown population slope we are inferring about.

• $\mathrm{SE}(b_1)$ — the standard error of the slope: how much $b_1$ would bounce from sample to sample.

• $t$ — the test statistic: how many standard errors the observed slope sits away from zero.

• $t^{\star}$ — the critical value from the $t$-distribution with $n-2$ degrees of freedom (e.g. qt(0.975, df) for 95% confidence).

• $\alpha$ — the significance level (the risk of a false alarm you accept, commonly 0.05).

The four conditions the test requires (remember them as L-I-N-E): Linearity (the scatter looks straight), Independence (observations gathered independently), Normal residuals (roughly symmetric, no skew), and Equal spread (residual scatter constant across $x$).

7.3R¶

The msummary(fit) table above already carries the slope’s standard error, $t$-statistic, and $p$-value in its crude_mmbbl row; confint() adds the 95% interval. Pull the slope-row pieces out for the sentence below:

confint(fit)                                        # 95% CIs for (Intercept) and slope
tstat <- coef(summary(fit))["crude_mmbbl", "t value"]      # t = b1 / SE(b1)
pval  <- coef(summary(fit))["crude_mmbbl", "Pr(>|t|)"]     # two-sided p-value
dfree <- df.residual(fit)                                  # n - 2
ci    <- confint(fit)["crude_mmbbl", ]                      # CI for beta1
c(t = tstat, df = dfree, p = pval)

For the Kern data: $t =$ Unexecuted inline expression for: round(tstat, 3) on Unexecuted inline expression for: dfree degrees of freedom, $p =$ Unexecuted inline expression for: signif(pval, 3), and the 95% confidence interval for the slope is (Unexecuted inline expression for: round(ci[1], 4), Unexecuted inline expression for: round(ci[2], 4)) (kern_energy_employment). Because $p < 0.05$ — and equivalently, because the interval excludes 0 — we reject $H_0$: there is statistically significant evidence of a positive linear relationship between crude production and oilfield employment in Kern County over 2015–2024.

8Worked examples¶

8.1Worked Example 1 — Correlation by hand (Kern energy)¶

Question. Using the 2015–2024 Kern data, find the correlation between crude production (million barrels) and oilfield jobs (thousands), and state direction and strength.

Intuition. The scatterplot tilts upward, so expect a positive $r$; the points hug a line fairly tightly, so expect it to be well above 0.5.

Formula. $r = S_{xy} / \sqrt{S_{xx}\,S_{yy}}$.

Computation.

x <- energy$crude_mmbbl
y <- energy$ces_mining_logging_thsd
xbar <- mean(x); ybar <- mean(y)
Sxx <- sum((x - xbar)^2)
Syy <- sum((y - ybar)^2)
Sxy <- sum((x - xbar) * (y - ybar))
r_manual <- Sxy / sqrt(Sxx * Syy)
r_manual
cor(x, y)   # check against the built-in

Interpretation. Both give $r =$ Unexecuted inline expression for: round(cor(x,y), 4) (kern_energy_employment). The relationship is positive (oil and jobs rise together) and strong (well above 0.7 in magnitude). The hand computation and cor() agree to the last digit, confirming we understand what the function does.

8.2Worked Example 2 — Fit, predict, and interpret (Kern energy)¶

Question. Fit the least-squares line predicting oilfield jobs from crude production. Interpret the slope, then predict the jobs for a year with 160 million barrels.

Intuition. A positive correlation forces a positive slope. With about 37 jobs per million barrels, 160 million barrels should predict somewhere near 9 thousand jobs.

Formula. $b_1 = S_{xy}/S_{xx}$, $b_0 = \bar{y} - b_1\bar{x}$, $\hat{y} = b_0 + b_1 x$.

Computation.

fit2 <- lm(ces_mining_logging_thsd ~ crude_mmbbl, data = energy)
b1_2 <- coef(fit2)[["crude_mmbbl"]]     # slope
b0_2 <- coef(fit2)[["(Intercept)"]]     # intercept
c(slope = b1_2, intercept = b0_2, pred_at_160 = b0_2 + b1_2 * 160)

Interpretation. The slope Unexecuted inline expression for: round(b1_2, 4) thousand jobs per million barrels means about 37 jobs per million barrels (multiply by 1000). At 160 million barrels the model predicts Unexecuted inline expression for: round(b0_2 + b1_2*160, 3) thousand jobs — about 8,920 jobs (kern_energy_employment). Because 160 sits inside the observed range (110–201 million barrels), this is an interpolation and is trustworthy.

8.3Worked Example 3 — A negative relationship (Kern energy)¶

Question. As oil production fell, what happened to the total Bakersfield-MSA payroll (all jobs, not just oilfield)? Compute the correlation and slope of total nonfarm employment on crude production.

Intuition. If the regional economy diversified as oil declined — adding warehouses, health care, and logistics jobs — total employment might have risen while oil fell, giving a negative correlation.

Formula. Same as before, now with $y =$ total nonfarm payroll.

Computation.

cor(ces_total_nonfarm_thsd ~ crude_mmbbl, data = energy)
fit3 <- lm(ces_total_nonfarm_thsd ~ crude_mmbbl, data = energy)
coef(fit3)[["crude_mmbbl"]]   # slope
rsquared(fit3)                # R^2

Interpretation. The correlation is Unexecuted inline expression for: round(cor(energy$crude_mmbbl, energy$ces_total_nonfarm_thsd), 4) — strong and negative (kern_energy_employment). The slope Unexecuted inline expression for: round(coef(fit3)[["crude_mmbbl"]], 4) says total payroll fell by about 0.45 thousand jobs (≈450 jobs) for every million-barrel increase in oil — i.e., as oil declined, total employment grew. With $R^2 =$ Unexecuted inline expression for: round(rsquared(fit3), 4), crude production tracks about 79% of the variation. The contrast with Example 2 is the lesson: oilfield jobs and total jobs moved in opposite directions over this decade.

8.4Worked Example 4 — Slope inference and conditions (Kern energy)¶

Question. Is the oilfield-jobs relationship from Example 2 statistically significant? Test $H_0: \beta_1 = 0$ at $\alpha = 0.05$ and give a 95% CI for the slope. Check the LINE conditions.

Intuition. The points hug the line tightly with $n=10$ years, so the slope is probably several standard errors from zero — expect a small p-value.

Formula. $t = b_1/\mathrm{SE}(b_1)$, df $= n-2 = 8$; reject if $p < \alpha$.

Computation.

tstat2 <- coef(summary(fit2))["crude_mmbbl", "t value"]
dfree2 <- df.residual(fit2)                              # n - 2 = 8
pval2  <- coef(summary(fit2))["crude_mmbbl", "Pr(>|t|)"]
ci2    <- confint(fit2)["crude_mmbbl", ]                 # 95% CI for the slope
c(t = tstat2, df = dfree2, p = pval2)
ci2

Interpretation. $t =$ Unexecuted inline expression for: round(tstat2, 3) on 8 df gives $p =$ Unexecuted inline expression for: signif(pval2, 3), far below 0.05, so we reject $H_0$. The 95% CI for the slope, (Unexecuted inline expression for: round(ci2[1],4), Unexecuted inline expression for: round(ci2[2],4)), excludes 0 — consistent with the test. Conditions: Linearity holds (Figure 1 is straight); Equal spread and Normal residuals look acceptable in the residual plot (ch13-fig-resid); Independence is the shakiest — these are consecutive years of one county, so a careful analyst flags possible year-to-year dependence as a caveat. (Result from kern_energy_employment.)

8.5Worked Example 5 — Regression on simulated crop data¶

Question. In the simulated Kern crops dataset, does almond yield per acre predict total crop value? Fit the line and report $r$, $R^2$, and the slope’s significance.

Intuition. Higher yield per acre across many fields could raise total value, so expect a positive slope — but simulated noise may make it less tight than the energy data.

Formula. Same least-squares machinery; $y=$ value (million USD), $x=$ yield per acre.

Computation.

crops <- read.csv("data/processed/kern_crops_sim.csv")
almonds <- subset(crops, commodity == "ALMONDS")
almonds$value_mil <- almonds$value_usd / 1e6
fit5 <- lm(value_mil ~ yield_per_acre, data = almonds)
r5   <- cor(value_mil ~ yield_per_acre, data = almonds)     # r
R2_5 <- rsquared(fit5)                                       # R^2
p5   <- coef(summary(fit5))["yield_per_acre", "Pr(>|t|)"]    # slope p-value
c(r = r5, R_squared = R2_5, slope_p = p5)

Interpretation. For the simulated almonds, $r =$ Unexecuted inline expression for: round(r5, 3) and $R^2 =$ Unexecuted inline expression for: round(R2_5, 3), with slope p-value Unexecuted inline expression for: signif(p5, 2) — a statistically significant positive relationship in the synthetic data (kern_crops_sim). The point of the example is the procedure, not the agronomy: the same lm(y ~ x, data =) call works on any pair of numerical variables, and msummary() walks you through every piece of the fit.

9Try it yourself¶

• Shiny — Statistics Explorer, “Correlation & Regression” module (shiny-explorer/, module mod_regression). Drop in any two numerical columns, watch the scatterplot, fitted line, $r$, $R^2$, and slope test update live — and copy the exact lm() / cor() R code each action generates. Launch locally with shiny::runApp("shiny-explorer").

• Jupyter lab — labs/lab13-regression.ipynb (R kernel). A guided walkthrough that re-creates this chapter’s Kern energy regression step by step, then turns you loose on kern_crops_sim and an openintro dataset of your choice with starter code and a reflection prompt.

Both tools use the same mosaic/lm() functions you used above, so the syntax you learn here is the syntax you use everywhere.

10Chapter summary¶

• Correlation $r$ measures the direction (sign) and strength (magnitude, 0 to 1) of a linear relationship. It is unit-free, symmetric, not resistant to outliers, and never proves causation.

• The least-squares regression line $\hat{y}=b_0+b_1x$ minimizes the sum of squared residuals. The slope $b_1$ is the predicted change in $y$ per one-unit increase in $x$; the intercept $b_0$ is $\hat{y}$ at $x=0$ (often an extrapolation). The line always passes through $(\bar{x},\bar{y})$.

• $R^2 = r^2$ is the fraction of variation in $y$ explained by the line; the residual standard error $s_e$ is the typical prediction error in $y$-units.

• A residual plot with no pattern supports the straight-line model. Outliers (big residual), leverage (extreme $x$), and influence (actually moves the line) are diagnosed by refitting without the suspect point.

• Inference on the slope uses $t = b_1/\mathrm{SE}(b_1)$ on $n-2$ df. A small p-value (or a CI excluding 0) means a statistically significant linear relationship. The conditions are LINE: Linearity, Independence, Normal residuals, Equal spread.

• For the real Kern energy data: $r =$ Unexecuted inline expression for: round(cor(energy$crude_mmbbl, energy$ces_mining_logging_thsd), 3), slope $=$ Unexecuted inline expression for: round(b1, 4) thousand jobs per million barrels, $R^2 =$ Unexecuted inline expression for: round(R2, 3), $p =$ Unexecuted inline expression for: signif(pval, 2).

11FAQ¶

Q1. What is the difference between $r$ and $R^2$? $r$ (from -1 to 1) carries the direction of the relationship; $R^2$ (from 0 to 1) is $r$ squared and gives the fraction of variation explained but loses the sign. Report $r$ to describe direction and strength; report $R^2$ to describe predictive accuracy.

Q2. Does a significant slope mean $x$ causes $y$? No. Significance means the slope is unlikely to be zero by chance — it says nothing about causation. Causal claims need an experiment with random assignment (Chapter 1, Chapter 8), not an observational regression.

Q3. Can I predict outside the range of my $x$ data? You can compute it, but you shouldn’t trust it. Predicting beyond the observed $x$ range is extrapolation, and the linear pattern may not hold there. Our intercept ($x=0$ barrels) is a stark example — no Kern year had zero production.

Q4. My residual plot has a clear curve. What now? A curved residual plot means a straight line is the wrong model. The relationship may be genuinely nonlinear; options include transforming a variable (e.g. taking a logarithm) or fitting a curve — topics beyond this chapter, but the residual plot is what tells you a line won’t do.

Q5. Why divide by $n-2$ for the residual standard error? Because two degrees of freedom are used up estimating the intercept and the slope. Dividing by $n-2$ (rather than $n$) corrects for that and gives an unbiased estimate of the typical residual size — the same idea as dividing by $n-1$ for a sample standard deviation in Chapter 2.

Q6. Does it matter which variable is $x$ and which is $y$? For correlation, no — $r$ is symmetric. For regression, yes — the line that predicts $y$ from $x$ is different from the line that predicts $x$ from $y$. Choose $y$ to be the variable you want to predict or explain.

Q7. I have only 10 data points. Is that enough to regress? It can be, if the relationship is strong and the conditions hold — our Kern example reaches significance with $n=10$. But small samples make every estimate shaky; always report the confidence interval so readers see how much the slope could plausibly vary.

12Practice problems¶

1. In one sentence, define the correlation coefficient $r$, naming both the quantity its sign conveys and the quantity its magnitude conveys.

2. A scatterplot of two variables shows a tight, perfectly U-shaped curve. Will $r$ be near +1, near -1, or near 0? Explain in terms of what $r$ can and cannot detect.

3. Compute cor(energy$crude_mmbbl, energy$kern_unemp_rate). State the value, its direction, and whether the strength is weak, moderate, or strong.

4. True or false, with a one-sentence reason: “If $r = 0$, the two variables are unrelated.”

5. Using the fitted line $\widehat{\text{jobs}} = 3.0411 + 0.0367 x$ (jobs in thousands, $x$ in million barrels), predict oilfield jobs for a year with 130 million barrels of production. Show the arithmetic.

6. In Problem 5’s model, interpret the slope 0.0367 in words and units, including the “per 1000” conversion to whole jobs.

7. Explain why the intercept $b_0 = 3.0411$ should not be reported as “the number of oilfield jobs when production is zero” for this dataset.

8. The regression line always passes through one particular point. Name it, and verify it by plugging $\bar{x}$ into the fitted equation and comparing to $\bar{y} =$ Unexecuted inline expression for: round(mean(energy$ces_mining_logging_thsd), 3).

9. For the Kern energy fit, $R^2 = 0.7321$. Write one sentence interpreting this value for a county economic-development officer.

10. The residual standard error for the oilfield-jobs model is about 0.70 thousand jobs. Explain what this number tells a planner who is using the model to forecast.

11. Define residual in a single sentence, and state what the residuals should look like in a residual plot when a straight-line model fits well.

12. Fit lm(ces_mining_logging_thsd ~ crude_mmbbl, data = energy) and read off the 95% confidence interval for the slope with confint(). Does it contain 0, and what does that imply for the hypothesis test?

13. Distinguish an outlier from a high-leverage point in regression, and say what makes a point influential.

14. The year 2015 has the largest residual and the most extreme $x$. Describe the procedure you would run to decide whether 2015 is influential, and what result would tell you it is not.

15. State the null and alternative hypotheses for the test of a regression slope, using the symbol $\beta_1$, and say in plain words what $H_0$ claims about the relationship.

16. A slope test returns $t = 4.68$ on 8 degrees of freedom with $p = 0.0016$. At $\alpha = 0.05$, state the decision and write a one-sentence conclusion in context for the Kern oilfield data.

17. List the four regression conditions (the LINE mnemonic) and, for the Kern energy data, name the one most open to question and why.

18. Correlation between two variables is $r = -0.89$. Without computing anything else, what is the sign of the least-squares slope, and why must they agree?

19. Compute the correlation between crude_mmbbl and mining_logging_share_pct (the oil sector’s share of total payroll). Is it stronger or weaker than the correlation with oilfield jobs (Unexecuted inline expression for: round(cor(energy$crude_mmbbl, energy$ces_mining_logging_thsd),3))? Speculate briefly why.

20. A colleague predicts oilfield jobs for a year with 300 million barrels of production using our line. Explain why this prediction is untrustworthy, naming the relevant term.

21. Using lm(), regress ces_total_nonfarm_thsd on crude_mmbbl. Report the slope and explain why its sign is the opposite of the oilfield-jobs slope, in terms of the regional economy.

22. Explain the difference between $r$ and the slope $b_1$. Can two datasets have the same $r$ but very different slopes? Give the reason.

23. A student says, “$R^2 = 0.73$, so 73% of the time the model predicts jobs correctly.” Identify the misconception and state what $R^2 = 0.73$ actually means.

24. For the simulated almond data (crops <- read.csv("data/processed/kern_crops_sim.csv"), almonds only), fit value_mil (value in million USD) on yield_per_acre and report $R^2$. Why must you label any conclusion as based on synthetic data?

25. Suppose you rescale crude production from million barrels to billion barrels (divide $x$ by 1000). Which of these change and which stay the same: $r$, $R^2$, the slope $b_1$, the slope’s p-value? Justify each.

26. The slope’s 95% confidence interval is about $(0.019,\ 0.055)$ thousand jobs per million barrels. Translate this interval into whole jobs per million barrels and write the plain-language sentence you would give a decision-maker.

27. Give a real-world pair of variables you would expect to be positively correlated but where one does not cause the other, and name a plausible lurking variable.

28. Why is the slope test in this chapter built on the $t$-distribution with $n-2$ degrees of freedom rather than the normal distribution? Connect your answer to one-sample $t$ inference from Chapter 10.

29. A regression has $r = 0.40$. Compute $R^2$ and write one sentence contrasting what the two numbers say about the relationship.

30. In two or three sentences, explain to a non-statistician why “the correlation between ice-cream sales and drowning deaths is high” does not mean ice cream is dangerous, using the vocabulary of this chapter.

13Resumen en español¶