15. Regression with time: autocorrelation and forecasting

In 1949 an international airline started keeping a simple monthly tally: how many passengers flew that month, counted in thousands. Twelve years of those counts, from January 1949 through December 1960, became one of the most studied datasets in statistics. The airline’s problem was concrete. To decide how many aircraft to buy, how many crews to hire, and how to schedule maintenance, the planners needed to forecast demand a year or two ahead. A good forecast saved money; a bad one grounded planes or stranded passengers.

Figure 1 plots the 144 monthly counts in order. Three features jump out. The series climbs steadily: air travel roughly tripled over the twelve years. It also cycles every year, with a tall summer peak (people fly on vacation) and a winter dip. And the swings get wider as the series rises: the gap between the summer peak and the winter trough was small in 1949 and large in 1960. A single straight line will not describe this, but a regression with a trend term and a set of monthly indicators just might.

Time-ordered data like this break a promise the earlier chapters relied on. Every inference you have done so far assumed the errors were uncorrelated: one observation’s luck told you nothing about the next. When observations come in time order, that assumption usually fails. A month that runs above the trend tends to be followed by another above the trend, because whatever pushed demand up (a booming economy, a new route) does not vanish overnight. This chapter shows what that correlation does to your standard errors, how to detect it, one classic way to remove it, and how to judge a forecast honestly by testing it on the future instead of the past.

15.1 A model for trend and seasonality¶

Intuition¶

We start where the workflow always starts: with the picture and the question. Figure 1 shows three ingredients, and a regression can carry one term for each. The steady climb is a trend, which we model with a straight line in the time index $t = 1, 2, \dots, 144$. The yearly cycle is seasonality, which we model with eleven indicator (dummy) variables for the months February through December, each measuring how that month sits relative to a January baseline. The growing swings are the reason we do not model the raw counts directly.

Look again at the raw series: in 1949 the summer peak stands maybe 30 above the winter trough, but by 1960 that same seasonal gap is over 150. The seasonal effect is not a fixed number of passengers; it is a fixed percentage of the current level. When an effect is multiplicative like that, taking logarithms turns it additive and, as a bonus, evens out the variance. Figure 2 shows the payoff: on the log scale the seasonal swings are about the same width every year. Working on the log scale is the transformation idea from 10.2 The log transformation and reading its coefficients, and it is what lets a single set of month effects fit the whole twelve years.

Formula¶

Let $Y_t$ be the passenger count in month $t$ and work with $\log Y_t$. The model carries one term for each feature of Figure 1: a linear trend, a fixed offset for each month, and an error.

The pieces, term by term:

• $t$ is the time index, 1 for January 1949 up to $n = 144$ for December 1960.

• $\beta_0$ is the intercept: the log level of a January at time $t = 0$ (a baseline anchor).

• $\beta_1$ is the trend slope: the change in $\log Y$ per month. Because $Y$ is logged, $\beta_1$ is an approximate proportional growth rate per month (from 10.2 The log transformation and reading its coefficients).

• $D_{mt}$ is the month-$m$ indicator: it equals 1 when month $t$ is calendar month $m$, and 0 otherwise. There are eleven of them, for February ($m=2$) through December ($m=12$).

• $\gamma_m$ is the seasonal coefficient for month $m$: how far month $m$ sits above or below January on the log scale, holding the trend fixed. January is the reference month, coded by all eleven indicators being zero, exactly the dummy-coding scheme of 11.1 From categories to numbers: indicator coding.

• $\varepsilon_t$ is the error in month $t$.

In words: the log passenger count is a straight-line trend plus a fixed offset for each month of the year plus a disturbance. For now we fit this by ordinary least squares, treating the $\varepsilon_t$ as if they satisfied the usual assumptions. Section 15.2 checks whether they do.

R¶

We read the tidy monthly file, build a time index and a log response, and make month a factor so lm expands it into indicator variables automatically.

air <- read.csv("data/airpassengers.csv")
air$t <- seq_len(nrow(air))
air$logpass <- log(air$passengers)
air$month <- factor(air$month)
dim(air)
head(air, 3)

[1] 144   5
  year month passengers t  logpass
1 1949     1        112 1 4.718499
2 1949     2        118 2 4.770685
3 1949     3        132 3 4.882802

Eleven coefficients are hard to picture from a table, so Figure 3 draws them as bars. Each bar is one month’s offset from January, and the shape tells the whole seasonal story at a glance: a summer hump peaking in July and August, a soft dip in late fall.

The trend slope deserves its own reading, because logs make it a growth rate.

Figure 4 lays the fitted values over the observed log series. The straight trend plus twelve monthly offsets reproduces the sawtooth shape closely, which is why $R^2$ is so high.

15.2 Why time breaks ordinary least squares¶

Intuition¶

We can now fit the line; the obvious next question is whether we should trust its standard errors. Recall the residual plot habit from Chapter 2: after any fit, look at the residuals. For time data, plot them in time order. Figure 5 does exactly that for the trend-plus-season model, and the picture is not the random band you want. The residuals move in long runs: a stretch of months above zero, then a stretch below, then above again. A month that overshoots the model is usually followed by another that overshoots. That is autocorrelation (Definition 15.2): the errors are correlated with their own recent past.

Autocorrelation matters because it breaks the “uncorrelated errors” assumption that every standard error formula in this book was built on. Under the Gauss-Markov conditions of 2.5 Sampling behavior and the Gauss-Markov theorem, distinct errors were uncorrelated. When they are not, least squares still lands on an unbiased estimate of the slope, but its reported standard error is wrong. For the positive autocorrelation you almost always see in time series, it is wrong in the dangerous direction: too small. The software prints a tight confidence interval and a tiny p-value, and you believe the trend is pinned down far more precisely than the data support.

Formula¶

The simplest and most common model for correlated errors is the first-order autoregressive model, written AR(1).

Reading the pieces:

• $\varepsilon_t$ is the regression error in month $t$, the thing we assumed was uncorrelated but is not.

• $\rho$ is the autocorrelation parameter: the correlation between an error and the one just before it. Positive $\rho$ means consecutive errors tend to share sign.

• $u_t$ is the innovation: the genuinely new, unpredictable shock in month $t$, and these are independent with constant variance.

• $|\rho| < 1$ keeps the process stable so the errors do not blow up.

In words: this month’s error is a fraction $\rho$ of last month’s error plus a fresh independent shock. Figure 6 contrasts independent errors with AR(1) errors of the same variance. The independent series has no memory; the AR(1) series wanders in runs, exactly like the residuals in Figure 5.

Derivation¶

Why does positive autocorrelation make the reported standard error too small? The cleanest place to see the mechanism is the simplest estimator, the sample mean, where the algebra is short and the lesson transfers directly to the regression slope.

Proof. Suppose $Y_t = \mu + \varepsilon_t$ for $t = 1, \dots, n$, where each error has mean zero and variance $\sigma^2$, and neighboring errors have correlation $\operatorname{Corr}\{\varepsilon_t, \varepsilon_{t+k}\} = \rho^{\,k}$, the AR(1) pattern. The estimator of $\mu$ is $\bar Y$. Its true variance is

The first sum is $n\sigma^2$. Every covariance is $\operatorname{Cov}\{\varepsilon_s,\varepsilon_t\} = \sigma^2 \rho^{\,|t-s|}$, so collecting terms by their lag $k = t - s$ gives

In words: the true variance of the mean is the familiar $\sigma^2/n$ multiplied by a bracket that depends on the autocorrelation. When the errors are uncorrelated ($\rho = 0$) the bracket is 1 and we recover $\sigma^2/n$. When $\rho > 0$ every term in the sum is positive, so the bracket exceeds 1 and the true variance is larger than $\sigma^2/n$. But $\sigma^2/n$ is exactly what the ordinary formula estimates, because it ignores the covariances. So ordinary least squares divides by too little and reports a variance, and a standard error, that are too small. The same cancellation of covariances inflates the true variance of the regression slope $b_1$; only the bookkeeping is longer. $\blacksquare$

How badly does this bite in practice? A simulation makes it concrete, and it is worth running yourself, in the spirit of the “check a formula by simulating it” habit.

The simulation below puts that whole argument under your finger: it holds the data and the shocks fixed and lets you turn the autocorrelation up, so you can watch the reported standard error stay put while the true one runs away from it.

15.3 Detecting autocorrelation: the Durbin-Watson test¶

Intuition¶

The residual plot warns you; a test quantifies the warning. Two pictures point the same way. A lag plot (Definition 15.5) graphs each residual $e_t$ against the previous residual $e_{t-1}$; positive autocorrelation shows up as an upward tilt, because a positive residual tends to sit next to another positive one. Figure 9 is that plot for the airline model, and the tilt is unmistakable.

The Durbin-Watson test turns that tilt into a single number and a p-value. You met this test as one item in the diagnostic toolkit of 9. Model diagnostics, where it flagged ordering in savings residuals; here is the time-ordered data it was built for, and this is where we finally derive why its scale runs the way it does.

Formula¶

The Durbin-Watson statistic measures how much consecutive residuals differ.

• $e_t$ is the residual in month $t$ from the fitted regression.

• The numerator adds up squared changes between neighboring residuals; the denominator is the usual residual sum of squares.

The statistic runs from 0 to 4. When neighboring residuals are nearly equal (strong positive autocorrelation) the numerator is small and $D$ is near 0. When residuals alternate sign (negative autocorrelation) the changes are large and $D$ is near 4. When residuals are uncorrelated, $D$ sits near 2. Figure 10 is the number line to keep in mind.

Derivation¶

The scale is not arbitrary. Chapter 9 stated the shortcut $D \approx 2(1 - \hat\rho)$ in its summary and used it to read the savings statistic; now that autocorrelation is the whole story, it is worth seeing where that identity comes from. The Durbin-Watson statistic is, up to a small-sample correction, just a rescaling of the lag-one residual autocorrelation, which explains why 2 is the neutral value.

Proof. Expand the square in the numerator:

For a reasonably long series the first two sums are each very close to the full residual sum of squares $\sum_{t=1}^n e_t^2$, because they each drop only one term (the last or the first). Write $r_1 = \big(\sum_{t=2}^n e_t e_{t-1}\big)\big/\big(\sum_{t=1}^n e_t^2\big)$ for the lag-one residual autocorrelation. Dividing the expansion by $\sum e_t^2$,

In words: the Durbin-Watson statistic is about 2 minus twice the lag-one autocorrelation. If $r_1 = 0$ then $D \approx 2$; if $r_1 \to 1$ then $D \to 0$; if $r_1 \to -1$ then $D \to 4$. This is why a value near 2 signals no autocorrelation. $\blacksquare$

Deciding whether an observed $D$ is far enough from 2 to be surprising is slightly awkward. The null distribution of $D$ depends on the predictor values, so Durbin and Watson published a pair of bounds, $d_L$ and $d_U$, that bracket the true critical value. Software sidesteps the tables by computing the p-value directly. Both appear below.

R and Python¶

A p-value below $2\times 10^{-16}$ is easy to read past. The simulation below makes it concrete by building the comparison the test is making: hundreds of series whose errors really are uncorrelated, so you can see for yourself how far 0.43 sits from anything chance produces.

15.4 A remedy: the Cochrane-Orcutt procedure¶

Intuition¶

If the errors follow an AR(1) pattern, we can transform them away. The trick is quasi-differencing: instead of the raw series, regress each observation minus $\rho$ times the previous observation. That combination cancels the correlated part of the error and leaves the fresh, independent innovations behind. Because the transformed model has uncorrelated errors, ordinary least squares on it is valid again. The one snag is that $\rho$ is unknown, so we estimate it from the residuals and iterate. That loop is the Cochrane-Orcutt procedure.

Formula¶

Start from the regression with AR(1) errors, written compactly with $\mathbf{x}_t$ for the row of predictors (the 1, the trend $t$, and the month indicators) in month $t$:

Lag the whole regression one step, multiply by $\rho$, and subtract:

• $Y_t^{\ast} = Y_t - \rho Y_{t-1}$ is the quasi-differenced response.

• $\mathbf{x}_t^{\ast} = \mathbf{x}_t - \rho\,\mathbf{x}_{t-1}$ is the quasi-differenced predictor row.

• $u_t$ is the innovation, and by the AR(1) definition it is independent across $t$ with constant variance.

The transformed model $Y_t^{\ast} = \mathbf{x}_t^{\ast\prime}\boldsymbol\beta + u_t$ has exactly the same coefficients $\boldsymbol\beta$ as the original, but now satisfies the ordinary least-squares assumptions, so fitting it gives trustworthy standard errors.

Derivation¶

Proof. The AR(1) definition says the error is its own lag scaled by $\rho$ plus an independent shock, $\varepsilon_t = \rho\varepsilon_{t-1} + u_t$, which rearranges to $u_t = \varepsilon_t - \rho\varepsilon_{t-1}$. Take the original regression $Y_t = \mathbf{x}_t'\boldsymbol\beta + \varepsilon_t$ and the same relation one step back, $Y_{t-1} = \mathbf{x}_{t-1}'\boldsymbol\beta + \varepsilon_{t-1}$. Multiply the lagged equation by $\rho$ and subtract it from the current one. On the left, $Y_t - \rho Y_{t-1}$; on the right, $(\mathbf{x}_t - \rho\mathbf{x}_{t-1})'\boldsymbol\beta$ plus $\varepsilon_t - \rho \varepsilon_{t-1}$, and that last piece is exactly $u_t$. So the quasi-differenced series obeys a regression with the same $\boldsymbol\beta$ and with the independent, constant-variance errors $u_t$. Ordinary least squares is back on solid ground, which is the whole point of the transform.

The obstacle is that $\rho$ is unknown. The Cochrane-Orcutt procedure estimates it and the coefficients together, by iterating:

1. Fit the original model by ordinary least squares and keep the residuals $e_t$.

2. Estimate $\rho$ by regressing $e_t$ on $e_{t-1}$ through the origin: $\hat\rho = \sum_{t=2}^n e_t e_{t-1} / \sum_{t=2}^n e_{t-1}^2$.

3. Quasi-difference $Y$ and every predictor using $\hat\rho$, and refit by least squares to get new $\boldsymbol\beta$.

4. Recompute residuals from the original-scale fit, re-estimate $\rho$, and repeat until $\hat\rho$ stops changing.

Each pass reduces the leftover autocorrelation; in practice a couple of iterations suffice. $\blacksquare$

Figure 12 lays the four steps out as a loop. Read it top to bottom, then follow the orange arrow back up whenever the estimated $\rho$ is still moving: fit, estimate, transform, refit, check, and stop once $\hat\rho$ settles.

R and Python¶

Try it 15.4 asks you to argue that $\rho = 1$ is too much. The simulation below lets you check the argument on the real airline model by choosing the $\rho$ yourself and reading off the autocorrelation that survives.

15.5 Forecasting honestly: test on the future¶

Intuition¶

A model that fits the past beautifully can still forecast the future badly. The only honest way to judge a forecast is to hide part of the future from the model, forecast it, and compare. For time data that means an out-of-time split (Definition 15.11): train on the earlier years, test on the later ones. This is the validation mindset of Chapter 12 with one strict rule added by the clock. You may never train on data that came after your test period, because at forecast time that data did not exist. Splitting at random, the way you would for cross-sectional data, would let the model peek at the future and flatter itself.

We train the trend-plus-season model on 1949 through 1957 and forecast 1958 through 1960, three years it has never seen. Figure 15 shows the result, and it is a useful humbling.

Formula¶

Two honest scorecards summarize test-set performance. The root mean squared error and the mean absolute percentage error on the held-out months are

• $Y_t$ is the actual passenger count in a test month; $\hat Y_t$ is the model’s forecast, back-transformed from the log scale by exponentiating.

• RMSE is in passengers (thousands); MAPE is a unit-free percentage that is easy to explain.

In words: RMSE is the typical size of a forecast miss in passengers, and MAPE is the average miss as a percent of the actual count.

Because the model is fitted on the log scale, we exponentiate the log forecast to get a count forecast before scoring. We also carry the ordinary regression prediction interval from 3.5 Prediction interval for a new observation, back-transformed the same way, and check how often it actually covers the truth.

R and Python¶

Why does the prediction interval fail so badly? The interval from 3.5 Prediction interval for a new observation was derived under independent, constant-variance, normal errors, and it accounts for only two sources of uncertainty: the spread of a single new error and the uncertainty in the fitted mean. For a forecast several years out, neither piece is the real story. The errors are strongly autocorrelated, so a run of bad months compounds instead of averaging out. The trend is extrapolated far beyond the data, so any error in the estimated slope grows with the horizon. And genuine forecast uncertainty should widen the further ahead you look, while the ordinary regression interval barely widens at all. An interval that ignores all of this is bound to be too narrow, which is exactly what the $19\%$ coverage shows.

The 1957 split is one choice among many, and the simulation below lets you move it. Watching the same model score well on the months it saw and badly on the months it did not, over and over at different splits, is the fastest way to make the lesson stick.

15.6 Chapter summary¶

You can now handle regression on time-ordered data from end to end. You fit a trend-plus-season model on the log scale (Definition 15.1), reading the trend as a growth rate and each month coefficient as a seasonal offset, reusing the log transform of 10.2 The log transformation and reading its coefficients and the dummy coding of 11.1 From categories to numbers: indicator coding. You can explain why time order usually correlates the errors, and you proved, for the sample mean, that positive autocorrelation makes the ordinary variance formula too small (Theorem 15.4), so least squares reports standard errors you cannot trust. You diagnose autocorrelation three ways, the residual-versus-time plot, the lag plot, and the Durbin-Watson test, and you derived why $D \approx 2(1 - r_1)$ makes 2 the neutral value (Theorem 15.7). You apply the Cochrane-Orcutt procedure and know why quasi-differencing restores independent errors (Theorem 15.10). And you evaluate a forecast honestly with an out-of-time split, computing test error and prediction-interval coverage, and you can say plainly why calibrated forecast intervals need time-series methods this course does not cover.

Key results at a glance.

Airline figures: $R^2 = 0.983$, $\hat\rho = 0.79$, $D$ moves from 0.43 to 2.19 after one Cochrane-Orcutt step, test RMSE 63 / MAPE $14\%$, and prediction-interval coverage 0.19 (7 of 36).

Key terms. time-ordered data, trend, seasonality, trend-plus-season model, autocorrelation, first-order autoregressive (AR(1)) error model, autocorrelation parameter, innovation, Durbin-Watson statistic, Durbin-Watson test, lag plot, Cochrane-Orcutt procedure, quasi-differencing, out-of-time split, mean absolute percentage error (MAPE), prediction-interval coverage.

You should now be able to.

• Fit and interpret a trend-plus-season model on the log scale, reading the trend as a growth rate and each month coefficient as a seasonal offset.

• Explain why time order correlates the errors, and show that positive autocorrelation makes ordinary least-squares standard errors too small.

• Diagnose autocorrelation with a residual-versus-time plot, a lag plot, and the Durbin-Watson test.

• Derive $D \approx 2(1 - r_1)$ and use it to read the Durbin-Watson scale.

• Apply the Cochrane-Orcutt procedure and explain why quasi-differencing restores independent errors.

• Evaluate a forecast with an out-of-time split, computing test RMSE, MAPE, and prediction-interval coverage.

• Explain why calibrated forecast intervals need time-series methods beyond ordinary regression.

Where this fits. This chapter serves the CHECK and USE stages of the modeling workflow introduced in The modeling workflow. We ASK a forecasting question and EXPLORE the series, FIT a trend and seasonal model, then spend most of the chapter on CHECK: the uncorrelated-errors assumption that every earlier inference relied on fails on time data, so we diagnose the failure with the Durbin-Watson test and repair the inference with Cochrane-Orcutt. The USE stage, forecasting, comes with a warning the earlier chapters could make cleanly but this one cannot: the prediction interval of 3.5 Prediction interval for a new observation, honest under independent errors, is badly miscalibrated here. That is the seam where an applied regression course hands off to a time-series course: the out-of-time split and the honest-forecast discipline you built here are the entry habits any such course assumes. Chapter 16 (16. Path analysis and a look ahead) closes the book by stepping back from any single model to ask what a chain of regressions can and cannot say about cause, and by walking the whole workflow one last time.

15.7 Frequently asked questions¶

Q1. Why log the passengers instead of modeling the counts directly? Because the seasonal swings and the error spread both grow with the level of the series, which is a multiplicative pattern. Taking logs turns the multiplicative seasonal effect into an additive one, so a single set of month coefficients fits all twelve years, and it stabilizes the variance so the constant-variance assumption is closer to true. This is the same reasoning as 10.2 The log transformation and reading its coefficients, applied to a series that grows over time.

Q2. If least squares is still unbiased under autocorrelation, why care? Unbiased means the estimate is right on average, but a single analysis gives you one estimate, and you need to know how far it might be off. That is the standard error’s job, and autocorrelation makes the reported standard error wrong, usually far too small. You end up with a point estimate that is fine and a confidence interval that lies about its own reliability.

Q3. My Durbin-Watson statistic is 2.6. Is that a problem? It is on the negative- autocorrelation side, since $D > 2$ means $r_1 < 0$. From $D \approx 2(1 - r_1)$, a $D$ of 2.6 implies $r_1 \approx -0.3$. Mild negative autocorrelation is less common than positive but does occur, often from over-differencing or from a variable measured as a change. Whether it is “a problem” depends on the p-value and on how much you rely on the standard errors.

Q4. Does Cochrane-Orcutt change the trend estimate or just the standard errors? Both, a little. The quasi-differenced fit is a generalized least-squares estimate, which weights the data differently from ordinary least squares, so the coefficients shift somewhat. The larger and more important change is to the standard errors, which now account for the correlation and are the ones you should report.

Q5. Why not just use the ordinary prediction interval and admit it is approximate? Because it is not approximately right, it is badly wrong: on the airline test set it covered $19\%$ of months instead of $95\%$. An interval that far off does not communicate uncertainty, it misstates it. If you cannot produce a calibrated interval, report the out-of-time test error instead, which is honest about how the forecast actually performs.

Q6. Could I add more predictors instead of modeling the error correlation? Sometimes. Autocorrelated residuals can be a symptom of a missing predictor, an omitted trend curvature or a skipped seasonal term, and adding the right predictor can remove the pattern at its source. That is usually the better fix when you can find the missing structure. Cochrane-Orcutt is for the residual correlation that remains after you have modeled the structure you can.

Q7. Is a high $R^2$ like 0.983 evidence the forecast will be good? No. That $R^2$ measures fit to the past, and this chapter’s whole point is that in-sample fit and out-of-time forecast accuracy are different things. The same model with $R^2 = 0.983$ missed the future by $14\%$ and its interval covered one test month in five. Judge a forecast on held-out data, never on $R^2$.

15.8 Practice problems¶

1. (A) In one sentence each, name the three features of the airline series in Figure 1 and say which term in the trend-plus-season model captures each.

2. (A) Explain why the errors in a regression on monthly data are likely to be correlated, and give a concrete reason two neighboring months would share the sign of their error.

3. (A) The trend slope estimate is $b_1 = 0.0101$ on the log scale. Interpret it as a monthly percentage growth rate, and explain why exponentiating is needed.

4. (A) A Durbin-Watson statistic equals 0.43. Without a table, state what it implies about the lag-one autocorrelation and about the reliability of the ordinary standard errors.

5. (A) Explain, to someone who has not taken this course, why splitting time-series data at random for a train/test evaluation is cheating, and what the correct split is.

6. (A) The naive $95\%$ prediction interval covered $19\%$ of the test months. State plainly what “95% coverage” is supposed to mean and why $19\%$ is a failure, not just a small miss.

7. (A) Give a wrong reading of the July coefficient $\hat\gamma_7 = 0.3006$ that a careless student might state, then correct it. (Hint: the coefficient is on the log scale, relative to January.)

8. (A) Why does positive autocorrelation, rather than negative, make ordinary least squares overconfident? Point to the sign of the covariance terms in the variance of the mean.

9. (B) Starting from $D = \sum_{t=2}^n (e_t - e_{t-1})^2 / \sum_{t=1}^n e_t^2$, derive the approximation $D \approx 2(1 - r_1)$ (Theorem 15.7), stating the step where the two truncated sums are replaced by the full residual sum of squares and why it is reasonable for large $n$.

10. (B) For $Y_t = \mu + \varepsilon_t$ with $\operatorname{Var}\{\varepsilon_t\} = \sigma^2$ and $\operatorname{Cov}\{\varepsilon_s, \varepsilon_t\} = \sigma^2 \rho^{|t-s|}$, derive $\operatorname{Var}\{\bar Y\} = \frac{\sigma^2}{n}\big(1 + 2\sum_{k=1}^{n-1}(1 - k/n)\rho^k\big)$ (Theorem 15.4), and conclude it exceeds $\sigma^2/n$ when $\rho > 0$.

11. (B) Using the AR(1) error model $\varepsilon_t = \rho\varepsilon_{t-1} + u_t$, derive the quasi-differencing transform (Theorem 15.10) and show that the transformed error $\varepsilon_t - \rho\varepsilon_{t-1}$ equals the independent innovation $u_t$.

12. (B) Show that $\operatorname{Var}\{\varepsilon_t\} = \sigma_u^2/(1 - \rho^2)$ for a stationary AR(1) process, starting from $\varepsilon_t = \rho\varepsilon_{t-1} + u_t$ and using $\operatorname{Var}\{\varepsilon_t\} = \operatorname{Var}\{\varepsilon_{t-1}\}$.

13. (B) In the AR(1) model, show that the correlation between errors $k$ steps apart is $\rho^k$, so the autocorrelation decays geometrically with the lag. (Use $\operatorname{Cov}\{\varepsilon_t, \varepsilon_{t-k}\} = \rho\,\operatorname{Cov}\{\varepsilon_{t-1}, \varepsilon_{t-k}\}$.)

14. (B) The Cochrane-Orcutt transform loses the first observation, since $Y_1^{\ast}$ has no predecessor. Explain why, and describe how the Prais-Winsten variant recovers it by rescaling the first row by $\sqrt{1 - \rho^2}$ (a one-paragraph argument, no full derivation needed).

15. (B) A student claims that because $b_1$ is unbiased under autocorrelation, the confidence interval $b_1 \pm t^{\ast} s\{b_1\}$ is still valid. Identify the error in the claim and state precisely which quantity in the interval is wrong.

16. (B) Show that if the true errors are uncorrelated ($\rho = 0$), the Cochrane-Orcutt first step estimates $\hat\rho \approx 0$ and the quasi-differenced fit reduces to ordinary least squares, so the procedure does no harm when there is no autocorrelation.

17. (C) Read airpassengers.csv, build t and logpass, fit the trend-plus-season model, and reproduce the trend slope, $R^2$, and residual standard error reported in Example 15.1.

18. (C) Compute the Durbin-Watson statistic from the residuals by hand (via $\sum (e_t - e_{t-1})^2 / \sum e_t^2$) and confirm it against lmtest::dwtest in R or durbin_watson in Python. Report both.

19. (C) Make the residual-versus-time plot and the lag plot for the fitted model. Describe what each shows and how they agree about the sign of the autocorrelation.

20. (C) Estimate $\rho$ from the residuals, quasi-difference the response and design matrix, refit, and report the Durbin-Watson statistic before and after. Confirm it moves from about 0.43 toward 2.

21. (C) Fit the model without logs (passengers ~ t + month) and compare its residual-versus-time plot and Durbin-Watson statistic to the log model’s. Does dropping the log make the autocorrelation better or worse, and does the residual spread look constant?

22. (C) Split the data by time (train on 1949 to 1957, test on 1958 to 1960), fit on the training years, forecast the test years, back-transform, and reproduce the test RMSE and MAPE from Example 15.6.

23. (C) For the same split, compute the naive $95\%$ prediction-interval coverage on the test set and confirm it is far below 0.95. Then compute the in-sample coverage on the training set and comment on the difference.

24. (C) Refit the trend-plus-season model dropping the seasonal dummies (trend only, logpass ~ t). Report the drop in $R^2$ and the new Durbin-Watson statistic, and explain how the unmodeled seasonal pattern shows up in the residuals of the trend-only fit.

25. (C) Add a squared trend term (logpass ~ t + I(t^2) + month) and report whether the curvature term is needed and whether it reduces the residual autocorrelation. Interpret what a negative coefficient on $t^2$ would mean for the growth rate.

26. (C) Using the training fit, forecast December 1958, 1959, and 1960 (months 120, 132, 144), back-transform, and compare each to the actual value. Describe how the forecast error grows with the horizon.

27. (C) Repeat the out-of-time evaluation with an earlier split (train 1949 to 1955, test 1956 to 1960). Report the test MAPE and compare it to the 1958 split’s $14\%$. Explain why a shorter training window can change the forecast error.

28. (C) Run the simulation of Example 15.3 for $\rho = 0$, $\rho = 0.4$, and $\rho = 0.8$, and report the ratio of the mean reported SE to the true standard deviation of $b_1$ for each. Describe how the ratio changes as $\rho$ grows and what that means for confidence intervals.

29. (A) The ordinary regression prediction interval from 3.5 Prediction interval for a new observation covered only $19\%$ of the airline test months. Give the three reasons this interval is structurally too narrow for a forecast several years ahead, and name one family of methods that builds the error correlation into the model so its forecast interval widens with the horizon.

15.9 Exam practice¶

These five questions are written in the style of the course exams: each one hands you output or a claim and asks you to explain, in full sentences with units, what it means. A correct number with no reasoning earns little credit here. Work each one before opening the model answer.

EP 15.1 (interpret this output in context). The trend-plus-season model was fitted to all 144 months of airpassengers.csv, and the software reported the following.

Coefficients (partial):
            Estimate  Std. Error  t value  Pr(>|t|)
t          0.0100688   0.0001193   84.40    < 2e-16 ***

Durbin-Watson test:  DW = 0.4252,  p-value < 2.2e-16
alternative hypothesis: true autocorrelation is greater than 0

Interpret the trend coefficient as a yearly growth rate (use $e^{12 \times 0.0101} \approx 1.128$), and then explain, given the Durbin-Watson result, whether you would report the printed standard error of 0.00012 and the tiny p-value on the trend at face value. State the direction in which those numbers are wrong and why.

EP 15.2 (a student claims X, evaluate). A student writes: “This model has $R^2 = 0.983$, so it explains almost all the variation in the series. That means it will forecast next year’s monthly passenger counts to within about $2\%$.” Using the out-of-time evaluation from this chapter, evaluate the student’s reasoning and state what the correct expectation is.

EP 15.3 (interpret this output in context). One Cochrane-Orcutt step was run on the airline model, producing the output below.

rho_hat (lag-one autocorrelation of residuals):  0.7918
Durbin-Watson before quasi-differencing:  0.4252
Durbin-Watson after  quasi-differencing:  2.1891

Explain what this procedure did to the model, what problem the “after” statistic shows it fixed, and name one thing this step does not fix. Write in full sentences.

EP 15.4 (what would change if). Suppose you had fitted the model to the raw passenger counts instead of their logs, passengers ~ t + month. The two fits compare as follows.

                    R^2     residual SE        DW      corr(|residual|, fitted)
raw counts        0.9559     26.33 pass       0.4502            +0.137
log counts        0.9835      0.0593 (log)    0.4252            -0.041

Explain what would change, and what would not, if you used the raw-count fit. Address both the behavior of the residual spread and whether the autocorrelation problem is cured, and say why the log scale is preferred here.

EP 15.5 (explain why). On the 1958 to 1960 test set, the model’s nominal $95\%$ prediction interval covered only 7 of the 36 test months, an actual coverage of $19\%$.

nominal coverage:  0.95
actual coverage:   0.194   (7 of 36 test months inside the 95% interval)

State what “$95\%$ coverage” is supposed to mean, explain why $19\%$ is a structural failure rather than a small miss, give the three reasons the ordinary regression interval is too narrow for a forecast several years ahead, and name one family of methods built to fix it.

Chapter game¶