Appendix F: Which Method When

Appendix F. Which method when¶

This appendix is a map. You bring a dataset and a question; it points you to the method this book teaches for that situation, and to the section where the method lives. It does not re-teach anything. Every arrow ends at an @anchor you can follow back into the chapter that explains the details, states the assumptions, and shows the R and Python code.

Read it as a companion to the workflow spine of The modeling workflow: ASK (what is the question and what are the data), EXPLORE (plot and summarize), FIT (estimate a model), CHECK (diagnostics and assumptions), USE (interpret, predict, decide, communicate). This guide mostly helps at ASK and FIT, points you to the right CHECK tools, and tells you what to do when a check fails.

Work through it in three questions, in order:

1. What kind of thing is your response variable?

2. What are you actually trying to find out?

3. Which assumptions hold, and what do you do when one breaks?

F.1 Start with the response: the master fork¶

The first fork is the type of the response variable $Y$, the thing you want to explain or predict. Almost everything else follows from it. Find your row.

In words: continuous responses are the home ground of ordinary least squares (most of this book); binary and count responses need the generalized linear model of 14.6 One family: the generalized linear model, which contains linear and logistic and Poisson regression as three members of one family; time order is not a response type so much as a warning that the independence assumption is in danger, handled in Chapter 15.

A common wrong turn: fitting ordinary linear regression to a 0/1 response because the software runs without complaint. 13.1 Why a straight line fails, and what to use instead shows why that gives fitted “probabilities” below 0 and above 1, and what to do instead.

F.2 Then ask what you want to find out¶

Once the response type has chosen a method family, your question chooses what you do inside it. The table reads within the continuous-response world first, since that is where the book spends most of its time; the last rows cross into the other families.

Two questions this book is careful about, because software will answer them even when it should not: “keep or drop these predictors” (12.3 The dangers of automatic selection warns that choosing predictors and then testing them on the same data inflates your confidence), and “does X cause Y” (16.4 What path analysis cannot claim about causation and the correlation-is-not- causation framing of 4.6 Four ways a correlation lies set the limits of what any regression can claim from observational data).

F.3 Continuous response, in detail¶

This is the largest branch, so it gets its own walk-through. Follow the questions top to bottom.

One predictor or many? With a single continuous predictor, fit the simple linear model $Y_i = \beta_0 + \beta_1 X_i + \varepsilon_i$ (2.1 The simple linear regression model), do inference on the slope with 3.2 Inference for the slope, and if you only want a direction-free measure of association use correlation (4.2 Correlation and the regression slope shows that $r$ and the slope carry the same sign and are two views of one fit). With two or more predictors, move to the matrix-form model $\mathbf{Y} = \mathbf{X}\beta + \varepsilon$ of 7.1 The model and least squares in matrix form and interpret coefficients with 8.1 Reading the coefficients of a multiple regression.

Are some predictors categorical? A group label (treatment vs. control, three regions) enters through dummy coding (11.1 From categories to numbers: indicator coding). If you suspect the effect of one predictor changes across groups, add an interaction (11.2 One categorical and one continuous predictor). If the relationship bends, use polynomial terms and center first (11.4 Polynomial regression and centering).

Do you have many predictors, possibly overlapping? Check for multicollinearity with VIF (12.1 Multicollinearity and the variance inflation factor); it does not bias the fit, but it inflates standard errors and makes single coefficients unstable. To choose a subset, use adjusted $R^2$, Mallows’ $C_p$, AIC, BIC, or PRESS (12.2 Choosing among models: selection criteria), and judge the result honestly with a train/test split or $k$-fold cross-validation (12.4 Validation: train, test, and cross-validate). When predictors are badly tangled, ridge and lasso shrinkage (12.5 A look at shrinkage: ridge and lasso) trade a little bias for much steadier estimates.

Do you need to predict, and how sure must you be? For the average response at a given $X$, use a confidence interval for the mean; for a single new observation, use the wider prediction interval (3.5 Prediction interval for a new observation). The gap between the two is one of the most-missed distinctions in the course.

F.4 When an assumption breaks: detect, then remedy¶

Ordinary least squares earns its good properties (unbiased, minimum variance among linear estimators) from the Gauss-Markov conditions of 7.6 The Gauss-Markov theorem: the mean is linear in the predictors, the errors have constant variance, and they are uncorrelated. Normality is added only for exact $t$ and $F$ inference. Chapter 9 is the toolbox for finding out which of these fails; Chapter 10 is the toolbox for fixing it. This table pairs the two.

Two habits keep you out of trouble. First, always CHECK before you USE: a slope with a tiny $p$-value is worthless if the residual plot shows the model is the wrong shape. Second, when a transformation fixes one problem it often changes the interpretation; 10.2 The log transformation and reading its coefficients shows how a log turns a slope into a percent-change statement, which is a feature, not a nuisance.

There is also the honest limit case. When the sample is small and you cannot trust the normal-theory $t$ and $F$ at all, do not pretend. The permutation test and the bootstrap (5.2 The permutation test for the slope, 5.4 The bootstrap for regression) ask for far less and give you a $p$-value or interval you can defend. 5.1 When normal-theory inference is on thin ice is the section that tells you when the classical machinery is standing on thin ice.

F.5 A worked mini-decision¶

Here is the guide used once, start to finish, on a real thread from the book: the Galapagos species data, where each island records the number of plant species it holds along with its area, elevation, and distance from other islands.

Question 1: what kind of response? The response is a count of species: 0, 1, 2, and up, never negative, never fractional. The master fork of Section F.1 sends counts to the Poisson / GLM family (14.1 Why counts break the linear model), not straight to linear regression.

But the book tried linear first, on purpose. In Chapter 9 the same data is fit with ordinary least squares, and the diagnostics of Section F.4 fire loudly: Isabela, a huge island, sits at an extreme leverage value, and the residual variance grows with the fitted count, breaking constant variance. That is the “variance grows with the mean” and “high-leverage points” rows of the remedy table.

Follow the remedy arrows. The constant-variance failure first suggests a transformation (10.3 Box-Cox: letting the data choose the power), and 10.5 When a transformation is not enough: the Galapagos counts shows that a log helps but does not truly fix a count response: islands with zero or few species strain any transformation of $Y$.

Land on the right family. The clean resolution is the one the response type pointed to all along: model the counts with Poisson regression and a log link (14.1 Why counts break the linear model), where the variance is allowed to grow with the mean by design. 14.5 The Galapagos thread, resolved closes the thread, and 14.4 Overdispersion and the quasi-Poisson fix adds the one more check you owe a count model: whether the spread is even larger than Poisson allows, in which case you widen to quasi-Poisson.

The lesson of the walk-through is the shape of the guide itself. The response type named the destination at Section F.1; the questions and the diagnostics explained why the detour through linear regression was worth taking and where it had to stop.

F.6 One-page routing table¶

When you just need the fast answer, this is the whole appendix in one view.

If two rows seem to fit, work the questions of Sections F.1 to F.3 in order. The response type wins ties: it decides the family, and the family decides the rest.