Appendix D: Python Reference

This appendix collects, in one place, every piece of Python you need for MATH 4210. Use it to look up a function you saw in a chapter but forgot the syntax for, to set up Python on your own machine, or to review the whole toolkit before an exam. It is a reference, not a tutorial: for the reasoning behind any function, go back to the chapter where it first appears (each table entry below names that chapter).

D.1 Setting up Python¶

The book uses Python 3.12 or later, with three packages: pandas for data, statsmodels for model fitting, and matplotlib for plots. Install them once, in a terminal:

pip install pandas statsmodels matplotlib scipy

scipy comes along because statsmodels leans on it for distributions (t, F, chi-square) and because a few chapters call scipy.stats directly. Chapter 12 makes one brief appearance by sklearn for cross-validated shrinkage; install it only if you reach that chapter:

pip install scikit-learn

Every chapter’s Python code starts with the same three imports. Learn this block once and you will type it at the top of every script for the rest of the course:

import pandas as pd
import numpy as np
import statsmodels.formula.api as smf

pd reads and holds your data as a DataFrame (a table with named columns). np gives you array math: square roots, sums, matrix algebra. smf is the formula API, the part of statsmodels that lets you write a model the same way you say it out loud, sales ~ lotsize, rather than building matrices by hand. Chapters that plot also import matplotlib:

import matplotlib.pyplot as plt

All code in this book reads data with a relative path, data/toluca.csv, assuming you run Python from the book’s root folder. If you run a script from inside a chapter folder instead, adjust the path or set your working directory first.

D.2 The core workflow¶

Almost every example in this book follows the same four-step pattern. Learn this pattern and you can read (and write) any statsmodels analysis in the course.

df = pd.read_csv("data/toluca.csv")            # 1. load the data
fit = smf.ols("hours ~ lotsize", data=df).fit()  # 2. state the model, fit it
print(fit.summary())                             # 3. inspect the fit
pred = fit.predict(pd.DataFrame({"lotsize": [70]}))  # 4. use the fit

In words: you load a table, describe the model as a formula string (response on the left of ~, predictors on the right), call .fit() to estimate the coefficients, and then pull whatever you need off the fitted object, a summary table, individual numbers, residuals, or predictions at new x-values. Every table in Section D.4 is a variation on step 3 or step 4: some attribute or method you call on fit once you have it.

A fit object is not just numbers; it is a live Python object with named pieces. The two you will reach for constantly:

• fit.params : a pandas Series of the estimated coefficients, indexed by name (Intercept, lotsize, ...).

• fit.summary() : a formatted report with coefficients, standard errors, t statistics, p-values, R-squared, and the F test, all at once. Print it whenever you want the full picture; pull individual pieces (fit.rsquared, fit.pvalues) when you need one number in code.

Formulas support the same shorthand throughout the book: y ~ x1 + x2 for multiple regression, y ~ x1 * x2 for two predictors plus their interaction, y ~ x1 + I(x1**2) for a squared term (the I() wrapper tells the formula parser “compute this, do not treat ** as formula syntax”), and y ~ C(group) to force a numeric-looking column to be treated as categorical.

D.3 Reading the tables below¶

Each chapter section lists the Python functions and methods that chapter uses, in the order they first appear, with a one-line description and a minimal runnable example. Examples use the book’s own datasets so you can paste them into a Python session and see real output. They are deliberately short: for the full worked analysis, see the chapter’s numbered examples and its code/chNN_examples.py file, which contains every code block from the chapter verbatim.

Some entries are pandas methods (Series.mean), some are numpy functions (numpy.sqrt), some are scipy distribution functions (scipy.stats.t.ppf), and some are methods on a fitted statsmodels object (fit.resid). The dotted path tells you where to find it: pandas.read_csv lives on the pd module, fit.resid lives on whatever object you got back from .fit().

D.4 Functions by chapter¶

Chapter 1: Regression as a way of thinking¶

Data: toluca.csv, salaries.csv, orings.csv, galton_heights.csv, anscombe.csv

Chapter 2: Simple linear regression¶

Data: toluca.csv, toluca_mini.csv (see 2.1 The simple linear regression model, 2.2 Least squares from first principles)

Chapter 3: Inference for simple linear regression¶

Data: toluca.csv, advertising.csv (see 3.6 The analysis of variance approach, 3.7 The F test and its equivalence to the t test, 3.5 Prediction interval for a new observation)

Chapter 4: Correlation¶

Data: savings.csv, anscombe.csv, galton_heights.csv (see 4.2 Correlation and the regression slope)

Chapter 5: Randomization and bootstrap inference for regression¶

Data: punting.csv, toluca.csv (see 5.2 The permutation test for the slope, 5.4 The bootstrap for regression)

Chapter 6: Matrix algebra for regression¶

Data: dwaine.csv (see 6.2 Transpose and matrix multiplication, 6.4 The inverse and the normal equations, 6.5 Idempotent and projection matrices, 6.7 Random vectors, expectation, and covariance matrices, 6.8 The multivariate normal, in brief)

Chapter 7: The general linear model¶

Data: dwaine.csv, toluca.csv (see 7.1 The model and least squares in matrix form, 7.3 The hat matrix, 7.6 The Gauss-Markov theorem, 7.2 The geometry of least squares)

Chapter 8: Multiple regression in practice¶

Data: dwaine.csv, bodyfat3.csv, savings.csv (see 8.3 Extra sums of squares, 8.4 The general linear test, 8.5 Added-variable plots)

Chapter 9: Model diagnostics¶

Data: gala.csv, savings.csv, toluca.csv (see 9.1 Leverage values and the hat matrix points, 9.2 Four flavors of residual, 9.3 Influence: which points actually change the fit)

Chapter 10: Remedial measures and transformations¶

Data: mammals.csv, cars.csv, strongx.csv, gala.csv, savings.csv

Chapter 11: Categorical predictors and interactions¶

Data: salaries.csv, cars.csv (see 11.1 From categories to numbers: indicator coding, 11.2 One categorical and one continuous predictor, 11.4 Polynomial regression and centering)

Chapter 12: Multicollinearity, variable selection, and validation¶

Data: fat.csv (see 12.1 Multicollinearity and the variance inflation factor, 12.2 Choosing among models: selection criteria, 12.4 Validation: train, test, and cross-validate, 12.5 A look at shrinkage: ridge and lasso)

Chapter 13: Logistic regression¶

Data: orings.csv, pima.csv (see 13.1 Why a straight line fails, and what to use instead, 13.3 Reading coefficients as odds ratios, 13.2 Fitting by maximum likelihood)

Chapter 14: Poisson regression and the GLM idea¶

Data: gala.csv, ships.csv, toluca.csv (see 14.1 Why counts break the linear model, 14.6 One family: the generalized linear model)

Chapter 15: Regression with time: autocorrelation and forecasting¶

Data: airpassengers.csv (see 15.2 Why time breaks ordinary least squares, 15.5 Forecasting honestly: test on the future)

Chapter 16: Path analysis and a look ahead¶

Data: duncan.csv (see 16.1 Drawing a causal story: path diagrams)

D.5 Common errors and fixes¶

• PatsyError: model is missing required outcome variables. Your formula string misspelled a column name, or you passed data= a DataFrame that does not contain that column. Check df.columns against your formula.

• A categorical column fits as if it were numeric. Wrap it: C(column_name). Without C(), a column of small integers (like a Likert rating) is treated as a continuous predictor, not a set of groups.

• fit.predict(new_row) raises a KeyError or gives the wrong answer. The new data must be a DataFrame with the same column names as the original fit, even if it has only one row: pd.DataFrame({"lotsize": [70]}), not a bare list or Series.

• Results differ slightly from a hand calculation. statsmodels uses QR decomposition internally rather than the textbook (X’X)^-1 X’y formula; the two agree to many decimal places but not to the last bit. Use numpy.allclose, not ==, to compare them.

• np.log of a negative or zero value gives nan or a warning. Check the variable’s range before transforming; a log transform requires strictly positive values (add a small constant only when the chapter’s method calls for it, and say so in your write-up).