2. Linear Models and Overfitting

Predictive modelling does not start with a new kind of mathematics. It starts with the linear model you already know, and a different question asked of it: not is this association real? but how close will my guess be for the next person?

Asking that question exposes a problem immediately — one that no amount of statistical training protects you from, and that this book spends the next three chapters dealing with.

In this chapter:

• Linear Models: a Brief Theory — least squares, predictions, and the two different questions a fitted model can answer.

• Linear Models in Action — fitting models in python, and meeting the first villain: overfitting.

• The First Villain: Overfitting — overfitting in other disguises, from polynomials to Google Flu Trends to Ptolemy.