Fundamentals of Machine Learning¶
What is machine learning?¶
Tom Mitchell's widely quoted definition (1997) reads: a computer program is said to learn from experience E with respect to some class of tasks T and performance measure P if its performance at tasks in T, as measured by P, improves with experience E. In most practical cases, the experience is a data set and the program learns a function \(\hat f\) that maps inputs \(\mathbf{x}\) to outputs \(y\).
Types of learning¶
- Supervised learning: each training example comes with the correct output (a label). If \(y\) is numerical the task is regression; if \(y\) is categorical it is classification. See Supervised Learning.
- Unsupervised learning: there are no labels; the aim is to find structure such as clusters or low-dimensional representations. See Unsupervised Learning.
- Reinforcement learning: an agent learns a strategy by interacting with an environment and receiving rewards. It is used, among other things, in RLHF.
- Self-supervised learning: the labels are derived from the data themselves, for example by predicting the next word of a text. It is the basis of modern language models.
Loss functions¶
Supervised learning is usually framed as minimizing the average loss over the training data, a principle called empirical risk minimization:
where \(\mathcal{F}\) is the class of candidate models. Common choices are the squared error \(L(y, \hat y) = (y - \hat y)^2\) for regression and the cross-entropy for classification. These correspond to the estimation criteria of linear regression and logistic regression.
Training, validation and test data¶
Error measured on the training data is optimistic, because the model has been fitted to exactly those data. The available data are therefore split:
- the training set is used to fit the model,
- the validation set is used to choose hyperparameters and compare candidate models,
- the test set is used once, at the very end, to estimate performance on new data.
When data are scarce, cross-validation replaces a single validation set.
Data leakage
Leakage occurs when information that would not be available at prediction time influences training. Typical examples are standardizing the full data set before splitting it, or using features that are recorded after the outcome. Leakage produces performance estimates that are far too optimistic. All preprocessing steps should be fitted on the training data only.
Overfitting and underfitting¶
- Underfitting: the model is too simple to capture the structure in the data; both training and test errors are high.
- Overfitting: the model is so flexible that it also fits the noise; the training error is low but the test error is high.
As model complexity increases, training error typically keeps decreasing, while test error first decreases and then increases again. The goal is a model near the minimum of the test error.
The bias-variance trade-off¶
For squared-error loss, suppose \(y = f(\mathbf{x}) + \varepsilon\) with \(E(\varepsilon) = 0\) and \(\operatorname{Var}(\varepsilon) = \sigma^2\). The expected prediction error at a point \(\mathbf{x}\), averaged over training sets and noise, decomposes as
Simple models tend to have high bias and low variance; flexible models have low bias and high variance. This is the predictive counterpart of the mean squared error decomposition of an estimator.
Double descent
Very large, overparameterized models such as modern neural networks do not always follow the classical U-shaped test error curve. Belkin et al. (2019) described a "double descent", in which test error decreases again once the model is large enough to fit the training data exactly. The classical trade-off remains a useful guide, but it is not the complete picture.