Statistical Inference¶
Statistical inference is the art of drawing conclusions about a population from a sample while being honest about the uncertainty involved. Descriptive statistics summarizes the data at hand; inference asks what those data imply beyond themselves.
This part focuses on the classical, frequentist approach, in which parameters are fixed but unknown constants and probability statements refer to what would happen under repeated sampling. It starts from random variables and ends with linear and logistic regression, the two models that lead naturally into machine learning.
Contents¶
- Fundamentals: random variables, expectation and variance, common distributions, sampling and the central limit theorem.
- Estimation: point estimators and their properties, confidence intervals, maximum likelihood.
- Hypothesis Testing: null and alternative hypotheses, p-values, type I and type II errors, common tests, multiple testing.
- Linear Regression: the linear model, least squares, assumptions and interpretation.
- Logistic Regression: modeling binary outcomes, odds ratios and maximum likelihood estimation.
What about Bayesian statistics?
In the Bayesian approach, parameters are treated as random quantities. Prior beliefs, expressed as a probability distribution, are updated with the data via Bayes' theorem to obtain a posterior distribution. This wiki currently covers the frequentist approach only; a Bayesian section may follow.
Notation used in this part
\(X\) denotes a random variable and \(x\) an observed value. Greek letters (\(\mu, \sigma, \theta, \beta\)) denote population parameters, and a hat marks an estimate (\(\hat\theta\)). The letter \(\sigma\) means a standard deviation, except in \(\sigma(\cdot)\), which denotes the logistic (sigmoid) function.
What comes next¶
Logistic regression closes this part. Machine learning reuses many of the same models but shifts the goal from explaining effects to predicting new observations.