Linear Regression
Predicting continuous values using a linear relationship.
For a small example, estimate the line through (1,2), (2,3), and (3,5). Compute slope and intercept, inspect residuals, and see how one point changes the least-squares fit.
Linear Regression
In practice
Temperature and occupancy predict daily consumption. Create leakage-safe time features, compare baseline and regularized models, inspect residual patterns, and report uncertainty.
Initializing Interactive Playground...
Experiment notes
Record your experiment
Before advancing, predict whether the line must rotate or shift to reduce the largest residuals.
Implementation
Repair a failing Linear Regression starter
The starter deliberately follows a tempting but flawed assumption: Interpreting association as causation
- Run the starter once and inspect the first wrong value, failed check, or incomplete branch.
- Repair the smallest relevant part of the code. Do not rewrite the whole solution.
- Check the solution, then explain why this correction works: State the estimand and causal assumptions separately from prediction.
Check: The browser checks pass and you can name the failure mechanism before opening the explanation.
Failing starter lab
Fit a line with gradient descent
Fit y = wx + b to a tiny noiseless dataset and verify that the learned line recovers its pattern.
Run the incomplete starter before opening hints. Use the first wrong output or failed check to form a repair hypothesis.
Knowledge check
10 questions
Explanations after each answer · 80% to pass · Linear Regression
Chapter 1 of 7 · Core
Formal Problem Statement & Data Generating Process
In the mathematical modeling of relationships between variables, the core objective of regression analysis is to characterize the conditional expectation of a scalar response variable given a vector of predictors . We assume that the observed data arises from a true, underlying data-generating process (DGP): where is the true population regression function, representing the systematic component of , and is an unobserved, random error term representing irreducible noise. The error term satisfies the strict exogeneity condition: which implies that the noise has zero mean and is completely uncorrelated with the predictors.
In the linear regression framework, we restrict the functional form of to the parametric family of affine functions. Thus, we model the conditional expectation as: where is the vector of population parameters (including the intercept), and the predictor vector is augmented with a constant as its first element, i.e., .
For a sample of independent and identically distributed (i.i.d.) observations, we stack the scalar equations to write the system in compact matrix notation: where is the vector of responses, is the design matrix, is the parameter vector, and is the vector of disturbances. We assume the standard homoscedasticity and spherical error structure: where is the constant noise variance, and is the identity matrix. This spherical error structure assumes that the disturbances are uncorrelated across observations: for all .
Key equations
Video explanation
Linear Regression, Clearly Explained!
StatQuest
See how a line is fitted, what the residuals mean, and how to interpret the resulting model.
References
Primary sources and free technical reading for Linear Regression.
An Introduction to Statistical Learning
Free R and Python editions with worked labs for regression, classification, trees, support vectors, and clustering.
BookFree
scikit-learn Linear Models
Official API concepts and examples for regression, regularization, and generalized linear models.
DocumentationFree