ModelRefs / Linear Regression — Tutorial
Linear Regression — Tutorial
The simplest supervised learning algorithm — and the perfect lens for understanding every model that follows. Covers What is Linear Regression.
Overview
The simplest supervised learning algorithm — and the perfect lens for understanding every model that follows.
Level: Intermediate. Estimated reading time: 35 minutes.
What is Linear Regression?
Linear regression is the simplest supervised learning algorithm. Given a set of data points, it finds the single straight line that passes as close as possible to all of them — then uses that line to predict new values.
Think of it like this: imagine you plot every apartment in a city on a chart — square footage on the X-axis, rent price on the Y-axis. Most points won't sit on a perfect line, but there's clearly a trend: bigger apartments cost more. Linear regression draws the best possible line through that cloud of points. That line is your model.
Once you have the line, predicting the rent for a 900 sq-ft apartment is as simple as reading off where X = 900 meets the line.
The Math (plain version first)
The equation for a line is:
ŷ = mx + b
Where: • ŷ (y-hat) = predicted value • x = input feature (e.g. square footage) • m = slope — how much Y changes per unit of X • b = intercept — the Y value when X is 0
The algorithm's job is to find the values of m and b that make the line fit the data as closely as possible. It does this by minimizing the sum of squared errors — the total squared distance between each actual data point and the line's prediction at that X value.
That minimization objective is called Ordinary Least Squares (OLS), and it has a closed-form solution — meaning the computer can solve it exactly rather than guessing iteratively.
When to use Linear Regression
✓ Your target is a continuous number (price, temperature, sales volume) ✓ You suspect a roughly linear relationship between inputs and output ✓ You need an explainable, auditable model (loan approval, medical dosing) ✓ You want a fast, low-memory baseline before trying complex models
✗ The target is a category (use logistic regression or a classifier) ✗ The relationship is highly non-linear (use polynomial regression or a neural network) ✗ You have millions of features with complex interactions (use gradient-boosted trees or deep learning)
Real-World Use Cases
1. House price prediction — sq-ft, bedrooms, location → price 2. Sales forecasting — ad spend, season, promotions → revenue 3. Risk scoring — health metrics → insurance premium 4. Energy demand — temperature, day-of-week → kWh consumption 5. Stock return estimation — market index → individual stock return 6. Dosage optimization — patient weight → drug dose
These aren't toy problems — linear regression (or its regularized variants Ridge / Lasso) powers large portions of pricing engines, risk models, and demand forecasts at major companies.
Model Evaluation: How Good Is the Line?
After fitting the model you need to measure how well it performs:
• MAE (Mean Absolute Error): average absolute distance between predictions and actuals. Easy to interpret — "predictions are off by $12k on average."
• MSE (Mean Squared Error): average squared error. Penalizes large errors more heavily.
• RMSE (Root MSE): square root of MSE — same units as the target, easier to read.
• R² (R-squared): fraction of variance in Y explained by X. R²=1 is a perfect fit; R²=0 means the model is no better than guessing the mean. R²=0.85 means the model explains 85% of the variation in the data.
Rule of thumb: always report at least RMSE and R² together.
Continue your research
Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to Linear Regression — Tutorial.