Logistic Regression — Tutorial

Not regression — it is classification

Despite its name, logistic regression is a classification algorithm. It models the probability that an input belongs to class 1: P(y=1|x) = σ(wᵀx + b), where σ is the sigmoid function that squashes any real number into (0, 1).

The decision boundary is where P=0.5 — a hyperplane in feature space. Points on one side are predicted class 1, points on the other are class 0. Logistic regression is the simplest member of the generalised linear model family and a strong baseline for binary classification.

The sigmoid function and log-odds

Sigmoid: σ(z) = 1 / (1 + e^−z). At z=0, σ=0.5. As z→∞, σ→1; as z→−∞, σ→0.

The model learns the log-odds: log(P/(1-P)) = wᵀx + b. This is why it's "regression" — the model is regressing on the log-odds. Training minimises binary cross-entropy loss: L = −[y·log(ŷ) + (1−y)·log(1−ŷ)].

Coefficients are interpretable: a positive weight for a feature means that feature increases the log-odds of class 1. Exponentiate coefficients to get odds ratios.

Regularisation prevents overfitting

sklearn's LogisticRegression uses L2 regularisation by default (penalty='l2', C=1.0). C is the inverse of regularisation strength — small C = strong regularisation = simpler model.

Use L1 (penalty='l1', solver='liblinear') for sparse feature selection: L1 drives some coefficients exactly to zero, effectively selecting features. Use ElasticNet ('elasticnet') for a mix of both.

Continue your research

Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to Logistic Regression — Tutorial.

• Browse models
• Review providers
• Inspect benchmarks
• Explore workflows
• Learn AI concepts
• Read decision guides
• Review trust and methodology
• Start a decision path