Probability & Statistics for ML — Tutorial

Why statistics underlies ML

Machine learning is applied statistics at scale. A loss function is a probabilistic statement about how wrong your model is. Regularisation encodes a prior belief about your parameters (Bayesian view). Cross-validation is hypothesis testing for model selection.

The three concepts you absolutely must know: probability distributions, expected value, and variance. Everything else builds on them.

Key distributions and what they model

Normal (Gaussian): continuous symmetric data, noise, measurement error. Most parameter distributions after sufficient data.

Bernoulli / Binomial: binary outcomes (spam/not-spam). Logistic regression models P(y=1|x) as a Bernoulli parameter.

Categorical / Softmax: multi-class output. The softmax function converts raw logits to a probability distribution over K classes.

Understanding which distribution governs your target tells you which loss function to use: cross-entropy for classification, MSE for Gaussian regression.

Mean, variance, and the bias-variance view

Mean (expected value) E[X]: the average outcome over many trials. Variance Var(X) = E[(X-μ)²]: how spread out the distribution is. Standard deviation = √Var — same units as the data.

In ML: bias is systematic error (your model consistently misses by a fixed amount). Variance is sensitivity to training data fluctuations. A high-bias model underfits; a high-variance model overfits. Statistics gives you the language to diagnose both.

Continue your research

Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to Probability & Statistics for ML — Tutorial.

• Browse models
• Review providers
• Inspect benchmarks
• Explore workflows
• Learn AI concepts
• Read decision guides
• Review trust and methodology
• Start a decision path