ModelRefs / Model Soup — AI Glossary

Model Soup — AI Glossary

Weight-averaging multiple fine-tuned checkpoints of the same base model to improve accuracy and reduce variance.

Overview

Wortsman et al. (2022) showed that averaging weights of models fine-tuned with different hyperparameters (a 'model soup') outperforms any individual soup ingredient. Uniform soup averages all; greedy soup greedily adds models that improve held-out accuracy. Low-overhead ensemble method requiring only weight averaging.

Reference details

Topic	training
Last reviewed	2026-06-24

Primary source

Wortsman et al., Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time (arXiv)

Continue your research

Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to Model Soup — AI Glossary.

Frequently asked questions

What is Model Soup?