ModelRefs / Curriculum Learning — AI Glossary

Curriculum Learning — AI Glossary

A training strategy presenting examples in increasing order of difficulty, mimicking human learning to improve convergence and generalization.

Overview

Curriculum learning (Bengio et al. 2009) orders training data by difficulty—simple examples first, hard ones later. For LLMs, curricula may progress from short to long contexts, or from common to rare vocabulary. Used in code generation models (EasyToHard curricula for algorithmic tasks) and math reasoning training.

Reference details

Topic	training
Last reviewed	2026-06-24

Example: You are probably already doing it, under another name

Pretraining data is generally shuffled, so the classic easy-to-hard ordering is not what most LLM training does. The curricula that are standard practice go unlabelled: training at a short context and then extending to a long one; supervised fine-tuning before preference optimisation; simple verifiable tasks before harder ones in reasoning training. Each is an ordered progression where the later stage depends on competence built in the earlier one. Recognising them as curricula is useful because it makes the design question explicit — what has to be true before this stage can work, and what happens if the order is reversed.

Commonly confused with

Curriculum learning orders the data; active learning chooses which examples to label next based on uncertainty; continual learning is about adding new tasks without losing old ones. All three concern what the model sees when, and they answer different questions — only the curriculum is decided entirely in advance.

When to use it

Reach for it when:

Context-length extension, where short-then-long is the established path
Reasoning training, where verifiable easy tasks establish a base for harder ones
Multi-stage post-training, where each stage assumes the previous one succeeded

Reach for something else when:

General pretraining, where shuffled data is the default for good reason
As a fix for insufficient or low-quality data — ordering bad data does not improve it
Without checking for forgetting: a late stage can erase what an early one taught

Continue your research

Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to Curriculum Learning — AI Glossary.

Frequently asked questions

What is Curriculum Learning?