Model Routing — AI Glossary

Reference details

Related terms

• Model Gateway (LLM Gateway)
• Fallback Chain
• Inference Cost

Example: Where the saving comes from

Suppose 80% of traffic is simple classification and 20% needs frontier reasoning. Sending everything to the expensive model costs 1.0 units. Routing the 80% to a model at one tenth the price gives 0.8 × 0.1 + 0.2 × 1.0 = 0.28 — roughly a 70% reduction, provided the router classifies correctly. The router's accuracy is the whole business case.

all-frontier : 1.00 x 1.0                = 1.00
routed       : 0.8 x 0.1  +  0.2 x 1.0   = 0.28

Commonly confused with

Routing is not the same as fallback. A router chooses a model up front based on the request; a fallback chain reacts after a failure. Systems often need both, and conflating them leads to fallbacks that quietly become the primary path.

When to use it

Reach for it when:

• Traffic is genuinely mixed in difficulty and volume is high enough to matter
• You can evaluate whether the cheap model was adequate for the routed requests
• Cost is a real constraint rather than a theoretical one

Reach for something else when:

• Volume is low — the routing layer costs more to run and maintain than it saves
• Requests are uniform; there is nothing to route
• You cannot detect misroutes, which silently degrade quality where it is least visible

Continue your research

Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to Model Routing — AI Glossary.

• Browse models
• Review providers
• Inspect benchmarks
• Explore workflows
• Learn AI concepts
• Read decision guides
• Review trust and methodology
• Start a decision path