What is LLM Routing?

Not every task needs the most expensive frontier model: a simple classification or a short reformatting task can be handled just as accurately by a small, fast, cheap model, while a request that requires deep multi-step reasoning or long-context understanding benefits from a larger model even at higher cost and latency. A routing layer sits in front of the model call and makes this decision automatically — sometimes with a lightweight classifier, sometimes with simple heuristics like input length or detected task type — rather than hardcoding every request to one model. Routing also serves reliability and vendor-diversification goals beyond pure cost optimization: if a preferred model provider has an outage or hits a rate limit, a routing layer can fail over to an alternate model rather than the whole system going down. Good routing implementations track outcome quality per model per task type over time, so the routing rules can be tuned as new models are released or as a given model's performance on a task type drifts.