A Cost-Aware Model Router for Claude in About 80 Lines of Python

The fastest way to cut an LLM bill without hurting quality is to stop sending every request to your most capable model. Most traffic in a typical agent workflow is classification, extraction, or short replies. A smaller, faster model handles those fine. The bigger model should be reserved for the steps where it measurably changes the outcome.

This post walks through the simple router I use with Claude models. It's not a framework. It's a pattern you can adapt in an afternoon.

Every request gets a cheap first pass that decides how hard it is. Easy requests go to a fast model. Hard ones go to a stronger model. If the fast model's answer fails a check, the request is retried on the stronger one.

import anthropic from dataclasses import dataclass client = anthropic. Anthropic () ROUTES = { " fast ": { " model ": " claude-sonnet-5 ", " max_tokens ": 1024 }, " strong ": { " model ": " claude-opus-5-5 ", " max_tokens ": 4096 }, } HARD_TASKS = { " multi_step_plan ", " contract_review ", " root_cause_analysis " } @dataclass class Request: task_type: str text: str needs_tools: bool = False def pick_route ( req: Request ) -> str: if req. task_type in HARD_TASKS: return " strong " if len ( req. text ) > 20_000: return " strong " if req. needs_tools and req. task_type == " multi_step_plan ": return " strong " return " fast " def call ( route: str, system: str, user: str ): cfg = ROUTES [ route ] resp = client. messages. create ( model = cfg [ " model " ], max_tokens = cfg [ " max_tokens " ], system = system, messages = [{ " role ": " user ", " content ": user }], ) return resp, cfg [ " model " ] Enter fullscreen mode Exit fullscreen mode The rules here are deliberately boring. Task type and input size go a long way before you need anything clever.

Check the current model names and pricing in Anthropic's documentation before copying this, since both change over time.

Routing up front catches the obvious cases. Escalation catches the rest:

import json def validate ( output_text: str, schema_keys: set ) -> bool: try: data = json. loads ( output_text ) except json. JSONDecodeError: return False return schema_keys. issubset ( data. keys ()) def route_and_run ( req: Request, system: str, schema_keys: set ): route = pick_route ( req ) resp, model = call ( route, system, req. text ) text = resp. content [ 0 ]. text if route == " fast " and not validate ( text, schema_keys ): resp, model = call ( " strong ", system, req. text ) text = resp. content [ 0 ]. text log_escalation ( req. task_type, reason = " validation_failed " ) log_usage ( req. task_type, model, resp. usage ) return text Enter fullscreen mode Exit fullscreen mode Validation can be anything cheap and deterministic: valid JSON, required fields present, a confidence field above a threshold, or a citation check for retrieval answers. Avoid using another LLM call as the validator for every request, or you lose most of the savings.

A router without measurement is a guess. I log four things per request:

Escalation rate per task type. If a task escalates more than about a third of the time, it probably belongs on the strong route from the start. Paying for two calls is worse than paying for one larger one.

Quality by route. Sample outputs from both routes each week and score them against the same rubric. The goal is equal quality at lower cost, not lower cost at any quality.

Routing on the wrong signal. Text length is a weak proxy for difficulty. A short question about a complex contract can be hard, and a long but simple form can be easy. Task type is usually the better signal.

Ignoring latency. The fast model is often fast as well as cheap. For user-facing steps, that matters as much as cost.

Forgetting tool use. Multi-step tool use with dependent calls is where larger models tend to be more reliable. I keep planning steps on the strong route even when the text is short.

Hard-coding models. Models change often. Keep routes in config so switching a model is a deploy-free change, and rerun your eval set when you do.

Rules get you surprisingly far, but eventually you'll want a better signal than task type. The cheapest upgrade I've found is to let your own logs train it.

After a few weeks of routing, you have labelled data for free: requests that the fast model handled and passed validation, and requests that escalated. A small classifier on simple features can learn which new requests are likely to escalate and send them straight to the strong route.

from sklearn.linear_model import LogisticRegression def features ( req: Request ): return [ len ( req. text ), req. text. count ( "? " ), int ( req. needs_tools ), int ( req. task_type in HARD_TASKS ), count_entities ( req. text ), # IDs, amounts, dates mentioned ] X = [ features ( r ) for r in logged_requests ] y = [ r. escalated for r in logged_requests ] clf = LogisticRegression (). fit ( X, y ) def pick_route_learned ( req: Request ) -> str: p = clf. predict_proba ([ features ( req )])[ 0 ][ 1 ] return " strong " if p > 0.4 else " fast " Enter fullscreen mode Exit fullscreen mode Keep the threshold in config and tune it against your eval set. A lower threshold sends more traffic to the strong model, raising cost and lowering escalations.

The right point depends on how much you value latency and how expensive a wasted fast call is for you.

Routing reduces average cost. It doesn't stop a single runaway task. I add two simple guards:

It depends entirely on your traffic mix. If most of your requests are simple, routing makes a large difference. If most are complex reasoning, it helps less. The only reliable way to know is to log a week of real traffic by task type, then simulate the routing rules against it before turning them on.

For a deeper comparison of the two models, including where the larger one tends to justify its price, I wrote up when Sonnet is enough and when Opus earns its cost.

I work on enterprise agent deployments at AI Hive, where model routing is one of the first things we set up for cost control. How do you decide which requests get your most expensive model?