The small model was the right call and everyone knew it

Nobody gets fired for calling the frontier model. That is exactly how a classification task that a small model handles fine ends up costing five times more per call, for two years, in a loop that runs forty thousand times a day. I have seen the invoice. It has feelings.

Claude Haiku 4.5 runs $1 per million input tokens and $5 per million output against much steeper rates up the model lineup, and the small tiers from other providers tell the same story. The question is never "which model is smartest". It is "which model is the cheapest one that passes my eval".

where small models just win

Classification and routing. Extraction into a schema. Summarization of things a human will skim anyway. Query rewriting for retrieval. Tool call formatting inside an agent loop where a big model already did the planning. These are the volume tasks, the ones that run per-request rather than per-decision, and volume is where unit price compounds.

the pattern is a router, not a religion

Big model plans, small model executes. One frontier call decomposes the job; a swarm of cheap calls does the repetitive middle; maybe one more expensive call reviews the result. Your pipeline probably has three tiers of task difficulty. Bill accordingly.

the part where I nag you about evals

The only honest way to downgrade a model is the eval set you already built. Run it against the small model. If it passes, take the discount and go buy the team lunch with a fraction of the savings. If it fails, you just learned exactly which cases need the big brain, and that list is usually shorter than anyone's intuition claimed.

the downgrade playbook, step by step

The migration itself deserves more respect than "swap the model string". Run shadow mode first: the small model answers every request in parallel, you log both outputs, nobody sees its answers. A week of shadow traffic tells you the disagreement rate and, more usefully, hands you every disagreement as a labeled case to read. Reading twenty disagreements takes an hour and always sorts into "small model is fine, outputs differ cosmetically" and "small model genuinely whiffs this shape". The second pile becomes routing rules, the first pile becomes confidence.

Then cut over with a fallback: low confidence, schema violation, or an explicit category match escalates to the big model. My last migration ended with 92 percent of volume on the small tier and the overflow costing less than the old logging bill. The invoice recovered. Emotionally, I mean.

Two traps for the road. Latency is not automatically better on small models under provider load, so measure p95 before promising anyone speed. And prompt formats do not transfer untouched; small models lean harder on examples than instructions, so the downgrade usually costs one rewrite of the few-shot block. Budget the afternoon.

The org-chart version of this mistake deserves its own sentence: some teams mandate the small tier everywhere to look thrifty, which is the same error with the sign flipped. The route exists so each task pays its own correct price. A planning step that saves three bad tool calls by thinking harder is worth frontier rates; a yes/no classification never was. Both halves of that sentence save money, and only the eval knows which half applies where you are standing.

Model snobbery is a real cost center. The eval doesn't care about prestige.