Fine-tuning is the last resort and that's a compliment

Once a quarter someone asks me to help fine-tune a model, and once a quarter I ask what the prompt looks like, and once a quarter the answer is two sentences long with a typo. Fine-tuning has main character energy. It is the option that sounds like real engineering, so it gets requested before the boring options that would have worked by lunch.

the order of operations

Knowledge problems are retrieval problems: if the model doesn't know your refund policy, put the policy in the context, don't burn it into weights that you now have to retrain when legal edits a paragraph (retrieval rant here). Behavior problems start as prompt problems: format, tone, procedure, all cheaper to iterate in text where a change ships in minutes.

Fine-tuning earns its slot when three things are all true. The behavior is stable (you will want it next quarter too). Prompting demonstrably plateaued, measured by an eval, not by fatigue. And you own hundreds to thousands of quality examples, per OpenAI's own guidance, because fifty examples buys you a model that memorized fifty examples.

what fine-tuning is actually great at

Style and format consistency at scale, tool-use conventions on smaller models, classification with lots of labeled history, and cost compression: teaching a cheap model behavior you currently rent from an expensive one, then taking the small-model discount on volume.

the middle rungs everyone skips

Between "edit the prompt" and "train the weights" sit four rungs, and the quarterly fine-tune request has usually skipped all of them.

Few-shot examples in the prompt, selected per request instead of pasted statically, close most format gaps: my last "we need fine-tuning for our output style" ticket died to eight examples and a retrieval step that picks the three most similar. Structured outputs kill another class: if the problem is malformed JSON, schema enforcement fixes it at the API layer for free. Prompt caching rewrites the economics of long prompts, so "our prompt is expensive" stopped being a training argument. And distillation-by-logging, where you run the expensive model for a month, keep the good traces, and only then ask whether those traces justify training the cheap one, turns the dataset problem into a byproduct of shipping.

Run the rungs in order and the quarterly conversation ends early, usually at rung one, occasionally at rung four with a dataset that built itself.

the eval gate, non-negotiable

One rule has saved me from every bad fine-tune I did not do: no training run without an eval that the prompted baseline already scored on. Not vibes, a number (evals first, always). The gate exists because fine-tuning always produces a model that feels better to the person who requested it, and the feeling survives contact with nothing. Half the time the eval shows the baseline was already at target and the request was aesthetic. The other half, you now have the dataset and the yardstick, which is exactly what the training run needed anyway. The gate costs a day. The ungated version costs a quarter and ends in a model nobody can explain that everyone is afraid to turn off.

the one fine-tune i defend without hedging

So it does not sound like I think training is always the wrong answer: the classification fine-tune remains excellent. Thousands of labeled historical tickets, a stable taxonomy, a small model, and a task where every basis point of accuracy has a dollar sign attached. The dataset already exists because the business created it by operating. The behavior is stable because the taxonomy changes yearly, not weekly. And the volume justifies owning the marginal cost curve instead of renting it. Notice how little that scenario resembles "the chatbot should sound more like us", which is the request the quarter usually actually brings.

The same shape shows up in tool-calling for small local models, in entity extraction against a fixed schema, and in safety filters tuned to a product's specific definition of trouble. Common thread: narrow task, existing labels, a metric someone already watches, and no ambition for the weights to be interesting. Training is at its best when it is boring.

the hidden invoice

A fine-tune is a snapshot with a maintenance contract. Base model deprecates, you retrain. Requirements shift, you rebuild the dataset. Meanwhile the prompt-based version of your competitor picked up every base-model improvement for free.

The invoice has a personnel line too. The person who ran the training becomes the only person who understands the model's quirks, and when they change teams the artifact becomes a haunted house: load-bearing, unexplainable, renewed annually out of fear. Prompts have diffs and blame. Weights have lore. Budget for the difference before you create it.

Last resort, in the medical sense: effective, appropriate, and not the first thing you reach for.