The Right Tool track gave you a framework for choosing between kinds of models. This page gives you the equivalent for adaptation methods. The short version: the escalation ladder runs from cheapest and most reversible to most expensive and most committed — and you should need evidence, not enthusiasm, to climb each rung.
Every adaptation problem should walk the same ladder, in order:
Default to LoRA or QLoRA, and escalate to full fine-tuning only when you have evidence that parameter-efficient tuning is insufficient. Not intuition, not "we want maximum quality," not a vendor's suggestion — a measured gap on your own evaluation between what an adapter achieves and what you need.
The asymmetry from the last page drives the whole decision. A LoRA experiment costs a modest GPU budget and produces a megabyte-scale, fully reversible artifact that can be merged for zero-latency inference. A full fine-tune costs 3–4× the model's size in training memory, produces a multi-gigabyte model copy you now own and maintain, and puts every capability the model has at risk of catastrophic forgetting. When one option is cheap, safe, and reversible and the other is expensive, risky, and committed, you try the cheap one first — especially since, for the most common fine-tuning goals (format, style, domain fluency, task specialization), adapters routinely match full fine-tuning in quality.
Starting with LoRA is also the better experiment. If an adapter gets you 90% of the way, you've learned your data works and can decide whether the last 10% justifies the heavy option. If you start with full fine-tuning and it disappoints, you've spent the large budget just to learn your dataset was the problem.
Escalation to full fine-tuning is justified when the situation genuinely looks like one of these:
A measured quality gap that resists the easy fixes. You trained an adapter, evaluated it, and it falls short of requirements — and increasing the adapter's capacity, improving the dataset, and tuning the training didn't close the gap. The dataset is the culprit far more often than the method; exhaust that explanation first.
A genuinely large domain shift. The target distribution is far from anything in pretraining — a low-resource language, a specialized corpus unlike web text, or non-text token domains like the network traffic and protein sequences from Beyond Text. When the model needs deep restructuring rather than a behavioral nudge, a low-rank update may be structurally too small, and full fine-tuning (or even continued pretraining) is the honest tool.
The economics actually invert. At sufficient scale, a small fully-tuned model that replaces a much larger general one can pay for its training many times over in serving costs. This is a real case — but it's an argument you make with arithmetic, after the adapter experiment, not before it.
"We want the best possible result" — untested assumption; adapters often match it. "We have the GPUs" — capacity isn't a reason. "The base model doesn't know our facts" — that's retrieval's job; fine-tuning is the wrong tool for knowledge injection no matter how many weights you update.
Is the problem missing knowledge? Feed the model — retrieval, not tuning. Is it behavior you can specify in instructions the model follows reliably? Prompt — you're done. Is it behavior the model won't hold from instructions alone, and you have (or can build) hundreds-to-thousands of good examples? Train a LoRA/QLoRA adapter and evaluate — both on your task and broadly enough to catch regressions. Did the adapter fall measurably short after honest iteration on data and settings, or is the domain shift genuinely large? Now full fine-tuning is on the table, with eyes open about memory, maintenance, and forgetting.
Notice what this ladder preserves from the rest of the course: the frozen model is a feature. Every rung you avoid climbing keeps more of the model's behavior stable, reproducible, and understood. Change the weights when the evidence says you must — and change as few of them as the problem allows.
Fine-tuning completes Part VI of Actually Understanding AI. The book puts the whole escalation ladder in one place — from prompt to LoRA to full fine-tune — alongside everything that led here.
Get the Book →