Distillation
Distillation is the training of a small model to reproduce the outputs of a larger one, so that the smaller student performs close to the teacher across a particular range of work while costing a fraction as much to serve.
The method is unglamorous: run a large model over a large body of inputs, keep what it produced, and train a small model on those pairs. The student learns the teacher's behaviour on that distribution — not its general capability, which is why a distilled model can look near-identical on the tasks it was distilled for and noticeably weaker just outside them.
It is most of the reason small open-weight models punch so far above their parameter count, and it explains a pattern buyers see constantly: the gap between a flagship and a small model is narrow on ordinary work and wide on unusual work. If your evaluation only contains ordinary work, it will recommend the small model correctly and for the wrong reason.
Whether you may train on a model's outputs is a licensing question rather than a technical one, and closed-model terms commonly forbid exactly this. Every model in this catalogue carries its licence on its own page, because that is the clause that decides it.
It is a different operation from its two neighbours. Quantization compresses the same model to lower precision; fine-tuning adapts an existing model to your data; distillation trains a genuinely different, smaller model to imitate a larger one.
What it costs you
The saving from distillation is not on the training run — it is on every request afterwards, because a task moved from a large model to a small one is repriced at the small model's rate for as long as the task runs. That is why the steepest price falls in open weights have come from small models behaving like large ones rather than from large models getting cheaper, and it is why the cheapest correct answer to "which model" is usually the smallest one that still passes your own evaluation rather than the one at the top of a leaderboard.
Related terms
- Fine-tuningFine-tuning is further training of an already-trained model on your own examples so that it adapts to a task, a format or a voice, producing a new set of weights that then has to be served somewhere.
- QuantizationQuantization is the compression of a model's weights from higher-precision numbers to lower-precision ones — typically 16-bit down to 8-bit or 4-bit — which reduces the memory and compute needed to serve it, at some cost to output quality.
- Mixture of experts (MoE)A mixture-of-experts model divides its parameters into many specialised sub-networks and routes each token through only a few of them, so the parameters actually used per token — the active parameters — are a small fraction of the model's total size.
Go deeper
Every rate quoted above is published in full on the pricing page, alongside the model publisher’s own official rate.