StoryboardAI Engineering

Fine-tuning

Training a private copy of a model on your own examples, so that it behaves a certain way without being told each time.

What is fine-tuning?

A model's behaviour can be changed in three ways. Telling it, in the prompt. Showing it, by fetching the right text at the moment of asking. And training it: giving a provider thousands of examples of an input and the ideal output, and receiving a private copy of the model adjusted to match.

Training suits behaviour that is hard to describe and easy to show: a house style, a fixed format, a narrow job done millions of times. It can let a small, cheap model do one task as well as a large one.

It is a poor way to teach facts. Facts change, a trained model cannot say where it learned something, and it still makes things up. Retrieval is the tool for knowledge.

Training has real costs: collecting and cleaning examples, a fee, a test set to prove it helped, and doing it all again when the base model is replaced. Most teams reach for it last.

The order to try things. Training comes last.1. A clearer prompt: minutes. 2. A few examples in the prompt: an hour. 3. Fetch the right text: days. 4. Give it tools: days. 5. Train a model: weeks, and again each year.A clearer promptminutesA few examples in the promptan hourFetch the right textdaysGive it toolsdaysTrain a modelweeks, and again each year
The order to try things. Training comes last.

Why does fine-tuning matter?

It is the first thing many people ask for, and usually the last thing they need. Fine-tuning can make a model behave in a specific way without long instructions, and can make a small model do a narrow job as well as a large one. It is also slow, costly to maintain and the wrong tool for teaching facts. Knowing when it pays saves months.

How does fine-tuning work?

  1. Collect thousands of examples: an input and the ideal output.
  2. Check and clean them, because the model will copy their flaws.
  3. The provider trains a private copy of a model on them, adjusting its inner settings.
  4. Test the trained model against the original on cases it has not seen.
  5. Use it like any other model, and repeat the process when the base model is replaced.

When should you use fine-tuning?

Use it when

  • A style or format is easier to show than to describe
  • A narrow task runs at very high volume
  • A small, cheap model must match a large one on one job

Think twice when

  • The goal is knowledge that changes. Use retrieval
  • A clearer prompt or a few examples has not been tried yet
  • There is no test set to prove it helped

Where does fine-tuning sit in a real system?

Training happens at the provider, from examples you supply. The result is a model with its own name that your system calls. The examples and the test set are kept and maintained by you.

What goes wrong with fine-tuning?

  • Training a model on facts and expecting it to stop making things up.
  • Skipping the cheaper options: prompt, examples, retrieval, tools.
  • Forgetting that every new base model means training again.

What does fine-tuning look like in practice?

A shop sorts a million messages a month into forty kinds. A large model does it for $4,000, right 94 times in 100. A small model trained on 50,000 messages, which the large one had labelled, does it for $200 and is right 95 times in 100.

Watch it happen

These stories show fine-tuning at work, one step at a time.