04.5 · Instruction tuning Same training — new data

Teach it
to answer.

The model from stage 04 only knows one move: continue the text. Ask it to do something and it doesn't obey — it keeps writing Shakespeare, because that is all it was ever shown. Instruction tuning changes nothing about the method: it is the exact same next-token training, run on a small pile of worked examples like Q: reverse honour → A: ruonoh. Pick a task below — on a word it was never tuned on — and watch the same prompt hit both models.

Held-out prompts · a word neither model saw during tuning
BASE MODELstage 04 · autocomplete only
AFTER TUNINGsame model · shown how to answer
It learns to obey
Share of held-out prompts answered exactly right, as tuning proceeds.
—→—
instruction-tuning steps →
Where the skill lands
Final held-out accuracy per task. Some transforms are easier for a tiny model than others — real numbers, not rounded up.

Nothing about the training changed — only the data did. Same next-token objective, same backprop as pretraining. Yet by simply showing the model what a good answer looks like, an autocompleter becomes something that reads your instruction and follows it — on words it never saw while tuning. This is the first, cheapest layer of "alignment": demonstration. The next two pages are about what happens when demonstration isn't enough and you start optimising against a judge.