Instruction tuning trains a model on many tasks phrased as natural language instructions, so that it can perform an unseen task from an instruction alone.
How it differs from the older paradigms
| Paradigm | Needs | Result |
|---|---|---|
| Pretrain then finetune, as in BERT and T5 | many task-specific examples | one specialised model per task |
| Prompting, as in GPT-3 | a prompt only | improves performance with no weight update |
| Instruction tuning, as in FLAN | many tasks with instructions | one model, evaluated on an unseen task |
- T5 casts every text processing task as text to text
- the output in instruction tuning is the response to the input, and not a completion of it
Scaling behaviour
- performance grows linearly as the number of tasks and the model size grow exponentially
- the number of instances per training task has little effect
- what matters is task count, model size and template diversity
Named collections
- FLAN, which gives 73 datasets, 146 task categories and 1,836 tasks
- Natural Instructions and Super-NaturalInstructions
- PromptSource and P3
- T5 plus instruction tuning on the FLAN collection gives FLAN-T5
- data can also be generated synthetically, as in Unnatural Instructions and Self-Instruct
Chat prompt templates
- a chat prompt is split into system, assistant and user
- templates are not shared across models, so Llama-2 Chat uses
[INST]and<<SYS>>while Alpaca uses### Instructionand### Response
This is the middle stage described in LLM Training Stages.
Note
Scale moves two things at once, and it helps models that were never instruction tuned as well.
- raising model scale by an order of magnitude, for example 8B to 62B, or 62B to 540B, improves performance substantially for both finetuned and non-finetuned models
- raising the number of finetuning tasks improves performance on its own
- instruction finetuning beats no finetuning by a large margin at every one of those scales
What tuning looks like from the outside:
- given a passage and the instruction
One-sentence Summary:, a base model continues the passage - an instruction tuned chat model of the same family returns a summary, even at a far smaller parameter count
- so the visible difference is the form of the answer, and not the size of the model