Intrinsic
This method of evaluation looks at the model's output distribution directly. This method doesn't care about the downstream tasks.
Extrinsic
This method measures a model by how well it does on a real downstream task where we can plug it into an actual application and see how it does.
Which metric is which
- intrinsic: Perplexity (PPL), because it looks only at the probability the model assigns to text
- extrinsic: word error rate for transcription, and BLEU for translation, because each one measures a real task
Note
- a lower perplexity does not prove a model is better for a specific downstream task
- the evaluation that measures how well a model captures probability itself is intrinsic
- the evaluation that measures usefulness in a downstream task is extrinsic