Word Error Rate (WER)

Word error rate measures how well a transcript matches the reference. It is the standard extrinsic metric for speech recognition.

Formula

$$WER=\frac{I+D+S}{N}$$

• $I$ is insertions, $D$ is deletions and $S$ is substitutions
• $N$ is the number of words in the reference transcript, and never the number of predicted words

Where it sits

• word error rate is extrinsic, because it measures usefulness on a real downstream task
• perplexity is intrinsic, so it says which model assigns higher probability to unseen text and not which model is better for a task
• a language model can be compared by perplexity, and the claim that one is better for transcription needs word error rate

The intrinsic counterpart is Perplexity (PPL).