A precision-recall curve shows how precision and recall change as the decision threshold of a classifier moves. It is the better choice than the ROC curve when the positive class is rare. It complements Area Under Curve (AUC-ROC).
Building The Curve
- the model gives a score for each sample
- for every threshold, label samples above it as positive
- compute precision and recall at that threshold, and plot recall on the axis and precision on the axis
- lowering the threshold usually raises recall and lowers precision
Why PR Beats ROC On Rare Positives
- the ROC curve uses the false positive rate,
- when negatives are many, is huge, so stays small even when false positives outnumber true positives
- precision has no in it, so it shows directly how many alarms are real
- example: finding a rare signal in hours of background noise, where almost every window is negative
Summary Numbers
- average precision (AP), close to the area under the PR curve
- a random classifier's PR curve is a flat line at the positive rate, not at 0.5 as with ROC
Choosing A Threshold
- fix the lowest precision (or the highest false alarm rate) that users accept, then choose the threshold with the highest recall
- or maximise when both errors cost the same
- the default threshold of 0.5 is rarely the best one, especially with class imbalance or label smoothing