Micro and macro evaluation metrics are 2 different approaches to aggregating and reporting performance measures such as precision, recall and F1 score.
They matter once there is more than 1 class, because a per class score has to be collapsed into a single number before it can be reported.
Micro metrics
- aggregate the contributions of all classes to compute the average metric
- micro precision calculates the precision for each class individually, sums up the numerators (true positives), and divides by the sum of the denominators (true positives and false positives) across all classes
Macro metrics
- evaluate the model's performance on each class independently and then average the results
- macro precision calculates the precision for each class individually and then takes the average of these precision scores
When to use
| Approach | Use it when |
|---|---|
| Micro | overall classification performance needs to be emphasised, giving equal importance to all instances, which is useful when class imbalance is present because all instances are considered collectively |
| Macro | each class is considered equally important, and the model's ability to perform well on all classes is to be assessed |
Where the weight lands
Micro gives every instance the same weight, so a large class dominates the result. Macro gives every class the same weight, so a small class counts as much as a large one.
Code
from sklearn.metrics import f1_score
micro = f1_score(test_sent, y_pred, average='micro')
macro = f1_score(test_sent, y_pred, average='macro')
print('F1 Micro: ' + str(micro))
print('F1 Macro: ' + str(macro))- the
averageargument is the only difference between the 2 calls - on a dataset with a balanced split in classes, the micro and macro scores do not differ much
- a large gap between the 2 is a signal of class imbalance, or of 1 class being handled badly
The metric being aggregated here is defined in F1 Score.