F1 Score

The F1 score is the harmonic mean of precision and recall values for a classification problem. It allows both values to be optimised at the same time, and it reports them as 1 number.

Formula

$$F1={\left(\frac{recal{l}^{-1}+precisio{n}^{-1}}{2}\right)}^{-1}=2\cdot \frac{precision\cdot recall}{precision+recall}$$

• the left form is the definition of a harmonic mean, which is the reciprocal of the average of the reciprocals
• the right form is the version used in practice, and both give the same value

Reason for Harmonic

The harmonic mean is used in place of an arithmetic mean because it gives a more balanced measure when dealing with extreme values.

Consider a model with a precision of 0.9 and a recall of 0.1:

• an arithmetic mean gives 0.5, which suggests moderate performance
• the harmonic mean gives 0.18, which indicates a significant issue in balancing between precision and recall

Main Idea

The harmonic mean is pulled toward the smaller of the 2 values, so a model cannot score well by being strong on 1 measure alone.

Code

from sklearn.metrics import f1_score
 
micro = f1_score(test_sent, y_pred, average='micro')
macro = f1_score(test_sent, y_pred, average='macro')
print('F1 Micro: ' + str(micro))
print('F1 Macro: ' + str(macro))

• f1_score takes the true labels first and the predictions second
• the average argument chooses how the per class scores are combined, which is covered in Micro and Macro Metrics

The precision and recall values that feed this formula come from the counts in a Confusion Matrix.