ML : ROC Curve, F1-Score, precision, choose the best metrics

Accuracy: 90 %

Which metrics to evaluate algorithms performance?

0. Quick answer

What do I expect from my model?

What mistakes my model can make? And what mistakes are forbidden?

1. Metrics and tools

Results of a Bradley Cooper detection

I. Confusion matrix

Confusion matrix for the Bradley Cooper detection

II. Accuracy

If our algorithm hadn’t detected a Bradley Cooper, we would have had 90% success too.

III. Precision

But… how do we know if we detected all the positive classes?

IV. Recall

Aoooright! We got metrics to estimate false negative and false positive impact. But how can we generally evaluate our model?

V. F1-Score

VI. Sensitivity, Specificity, ROC Curve

What is this threshold?

y_pred = (clf.predict_proba(X_test)[:,1] >= THRESHOLD).astype(bool)
Source

How to compare models?

Source : Sensitivity, specificity, CNAM

How to choose the ideal threshold?

We will choose point B as the threshold! Source : Sensitivity, specificity, CNAM

Conclusion

400 views in a month!!

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store