Part 7: Scoring
Evaluating your machine learning algorithm, and the results may give mix signals. Your model may give you a mix of results, and this is the opportunity to perform a sanity test and select another algorithm. The different score provides different insights about your model, so they are all valuable.
- Classification Accuracy
- Logarithmic Loss
- Confusion Matrix
- Area under Curve
- F1 Score
- Mean Absolute Error
- Mean Squared Error
Most of the time, I tend to use the classification accuracy to measure the performance of my model. I know that this is not enough to truly judge the model but I have limited resources.
Accuracy = Number of correctly classified points / Total number of points
This is simple and easy to understand. The accuracy value is between 0 and 1. “Zero” implies that the model is “questionable” and “one” means that the model is much better.
I am aware that accuracy fails in case of imbalanced data. In the case of imbalanced data-set, the DUMB model can give high accuracy score.