Answer by augustin4 for Why is accuracy not the best measure for assessing...
One of the problems of accuracy is that it ignores the intrinsic difficulty in the data-generating mechanism. Here, the difficulty refers to the uncertainty of the label, which can be measured by its...
View ArticleClik here to view.
Answer by Dikran Marsupial for Why is accuracy not the best measure for...
Here is a somewhat adversarial counter-example, where accuracy is better than a proper scoring rule, based on @Benoit_Sanchez's neat thought experiment,You own an egg shop and each egg you sell...
View ArticleAnswer by vonjd for Why is accuracy not the best measure for assessing...
I wrote a whole blog post on the matter:https://blog.ephorie.de/zeror-the-simplest-possible-classifier-or-why-high-accuracy-can-be-misleadingZeroR, the simplest possible classifier, just takes the...
View ArticleAnswer by brethvoice for Why is accuracy not the best measure for assessing...
After reading through all the answers above, here is an appeal to common sense. Optimality is a flexible term and always needs to be qualified; in other words, saying a model or algorithm is "optimal"...
View ArticleAnswer by jeza for Why is accuracy not the best measure for assessing...
Classification accuracy is the number of correct predictions divided by the total number of predictions.Accuracy can be misleading. For example, in a problem where there is a large class imbalance, a...
View ArticleAnswer by Benoit Sanchez for Why is accuracy not the best measure for...
DaL answer is just exactly this. I'll illustrate it with a very simple example about... selling eggs.You own an egg shop and each egg you sell generates a net revenue of $2$ dollars. Each customer who...
View ArticleAnswer by Mayou36 for Why is accuracy not the best measure for assessing...
Imbalanced classes in your datasetTo be short: imagine, 99% of one class (say apples) and 1% of another class is in your data set (say bananas). My super duper algorithm gets an astonishing 99%...
View ArticleAnswer by mroman for Why is accuracy not the best measure for assessing...
The problem with accuracyStandard accuracy is defined as the ratio of correct classifications to the number of classifications done.\begin{align*}accuracy := \frac{\text{correct...
View ArticleAnswer by James for Why is accuracy not the best measure for assessing...
You may view accuracy as the $R^2$ of classification: an initially appealing metric with which to compare models, that falls short under detailed examination.In both cases overfitting can be a major...
View ArticleAnswer by Stephan Kolassa for Why is accuracy not the best measure for...
Most of the other answers focus on the example of unbalanced classes. Yes, this is important. However, I argue that accuracy is problematic even with balanced classes.Frank Harrell has written about...
View ArticleAnswer by DaL for Why is accuracy not the best measure for assessing...
When we use accuracy, we assign equal cost to false positives and false negatives. When that data set is imbalanced - say it has 99% of instances in one class and only 1 % in the other - there is a...
View ArticleWhy is accuracy not the best measure for assessing classification models?
This is a general question that was asked indirectly multiple times in here, but it lacks a single authoritative answer. It would be great to have a detailed answer to this for the reference.Accuracy,...
View Article
