Imbalanced classes in your dataset
To be short: imagine, 99% of one class (say apples) and 1% of another class is in your data set (say bananas). My super duper algorithm gets an astonishing 99% accuracy for this data set, check it out:
return "it's an apple"
He will be right 99% of the time and therefore gets a 99% accuracy. Can I sell you my algorithm?
Solution: don't use an absolute measure (accuracy) but a relative-to-each-class measure (there are a lot out there, like ROC AUC)
