BIBA - K Nearest neighbours - Judging Classifier Performance I.e., categorical variables
8 important questions on BIBA - K Nearest neighbours - Judging Classifier Performance I.e., categorical variables
What is misclassification and how can misclassification happen?
- Misclassification is when a record belongs to one class but the model classifies it as a member of a different class
- But the real world has “noise” and not all the information needed to classify records precisely
Give an example of a confusion/classification matrix and explain the variables in the matrix.
- Correct classifications:
- True Positive and True Negative
- Incorrect classifications:
- False Positive, i.e., outcome incorrectly predicted as yes /positive
- False Negative: i.e., outcome incorrectly predicted as no / negative
Give an example where there is a limitation of accuracy?
- Consider a 2-class problem
- Number of Class 0 examples = 9990
- Number of Class 1 examples = 10
- If model predicts everything to be class 0 (due to that it is a bad model), accuracy is 9990/10000 = 99.9 % even though the model isn't very flexible if the number of class 1 rises and therefore isn't really accurate.
- Accuracy is misleading because model does not detect any class 1 example
- Higher grades + faster learning
- Never study anything twice
- 100% sure, 100% understanding
Explain how cost and accuracy are proportional.
Explain how multiclass prediction works and what kind of confusion matrice it has.
- Two confusion matrices for a 3-class problem: actual predictor(left) vs. random predictor (right)
- Number of successes: sum of entries in diagonal (D)
What is a kappa statistic and what is the formula?
- Kappa statistic measures relative improvement on random predictor: 1 means perfect accuracy, 0 means we are doing no better than random
- Kappa statistic: (success rate of actual predictor - success rate of random predictor) / (1 - success rate of random predictor)
What is the formula of precision and recall in a table?
What is the formula of precision and recall in a matrix?
The question on the page originate from the summary of the following study material:
- A unique study and practice tool
- Never study anything twice again
- Get the grades you hope for
- 100% sure, 100% understanding