BIBA - K Nearest neighbours - Evaluating Predictive Performance Ie., numerical (continuous) variables
8 important questions on BIBA - K Nearest neighbours - Evaluating Predictive Performance Ie., numerical (continuous) variables
How can you generate nummeric predictions?
What is the formula for calculating the prediction error for a record i?
Give the definition and formula of the mean absolute error/deviation (MAE)?
- Higher grades + faster learning
- Never study anything twice
- 100% sure, 100% understanding
Give the definition, implications and formula of the mean error?
- Same as MAE but no absolute value
- Negative errors cancel positive of same magnitude
- Indication of whether predictions are on average over-/under-predicting the outcome variable
Give the definition, implications and formula of the mean percentage error (MPE)?
- Percentage score of how predictions deviate from the actual values (on average)
- Takes into account the direction of the error
Give the definition and formula of the mean absolute percentage error (MAPE)?
- Percentage score of how predictions deviate (on average)from the actual values
Give the definition, implications and formula of the root mean squared error (RMSE)?
- Intuition: (normalized) distance between the vector of predicted values and the vector of actual value
- Same units as the outcome variable
How can you generate a lift chart?
- Take sample data set
- Apply the selected model
- Sort according to predicted probability of a yes response, i.e.,
- First instance is the one the model thinks is more likely a yes
- Next instance is the next most likely Etc.
- Intuition: more yes at the beginning of the list
- x-axis is sample percentage, i.e., 20% from the start of the list
- y-axis is response number, i.e., percentage where the model correctly predicts the positive class
The question on the page originate from the summary of the following study material:
- A unique study and practice tool
- Never study anything twice again
- Get the grades you hope for
- 100% sure, 100% understanding