BIBA - K Nearest neighbours - Evaluating Predictive Performance Ie., numerical (continuous) variables

8 important questions on BIBA - K Nearest neighbours - Evaluating Predictive Performance Ie., numerical (continuous) variables

How can you generate nummeric predictions?

Models are trained on the training data applied to the validation data and measures of accuracy, then use the prediction errors on that validation set

What is the formula for calculating the prediction error for a record i?

Error = actual outcome - predicted outcome

Give the definition and formula of the mean absolute error/deviation (MAE)?

Gives the magnitude of the average absolute error
  • Higher grades + faster learning
  • Never study anything twice
  • 100% sure, 100% understanding
Discover Study Smart

Give the definition, implications and formula of the mean error?

  • Same as MAE but no absolute value
  • Negative errors cancel positive of same magnitude
  • Indication of whether predictions are on average over-/under-predicting the outcome variable

Give the definition, implications and formula of the mean percentage error (MPE)?

  • Percentage score of how predictions deviate from the actual values (on average)
  • Takes into account the direction of the error

Give the definition and formula of the mean absolute percentage error (MAPE)?

  • Percentage score of how predictions deviate (on average)from the actual values

Give the definition, implications and formula of the root mean squared error (RMSE)?

  • Intuition: (normalized) distance between the vector of predicted values and the vector of actual value
  • Same units as the outcome variable

How can you generate a lift chart?


  • Take sample data set
  • Apply the selected model
  • Sort according to predicted probability of a yes response, i.e.,
    • First instance is the one the model thinks is more likely a yes
    • Next instance is the next most likely Etc.
  • Intuition: more yes at the beginning of the list
  • x-axis is sample percentage, i.e., 20% from the start of the list
  • y-axis is response number, i.e., percentage where the model correctly predicts the positive class

The question on the page originate from the summary of the following study material:

  • A unique study and practice tool
  • Never study anything twice again
  • Get the grades you hope for
  • 100% sure, 100% understanding
Remember faster, study better. Scientifically proven.
Trustpilot Logo