Prediction Models

13 important questions on Prediction Models

What is the problem with both selection methods?

They lead to different models and there is a risk of overfitting regression coeffecient

When are variables kept or deleted in fw or bw selection?

--> if Pentry=Premoval then FW and BW selections lead to = models
-->if Pentry DIFFP removal then FW and BW selections lead to DIFF models

Events per Variable Problem

Too many variables(df) with respect to the number of people who are positive/negative on the outcome.
How to solve that?
10/15 people *each prediction variable(dof)
  • Higher grades + faster learning
  • Never study anything twice
  • 100% sure, 100% understanding
Discover Study Smart

Quality of Prediction model, how to assess that?


•Explained variance(R2, values 0-100%, the higher the better)
•Calibration(=agreement between predicted probabilities of outcome and observed outcome)
•Discrimination(= how well the model can distinguish between the people who have the outcome and those who don't)

What types of calibration are there?

Calibration in the large - mean predicted=mean observed, same for sum predicted/observed
Calibration curve - predicted probabilities against observed probabilities. Ideally in a linear line. scale is 0,1 in both x and y
Hosmer and Lemeshow- agreement predicted and expected numbers

What is the linear predictor?

Combi of regression coefficients with predictor scores. Resulting in the deviation between observed and predicted. You need this for external validation, after freezing coefficients

Calibration slope and what can you use it for?

re-estimate model in the same data, now with lp as the predictor.

It is used to correct overfitting(optimism) of the regression coefficient.

What is the shrinkage factor?

When the slope value is used to correct for deviation of the model in the new dataset

Why do you need to adjust the regression coefficient?

Adjustment of the regression coefficients also leads to adjustment of the intercept

Internal validation of Prediction model

-To determine to what extent the prediction model is feasible/optimistic.
- Internal validation is important to obtain a honest estimate of performance for patients that are similar to those in the development sample
-Methods to internally validate: bootstrapping-estimation optimism-adjustment slope accordingly.

External Validation of Prediction model

Predictions are calculated from the previously developed model, and tested in new data that are different from the development population (e.g. from another hospital).

External validity depends on quality of internal validity and different distribution of predictors/outcome values between development and validation samples

3 steps for external validation of Prediction models?

1. Freeze coefficient=transporting regression coeff to external dataset
2.Calculation new Linear predictor (LP)= multiply regression coeff with values predictors in external dataset
3. Quality model evaluation (discrimination, calibration)

What is the wald value?

The Wald test (also called the Wald Chi-Squared Test) is a way to find out if explanatory variables in a model are significant. “Significant” means that they add something to the model; variables that add nothing can be deleted without affecting the model in any meaningful way.

The question on the page originate from the summary of the following study material:

  • A unique study and practice tool
  • Never study anything twice again
  • Get the grades you hope for
  • 100% sure, 100% understanding
Remember faster, study better. Scientifically proven.
Trustpilot Logo