Home / Summaries / Class notes - Part 2: Quantitative Analysis / data-outliers-population

Linear Regression with One Regressor

4 important questions on Linear Regression with One Regressor

How do we estimate coefficients of the linear regression model?

Us data to estimate the slope and intercept to get the population regression line. As we don't have the full population, we use a sample. Check the sample for outliers before using them all. R2 is the correlation squared and tells us how much of Ys variation is linearly explained by X.

Do eyeball to get the line? Very unscientific, so we will use "ordinary least squares".

Assumption #1 - Error given X has a mean of zero

The conditional distribution of the error given x, has a mean of zero. Meaning that the other factors (error) are unrelated to X in sense that, given a value of X, the mean of the distribution of these other factors is zero. Like random assignment. Zero covariance, meaning uncorrelated but that is not enough (only linear association).

Often convenient to discuss the conditional mean assumption in terms of possible correlation. IF they are correlated, the assumption is violated.

This implies that on average over the population the prediction is right.

Assumption #2 - X and Y are i.i.d.

Independent and identically distributed. At random. Nonrandom could be when a researcher chooses its own data or when we deal with timeseries. Those clearly violate i.i.d.

Assumption #3 - Large outliers are unlikely

Far outside the usual range of the data is unlikely. Such outliers could be misleading and shift the entire slope upward or downward. To state/test this assumption, do test for kurtosis, the fourth moment, it should be finite. Do plot your data.

The question on the page originate from the summary of the following study material:

Part 2: Quantitative Analysis

View summary