Regression with a Single Regressor

7 important questions on Regression with a Single Regressor

What about two-sided hypotheses concerning B1?

H0 = certain value, H1 = Unequal to this value, three steps:
- Compute standard error
- Compute t-statistic
- Compute P-value (smallest significance level) or compare with critical value

T-statistic = ( estimator - Hypothesis value ) / Standard error of the estimator
In large samples the sampling distribution of Y is approximately normal.

Difficult formula for standard error: 1/2 * ((1/(n-2)* sum(Xi-Mean(X))^2* error^2) / (1/n * SUM(Xi - MEan(x))^2)^2.

What about one-sided hypotheses concerning B1?

Because under the alternative B1 could be either larger or smaller than B10. So sometimes it might me appropriate to use a one-sided hypothesis.
H0= certain value, H1 < or > then a certain value

In practice, one-sided alternative hypotheses should be used only when there is a clear reason for doing so (economic theory). But ambiguity/skepticism often leads econometricians to use two-sided tests.

What about confidence levels?

Sampling always goes with uncertainty. Based on the OLS estimator and its standard error you can construct a confidence interval for slope and intercept.

95% CL (+/- 1.96*SE) means the set of values that cannot be rejected using a two-sided hypothesis test with a significance of 5%. Or 95% probability that we have the "true" value.

Can also be used to make a predicted effect by multiplying low level and high level by delta X.
  • Higher grades + faster learning
  • Never study anything twice
  • 100% sure, 100% understanding
Discover Study Smart

What about a binary variable?

Binary is 0 or 1, implying the use of a dummy (indicator) variable. Mechanics are exactly the same but the interpretation of B1 should be seen as the mean difference of 0 and 1. Like a mean difference analysis, there is no slope or line. Simply the coefficient (multiplying) on Dummy. If groups are the same the B1 should be zero.

What about hetero- and homoskedasticity?

If besides the assumption that the error conditional on X has a mean zero, we also observe the variance of the error not to be dependent of X, we call it homoskedasticity. Meaning that the variance is constant over the distribution. Like pay checks among male and female, this will be heterosekdastic as the distribution for men is much larger.

OLS estimators remain unbiased and consistent, even if homoskedastic or heteroskedastic. Moreover, in large samples asymptotically normal.

What about efficient estimator (Gauss-Markov)?

If leas square assumptions hold and the errors are homoskedastic the estimators B0 and B1 are efficient (all linear). In such a case a homoskedasticity-only standard error can be used Variance of error/SUM(Xi-Mean(X))^2

Note: some use this as a default!! Underestimate t-statistic, even in normal distribution and large samples. Therefore heteroskedastic-robust SE always more conservative.

Economic theory rarely gives any reason to believe errors are homoskedastic. Therefore be more prudent and conservative unless you have compelling reasons to believe otherwise.

What if the sample size is small?

The exact distribution on the t-statistic is complicated and depends on the unknown population distribution of the data. But if the three assumptions hold, errors are normally distributed, homoskedastic, then the estimators are normal and the t-statistic has a STUDENT T-distribution. 

STUDENT is more prudent but in larger samples the difference is negligible. Moreover rarely homoskedastic and normally distributed so get large datasets :p

The question on the page originate from the summary of the following study material:

  • A unique study and practice tool
  • Never study anything twice again
  • Get the grades you hope for
  • 100% sure, 100% understanding
Remember faster, study better. Scientifically proven.
Trustpilot Logo