Regression Model
•The relationship b/n X and Y is linear
•The X’s are fixed (non-random)
•The error has zero expectation
•Constant variance
•The errors are independent
•The errors are normally distributed
i i i
Y X
( ) 0E
2 2
( )E
( ) 0
i j
E
How Good is Linear Regression?
•It is the Best Linear Unbiased Estimate
(BLUE)
•Linear because all estimates are ultimately
weighted sums of the observations
•Unbiased because
•Best because regression estimates have the
smallest variance of all linear unbiased
estimates
ˆ
ˆ
( ) ( )E E
Distribution of the Estimates
•The estimates are normally distributed
around the true parameters:
2
2
2
2
2
2
2
ˆ~ ,
ˆ~ ,
ˆ
ˆ
( , )
i
i
i
i
Nx
X
NN x
X
Cov x
Sample Estimate of the Variance
•Since we don’t observe the true variance, we
estimate it from the sample:
•Then, the standard errors of the estimates are:
2
2 2
ˆ
ˆ2
i
sN
2
2
ˆ2
2
2 2
ˆ2
i
i
i
s
sx
X
s s N x
Hypothesis Testing and
Confidence Intervals
•Since we estimate the variance, we use
t – distribution:
0
ˆ
