Simple linear regression (3/5)- standard error of slope and intercept
Previous post!!
□ Simple linear regression (1/5)- correlation and covariance
□ Simple linear regression (2/5)- slope and intercept of linear regression model
In my previous post, I explained how to calculate slope (β1) and intercept (β0) of linear regression model.
x=c(10,20,30,40,50)
y=c(100,120,140,150,160)
dataA=data.frame(x,y)
summary(lm(y~x))
If you well followed my previous posts, you will get the above result, y= 89.0 + 1.5x
Now our interest is how to calculate standard error in the intercept and slope (Red box). Here is the equation to obtain standard error of intercept (β0) and slope (β1).
The following question would be what ei2 is. Let’s calculate ei2 which is the sum of square of error.
| n | xi | (xi – x̄) | (xi – x̄)2 | yi | ŷ (ŷ= 89.0+1.5x) | (yi – ŷ) | (yi – ŷ)2 |
| 1 | 10 | -20.0 | 400.0 | 100 | 104.0 | -4.0 | 16.0 |
| 2 | 20 | -10.0 | 100.0 | 120 | 119.0 | 1.0 | 1.0 |
| 3 | 30 | 0.0 | 0.0 | 140 | 134.0 | 6.0 | 36.0 |
| 4 | 40 | 10.0 | 100.0 | 150 | 149.0 | 1.0 | 1.0 |
| 5 | 50 | 20.0 | 400.0 | 160 | 164.0 | -4.0 | 16.0 |
| x̄ = 30.0 | Σ (xi – x̄)2 = 1000.0 | Σ (yi – ŷ)2 = 70.0 |
Σei2 is 70.0. This is Sum Squared Error (SSE), Σ(yi - ŷ)2
If we divide this value by degree of freedom (n-2), it will be variance of error (also we call Mean squared error, MSE). Let’s divide 70.0 by 3 (=5-2).
It would be 23.3 (=70.0 / 3)
Then let’s calculate square root of 23.3.
√ 23.3 = 4.83
We calculate square root of variance. This is the concept of standard deviation.
Therefore, the above equation will be calculated as √(Σ(yi - ŷ)2 / (n-2)) = √70.0 / (5-2) ≈ 4.83
Let’s calculate the other equation.
√(1/5 + 30.02 / 1000.0) ≈ 1.05
Therefore, the standard error of intercept would be
√(Σ(yi - ŷ)2 / (n-2)) * √(1/n + x̄2 / Σ(xi – x̄)2)
= 4.83 * 1.05 ≈ 5.07
The standard error of intercept is 5.07. In R, it was 5.0662. It’s the same value.
Now, let’s calculate standard error of slope.
We already know variance of error; Σ(yi - ŷ)2 / (n-2)
It’s 23.3
We already calculated Σ (xi – x̄)2
It’s 1,000
Then SE(b1) will be √ 23.3 / 1000 ≈ 0.153
It’s the same as R provided.
Mean squared error, MSE
I told you when Sum Squared Error (SSE), Σ(yi - ŷ)2 is divided by degree of freedom (n-2), it will be variance of error (also we call Mean squared error, MSE); Σ(yi - ŷ)2 / (n-2)
Statistical programs automatically calculate this value.
x<- c(10,20,30,40,50)
y<- c(100,120,140,150,160)
dataA<- data.frame(x,y)
linear<-lm(y~x)
summary(linear)
anova(linear)
If you run statistical programs, you will find two results. One is the table for coefficients, and another is the ANOVA table. In ANOVA table, you can find MSE. Here, it’s 23.33 which is the same value we calculated by hand. About ANOVA table in regression model, I’ll explain 5th chapter; R-squared (Coefficient of determination).
The next question is how to calculate t-value of slope and intercept in the table for coefficients?
The answer will be in the next post!!
Follow up!!
□ Simple linear regression (4/5)- t value on the slope and intercept
