---

Previous post!!
□ Simple linear regression (1/5)- correlation and covariance
□ Simple linear regression (2/5)- slope and intercept of linear regression model

---

---

---

In my previous post, I explained how to calculate slope (β₁) and intercept (β₀) of linear regression model.

x=c(10,20,30,40,50)
y=c(100,120,140,150,160)
dataA=data.frame(x,y)
summary(lm(y~x))

If you well followed my previous posts, you will get the above result, y= 89.0 + 1.5x 

Now our interest is how to calculate standard error in the intercept and slope (Red box). Here is the equation to obtain standard error of intercept (β₀) and slope (β₁).

The following question would be what ei2 is. Let’s calculate ei2 which is the sum of square of error.

n	xi	(xi – x̄)	(xi – x̄)2	yi	ŷ (ŷ= 89.0+1.5x)	(yi – ŷ)	(yi – ŷ)2
1	10	-20.0	400.0	100	104.0	-4.0	16.0
2	20	-10.0	100.0	120	119.0	1.0	1.0
3	30	0.0	0.0	140	134.0	6.0	36.0
4	40	10.0	100.0	150	149.0	1.0	1.0
5	50	20.0	400.0	160	164.0	-4.0	16.0
	x̄ = 30.0		Σ (xi – x̄)2 = 1000.0				Σ (yi – ŷ)2 = 70.0

Σei2 is 70.0. This is Sum Squared Error (SSE), Σ(yi - ŷ)2

If we divide this value by degree of freedom (n-2), it will be variance of error (also we call Mean squared error, MSE). Let’s divide 70.0 by 3 (=5-2).

It would be 23.3 (=70.0 / 3)

Then let’s calculate square root of 23.3.

√ 23.3 = 4.83

We calculate square root of variance. This is the concept of standard deviation.

Therefore, the above equation will be calculated as √(Σ(yi - ŷ)2 / (n-2)) = √70.0 / (5-2) ≈ 4.83

Let’s calculate the other equation.

√(1/5 + 30.02 / 1000.0) ≈ 1.05

Therefore, the standard error of intercept would be

√(Σ(yi - ŷ)2 / (n-2)) * √(1/n + x̄2 / Σ(xi – x̄)2)
= 4.83 * 1.05 ≈ 5.07

The standard error of intercept is 5.07. In R, it was 5.0662. It’s the same value.

---

---

Now, let’s calculate standard error of slope.

We already know variance of error; Σ(yi - ŷ)2 / (n-2)

It’s 23.3

We already calculated Σ (x_i – x̄)²

It’s 1,000

Then SE(b1) will be √ 23.3 / 1000 ≈ 0.153

It’s the same as R provided.

---

---

Mean squared error, MSE

I told you when Sum Squared Error (SSE), Σ(yi - ŷ)2 is divided by degree of freedom (n-2), it will be variance of error (also we call Mean squared error, MSE); Σ(yi - ŷ)2 / (n-2)

Statistical programs automatically calculate this value.

x<- c(10,20,30,40,50)
y<- c(100,120,140,150,160)
dataA<- data.frame(x,y)
linear<-lm(y~x)
summary(linear)
anova(linear)

If you run statistical programs, you will find two results. One is the table for coefficients, and another is the ANOVA table. In ANOVA table, you can find MSE. Here, it’s 23.33 which is the same value we calculated by hand. About ANOVA table in regression model, I’ll explain 5th chapter; R-squared (Coefficient of determination).

---

---

The next question is how to calculate t-value of slope and intercept in the table for coefficients?

The answer will be in the next post!!

---

Follow up!!
□ Simple linear regression (4/5)- t value on the slope and intercept

---

---