5
$\begingroup$

I have a bunch of data that I fit a linear regression to, and now I need to find the variance of my slope. Is there an analytical way to get this?

If an example is necessary, consider this my data in R:

x <- c(1:6) y <- c(18, 14, 15, 12, 7, 6) lm(y ~ x)$coefficients

So I have a slope estimate of -2.4, but I want to know the variance of that estimate.

After looking at previous questions, I've seen a few equations for estimating the slope parameter, but I'm a little confused about what the differences between equations are and what approach is valid for my problem.

For example, the answers in this question say that $\newcommand{\Var}{\rm Var}\newcommand{\slope}{\rm slope}\Var[\slope] = \frac{V[Y]}{\sum\left(\frac{x_i-\bar{x}}{\sum(x_i-\bar{x})^2}\right)}$.

This question says that $\Var[\slope] = \frac{V[Y]}{\sum(x_i-\bar{x})^2}$.

And if I look at the output in R (as a "check" mechanism), I'm given two other ways I could potentially calculate the slope variance (one using the standard error, another given the covariance matrix). I feel like I'm missing something key because all these estimates give me similar (but not the same) answer.

Improve this question
$\endgroup$
3
  • 1
    $\begingroup$ Are you wondering how to get R to give this to you? (You would use ?vcov(model).) Or are you wondering where that comes from / why? $\endgroup$ Commented Mar 30, 2015 at 15:50
  • $\begingroup$ I'm less interested in R tricks, and more interested in how I would calculate the variance myself (but using R as a calculator of sorts). Does that make sense? I want to know how to find the variance of a slope, what formula makes sense, and not so much how can I find the answer using R. $\endgroup$ Commented Mar 30, 2015 at 15:54
  • 1
    $\begingroup$ I think the answer you want may be at the linked thread. Please read it. If it isn't what you're want / you still have a question afterwords, come back here & edit your Q to state what you learned & what you still need to know. Then we can provide the information you need w/o just duplicating material elsewhere that already didn't help you. $\endgroup$ Commented Mar 30, 2015 at 16:28

2 Answers 2

Reset to default
5
$\begingroup$

For a standard linear regression that meets the normal assumptions, the variances of your parameter estimates can be taken from the variance covariance matrix, $\Sigma$. For example, the variance of the intercept is the first element on the main diagonal, $\Sigma_{11}$. The variance of the slope on $X_1$ is the second element on the main diagonal, $\Sigma_{22}$, and so on.

There are probably many ways to skin a cat, but the standard calculation for the variance covariance matrix uses the residual variance from your model and your design matrix. Then it is:
$$ \rm{VCov(model)} = s^2(X' X)^{-1} $$ Here's a worked example of the calculations with your data and R: 

x = c(1:6); y = c(18, 14, 15, 12, 7, 6); m = lm(y ~ x) summary(m) # Coefficients: # Estimate Std. Error t value Pr(>|t|) # (Intercept) 20.4000 1.4119 14.45 0.000133 *** # x -2.4000 0.3625 -6.62 0.002700 ** # # Residual standard error: 1.517 on 4 degrees of freedom s = summary(m)$sigma; s # [1] 1.516575 dm = model.matrix(m); dm # (Intercept) x # 1 1 1 # 2 1 2 # 3 1 3 # 4 1 4 # 5 1 5 # 6 1 6 s^2*solve(t(dm)%*%dm) # (Intercept) x # (Intercept) 1.993333 -0.4600000 # x -0.460000 0.1314286 vcov(m) # you can see that this is the same as the manual calculation above # (Intercept) x # (Intercept) 1.993333 -0.4600000 # x -0.460000 0.1314286 sqrt(diag(vcov(m))) # these are the same standard errors as the summary output # (Intercept) x # 1.4118546 0.3625308
Improve this answer
$\endgroup$
2
  • $\begingroup$ I follow that, and I understand and agree that that's how you would manually get the variance/covariance matrix, but I still don't understand the equations in those other questions. The method in your answer gives one result, but those two equations give a different one. Is there some scaling factor the variance covariance matrix doesn't take into account? $\endgroup$ Commented Mar 31, 2015 at 14:14
  • $\begingroup$ @user72320, you can see that both the variance covariance matrix & the manual calculation give exactly the right answer. It the equations from the other Qs differ, they are wrong. Note also that Glen_b's answer below gives the proof for the formula used here. $\endgroup$ Commented Mar 31, 2015 at 14:56
4
$\begingroup$

It's easy to do this for the multiple regression case: \begin{align} \text{Var}((X'X)^{-1}X'y) &= (X'X)^{-1}X'\text{Var}(y)X'(X'X)^{-1} \\ &=(X'X)^{-1}X'(\sigma^2I)X'(X'X)^{-1} \\ &=\sigma^2(X'X)^{-1}X'X'(X'X)^{-1} \\ &=\sigma^2(X'X)^{-1} \end{align} which is usually estimated by $s^2(X'X)^{-1}$

Improve this answer
$\endgroup$
0

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.