1
$\begingroup$

I've come across some equations on regression coefficients for multiple linear regression(Fig.1 and Fig.2) with 2 independent variables and was wondering how they were derived.

enter image description here

FIG.1

enter image description here

FIG.2

I'm aware of how to derive the regression coefficients using Matrix Notation, but this intrigued me(and seeing the derivation of the equations in non-matrix form always develops my intuition better than the matrix form proofs). When I saw figure 2 I figured(no pun intended) it's just an extrapolated version of the single linear regression equation(Fig.3)

enter image description here

Fig.3

and assumed that one would have to use something along the lines of a Multiple Correlation coefficient and just multiply that by the (StdevY)/(Stdev of Respective Independent Variable of Regression Coefficient). But then I realized the equation in Fig.2, while it might be using correlation coefficients, isn't actually the multiple correlation coefficient itself. I can't find any sources online(I pretty much only use online sources. I am not a stats student. Someone who is just genuinely curious about statistics.) that actually detail how they derived these equations for the regression coefficients in Fig.1 and Fig.2 let alone explain the intuition behind it(which is very important for me and was very useful in understanding the reasoning behind the Single Linear Regression equations.). If anyone is aware of such sources/books that perhaps even explain how to derive in them in non-matrix form as the number of independent variables gets bigger (although at that Point I understand it becomes very cumbersome and Linear Algebra is a much neater trick) I'd be very grateful. Thank you very much

$\endgroup$
1
  • $\begingroup$ Nothing? Anyone? $\endgroup$ Commented Mar 29, 2024 at 15:56

1 Answer 1

0
$\begingroup$

Check out the Frisch-Waugh-Lovell Theorem.

Let $$y = X\beta + u$$ be the regression model in matrix notation. Write $$y = Z\alpha + Q\delta + u,$$ where $Z$ is $2\times n$ matrix with the first column being a vector of $n$ ones and the second column is the vector of your regressor of interest (e.g., $x_2$). Put $M = I_n - Q(Q'Q)^{-1}Q'$. Then $$\underbrace{My}_{=:\epsilon} = MX\beta + Mu = \underbrace{MZ}_{=:W}\alpha + \underbrace{MQ}_{=0}\delta + \underbrace{Mu}_{=:v} = W\alpha + v.$$ Here $\epsilon$ are the residuals from a regression of $y$ on $Q$, and $W$ are the residuals from a regression of $Z$ on $Q$. Now observe that $$\epsilon = W\alpha + v$$ is a univariate regression and you can obtain the estimate for the intercept and slope as you have described. Note that the correlation of $\epsilon$ and the column of interest in $Q$ is called partial correlation.

See also this thread: Partial Correlation and Partial (Linear) Regression

$\endgroup$

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.