In a univariate regression, $Y=a+bX+e$, the solution for slope b is given by $COV(X,Y)/VAR(X)$.
Is there a similar expression for a bivariate regression $Y=a+bX+cZ+e$. What is the closed form solution for b and c in this case?
1
- $\begingroup$ Terminology: bivariate regression would normally refer to the case where there are a pair of response variates ($Y_1$ and $Y_2$, say), rather than two predictors; note that when there are multiple predictors ("x"-variables) it is called multiple regression, not multivariate regression (see item 2 there). $\endgroup$Glen_b– Glen_b2015-11-19 22:43:22 +00:00Commented Nov 19, 2015 at 22:43
Add a comment |
2 Answers
$\begingroup$ $\endgroup$
Given $$y=\beta_1+\beta_2 x_2 + \beta_3 x_3 +\varepsilon $$ with the usual assumptions, OLS would correspond to solving these normal equations
$$ n \cdot b_1 + b_2 \sum x_2 + b_3 \sum x_3 = \sum y \\ b_1 \sum x_2 + b_2 \sum x_2^2 + b_3 \sum x_2 \cdot x_3= \sum x_2 \cdot y \\ b_1 \sum x_3 + b_2 \sum x_2 \cdot x_3 + b_3 \sum x_3^2= \sum x_3 \cdot y, $$
where $n$ is the sample size. I don't think there's a neat interpretation like there is with the 2D case.
$\begingroup$ $\endgroup$
3 Yes, it is called ordinary least squares (OLS) and it is covered in most handbooks:
$$ \hat{\boldsymbol\beta} = (\mathbf{X}^{\rm T}\mathbf{X})^{-1} \mathbf{X}^{\rm T}\mathbf{y} $$
where $\hat{\boldsymbol\beta}$ and $\mathbf{X}$ are matrices and $\mathbf{y}$ is a vector. See also this thread.
- $\begingroup$ Thanks, this is pointing in the right direction. What would be the result for a bivariate regression (with 1 intercept and 2 regressors) without using matrix notation? I am looking for an expression just in terms of covariances and variances. $\endgroup$user95626– user956262015-11-19 14:39:14 +00:00Commented Nov 19, 2015 at 14:39
- $\begingroup$ You have covariance of three variables (X,Y,Z) and their variances, you are interested in their multivariate relations and so things are getting messy, so we are using mathematical tools like matrix algebra in here. $\endgroup$Tim– Tim2015-11-19 14:45:31 +00:00Commented Nov 19, 2015 at 14:45
- 3$\begingroup$ It's actually much simpler to use the matrix algebra; indeed, once you follow it, it's also the easier way to look at simple linear regression. $\endgroup$Glen_b– Glen_b2015-11-19 22:45:29 +00:00Commented Nov 19, 2015 at 22:45