2
$\begingroup$

Assuming $X$ is a full-rank $n \times p$ design matrix, and $y$ is an $n$ vector, it is possible to solve the regression coefficient vector $\hat{\beta}$ using either $\hat{\beta}=(X^T X)^{-1} X^T y$ ie the OLS method, or by calculating the covariance matricies $S_{XX}$ and $S_{Xy}$, and then $\hat{\beta} = S_{XX}^{-1} S_{Xy}$.

I would like to prove that both expressions are equivalent to each other. The step I can do is expand the sample covariance terms to:

$$\begin{aligned} \hat{\beta} &= S_{XX}^{-1} S_{Xy}\\ &= \left( \frac{1}{n-1} X^TX - \bar{X}^T\bar{X} \right)^{-1} \left( \frac{1}{n-1} X^Ty - \bar{X}^T\bar{y} \right)\\ &=\left( X^TX - \bar{X}^T\bar{X} \right)^{-1} \left( X^Ty - \bar{X}^T\bar{y} \right) \end{aligned}$$

Where I'm stuck is proving that these mean terms $\bar{X}^T\bar{X}$ and $\bar{X}^T\bar{y}$ cancel out to result in the above expression for the OLS coefficients. How can this be done?

$\endgroup$
10
  • $\begingroup$ What does it mean to calculate the covariance between $X$ and $y?$ $\endgroup$ Commented Nov 14, 2021 at 4:09
  • $\begingroup$ en.wikipedia.org/wiki/Cross-covariance_matrix $\endgroup$ Commented Nov 14, 2021 at 4:10
  • $\begingroup$ Although we are dealing with empirical covariances in this case $\endgroup$ Commented Nov 14, 2021 at 4:12
  • $\begingroup$ What are $\bar X \bar X$ and $\bar X \bar y?$ Are these supposed to be vectors? Scalars? Matrices? $\endgroup$ Commented Nov 14, 2021 at 4:31
  • 1
    $\begingroup$ Oh whoops I forgot the transpose there $\endgroup$ Commented Nov 14, 2021 at 4:37

0

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.