Skip to main content
added 20 characters in body
Source Link
Neurobro
  • 715
  • 1
  • 10
  • 24

I'd like to calculate the posterior distribution given the prior distribution $w\sim N(0,\Sigma_p)$ and the likelihood $y|X,w\sim N(X^\top w,\sigma_n^2I).$

Ignoring everything that does not contain w:

$p(w|X,y)\propto p(y|X,w)p(w) \propto exp(-{1 \over 2} (\sigma_n^{-2}(y-X^\top w)^\top (y-X^\top w) -w^\top\Sigma_p^{-1}w)= \sigma_n^{-2}(w^\top Xy + y^\top X^\top w)-w^\top(\sigma_n^{-2}XX^\top -\Sigma_p^{-1} )w$$p(w|X,y)\propto p(y|X,w)p(w) \propto exp(-{1 \over 2} (\sigma_n^{-2}(y-X^\top w)^\top (y-X^\top w) -w^\top\Sigma_p^{-1}w))= exp(-{1 \over 2}(\sigma_n^{-2}(w^\top Xy + y^\top X^\top w)-w^\top(\sigma_n^{-2}XX^\top -\Sigma_p^{-1} )w))$

Now the answer should be $exp(-{1 \over 2}(w - \bar w)^\top(\sigma_n^{-2}XX^\top -\Sigma_p^{-1})(w - \bar w))$ where $\bar w =\sigma_n^{-2}(\sigma_n^{-2}XX^\top -\Sigma_p^{-1})^{-1}Xy.$

Could anyone help how to do this?

Thanks!

I'd like to calculate the posterior distribution given the prior distribution $w\sim N(0,\Sigma_p)$ and the likelihood $y|X,w\sim N(X^\top w,\sigma_n^2I).$

Ignoring everything that does not contain w:

$p(w|X,y)\propto p(y|X,w)p(w) \propto exp(-{1 \over 2} (\sigma_n^{-2}(y-X^\top w)^\top (y-X^\top w) -w^\top\Sigma_p^{-1}w)= \sigma_n^{-2}(w^\top Xy + y^\top X^\top w)-w^\top(\sigma_n^{-2}XX^\top -\Sigma_p^{-1} )w$

Now the answer should be $exp(-{1 \over 2}(w - \bar w)^\top(\sigma_n^{-2}XX^\top -\Sigma_p^{-1})(w - \bar w))$ where $\bar w =\sigma_n^{-2}(\sigma_n^{-2}XX^\top -\Sigma_p^{-1})^{-1}Xy.$

Could anyone help how to do this?

Thanks!

I'd like to calculate the posterior distribution given the prior distribution $w\sim N(0,\Sigma_p)$ and the likelihood $y|X,w\sim N(X^\top w,\sigma_n^2I).$

Ignoring everything that does not contain w:

$p(w|X,y)\propto p(y|X,w)p(w) \propto exp(-{1 \over 2} (\sigma_n^{-2}(y-X^\top w)^\top (y-X^\top w) -w^\top\Sigma_p^{-1}w))= exp(-{1 \over 2}(\sigma_n^{-2}(w^\top Xy + y^\top X^\top w)-w^\top(\sigma_n^{-2}XX^\top -\Sigma_p^{-1} )w))$

Now the answer should be $exp(-{1 \over 2}(w - \bar w)^\top(\sigma_n^{-2}XX^\top -\Sigma_p^{-1})(w - \bar w))$ where $\bar w =\sigma_n^{-2}(\sigma_n^{-2}XX^\top -\Sigma_p^{-1})^{-1}Xy.$

Could anyone help how to do this?

Thanks!

Source Link
Neurobro
  • 715
  • 1
  • 10
  • 24

completing the square for matrices

I'd like to calculate the posterior distribution given the prior distribution $w\sim N(0,\Sigma_p)$ and the likelihood $y|X,w\sim N(X^\top w,\sigma_n^2I).$

Ignoring everything that does not contain w:

$p(w|X,y)\propto p(y|X,w)p(w) \propto exp(-{1 \over 2} (\sigma_n^{-2}(y-X^\top w)^\top (y-X^\top w) -w^\top\Sigma_p^{-1}w)= \sigma_n^{-2}(w^\top Xy + y^\top X^\top w)-w^\top(\sigma_n^{-2}XX^\top -\Sigma_p^{-1} )w$

Now the answer should be $exp(-{1 \over 2}(w - \bar w)^\top(\sigma_n^{-2}XX^\top -\Sigma_p^{-1})(w - \bar w))$ where $\bar w =\sigma_n^{-2}(\sigma_n^{-2}XX^\top -\Sigma_p^{-1})^{-1}Xy.$

Could anyone help how to do this?

Thanks!