Update README.md

Author: flowel1

README.md

@@ -46,3 +46,19 @@ which is satisfied when the displacement <ins>&delta;</ins> solves the following
 <a href="https://www.codecogs.com/eqnedit.php?latex=(J^{(0)}^T&space;\cdot&space;J^{(0)})&space;\cdot&space;\underline{\delta}&space;=&space;J^{(0)}&space;\cdot&space;\underline{\epsilon}^{(0)}" target="_blank"><img src="https://latex.codecogs.com/gif.latex?(J^{(0)}^T&space;\cdot&space;J^{(0)})&space;\cdot&space;\underline{\delta}&space;=&space;J^{(0)}&space;\cdot&space;\underline{\epsilon}^{(0)}" title="(J^{(0)}^T \cdot J^{(0)}) \cdot \underline{\delta} = J^{(0)} \cdot \underline{\epsilon}^{(0)}" /></a>
 
 ![alt-text](https://github.com/flowel1/nonlinear-regression/blob/master/pictures/quadratic-approx.png)
+
+## Extension: weighted observations and priors
+
+<a href="https://www.codecogs.com/eqnedit.php?latex=\text{Gaussian&space;prior:&space;}&space;p(\theta_j)&space;=&space;\frac{\beta_j}{\sqrt{2&space;\pi}}&space;\cdot&space;\exp{&space;\left&space;(&space;-\frac{1}{2}&space;\beta_j^2&space;(\theta_j&space;-&space;\mu_j)^2&space;\right&space;)&space;}" target="_blank"><img src="https://latex.codecogs.com/gif.latex?\text{Gaussian&space;prior:&space;}&space;p(\theta_j)&space;=&space;\frac{\beta_j}{\sqrt{2&space;\pi}}&space;\cdot&space;\exp{&space;\left&space;(&space;-\frac{1}{2}&space;\beta_j^2&space;(\theta_j&space;-&space;\mu_j)^2&space;\right&space;)&space;}" title="\text{Gaussian prior: } p(\theta_j) = \frac{\beta_j}{\sqrt{2 \pi}} \cdot \exp{ \left ( -\frac{1}{2} \beta_j^2 (\theta_j - \mu_j)^2 \right ) }" /></a>
+
+<a href="https://www.codecogs.com/eqnedit.php?latex=\text{Lognormal&space;prior:&space;}&space;p(\theta_j)&space;=&space;\frac{\beta_j}{\theta_j&space;\cdot&space;\sqrt{2&space;\pi}}&space;\cdot&space;\exp{&space;\left&space;(&space;-\frac{1}{2}&space;\beta_j^2&space;(\log\theta_j&space;-&space;\log\mu_j)^2&space;\right&space;)&space;}" target="_blank"><img src="https://latex.codecogs.com/gif.latex?\text{Lognormal&space;prior:&space;}&space;p(\theta_j)&space;=&space;\frac{\beta_j}{\theta_j&space;\cdot&space;\sqrt{2&space;\pi}}&space;\cdot&space;\exp{&space;\left&space;(&space;-\frac{1}{2}&space;\beta_j^2&space;(\log\theta_j&space;-&space;\log\mu_j)^2&space;\right&space;)&space;}" title="\text{Lognormal prior: } p(\theta_j) = \frac{\beta_j}{\theta_j \cdot \sqrt{2 \pi}} \cdot \exp{ \left ( -\frac{1}{2} \beta_j^2 (\log\theta_j - \log\mu_j)^2 \right ) }" /></a>
+
+<a href="https://www.codecogs.com/eqnedit.php?latex=L(\underline{\theta})&space;=p(\underline{\theta}&space;|&space;X,&space;\underline{y})&space;=&space;\frac{p(\underline{y}&space;|&space;X,&space;\underline{\theta})&space;\cdot&space;p(\underline{\theta})}{p(\underline{y}&space;|&space;X)}" target="_blank"><img src="https://latex.codecogs.com/gif.latex?L(\underline{\theta})&space;=p(\underline{\theta}&space;|&space;X,&space;\underline{y})&space;=&space;\frac{p(\underline{y}&space;|&space;X,&space;\underline{\theta})&space;\cdot&space;p(\underline{\theta})}{p(\underline{y}&space;|&space;X)}" title="L(\underline{\theta}) =p(\underline{\theta} | X, \underline{y}) = \frac{p(\underline{y} | X, \underline{\theta}) \cdot p(\underline{\theta})}{p(\underline{y} | X)}" /></a>
+
+Since
+
+<a href="https://www.codecogs.com/eqnedit.php?latex=p(\underline{y}&space;|&space;X,&space;\underline{\theta})&space;=&space;\prod_{i&space;=&space;1}^N&space;p(y_i&space;|&space;\underline{x}_i,&space;\underline{\theta})&space;=&space;\prod_{i&space;=&space;1}^N&space;\frac{1}{\sigma&space;\sqrt{\frac{2&space;\pi}{w_i}}}&space;\cdot&space;\exp{&space;\left(&space;-\frac{1}{2}&space;w_i&space;\frac{(y_i&space;-&space;f(\underline{x}_i,&space;\underline{\theta}))^2}{\sigma^2}{}\right)}" target="_blank"><img src="https://latex.codecogs.com/gif.latex?p(\underline{y}&space;|&space;X,&space;\underline{\theta})&space;=&space;\prod_{i&space;=&space;1}^N&space;p(y_i&space;|&space;\underline{x}_i,&space;\underline{\theta})&space;=&space;\prod_{i&space;=&space;1}^N&space;\frac{1}{\sigma&space;\sqrt{\frac{2&space;\pi}{w_i}}}&space;\cdot&space;\exp{&space;\left(&space;-\frac{1}{2}&space;w_i&space;\frac{(y_i&space;-&space;f(\underline{x}_i,&space;\underline{\theta}))^2}{\sigma^2}{}\right)}" title="p(\underline{y} | X, \underline{\theta}) = \prod_{i = 1}^N p(y_i | \underline{x}_i, \underline{\theta}) = \prod_{i = 1}^N \frac{1}{\sigma \sqrt{\frac{2 \pi}{w_i}}} \cdot \exp{ \left( -\frac{1}{2} w_i \frac{(y_i - f(\underline{x}_i, \underline{\theta}))^2}{\sigma^2}{}\right)}" /></a>
+
+setting <a href="https://www.codecogs.com/eqnedit.php?latex=W&space;=&space;\text{diag}\{\sqrt{w_1},&space;...&space;\sqrt{w_N}&space;\}" target="_blank"><img src="https://latex.codecogs.com/gif.latex?W&space;=&space;\text{diag}\{\sqrt{w_1},&space;...&space;\sqrt{w_N}&space;\}" title="W = \text{diag}\{\sqrt{w_1}, ... \sqrt{w_N} \}" /></a>, we have
+
+<a href="https://www.codecogs.com/eqnedit.php?latex=\log&space;L(\underline{\theta})&space;=&space;\text{const}&space;-\frac{N}{2}&space;\log(\sigma^2)&space;-\frac{1}{2\sigma^2}&space;\left&space;\|&space;W&space;\cdot&space;(\underline{y}&space;-&space;f(X,&space;\underline{\theta}))&space;\right&space;\|^2&space;&plus;&space;\newline&space;-\frac{1}{2}&space;\cdot&space;\sum_{\{j&space;|&space;p(\theta_j)&space;\text{&space;gaussian}\}}&space;\beta_j^2&space;(\theta_j&space;-&space;\mu_j)^2&space;\newline&space;-\frac{1}{2}&space;\cdot&space;\sum_{\{j&space;|&space;p(\theta_j)&space;\text{&space;lognormal}\}}&space;\{&space;-\log\theta_j&space;&plus;&space;\beta_j^2&space;(\log\theta_j&space;-&space;\log\mu_j)^2\}" target="_blank"><img src="https://latex.codecogs.com/gif.latex?\log&space;L(\underline{\theta})&space;=&space;\text{const}&space;-\frac{N}{2}&space;\log(\sigma^2)&space;-\frac{1}{2\sigma^2}&space;\left&space;\|&space;W&space;\cdot&space;(\underline{y}&space;-&space;f(X,&space;\underline{\theta}))&space;\right&space;\|^2&space;&plus;&space;\newline&space;-\frac{1}{2}&space;\cdot&space;\sum_{\{j&space;|&space;p(\theta_j)&space;\text{&space;gaussian}\}}&space;\beta_j^2&space;(\theta_j&space;-&space;\mu_j)^2&space;\newline&space;-\frac{1}{2}&space;\cdot&space;\sum_{\{j&space;|&space;p(\theta_j)&space;\text{&space;lognormal}\}}&space;\{&space;-\log\theta_j&space;&plus;&space;\beta_j^2&space;(\log\theta_j&space;-&space;\log\mu_j)^2\}" title="\log L(\underline{\theta}) = \text{const} -\frac{N}{2} \log(\sigma^2) -\frac{1}{2\sigma^2} \left \| W \cdot (\underline{y} - f(X, \underline{\theta})) \right \|^2 + \newline -\frac{1}{2} \cdot \sum_{\{j | p(\theta_j) \text{ gaussian}\}} \beta_j^2 (\theta_j - \mu_j)^2 \newline -\frac{1}{2} \cdot \sum_{\{j | p(\theta_j) \text{ lognormal}\}} \{ -\log\theta_j + \beta_j^2 (\log\theta_j - \log\mu_j)^2\}" /></a>