1
$\begingroup$

Consider the following linear regression model: $Y=\beta_0 + \beta_1 x_i + error $, where $y_i,...,y_n$ are normally distributed and $error \sim N(0,\sigma^2)$. I am trying to find the MLEs for $\beta_0$ and $\beta_1$.

The likelihood function is:

$$ L(\beta_0,\beta_1,\sigma^2) = \prod_{i=1}^{n} \frac{e^{-\frac{(y_i - (\beta_ + \beta_1 x_i))^2}{2\sigma^2}}}{\sqrt{2\pi\sigma^2}} $$

The log-likelihood function is:

$$ l(\beta_0,\beta_1,\sigma^2) = \sum_{i=1}^{n} {-\frac{(y_i - (\beta_0 + \beta_1 x_i))^2}{2\sigma^2}} - \frac{1}{2}log(2\pi\sigma^2) $$

Next, I differentiate to find the MLEs:

$$ \frac{\partial}{\partial\beta_0} l(\beta_0,\beta_1,\sigma^2) = 0 \rightarrow n\bar{y} - n\beta_0 - n\beta_1\bar{x} = 0 \rightarrow \hat{\beta_0} = \bar{y} - \hat{\beta_1}\bar{x} $$

$$ \frac{\partial}{\partial\beta_1} l(\beta_0,\beta_1,\sigma^2) = 0 \rightarrow \sum_{i=1}^{n} y_i x_i - n \beta_0\bar{x} - \beta_1\sum_{i=1}^{n} x_i^2 = 0 $$

Now plugging $\hat{\beta_{0}}$ from equation 1 into equation 2:

$$ \sum_{i=1}^{n} y_i x_i - n\big(\bar{y} - \hat{\beta_1}\bar{x}\big)\bar{x} - \hat{\beta_1}\sum_{i=1}^{n} x_i^2 = 0 \\ \sum_{i=1}^{n} y_i x_i - n\bar{y}\bar{x} = \hat{\beta_1}\sum_{i=1}^{n} x_i^2 - n\hat{\beta_1}\bar{x_i}^2 \\ \hat{\beta_1} = \frac{\sum_{i=1}^{n} y_i x_i - n\bar{y}\bar{x}}{\sum_{i=1}^{n} x_i^2 - n\bar{x_i}^2} $$

My answer for $\hat{\beta_0}$ appears correct. However, every textbook that I have consulted shows that $\hat{b_1} = \frac{\sum_{i=1}^{n} (x_i - \bar{x})(y_i - \bar{y}) }{\sum_{i=1}^{n} (x_i - \bar{x})^2} $. Where have I gone wrong?

$\endgroup$

1 Answer 1

2
$\begingroup$

We have $$\begin{eqnarray}\sum_{i=1}^n(x_i-\bar x)(y_i-\bar y) &=&\sum_{i=1}^n x_iy_i- \bar x\sum_{i=1}^ny_i - \bar{y}\sum_{i=1}^nx_i + \bar x\bar y \sum_{i=1}^n (1) \\&=& \sum_{i=1}^nx_iy_i-n\bar x\bar y - n \bar x \bar y + n\bar x \bar y \\ &=&\sum_{i=1}^n x_i y_i - n \bar x \bar y\end{eqnarray}$$

so your numerator's right. Maybe your denominator is too.

$\endgroup$

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.