15
$\begingroup$

I am learning Maximum Likelihood Estimation.

Per this post, the log of the PDF for a normal distribution looks like this:

$$ \log{\left(f\left(x_i;\,\mu,\sigma^2\right)\right)} = - \frac{n}{2} \log{\left(2 \pi\right)} - \frac{n}{2} \log{\left(\sigma^2\right)} - \frac{1}{2 \sigma^2} \sum{{\left(x_i - \mu\right)}^2} \tag{1} $$

According to any Probability Theory textbook, the formula of the PDF for a normal distribution: $$ \frac {1}{\sigma \sqrt {2\pi}} e^{-\frac {(x - \mu)^2}{2\sigma ^2}} \rlap{\qquad \text{where}~-\infty <x<\infty} \tag{2} $$

Taking the log of Expression 2 produces

\begin{align} \ln\left(\frac{1}{\sigma \sqrt {2\pi}} e^{-\frac{\left(x - \mu\right)^2}{2\sigma ^2}}\right) &= \ln\left(\frac {1}{\sigma \sqrt {2\pi}}\right)+\ln{\left(e^{-\frac {(x - \mu)^2}{2\sigma ^2}}\right)} \tag{3} \\[5px] &=-\ln\left(\sigma\right)-\frac{1}{2} \ln\left(2\pi\right) - \frac{\left(x - \mu\right)^2}{2\sigma ^2} \tag{4} \end{align}

which is very different from Equation 1.

Is Equation 1 right? What am I missing?

Improve this question
$\endgroup$
2
  • 10
    $\begingroup$ Your first equation is the joint log-pdf of a sample of n iid normal random variables (AKA the log-likelihood of that sample). The second equation is the the log-pdf of a single normal random variable $\endgroup$ Commented Apr 20, 2019 at 22:26
  • 3
    $\begingroup$ @ArtemMavrin, I think your comment would be a perfectly good answer if you expanded on just a bit to make it slightly more clear. $\endgroup$ Commented Apr 20, 2019 at 22:47

1 Answer 1

Reset to default
20
$\begingroup$

For a single observed value $x$ you have log-likelihood:

$$\ell_x(\mu,\sigma^2) = - \ln \sigma - \frac{1}{2} \ln (2 \pi) - \frac{1}{2} \Big( \frac{x-\mu}{\sigma} \Big)^2.$$

For a sample of IID observed values $\mathbf{x} = (x_1,...,x_n)$ you then have:

$$\ell_\mathbf{x}(\mu,\sigma^2) = \sum_{i=1}^n \ell_{x_i}(\mu,\sigma^2) = - n \ln \sigma - \frac{n}{2} \ln (2 \pi) - \frac{1}{2 \sigma^2} \sum_{i=1}^n (x_i-\mu)^2.$$

(Note that in both cases you can remove the constant term from the log-likelihood, but then it is not the logarithm of the sampling density.)

Improve this answer
$\endgroup$

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.