I'm hoping that someone could help better explain why $\sigma$ (equation 2.19) must be multiplied by $\frac{4n}{4n + 1}$. Obviously all the math is there. Perhaps someone can make this easier to understand.
Reference is here:
I'm hoping that someone could help better explain why $\sigma$ (equation 2.19) must be multiplied by $\frac{4n}{4n + 1}$. Obviously all the math is there. Perhaps someone can make this easier to understand.
Reference is here:
Before answering our question, I would like to remind you of the following definition:
Unbiased Estimator: $\hat{\theta}$ is an unbiased estimator of $\theta$ if $\mathbb{E}[\hat{\theta}]=\theta$.
Let's assume having a sample $\{x_1, x_2,..., x_n\}$, and you want an Unbiased Estimator of it's variance $\sigma^2$, then the following estimator satisfies this property :
$\hat{\sigma^2} = \frac{1}{n-1}\sum_{i=1}^{n}(x_i - \hat{\bar{x}})^2$, where $\hat{\bar{x}}=\frac{1}{n}\sum_{i=1}^{n}x_i$ (which is also an unbiased estimator of the mean $\bar{x}$).
But if you want an unbiased estimator of the standard deviation $\sigma$, then you cannot just take the square root of $\hat{\sigma^2}$, because due to the Jensen's Inequality applied to the case of a concave function $x \rightarrow \phi(x)$ (which the case of $x \rightarrow \sqrt x$) taking the square root of the previous estimator leads to an biased estimator of the standard deviation.
But fortunately, an application of Cochran's Theorem shows that:
if we denote $s = \sqrt{\frac{1}{n-1}\sum_{i=1}^{n}(x_i - \hat{\bar{x}})^2}$ then $\sqrt{n-1}\frac{s}{\sigma}$ has a $\chi_{n-1}$ distribution. Which leads to :
$\mathbb{E}[s]= (\sqrt{\frac{2}{n-1}}\frac{\Gamma(\frac{n}{2})}{\Gamma(\frac{n-1}{2})}) \sigma = (1 + \frac{1}{4n} + O(\frac{1}{n^2})) \sigma = (\frac{4n+1}{4n} + O(n^{-2}))\sigma\approx\frac{4n+1}{4n}\sigma$
Hence if you want to get an unbiased estimator of the standard deviation (to a certain approximation) you need to use:
$s' = \frac{4n}{4n+1} s$ and thus $$\mathbb{E}[s']= \frac{4n}{4n+1} \mathbb{E}[s]=(\frac{4n}{4n+1})(\frac{4n+1}{4n})\sigma = \sigma$$
I hope this will help!