5
$\begingroup$

When you add two independent normal distributions the resulting distributions' variance is the sum of the variances i.e. it gets larger. However, the Central Limit Theorem states that

when independent random variables are added, their properly normalized sum tends toward a normal distribution (informally a bell curve) even if the original variables themselves are not normally distributed. https://en.wikipedia.org/wiki/Central_limit_theorem

and also says that as you add more variables the variance gets smaller.

Why does the variance increase in one case but decrease in the other? Is it because in the CLT case the random variables come from the same underlying distribution?

Improve this question
$\endgroup$
2
  • 5
    $\begingroup$ The CLT deals with sample mean $\bar X,$ for which the standard error is $SD(\bar X) = \sigma/\sqrt{n}.$ $\endgroup$ Commented Nov 23, 2020 at 19:45
  • 1
    $\begingroup$ True, but how's this a comment and not an answer? $\endgroup$ Commented Nov 23, 2020 at 21:16

2 Answers 2

Reset to default
5
$\begingroup$

Let $X_1, X_2\overset{iid}{\sim}F_X(x)$ with $\mu$ and $\sigma^2$ defined and finite.

$$var(X_1+X_2) = var(X_1) + var(X_2) = 2\sigma^2$$

But...

$$var(\bar{X}) = var\bigg(\dfrac{X_1 + X_2}{2}\bigg) = \dfrac{1}{4}var(X_1 + X_2) = \dfrac{1}{2}\sigma^2 $$

(Remember that variance is not linear; the constant pulls out of the variance operator as a squared value.)

Using more than two $iid$ variables results in that familiar $var(\bar{X}) = \dfrac{\sigma^2}{n}$.

$$var(\bar{X}) = var\bigg(\dfrac{1}{n}\sum_{i=1}^n X_i\bigg) = \dfrac{1}{n^2}var\bigg(\sum_{i=1}^n X_i\bigg) = \dfrac{1}{n^2} n\sigma^2 = \dfrac{\sigma^2}{n} $$

Improve this answer
$\endgroup$
0
$\begingroup$

What I understand is that this question is asking why for

$X_1, X_2, ...\overset{iid}{\sim}F_X(x)$ with $X \sim N(\mu,\sigma^2)$

we have that

$Var(X_1), Var(X_1+X_2), Var(X_1+X_2+X_3), ...$ is increasing while

$Var\bigg(\dfrac{X_1}{1}\bigg), Var\bigg(\dfrac{X_1 + X_2}{2}\bigg), Var\bigg(\dfrac{X_1 + X_2 + X_3}{3}\bigg), ...$

is decreasing or at least non-increasing. I guess OP just overlooked that there's a fraction $\frac{\cdot}{n}$ multiplied to the sum.

Dave's answer explains why the 2nd sequence is decreasing, but actually even if variance were for example linear in taking out constants, the sequence is non-increasing due to being constant:

$(Var\bigg(\dfrac{X_1}{1}\bigg), Var\bigg(\dfrac{X_1 + X_2}{2}\bigg), Var\bigg(\dfrac{X_1 + X_2 + X_3}{3}\bigg), ...) = (\dfrac{1\sigma}{1}, \dfrac{2\sigma}{2}, \dfrac{3\sigma}{3}, ...)$

This post actually is not mainly to answer the question but to see if indeed this is what OP was overlooking: that there's a fraction $\frac{\cdot}{n}$ multiplied to the sum?

P.S. Re the wikipedia quote of OP the 'normalised' part links to this page which says

There are different types of normalizations in statistics – nondimensional ratios of errors, residuals, means and standard deviations, which are hence scale invariant – some of which may be summarized as follows.

So I guess the relevant part of the above quote is 'means' and thus the 'normalised sum' in OP's quote refers to taking the mean, i.e. refers to the missing fraction $\frac{\cdot}{n}$ multiplied to the sum.

Improve this answer
$\endgroup$

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.