3
$\begingroup$

Consider the $N$ i.i.d. values

$$ X_i \sim \mathcal{N}(0, \sigma^2) $$

such that

$$ Z_i = \sum_{j=1}^i X_j $$

I am interested in the distribution

$$ f(X_i | Z_N = z) $$

Mean

Under the condition, the $X_i$ are no longer independent, but they should remain identically distributed. Hence

$$ Z_N = \sum_{i=1}^N X_i = z $$

implies

$$ \sum_{i=1}^N \mathbb{E}[X_i] = z $$

and therefore

$$ \mathbb{E}[X_i] = z/N $$

Variance

Empirically I found the variance to be

$$ var(X_i) = \frac{N-1}{N} \sigma^2 $$

Distribution

If you start doing the calculations to find the distribution working backwards from $X_N$ I believe you are simply finding the product of many Gaussian distributions so I believe the distribution is going to be Gaussian. However I ran into issues with this after the first step.

Question

How do I show rigorously that the marginal distribution

$$ (X_i | Z_N = z) \sim \mathcal{N} \left (\frac{z}{N}, \frac{N-1}{N} \sigma^2 \right) $$

Improve this question
$\endgroup$

2 Answers 2

Reset to default
1
$\begingroup$

It all follows from the properties of multivariate normals. Since $X_i$ are independent and normally distributed, they're jointly normal, which means any linear combination of them is also jointly normal with them. So, $p_{\mathbf{X},Z_N}(\mathbf{x},z)$ is a multivariate normal, which in turn means $p_{X_i,Z_N}(x,z)$ is multivariate normal with

$$\mu=\begin{bmatrix}0\\0\end{bmatrix},\Sigma=\begin{bmatrix}\sigma^2&\sigma^2\\\sigma^2&N\sigma^2\end{bmatrix}$$

Because $p_{X_i,Z_N}(x,z)$ is MV normal, the conditional distributon $p_{X_i|Z_N}(x,z)$ is univariate normal, and conditional expectation and the variance can be found (same as your answer) via the formulas under "Conditional distributions" section in the wiki page linked above.

Improve this answer
$\endgroup$
1
$\begingroup$

It's a bit messy, which might explain why it isn't seen more. Here is a sketch starting from the bivariate case, which generalizes. I'll use $X$ and $Y$ and $Z=X+Y.$

First let's find the conditional cdf for $Z$ given $X=x.$

$$F_{Z|X=x}=P \left[ X+Y \leq z \ | \ X=x\right]=P[Y \leq z-x]=F_Y(z-x)$$

Then the conditional pdf is found by differentiating: $$f_{Z|X=x}=f_{Z|X}(z|x)=f_Y(z-x)$$

That in turn means the joint density function is

$$f_{X,Z}(x,z) = f_{Z|X}(z|x)f_X(x)=f_Y(z-x)f_X(x)$$

Finally, we get the conditional density formula

$$ f_{X|X+Y=z}(x,z)=\frac{f_Y(z-x)f_X(x)}{f_Z(z)} \ \ \ \ \ \ [1]$$

Now let's look at the normal case. Let $$Z=X_1 + X_2 + \cdots + X_N$$ and $$Y = X_1 + X_2 + \cdots + X_{N-1},$$ where

$$ X_i \sim \mathcal{N} \left( \mu, \sigma^2 \right) $$ and $$ Y \sim \mathcal{N} \left( \left( N - 1 \right) \mu, \left( N-1 \right) \sigma^2 \right) $$

Now what is the pdf $f_{X_N|Z}(x,z)$?

Using $[1],$

$$f_{X|Z=z}(x,z) = \frac{ \left( \frac{1} {\sqrt{{2 \pi \left(N-1 \right)\sigma^2} } } \right) \left( e^{\frac{- \left( z-x- \left( N-1 \right) \mu \right)^2 } {2 \left( N - 1 \right) \sigma^2}} \right) \left( \frac{1}{\sqrt{2 \pi \sigma^2}} \right) \left( e^{\frac{- \left( x- \mu \right)^2}{2 \sigma^2}} \right) } {\left( \frac{1} {\sqrt{{2 \pi N \sigma^2} } } \right) \left( e^{\frac{- \left( z - N \mu \right)^2}{2 N \sigma^2}} \right)}$$

After much simplifying (probably worth going throug once, but I'm not putting the details in here at this time), this can be expressed as

$$f_{X|Z=z}(x,z) =\left( \frac{1}{\sqrt{2 \pi s^2 }} \right) \left( e^{\frac{- \left( x - \frac{z}{N} \right)^2}{2 s^2}}\right), $$

where $$s^2 = \frac{\left( N - 1 \right) \sigma^2}{N}$$

This is now recognizable as a normal pdf and we can see that $$X_i|Z \sim \mathcal{N} \left( {\frac{z}{N}}, \frac{N-1}{N} \sigma^2 \right) $$ as you found empirically.

Improve this answer
$\endgroup$

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.