Does Independence Imply Conditional Independence in regression?

Consider a single underlying probability space with a measure $\mu_\theta$ that governs all possible realizations of $(X_i, Y_i)_{i=1}^n$. Full independence of each pair $(X_i, Y_i)$ means that for any measurable sets $A_i \subseteq \mathcal{X}$ and $B_i \subseteq \mathcal{Y}$, the measure satisfies:

$$ \mu_\theta\Bigl(\prod_{i=1}^n (A_i \times B_i)\Bigr) = \prod_{i=1}^n \mu_\theta\bigl(A_i \times B_i\bigr) $$

From the chain rule applied to each factor $P(x_i,y_i\mid\theta)$, one obtains:

$$ P(x_1,y_1,\dots,x_n,y_n\mid\theta) = \prod_{i=1}^n P(y_i\mid x_i,\theta)P(x_i\mid\theta) $$

Conditional independence of $Y_1,\dots,Y_n$ given $X_1,\dots,X_n$ requires that for almost every configuration of $(x_1,\dots,x_n)$, the conditional distribution factorizes as:

$$ P(y_1,\dots,y_n\mid x_1,\dots,x_n,\theta) = \prod_{i=1}^n P(y_i\mid x_i,\theta) $$

This imposes no factorization on $P(x_1,\dots,x_n\mid\theta)$, so the joint distribution becomes:

$$ P(x_1,y_1,\dots,x_n,y_n\mid\theta) = \Bigl[\prod_{i=1}^n P(y_i\mid x_i,\theta)\Bigr] P(x_1,\dots,x_n\mid\theta) $$

These factorizations match exactly the two scenarios discussed: full independence treats $(X_i,Y_i)$ as mutually independent pairs, while conditional independence concerns only the conditional structure of $Y_i$ given $X_i$. The former implies the latter, because if each pair is mutually independent, then:

$$ P(x_i,y_i\mid\theta) = P(y_i\mid x_i,\theta)P(x_i\mid \theta) $$

and the product over all $i$ yields:

$$ \prod_{i=1}^n P(x_i,y_i\mid\theta) = \Bigl[\prod_{i=1}^n P(y_i\mid x_i,\theta)\Bigr]\Bigl[\prod_{i=1}^n P(x_i\mid\theta)\Bigr] $$

which directly recovers the conditional factorization of the outputs.

We can also view this from the perspective of filtrations and martingale theory. Define two filtrations:

$$ \mathcal{F}_k = \sigma\bigl((X_1,Y_1),\dots,(X_k,Y_k)\bigr), \quad \mathcal{G}_k = \sigma\bigl(X_1,\dots,X_n,Y_1,\dots,Y_k\bigr) $$

In both independence regimes, one can show that:

$$ \mathbb{E}[Y_k\mid \mathcal{F}_{k-1}] = \mathbb{E}[Y_k\mid X_k], \quad \mathbb{E}[Y_k\mid \mathcal{G}_{k-1}] = \mathbb{E}[Y_k\mid X_k] $$

Hence both full independence and conditional independence imply that the sequence:

$$ M_k = \sum_{i=1}^k \bigl(Y_i - \mathbb{E}[Y_i\mid X_i]\bigr) $$

is a martingale with respect to its respective filtration. The difference emerges in the variability of this martingale: under full independence,

$$ \langle M\rangle_k = \sum_{i=1}^k \mathrm{Var}(Y_i\mid X_i) $$

whereas conditional independence allows:

$$ \langle M\rangle_k = \sum_{i=1}^k \mathrm{Var}(Y_i\mid X_i,\{X_j\}_{j\neq i}) $$

which reduces to the same sum as above if and only if:

$$ \mathrm{Var}(Y_i\mid X_i) = \mathrm{Var}(Y_i\mid X_i,\{X_j\}_{j\neq i}) $$

Full independence guarantees that no additional knowledge of other $(X_j,Y_j)$ can further reduce $\mathrm{Var}(Y_i\mid X_i)$. Under mere conditional independence, the first-order (martingale) structure is preserved but higher-order moment relationships may differ, revealing potential correlations among the residuals. Letting:

$$ \eta_k = Y_k - \mathbb{E}[Y_k\mid X_k] $$

yields a sequence of martingale differences under both structures, yet full independence enforces:

$$ \mathrm{Cov}(\eta_i,\eta_j\mid X_i,X_j)=0 \quad (i\neq j) $$

whereas conditional independence alone does not. The innovation process $\{\eta_k\}$ thus remains uncorrelated across $i,j$ only in the fully independent regime.

In the case of independent pairs

if the training observations $(x_i,y_i)$ represent independent random draws

then for a specific $k$, the distribution of $y_k,x_k$ is independent from the other $y_i$ and $x_i$. Conditioning on the other $x_i$ doesn't change anything about that.

You get that in both cases the distribution is just a product of individual terms

Unconditional

$$f(\mathbf{y},\mathbf{x}) = \prod_i f_i(y_i,x_i) = \prod_i g_i(y_i|x_i) h_i(x_i)$$

Conditional

$$f(\mathbf{y}|\mathbf{x}) = \frac{f(\mathbf{y},\mathbf{x})}{h(\mathbf{x})} = \frac{ \prod_i f_i(y_i,x_i)}{\prod_i h_i(x)} = \prod_i g_i(y_i|x_i)$$

---

In terms of your cases

For iid $x_i$ we have

$$P(x_1, x_2, \dots, x_n \mid \theta) = \prod_{i=1}^n P(x_i \mid \theta)$$

and the case 1 can always be written in the form of case 2.

---

More generally

A way how conditioning can change independence of $y_i$ into dependence of the $y_i$ is when the condition $x_i$ is not independent of the $y_k,x_k$ with $k\neq i$. Collider bias is an example (see also here: Can spurious correlations exist in the (theoretical) population?).