How to formalize "conditional random variables"

But is this abuse of notation sound?

As others have noted in the comments, the answer is not quite. But to inform your understanding of why not, it may be helpful for you to read about the concept of conditional expectation, which may be the closest formal approximation of what you're trying to get at.

The setup for the definition requires you to brush of on your measure theoretic probability, and consists of:

• A probability space $(\Omega, \mathcal{F}, P)$.
• A random variable $X : \Omega \to \mathbb{R}^n$.
• Another random variable $Y : \Omega \to U$ (where $(U, \Sigma)$ is some other measure space).

The conditional expecation $\mathbb{E}( X \mid Y )$ is, in a precise sense, the $L_2$-closest $Y^{-1}(\Sigma)$-measurable approximation of $X$. That is, it answers the question What is the most that we can know about $X$ given information that we can glean from observing $Y$?

More formally, letting $\mathcal{H} = Y^{-1}(\Sigma)$, $\mathbb{E}(X \mid Y)$ is an $\mathcal{H}$-measurable random variable (i.e. it is as "coarse" as $Y$) which is is guaranteed to agree with $X$ on any event $H \in \mathcal{H}$:

$$ \int_H \mathbb{E}(X \mid Y) \, dP = \int_H X \, dP. $$ Its existence is proved via the Radon-Nikodym theorem.

And furthermore,

Perhaps tangentially, is there a categorical interpretation?

while I don't have a strong grasp of category theory and so I won't try to explain it it categorical terms, conditional expectation does have a nice interpretation in terms of factorization / commutative diagrams, as can be seen on the wikipedia page :)

In the paper (Jardin et al., 2006) the next notation was used:

It is defined as the conditional random variable: $$T - t|T>t, Z(t),$$ where $T$ denotes the random variable of time to failure, $t$ is the current age and $Z(t)$ is the past condition profile up to the current time.

Reference. Jardine A K, Lin D and Banjevic D 2006 Mechanical Systems and Signal Processing 20 1483–1510

The standard construction of $X$ as a function of $Y$ and another uniform random variable $E$ on $[0,1]$ independent of $Y$, such that $X=f_X(Y,E)$, is shown in the paper Distinguishing cause from effect using observational data: methods and benchmarks within the proof of Proposition 4.

The idea is to use the conditional cumulative distribution function

$$F_{X|y}(x)=P(X \leq x|Y=y):=\lim _{h \to 0} \frac{P(X\leq x, \ y-h<Y\leq y+h)}{P(y-h<Y\leq y+h)}$$

to transform the conditional random variable into a uniform random variable on $[0,1]$ under absolute continuous assumption, $E:=F_{X|Y}(X)$.

Then $f_X$ can be defined by

$$f_X(y,e)=F^{-1}_{X|y}(e):=inf\{x\in \mathbb{R}\ |\ e\leq F_{X|y}(x) \}, \ e\in [0,1].$$

This should be the way to interpret $X$ as a random random variable. More explicitly, given $y$, you can transform a uniform distribution to any distribution through the generalized inverse of its cumulative distribution function (the quantile function) and attach it to $y$ as the conditional distribution of $X$, together forming the full random variable $X$.

This was also used in the linkage concept for copula and can have applications even in finance.

If you have studied information theory you should be familiar with information diagrams like this one:

In somne sense, there is a kind of "algebra of set" for (conditional or not) random variables. In fact the entropy acts as a "functor" with values in $(\mathbb{R^+}, \leq)$. The question is then, what is the domain of this functor. As explained by Adam, conditional random variables aren't really random variables.

Shannon's channel coding theorem is hinting at what could be the right objects. There is this concept of source represented by discrete random variables with values in an alphabet. These sources can be compressed, encoded, transmitted and decoded to form some other sources.

For example, given some source, you can encode by bloc of size $n$ this source as a prefix distinct binary code using Huffman algorithm.

But you can also encode the conditional realization of a source $Y$ relative to an other source $X$. Using the asymptotic equipartition property (AEP), you can show that these encodings $B_{Y^n|X^n=x^n}$ are of length $nH(Y|X)\pm o(n)$ (using Landau's notation here).

Using some binary tree rebalancing you can ensure that all your encodings have a minimal length of $nH(Y|X) - |o(n)|$, such that their prefixes of length $nH(Y|X) - |o(n)|$ form a family of surjections indexed by $x^n$. By choosing right inverses for these surjections one can encode $B_{Y^n|X^n=x^n}$ with a prefix $B_{Y^n|X^n}$ which is only a function of $y^n$ and some suffix which depends upon $x^n$ (and $y^n$) whose entropy is an $o(n)$.

Once $B_{Y^n|X^n}$ is defined you can for example speak of $B_{Y^n|B_{X^n|Y^n}}$ whose entropy is $nI(X,Y)+o(n)$. You get a real category with products and coproducts.

Disclaimer, I own no math degree and have no academic experience but the channel coding theorem had quite an impression on me so I tried to define this category and found a new constructive proof of this theorem as a result.

I've written down everything more clearly here, feel free to have a look: https://matovitch.github.io/itccc/.

This hasn't been reviewed so any comment/feedback is welcome.