Premise: I have an $n × q$ matrix $X$ and a $q × a$ matrix $C$ with $n > q > a$.

I'm interested in the structure of the matrix $$ M = X X^+ - X_0 X_0^+ $$ where the superscript $^+$ indicates the Moore–Penrose pseudoinverse and $$ X_0 = X (I_q - C C^+). $$

I assume that $X$ is of full column rank and therefore $X^+ = (X' X)^{-1} X$ (where $'$ indicates the transpose).

$M$ is the difference of two orthogonal projection matrices, where the second projects into a subspace of the subspace the first projects into. This makes the difference an orthogonal projection matrix itself (symmetric and idempotent), which means it has a representation $$ M = X_\Delta X_\Delta^+. $$

Question: How do I obtain $X_\Delta$?

My approach: I am guessing that $$ X_\Delta = X - X_0 X_0^+ X, $$ and this seems to be confirmed by numerical tests. But I am unable to come up with a proof, i.e. to show that $$ (X - X_0 X_0^+ X) (X - X_0 X_0^+ X)^+ = X X^+ - X_0 X_0^+. $$

The problem is how to deal with the pseudoinverse of a difference. One can write $$ X_\Delta = (I_n - X_0 X_0^+) X, $$ and according to Wikipedia, in the pseudoinverse of a product where one factor is an orthogonal projection, the orthogonal projection can be redundantly multiplied to the opposite side, meaning here $$ X_\Delta^+ = [(I_n - X_0 X_0^+) X]^+ = [(I_n - X_0 X_0^+) X]^+ (I_n - X_0 X_0^+) = X_\Delta^+ (I_n - X_0 X_0^+), $$ but that doesn't seem to help.

I can prove that $M$ is symmetric and idempotent, using the relations $$ X X^+ X_0 = X_0 \quad \text{and} \quad X_0 X_0^+ X X^+ = X_0 X_0^+, $$ which derive from the definition of $X_0$ and the properties of the pseudoinverse. I can also show that $$ X X_0^+ = X_0 X_0^+ $$ using the property of the pseudoinverse of a product involving an orthogonal projection (see above). But none of that helps either.

Premise: I have an $n × q$ matrix $X$ and a $q × a$ matrix $C$ with $n > q > a$.

I'm interested in the structure of the matrix $$ M = X X^+ - X_0 X_0^+ $$ where the superscript $^+$ indicates the Moore–Penrose pseudoinverse and $$ X_0 = X (I_q - C C^+). $$

I assume that $X$ is of full column rank and therefore $X^+ = (X' X)^{-1} X$ (where $'$ indicates the transpose).

$M$ is the difference of two orthogonal projection matrices, where the second projects into a subspace of the subspace the first projects into. This makes the difference an orthogonal projection matrix itself (symmetric and idempotent), which means it has a representation $$ M = X_\Delta X_\Delta^+. $$

Question: How do I obtain $X_\Delta$?

My approach: I am guessing that $$ X_\Delta = X - X_0 X_0^+ X, $$ and this seems to be confirmed by numerical tests. But I am unable to come up with a proof, i.e. to show that $$ (X - X_0 X_0^+ X) (X - X_0 X_0^+ X)^+ = X X^+ - X_0 X_0^+. $$

The problem is how to deal with the pseudoinverse of a difference. One can write $$ X_\Delta = (I_n - X_0 X_0^+) X, $$ and according to Wikipedia, in the pseudoinverse of a product where one factor is an orthogonal projection, the orthogonal projection can be redundantly multiplied to the opposite side, meaning here $$ X_\Delta^+ = [(I_n - X_0 X_0^+) X]^+ = [(I_n - X_0 X_0^+) X]^+ (I_n - X_0 X_0^+) = X_\Delta^+ (I_n - X_0 X_0^+), $$ but that doesn't seem to help.

I can prove that $M$ is symmetric and idempotent, using the relations $$ X X^+ X_0 = X_0 \quad \text{and} \quad X_0 X_0^+ X X^+ = X_0 X_0^+, $$ which derive from the definition of $X_0$ and the properties of the pseudoinverse. I can also show that $$ X X_0^+ = X_0 X_0^+ $$ using the property of the pseudoinverse of a product involving an orthogonal projection (see above). But none of that helps either.

Premise: I have an $n × q$ design matrix $X$ of a linear model and a $q × a$ contrast matrix $C$ specifying an effect of interest, with $n > q > a$.

I'm interested in the structure of the matrix $$ M = X X^- - X_0 X_0^- $$ where the superscript $^-$ indicates the Moore–Penrose pseudoinverse and $$ X_0 = X (I_q - C C^-) $$ is the reduced design matrix corresponding to the case that the effect of interest is 0. This matrix $M$ occurs in the formulation of standard test statistics like univariate $F$ and multivariate Hotelling's trace, and can be seen as extracting that part of the data that is accounted for by the effect indicated by $C$.

I assume that the contrast is estimable, meaning that $$ X^- X C = C, $$ or that $C'$ lies in the row space of $X$, and that $X$ is of full rank and therefore $X^- = (X' X)^{-1} X$ (where $'$ indicates the transpose).

Question: $M$ is the difference of two orthogonal projection matrices, and because $C$ is estimable, it is an orthogonal projection matrix itself (symmetric and idempotent). $M$ is an $n × n$ matrix, but it is of the same rank as $C$, $$ \DeclareMathOperator{\rk}{rank} \rk M = \rk C = c \leq a < n, $$ which means its spectral representation can be written in terms of $c$ $n$-dimensional vectors (eigenvectors whose eigenvalues are 1).

Is there a way to obtain these $c$ vectors without explicitly calculating $M$ and then its spectral decomposition?

The question is motivated by an application in data analysis, where I need to store a large number of such matrices.

My own attempts: In my view, the key to this question lies in the simplification of the expression $$ X_0^- = [X (I_q - C C^-)]^-. $$ If I could reformulate $X_0^-$ in terms of $X^-$ and a factor depending on $C$ only, I could unify the two terms in the definition of $M$ and hopefully "guess" its spectral decomposition. In other words, how can I understand the effect of $C$ not on $X$ (leading to $X_0$) but on $X^-$ leading to $X_0^-$?

I tried to use the properties of the Moore–Penrose pseudoinverse with respect to products and with respect to projections, but as far as I can tell, none of them help.

Premise: I have an $n × q$ matrix $X$ and a $q × a$ matrix $C$ with $n > q > a$.

I'm interested in the structure of the matrix $$ M = X X^+ - X_0 X_0^+ $$ where the superscript $^+$ indicates the Moore–Penrose pseudoinverse and $$ X_0 = X (I_q - C C^+). $$

I assume that $X$ is of full column rank and therefore $X^+ = (X' X)^{-1} X$ (where $'$ indicates the transpose).

$M$ is the difference of two orthogonal projection matrices, where the second projects into a subspace of the subspace the first projects into. This makes the difference an orthogonal projection matrix itself (symmetric and idempotent), which means it has a representation $$ M = X_\Delta X_\Delta^+. $$

Question: How do I obtain $X_\Delta$?

My approach: I am guessing that $$ X_\Delta = X - X_0 X_0^+ X, $$ and this seems to be confirmed by numerical tests. But I am unable to come up with a proof, i.e. to show that $$ (X - X_0 X_0^+ X) (X - X_0 X_0^+ X)^+ = X X^+ - X_0 X_0^+. $$

The problem is how to deal with the pseudoinverse of a difference. One can write $$ X_\Delta = (I_n - X_0 X_0^+) X, $$ and according to Wikipedia, in the pseudoinverse of a product where one factor is an orthogonal projection, the orthogonal projection can be redundantly multiplied to the opposite side, meaning here $$ X_\Delta^+ = [(I_n - X_0 X_0^+) X]^+ = [(I_n - X_0 X_0^+) X]^+ (I_n - X_0 X_0^+) = X_\Delta^+ (I_n - X_0 X_0^+), $$ but that doesn't seem to help.

I can prove that $M$ is symmetric and idempotent, using the relations $$ X X^+ X_0 = X_0 \quad \text{and} \quad X_0 X_0^+ X X^+ = X_0 X_0^+, $$ which derive from the definition of $X_0$ and the properties of the pseudoinverse. I can also show that $$ X X_0^+ = X_0 X_0^+ $$ using the property of the pseudoinverse of a product involving an orthogonal projection (see above). But none of that helps either.

Premise: I have an $n × q$ design matrix $X$ of a linear model and a $q × a$ contrast matrix $C$ specifying an effect of interest, with $n > q > a$.

I'm interested in the structure of the matrix $$ M = X X^- - X_0 X_0^- $$ where the superscript $^-$ indicates the Moore–Penrose pseudoinverse and $$ X_0 = X (I_q - C C^-) $$ is the reduced design matrix corresponding to the case that the effect of interest is 0. This matrix $M$ occurs in the formulation of standard test statistics like univariate $F$ and multivariate Hotelling's trace, and can be seen as extracting that part of the data that is accounted for by the effect indicated by $C$.

I assume that the contrast is estimable, meaning that $$ X^- X C = C, $$ or that $C'$ lies in the row space of $X$, and that $X$ is of full rank and therefore $X^- = (X' X)^{-1} X$ (where $'$ indicates the transpose).

Question: $M$ is the difference of two orthogonal projection matrices, and because $C$ is estimable, it is an orthogonal projection matrix itself (symmetric and idempotent). $M$ is an $n × n$ matrix, but it of the same rank as $C$, $$ \DeclareMathOperator{\rk}{rank} \rk M = \rk C = c \leq a < n, $$ which means its spectral representation can be written in terms of $c$ $n$-dimensional vectors (eigenvectors whose eigenvalues are 1).

Is there a way to obtain these $c$ vectors without explicitly calculating $M$ and then its spectral decomposition?

The question is motivated by an application in data analysis, where I need to store a large number of such matrices.

My own attempts: In my view, the key to this question lies in the simplification of the expression $$ X_0^- = [X (I_q - C C^-)]^-. $$ If I could reformulate $X_0^-$ in terms of $X^-$ and a factor depending on $C$ only, I could unify the two terms in the definition of $M$ and hopefully "guess" its spectral decomposition. In other words, how can I understand the effect of $C$ not on $X$ (leading to $X_0$) but on $X^-$ leading to $X_0^-$?

I tried to use the properties of the Moore–Penrose pseudoinverse with respect to products and with respect to projections, but as far as I can tell, none of them help.

Premise: I have an $n × q$ design matrix $X$ of a linear model and a $q × a$ contrast matrix $C$ specifying an effect of interest, with $n > q > a$.

I'm interested in the structure of the matrix $$ M = X X^- - X_0 X_0^- $$ where the superscript $^-$ indicates the Moore–Penrose pseudoinverse and $$ X_0 = X (I_q - C C^-) $$ is the reduced design matrix corresponding to the case that the effect of interest is 0. This matrix $M$ occurs in the formulation of standard test statistics like univariate $F$ and multivariate Hotelling's trace, and can be seen as extracting that part of the data that is accounted for by the effect indicated by $C$.

I assume that the contrast is estimable, meaning that $$ X^- X C = C, $$ or that $C'$ lies in the row space of $X$, and that $X$ is of full rank and therefore $X^- = (X' X)^{-1} X$ (where $'$ indicates the transpose).

Question: $M$ is the difference of two orthogonal projection matrices, and because $C$ is estimable, it is an orthogonal projection matrix itself (symmetric and idempotent). $M$ is an $n × n$ matrix, but it is of the same rank as $C$, $$ \DeclareMathOperator{\rk}{rank} \rk M = \rk C = c \leq a < n, $$ which means its spectral representation can be written in terms of $c$ $n$-dimensional vectors (eigenvectors whose eigenvalues are 1).

Is there a way to obtain these $c$ vectors without explicitly calculating $M$ and then its spectral decomposition?

The question is motivated by an application in data analysis, where I need to store a large number of such matrices.

My own attempts: In my view, the key to this question lies in the simplification of the expression $$ X_0^- = [X (I_q - C C^-)]^-. $$ If I could reformulate $X_0^-$ in terms of $X^-$ and a factor depending on $C$ only, I could unify the two terms in the definition of $M$ and hopefully "guess" its spectral decomposition. In other words, how can I understand the effect of $C$ not on $X$ (leading to $X_0$) but on $X^-$ leading to $X_0^-$?

I tried to use the properties of the Moore–Penrose pseudoinverse with respect to products and with respect to projections, but as far as I can tell, none of them help.