State space system, with a state space system as feedback

Write $u_1=r-y_2$ and $u_2 = y_1$, then

\begin{align*} y_1 &= C_1 x_1 + D_1 (r - y_2)\\ &= C_1 x_1 - D_1 C_2 x_2 - D_1 D_2 u_2 + D_1 r \\ (I + D_1 D_2) y_1 &= \begin{bmatrix} C_1 & -D_1 C_2 \end{bmatrix} \begin{bmatrix} x_1 \\ x_2 \end{bmatrix} + D_1 r \\ y_1 &= \begin{bmatrix} R C_1 & -R D_1 C_2 \end{bmatrix} \begin{bmatrix} x_1 \\ x_2 \end{bmatrix} + R D_1 r \\ \end{align*}

where $R = (I + D_1 D_2)^{-1}$ provided that it exists (the system is admissible).

\begin{align*} \dot{x}_1 &= A_1 x_1 + B_1 (r - y_2) \\ &= A_1 x_1 - B_1 C_2 x_2 - B_1 D_2 u_2 + B_1 r \\ &= \begin{bmatrix} A_1 - B_1 D_2 R C_1 & -B_1 C_2 + B_1 D_2 R D_1 C_2 \end{bmatrix} \begin{bmatrix} x_1 \\ x_2 \end{bmatrix} + (- B_1 D_2 R D_1 + B_1) r \\ \dot{x}_2 &= A_2 x_2 + B_2 y_1 \\ &= \begin{bmatrix} B_2 R C_1 & A_2 - B_2 R D_1 C_2 \end{bmatrix} \begin{bmatrix} x_1 \\ x_2 \end{bmatrix} + B_2 R D_1 r \end{align*}

Therefore, the overall system can be written as

\begin{align*} \begin{bmatrix} \dot{x}_1 \\ \dot{x}_2 \end{bmatrix} &= \begin{bmatrix} A_1 - B_1 D_2 R C_1 & B_1 (D_2 R D_1 - I) C_2 \\ B_2 R C_1 & A_2 - B_2 R D_1 C_2 \end{bmatrix} \begin{bmatrix} x_1 \\ x_2 \end{bmatrix} + \begin{bmatrix} B_1(I - D_2 R D_1) \\ B_2 R D_1 \end{bmatrix} r \\ y_1 &= \begin{bmatrix} R C_1 & -R D_1 C_2 \end{bmatrix} \begin{bmatrix} x_1 \\ x_2 \end{bmatrix} + R D_1 r \end{align*}