Please always post complete documents showing all packages used. I couldn't locate underbracket. there appears to be a spurious right hand column affecting the spacing of the delimiter, also I removed some superfluous braces.

\documentclass{article} \usepackage{amsmath} \begin{document} $ D = \begin{pmatrix} D_1 & 0 & 0 & & \  0 & \ddots & & & \textrm{\huge 0} \  0 & 0 & D_n & & &  \  \multicolumn{3}{c}{$\upbracefill$}&\ \multicolumn{3}{c}{\scriptstyle n}&\  & & & &  \ & \textrm{\huge 0} & & & \textrm{\huge 0} \  \multicolumn{6}{c}{$\upbracefill$}\ \multicolumn{6}{c}{\scriptstyle N}\ \noalign{\vspace{-2\normalbaselineskip}} \end{pmatrix} $\vspace{2\normalbaselineskip} \end{document} 

Please always post complete documents showing all packages used. I couldn't locate underbracket. there appears to be a spurious right hand column affecting the spacing of the delimiter, also I removed some superfluous braces.

\documentclass{article} \usepackage{amsmath} \begin{document} $ D = \begin{pmatrix} D_1 & 0 & 0 & & \\  0 & \ddots & & & \textrm{\huge 0} \\  0 & 0 & D_n & & & \\ \multicolumn{3}{c}{$\upbracefill$}&\\ \multicolumn{3}{c}{\scriptstyle n}&\\  & & & & \\  & \textrm{\huge 0} & & & \textrm{\huge 0} \\ \multicolumn{6}{c}{$\upbracefill$}\\ \multicolumn{6}{c}{\scriptstyle N}\\ \noalign{\vspace{-2\normalbaselineskip}} \end{pmatrix} $\vspace{2\normalbaselineskip} \end{document} 

Please always post complete documents showing all packages used. I couldn't locate underbracket. there appears to be a spurious right hand column affecting the spacing of the delimiter, also I removed some superfluous braces.

\documentclass{article} \usepackage{amsmath} \begin{document}

$ D = \begin{pmatrix} D_1 & 0 & 0 & & \ 0 & \ddots & & & \textrm{\huge 0} \ 0 & 0 & D_n & & & \ \multicolumn{3}{c}{$\upbracefill$}&\ \multicolumn{3}{c}{\scriptstyle n}&\ & & & & \ & \textrm{\huge 0} & & & \textrm{\huge 0} \ \multicolumn{6}{c}{$\upbracefill$}\ \multicolumn{6}{c}{\scriptstyle N}\ \noalign{\vspace{-2\normalbaselineskip}} \end{pmatrix} $\vspace{2\normalbaselineskip} \end{document} 

Please always post complete documents showing all packages used. I couldn't locate underbracket. there appears to be a spurious right hand column affecting the spacing of the delimiter, also I removed some superfluous braces.

\documentclass{article} \usepackage{amsmath} \begin{document} $ D = \begin{pmatrix} D_1 & 0 & 0 & & \ 0 & \ddots & & & \textrm{\huge 0} \ 0 & 0 & D_n & & & \ \multicolumn{3}{c}{$\upbracefill$}&\ \multicolumn{3}{c}{\scriptstyle n}&\ & & & & \ & \textrm{\huge 0} & & & \textrm{\huge 0} \ \multicolumn{6}{c}{$\upbracefill$}\ \multicolumn{6}{c}{\scriptstyle N}\ \noalign{\vspace{-2\normalbaselineskip}} \end{pmatrix} $\vspace{2\normalbaselineskip} \end{document}