Matrix multiplication is defined as:
Let $A$ be a $n \times m$ matrix and $B$ a $m\times p$ matrix, the product $AB$ is defined as a matrix of size $n\times p$ such that $(AB)_i,_j = \sum\limits_{k=1}^mA_i,_kB_k,_j$.
For what good reason did mathematicians define it like this?
A matrix is nothing but a particular representation of a linear map (with respect to a choice of basis in source and target space). The formula is what results (naturally) if you look at the composition of such maps and write them down using a matrix.
One reason is that it gives you associativity with a vector: If $A$ and $B$ are matrices and $x$ is a vector, then
$$ (AB)x = A(Bx) $$
ETA: This doesn't say anything more than Thomas's answer, by the way; I thought it might help if it were presented in a more tangible way, though.
Let $V$ be a finite-dimensional vector space, $\mathcal{B}$ be a basis of $V$ and $f,g\in\textrm{End}(V)$. One has: $$\textrm{mat}_{\mathcal{B}}(f\circ g)=\textrm{mat}_{\mathcal{B}}(f)\times\textrm{mat}_{\mathcal{B}}(g).$$ I think the definition of matrix multiplication derives from there.
