$A$ has the eigenbasis $\{| a \rangle \}$.
Let $\lambda$ be the eigenvalue of $B$ and $v \in V_{\lambda}$ an eigenvector ($Bv = \lambda v$).
Since $A$ and $B$ commute and are hermitian, following must be true:
$A^\dagger = A$, $B^\dagger = B$ and they are both diagonizable (source).
$$\Rightarrow Bv = \lambda v$$
$$\Rightarrow BAv = ABv = \lambda Av$$ $\rightarrow Av$ is also eigenvector of $B$ for the eigenvalue $\lambda$ and $Av \in V_{\lambda}$.
This is where I am stuck, I am not sure how to proceed and show that they must have the same eigenbasis but I think this must be the right trail.