The Solovay-Kitaev algorithm gives a construction to $\epsilon$-approximate any $m$-qubit unitary $U$ with $O(m \log(m/\epsilon))$ elementary gates, provided $m$ is a constant.
My question is: if the dimension of $U$ is polynomial in the input size, is it possible to approximate it in polynomial time?