This does have a closed-form solution, which I will state but not derive. First, define $$f(u,v,w) \equiv uv\operatorname{arsinh}\frac{w}{\sqrt{u^2+v^2}} + vw\operatorname{arsinh}\frac{u}{\sqrt{v^2+w^2}} + wu\operatorname{arsinh}\frac{v}{\sqrt{w^2+u^2}} - \frac{u^2}{2}\arctan\frac{vw}{u\sqrt{u^2+v^2+w^2}} - \frac{v^2}{2}\arctan\frac{wu}{v\sqrt{u^2+v^2+w^2}} - \frac{w^2}{2}\arctan\frac{uv}{w\sqrt{u^2+v^2+w^2}}.$$ The gravitational potential of a cuboid of uniform density $\rho$ centered at the origin, aligned with the Cartesian axes and having dimensions $2a$, $2b$, $2c$ in the $x$, $y$, $z$ directions respectively is $$\phi(x,y,z) = -G\rho \sum_{k_1,k_2,k_3 \in \{-1,+1\}} k_1 k_2 k_3 f(x+k_1 a,y+k_2 b,z+k_3 c) \\ = -G\rho[f(x+a,y+b,z+c) + f(x+a,y-b,z-c) + f(x-a,y+b,z-c) + f(x-a,y-b,z+c) - f(x-a,y+b,z+c) - f(x+a,y-b,z+c) - f(x+a,y+b,z-c) - f(x-a,y-b,z-c)]$$ and the gravitational field is $-\nabla\phi$. Sample plots are shown below:
3D gravitational field

Larger, square face (i.e. top and bottom)
- Normal component:

- Tangential component:

- Normal component:
Smaller, square face (i.e. sides)
- Normal component:

- Tangential component:

- Normal component:
So your intuition that the gravitational field away from the center of a face has a tangential component is correct. However, for your dimensions, the gravitational field at the center of a larger, square face is actually about 8% weaker than that at the center of a smaller, rectangular face. In addition, within each of the larger, square faces, the point with the strongest magnitude of gravity is not the center, but rather near the center of each of the four edges.