MATL, 2424 23 bytes
:Q+"@X+"@X!XJ_JXzJ32>S(c@_X! Order is URDL, 01-based. So (as in the example in the challenge)1 is Ù, 2isR` etc.
Inputs are: number of times, initial direction, char matrix (using ; as row separator).